Getting Started with Web Services
This chapter provides an approach to getting started with Web services. It provides three basic experiments that use Web services and then uses the story about C. R.’s business trip to address more advanced uses of Web services. At the end is a vision of what Web services might mean for the future.
All Web Services Connections Look the Same
By now, you probably have noticed that the Web services protocols for connecting internal services are no different than the protocols needed for connecting internal services to external services. Web services and the pervasiveness of HTTP connections make it relatively easy to connect internal and external services together.
The Impact of Web Services
For many companies, the initial impact that Web services will have is to make existing forms of integration simpler. This will create more opportunities for integration and data exchange. These opportunities may occur within an organization or between organizations.
The story of C. R.’s business trip illustrates some examples of what Web services (along with service-oriented architecture and cloud computing) might mean for all of us. In addition to connectivity, we are seeing businesses provide all sorts of services that can be integrated1 with internal systems. (This is the blurring of internal and external services mentioned on page 37.) Advances in technology can take advantage of these services and will eventually be able do to such things as handle travel arrangements and help us manage our lives (as illustrated by the virtual personal assistant in the story of C. R.’s business trip).
The integration opportunities presented by Web services are making the use of Web services a requirement for many organizations. The software affected will range from desktop systems and mobile devices to distributed enterprise systems and sophisticated cloud-based systems.
Use of Web Services will Likely Spur Innovation
The problem with predicting how Web services will affect our systems is that the effect is not always immediately apparent. In some ways, it’s fair to compare the evolution of the use of Web services to the evolution of the use of the Internet in how it affects all that we do. For example, when the Internet first became available, who knew that online shopping would be setting records,2 that we would connect with friends and acquaintances via social networks, that the Internet would make it possible to stream movies and video to our desktop or TV, or that grandparents would buy personal computers to exchange email or video chat with their grandchildren? Web services constitute a similar situation in that businesses will think of all sorts of new and creative ways to use this capability.
Another way to look at C. R.’s business trip is the importance of the Internet/cloud and the services offered. The prevalence of connections in the cloud is enticing developers to leverage those services into all sorts of new creative services that, in turn, make adoption of a service-oriented architecture (SOA) an offer few businesses can afford to refuse.
Start by Experimenting with Web Services
One way to get started with Web services is to consider small projects that have a high chance of success. Keeping the use of Web services to something basic further enhances the chance of success. Choose a project that will be helpful but not vital. Choose team members who like to play with possibilities. Be sure to communicate that the project is an experiment.
Use an external service
Probably the best place to start is using an external service. There are many simple external services from which to choose. For example, you could use weather forecasts, stock information, or news feeds. More examples of relatively simple external services can be found at http://www.programmableweb.com/apis/directory.
Perhaps the easiest project is to create a webpage that displays something available from an external service. This project would provide experience at using Web services for sending and receiving messages. It will give you an idea of how Web services work and where you might want to try your hand at developing an internal service. Figure 11.1 illustrates using an external service to display content on a webpage.
Figure 11.1 Using an external service to display content on a webpage.
Develop an internal service
Experimenting with the development of an internal service allows your organization to get more deeply into the details of Web services. There are two options for developing an internal service:
If you have existing systems that you would like to use with Web services, the second option might be more useful.
This project is similar to the previous one. The difference is that the content displayed comes from an internal system. Examples of such access include obtaining customer contact information or internal employee telephone numbers.
This requires the development of an adapter. It will do two things:
1. Transform a Web services message into a request format that can be accepted by the internal system.
2. Transform the response from the internal system into a Web services message.
There are adapter toolkits to help you build an adapter. Of course, if your organization is more likely to use existing adapters, consider either an open-source adapter or a commercial adapter for an existing system and incorporate that adapter into this project. Figure 11.2 illustrates using an internal service to display content on a webpage.
Figure 11.2 Using an internal service to display content on a webpage.
Exchange data between existing systems
If your organization is likely to use Web services to exchange data between existing internal systems, then it would be appropriate to add an experimental project that does just that. Figure 11.3 illustrates this exchange of data between internal systems.
Figure 11.3 Using Web services to exchange data between internal systems.
This project uses the experience from the previous experimental project of using an adapter. In this project, however, two internal systems exchange data. For example, both systems A and B may allow users to enter customer address and contact information. If one system updates this customer information, the other system should also receive the update. Another example might be that system A has the master account for customer information and system B may request system A to validate that a customer identification number is correct.
Both systems A and B would need adapters. The development would require agreement on the semantics of the vocabulary in the messages exchanged by the adapters. This would create the opportunity to investigate and possibly use the semantic vocabulary developed by standards efforts in your industry.
Use an ESB
Many organizations are also likely to use an enterprise service bus (ESB). If your organization is planning to use an ESB, then the previous experiment can be modified to include one. Figure 11.4 shows the use of Web services with an ESB to exchange data between internal systems.
Figure 11.4 Using Web services with an ESB to exchange data between internal systems.
The intent of this experimental project is to gain appreciation of the issues related to message routing as well as experience in using an ESB.
You probably won’t need to buy an ESB for this experiment. There are open-source ESBs, some vendors may let you try their ESBs, and—perhaps even simpler—some ESBs are available as a service in the cloud.
Staffing issues
It is important to pick the right people to do this experimentation. Frankly, in most situations it is risky to involve people who have never expressed much interest in trying something new. Instead, choose people who like to experiment and take risks. For many organizations, it would be good to bring in someone from outside the organization who is familiar with Web services to mentor developers during these efforts. The mentor would be a “second set of eyes” during this experimentation stage and would be a great source of information. Keep the project team small. A few people would be appropriate for most organizations.
Likely change issues
The most likely change issues you will encounter are:
Lack of training/understanding. This is a rational concern. People will need training on Web services. You will need to find the appropriate training for those involved in the experimentation. Also, you need to be ready to dispel any misunderstandings concerning the use of Web services by your organization.
Inertia—why change? Be prepared to communicate on many levels and in many ways why you want this experimentation to occur. Be available for personal chats. Be prepared, as well, to really listen to concerns expressed.
Adapt Existing Systems to Use Web Services
Once you have some experience using Web services, look for some places in your existing systems where Web services could save time and money in the short term. At this point, you might consider trying the incremental SOA analysis introduced in Chapter 10. It might help you ensure that you are taking on the smallest, shortest-duration project.
To illustrate adapting existing systems to Web services, I’m going to return to the story about C. R. His organization had a repository that was originally an enterprise data warehouse (EDW). Like many organizations, C. R.’s had common data that was replicated in multiple systems, creating an opportunity. For example, his organization had common customer data in multiple systems. These systems were either developed over time in separate departments or they were purchased software packages. Some were systems used by other organizations that his organization had acquired over time. In any case, the systems were different, had replicated data, and in some cases, inconsistent data. C. R.’s organization saw business advantages to creating more visibility of customers for such purposes as cross-selling among departments, creating new packages of products for specific customers, and simply reducing waste in misrouted or duplicated mail.
Enterprise database warehouse
For some people, the very idea of creating an EDW can be discouraging. Many of us have had the experience of failed efforts to create master files like an EDW (see page 100). Here are some tips:
Use an existing master file. You might already have a master file that is part of packaged software your organization owns. It might make sense to adopt that as the master file. If you do not have a master file in packaged software your organization owns but are considering the purchase of packaged software, check to see if the software being considered includes a master file that could be used as an EDW.
Buy a model. This option is often overlooked. Many models can be purchased. Sometimes they are referred to as universal data models (see page 108). The fact is that, although every organization is unique in some way, most of the data is pretty standard. For example, there are practical and flexible models for keeping basic customer information such as addresses and other contact information. Often, these models are simply better than anything an organization might build itself. Experienced modelers who have created models for many organizations usually are the people who design these models. If you buy a model, you should resist any efforts to extensively modify it. See the next tip.
Don’t start a modeling project. A modeling project opens your organization to any number of restraining forces, including our problems are special, power of an internal expert, and lack of training and understanding. The lack of training and understanding is significant. Data modeling appears deceptively simple until you get into it. Even if you are doing something as basic as a customer master, you can get yourself pretty knotted up in the options of data modeling. A modeling project also is an opportunity to add “bells and whistles” to a basic model. Starting a modeling project is essentially creating an environment for “analysis paralysis.”
Start small. Your EDW does not need to be perfect. It simply needs to be useful or effective. Also, you can always add to your EDW at some later date. So, if either an existing master file or a purchased model has many fields you could populate, try to limit the data to what might be most useful. Don’t make the project any larger than it needs to be. You can always add more data to the EDW later.
C. R.’s organization took on the task of data cleansing to populate the EDW. Figure 11.5 illustrates this. An EDW is at the left in the figure. At the right is an existing internal system, and existing applications are above the existing system.
Figure 11.5 Creating a customer data warehouse.
It may seem that I have oversimplified what needs to be done. In a sense I have, but only because at this point the EDW is meant to achieve a limited goal of consolidating a small amount of data—in this example, customer master data. In C. R.’s story, the EDW did eventually grow to be very large. C. R.’s organization, like most organizations, had multiple sources of customer data in its existing systems. This process started with one existing system and then moved on to others.
Connect components to Web services
Figure 11.6 shows the three components that C. R.’s organization connected using Web services:
EDW that was populated after data cleansing along with associated business intelligence (BI)/analytics software
Figure 11.6 Connecting a data warehouse and an internal system with Web services.
C. R.’s organization decided that to keep the EDW updated, changes were needed to internal system A in Figure 11.6. Internal system A needed to send updates to the EDW as they came in from the existing applications. Those updates were routed to the data warehouse using the ESB. Internal system A was also updated so that the data going to the data warehouse was of the highest quality.
Additional systems
C. R.’s organization repeated the data cleansing and populating of the EDW for each of their additional systems that would provide data for the EDW. Eventually, the systems architecture looked like Figure 11.7.
Figure 11.7 Adding additional systems.
With the data from the additional systems, the EDW is the source of data for the development of future services. That reduced the impact on the existing internal operational systems.
In summary, for C. R.’s organization, the EDW and ESB:
Reduced the risk of any one internal system not being able to complete processing that is dependent on data from another system.
Required using fewer adapters since each internal system needs only to have an adapter that works with the ESB.
Reduced the possible negative impact of requests for data that is outside the normal processing of the internal systems.
Staffing issues
For this type of effort, costs could start to increase significantly because more people are getting involved. At this point, you could have at least one person but perhaps two or three people who have a reasonable understanding of Web services. They can form the core of this team along with a few new people. The entire team, however, should be under seven people. This is also a good time to establish the methodology that will be used going forward.
Likely change issues
The most likely change issues you will encounter in this type effort are:
Lack of training/understanding. This is still a rational concern. The new people will likely need training. Don’t assume that the people who have been doing the experimentation are the right ones to do the training. It would be best to have the training done professionally so that any bad habits that may have crept in aren’t passed along. Also, be ready to dispel any misunderstandings concerning Web services and SOAs. This will likely require communication in many ways on many levels, including upper management.
Power of the internal “expert.” Be careful that an internal expert does not sink the project in this stage. It is important that you select the right people and plan for a second set of eyes for each person involved. This may help counter the internal expert if he or she is on the team.
Our problems are special. This will show up if you buy a model. It will be important to get this resistance out in the open as soon as possible so that you can deal with it. Recall the “But it’s so complicated!” scenario on page 95. It could happen to you.
Vision of the Future
The effect of Web services means we are going to have fewer people involved in IT. The jobs in IT will also generally change to creating architectures and often realizing those architectures by making the connections to services in the cloud. At the same time, the quality of software will improve because progressively less new code will need to be written.
The industry will standardize on the capabilities of various services. An analogy was provided earlier to how the audio-video (AV) industry eventually settled on the capabilities of various AV components. The same will happen with services. As this happens, it will become easier to find services in the cloud and connect them with internal services. Already, fewer people build custom software because it is cheaper to purchase commercial off-the-shelf software and tailor it to an organization’s needs. This is a trend that will continue with services. For many organizations, staying competitive will mean taking advantage of the services available in the cloud. Some organizations may find that they have unique services that they can provide, and IT staff will be needed to create those services. Nevertheless, there will be fewer jobs involving custom development.
With the eventual standardization of services, it will become easier to replace one service with another. (This would be similar to replacing one AV receiver component with another that has more capabilities.) This should be a clarion call to service providers to protect the quality of their product. There will be fewer reasons for organizations to put up with inferior software if it is easy to swap in a service from a different vendor.
Summary
This chapter provided an approach to getting started with Web services with three basic experiments meant to create a familiarity with using Web services. Following that, the chapter used the story about C. R.’s business trip to address more advanced uses of Web services. A vision of what Web services might mean for the future was provided at the end.
Although it might have looked like it, this chapter did not address SOA. Chapter 12 provides suggestions for getting started with SOAs.
1With Web services, it is sometimes difficult to come up with the correct descriptive phrases. Integrated is not exactly the best term because the services are provided in a seamless way at many locations on the Internet. Another term often used is mashup, but that term does not give the sense that there is an architecture. For the purposes of this book, I use integrated.
2“Holiday Shoppers Flocking Online Create Record Breaking Sales,” http://www.forbes.com/sites/anthonydemarco/2011/11/27/holiday-shoppers-flocking-online-create-record-breaking-sales/.