In this chapter, we’ll look at a strategic application of webbots—monitoring online prices. There are many reasons one would do this. For example, a webbot might monitor prices for these purposes:
Notifying someone (via email or text message[29]) when a price drops below a preset threshold
Predicting price trends by performing statistical analysis on price histories
Establishing your company’s prices by studying what the competition charges for similar items
Regardless of your reasons to monitor prices, the one thing that all of these strategies have in common is that they all download web pages containing prices and then identify and parse the data.
In this chapter, I will describe methods for monitoring online prices on e-commerce websites. Additionally, I will explain how to parse data from tables and prepare you for the webbot strategies revealed in Chapter 18.
The practice store, available at this book’s website,[30] will be the target for our price-monitoring webbot. A screenshot of the store is shown in Figure 8-1.
This practice store provides a controlled environment that is ideal for this exercise. For example, by targeting the example store you can do the following:
Experiment with price-monitoring webbots without the possibility of interfering with an actual business
Control the content of this target, so you don’t run the risk of someone modifying the web page, which could break the example scripts[31]
The prices change on a daily basis, so you can also use it to practice writing webbots that track and graph prices over time.
[29] Chapter 15 describes how webbots send email. Appendix C describes how to use email to send text messages.
[30] The URL for this store is found at http://www.WebbotsSpidersScreenScrapers.com.
[31] The example scripts are resistant to most changes in the target store.