Chapter 13

Search Engines: What You Need to Know

In This Chapter

Analysing how search engines find your site

Focusing on ways to improve your coverage on Google

Adding keywords and registering your site with search engines

Tracing referrals and visits to focus on the search services that count

The other day, Greg took some old radios to a local repair shop. The shop has been in business for more than three decades but never seemed to be busy. This time, however, the owner told Greg he was overwhelmed with hundreds of back orders and wouldn’t be able to get to Greg’s jobs for several weeks. His shop had just been featured on a television show, and now people were driving long distances to bring him retro audio equipment to fix.

If you can get your business mentioned in just the right place, customers will find you more easily. On the Web, search engines are the most important places to get yourself listed. One of the key requirements for any business is the ability to match up your products or services with potential customers and to ensure that your company shows up in lots of search results and that your site is near the top of the first page. You do have a measure of control over the quality of your placement in search results, and this chapter describes strategies for improving it.

Search engines have created a huge industry for themselves and the search engine optimisation businesses that feed off them. People around the world lodge billions of search enquiries every month, which lead to billions of results. You can see why it’s easy for your Web site to get lost in the jumble of businesses who are vying for attention.

Understanding How Search Engines Find You

Have you ever wondered why some companies manage to find their way to the top of a page of search engine results – and occasionally pop up several times on the same page – while others get buried deep within pages and pages of Web site listings? In an ideal world, search engines would rank e-commerce sites by their design, functionality, and whether the businesses behind them give the best possible deals. But with so many millions of Web sites crowding the Internet, the job of processing searches and indexing Web site URLs and contents has to be automated. Because it’s computerised, you can perform some magic with the way your Web pages are written that can help you improve your placement in a set of search results.

Your site doesn’t necessarily need to appear right at the top of the first search results page. The important thing is to ensure that your site appears before that of your competition. You need to think like a searcher, which is probably easy because you probably do plenty of Web-based searches yourself. How do you find the Web sites you want? Two things are of paramount importance: keywords and links.

Keywords are key

A keyword is a word describing a subject that you enter in a search box in order to find information on a Web site or on the wider Internet. Suppose that you’re trying to find a source for an herbal sleep aid called Nightol. You’d naturally enter the term Nightol in the search box on your search service of choice, click a button called Search, Search Now, Go, or something similar, and wait a few seconds for search results to be gathered.

When you send a keyword to a search service, you set a number of possible actions in motion. One thing that happens for sure is that the keyword is processed by a script on a Web server that is operated by the search service. The script makes a request (which is called, in computerspeak, a query) to a database file. The database contains contents culled from millions (even billions, depending on the service) of Web pages.

The database contents are gathered from two sources. In some cases, search services employ human editors who record selected contents of Web pages and write descriptions for those pages. But Web pages are so ubiquitous and changeable that most of the work is actually done by computer programs that automatically scour the Web. These programs don’t record every word on every Web page. Some take words from the headings; others index the first 50 or 100 words on a Web site. Accordingly, when Dan did a search for Twix on Google.co.uk, the sites that were listed at the top of the first page of search results had two attributes:

Some sites had the brand name Twix in the URL, such as www.twix.com or en.wikipedia.org/wiki/Twix.

Other sites had the word Twix mentioned several times at the top of the home page.

A service called Wordtracker (wordtracker.com) does daily surveys of the keyword queries made to various search engines. It creates lists of what it finds to be the most popular search terms. It’s not likely those terms apply to your own e-commerce Web site, of course. But if you want to maximise the number of visits to your site, or just make your site more prominent in a list of search results, you may do well to know what’s trendy and write your text accordingly.

Adding your site’s most important keyword to the URL is one solution to better search placement. But you can’t always do this. When it comes to keywords, your job is to load your Web site’s headings with as many words as you can find that are relevant to what you sell. You can do so by:

Registering your site with one or more of the services (see the ‘Going Gaga over Google’ section, later in this chapter).

Burying keywords in the <META> tag in the HTML for your home page so that they aren’t visible to your visitors but appear to the spider programs that index Web pages (see the ‘Adding keywords to your HTML’ section, later in this chapter).

Adding keywords to the headings and initial body text on your pages, as described in the ‘Adding keywords to key pages’ section, later in this chapter.

A keyword doesn’t have to be a single word. You can also use a phrase containing two or more words. Think beyond single words to consider phrases people may enter when they’re trying to find products or services you’re offering.

Links help searchers connect to you

Keywords aren’t the only things that point search services to Web sites. Services like Google keep track of the number of links that point to a site. The greater the number of links, the higher that site’s ranking in a set of Google search listings. It’s especially good if the URLs that form the links make use of your keywords.

Suppose that your ideal keywords are ‘Dan’s Shoe Shop’. The ideal URL would be www.dansshoesshop.co.uk, www.dansshoeshop.com, and so on. You can create the following HTML link to your e-commerce Web site on a personal Web page, or an eBay About Me page (see Chapter 10):

<a href=’http://www.dansshoeshop.com’> Visit Dan’s Shoe Shop </a>

Such a link would be doubly useful: A search service such as Google.co.uk would find your desired keywords (‘Dan’s Shoe Shop’) in the visible, clickable link on your Web page, as well as in the HTML for the link.

Don’t forget the human touch

We don’t want to suggest that search engines work solely by means of computer programs that automatically scour Web pages and by paid advertisements. Computer programs are perceived to be the primary source, but the human factor still plays a role. Yahoo!, one of the oldest search engines around, originally compiled its directory of Web sites by means of real live employees. These days, its Web directory is hard to spot on Yahoo.co.uk. But editors still index sites and assign them to a category called New and Notable Sites, which includes sites that are especially cool in someone’s opinion.

There’s almost no way to make sure that a human editor indexes your Web site. The only thing you can do is to make your site as unique and content rich as possible, which helps your business not only show up in directories and search results but also drum up more paying customers for you, too.

Taking the initiative: Paying for ads

You can’t get much better placement than right at the top of the first page of a set of search results, either at the top of the page or in a column on the right-hand side. It’s even better if your site’s name and URL are highlighted in a colour.

Unfortunately, the only way to get such preferred treatment is to pay for it. And that’s just what a growing number of online businesses are doing – paying search engines to list their sites in a prominent location. See the ‘Paying for search listings can pay off’ sidebar, later in this chapter, for more information.

Knowing who supplies the search results

Another important thing to remember about search engines is that they often gather results from other search services. You may be surprised to find out that, if you do a search of the Web on AOL, your search results are primarily gathered from Google. That’s because AOL has a contract from Google to supply such results. The same applies to thousands of major Web sites that have taken advantage of Google’s powerful search capabilities (Myspace.com being another example). Not only that, but many search services are owned by parent search services.

Just what are the most popular search services in the world? A rundown appears in Table 13-1. The services are presented in rank order, beginning in the first row with Google, which is No. 1. Rankings were reported by Nielsen NetRatings in July 2006.

Table 13-1 Internet Search Services

Search Service	URL	Proportion of searches
Google	www.google.com		49.2%
Yahoo!	www.yahoo.com,		23.8%
MSN Search	search.msn.com		9.6%
AOL Search,	search.aol.com		6.3%
Ask.com	www.ask.com		2.6%
Others	none		8.5%

These search services are by no means the only ones around. Note that the ‘other’ search engines focus on Web sites and Internet resources in specific countries and account for 8.5 per cent of global searches. You can find more of them at www.searchenginewatch.com/links/article.php/2156121.

Going Gaga over Google

When it comes to search engines, Google is at the top of the heap. A few years ago, it was Yahoo! that was setting the pace, but Google’s lightening quick searches and its comprehensive documenting of the Web has made it favourite.

Google is a runaway success thanks to its effectiveness. You’re simply more likely to find something on Google, more quickly, than you are on its competitors. Any search engine placement strategy has to address Google first and foremost. But that doesn’t mean you should ignore Google’s competitors, such as Yahoo! and MSN.

Googling yourself

If you want to evaluate the quality of your search results placement on Google, you have to start by taking stock of where you currently stand. That’s easily done: Just go to Google’s UK home page (www.google.co.uk) and ‘Google’ yourself. (In other words, do a search for your own name or your business’s name – a pastime that has also been called egosurfing.) See where your Web site turns up in the results and also make note of which other sites mention yours.

Next, click Advanced Search or go directly to www.google.co.uk/advanced_search?hl=en. Under the heading Page-Specific Search, enter the URL for your e-commerce site in the Links text box and then click Search. The results that appear in a few seconds consist of Web sites that link to yours. The list should suggest to you the kinds of sites you should approach to solicit links. It should also suggest the kinds of informational Web sites you may create for the purpose of steering business to your Web site. (See the ‘Maximising links’ section, later in this chapter, for a specific example.)

Playing Google’s game to reach No. 1

Not long ago, some bloggers got together and decided to play a game called Google bombing. The game is simple: It consists of making links to a particular Web site in an attempt to get that site listed on Google. The more links the site has pointing to it, the higher that site appears in a set of search results. Of course, the links that are made all have to be connected with a particular keyword or phrase. In the game we’re recalling, one phrase used was ‘miserable failure’. The words ‘miserable failure’ were hyperlinks pointing to the Web site of the White House. The story went that if you went to Google, typed the words miserable failure, and clicked the I’m Feeling Lucky button, you would be taken to President Bush’s biography on the White House Web site – this still worked at time of writing. (Incidentally, if you type those words and click Google Search rather than I’m Feeling Lucky, the No. 2 hit takes you to Jimmy Carter’s biography on the White House Web site; and the No. 3 hit takes you to a story about search engine manipulation on the BBC News Web site and the No.4 hit brings up Michael Moore’s Web site.) You can find out more about this interesting pastime on a Web site called The Word Spy (www.wordspy.com/words/Googlebombing.asp).

The Google game applies to your e-commerce Web site, too. Suppose that you sell yo-yos, and your Web site URL is www.yoyoplay.com. The game is to get as many other Web sites as possible to link to this URL. The terms that a visitor clicks to get to this URL can be anything: Yo-Yos, Play Yo-Yos, and so on. The more links you can make, the better your search results will be.

Leaving a Trail of Crumbs

In order to improve your site’s search placement, you need to make it easy for searchers to find you. You leave a trail of digital crumbs. You add keywords to the HTML for your Web pages, and you make sure that your site is included in the databases of the most popular services.

Keep in mind that most Web surfers don’t enter single words in search boxes. They tend to enter phrases. Combinations of keywords are extra effective. If you sell tools, don’t just enter tools as a keyword. Enter keywords such as tool box, power tool, tool caddy, pneumatic tool, electric tool, and so on.

Adding keywords to your HTML

What keywords should you add to your site? Take an old-fashioned pencil and paper and write down all the words you can think of that are related to your site, your products, your services, or you – whatever you want to promote, in other words. You may also enlist the help of a printed thesaurus or the one supplied online at Dictionary.com (www.dictionary.com). Look up one term associated with your goods or services, and you’re likely to find a number of similar terms.

After you have a set of keywords, you need to add them to the HTML for your Web pages. Keywords and Web site descriptions are contained within HTML commands that begin with <META>. If you type the commands by hand using a text editor, you need to locate the commands in between the <HEAD> and </HEAD> tags at the head of the document. They look like this:

<META NAME=’description’>

<META NAME=’keywords’>

Some Web page editors make this user friendly for you: You can type your information in specially designated boxes. Figure 13-1 shows Adobe Dreamweaver’s commands, which are accessed by opening the Objects panel, clicking Keywords, and then typing the words in the Keywords dialog box.

Figure 13-1: Some Web page editors make it easy to add keywords and descriptions for search services to find.

You can also spy on your competitors’ Web sites to see whether they have added any keywords to their Web pages by following these steps:

1. Go to your competitor’s home page and choose View ⇒Source if you’re using Internet Explorer.

A new window opens with the page source supplied.

2. Scroll through the code, looking for the <META> tags if they’re present. (Press Ctrl+F, enter META, and click the Find button if you can’t find them on your own.)

If the page’s author used <META> tags to enter keywords, you’ll see them on-screen.

3. Make a note of the keywords supplied and see whether any may be applied to your own Web site.

Keywords, like Web page addresses, are frequently misspelled. Make sure that you type several variations on keywords that may be subject to typos: for example, Aberdeen, Abberdeen, Aberdeene, and so on. Don’t worry about getting capitalisation just right, however; most searchers simply enter all lowercase characters and don’t bother with capital letters at all.

Besides keywords, the <META> tag is also important for the Description command, which enables you to create a description of your Web site or Web page that search engines can index and use in search results. Some search services also scan the description for keywords, too, so make sure that you create a description at the same time you type your keywords in the <META> tags.

Spying on the Web searchers

If you’ve ever wondered what people search for every day on the Internet, you’re in luck. A site called Metaspy (www.metaspy.com) lets visitors look ‘behind the scenes’ at a list of keywords that visitors to its search service, MetaCrawler, are entering in near-real time. (The list of search services is refreshed every 15 seconds.) It doesn’t help you make your e-commerce site more visible, but it does tell you how diverse the topics are that are searched on Web sites. If nothing else, it makes you aware that some of your keywords need to be general and universal as well as topic specific.

Registering your site with Google.co.uk

Google has a program that automatically indexes Web pages all over the Internet. The program actually has a name: Googlebot. However, you don’t have to wait for Googlebot to find your site: You can fill out a simple form that adds your URL to the sites that are indexed by this program. Go to www.google.co.uk/addurl.html, enter your URL and a few comments about your site, and click the Add URL button. That’s all there is to it. Expect to wait a few weeks for your site to appear among Google’s search results if it doesn’t appear there already.

Getting listed in the Yahoo! index

Yahoo! won’t guarantee to list just any Web site, but don’t fret, there are two ways to give yourself a fighting chance. One is building a vibrant and content rich Web site; the other is buying your way into its search pages.

For £60, Yahoo! sets you up with a pay-per-click search package. That means you pay Yahoo a small amount of money every time someone finds your Web pages through its search engine. To start with, this charge comes out of your initial £60 deposit; when that runs out, you can decide to top it up and continue to get your Web site listed prominently, or you can simply stop there. Follow this link for more information: searchmarketing.yahoo.com/en_GB/arp/srch.php?o=GB0176.

Search Engine Watch (searchenginewatch.com) is a great place to go for tips on how search engines and indexes work, and how to get listed on them. The site includes an article about one company’s problems getting what it considers to be adequate Yahoo! Coverage.

What else can you do to get listed on Yahoo!? We have a three-step suggestion:

1. Make your site interesting, quirky, or somehow attention grabbing.

You never know; you may just stand out from the sea of new Web sites and gain the attention of Yahoo’s indexing software.

2. Submit your Web site to the search engine.

a. Go to www.yahoo.co.uk, and click the How To Suggest A Site link at the very bottom of the page.

The Yahoo! Submit Your Site page appears.

b. Click the Submit Your Site For Free button.

c. Input your main domain name and the address of one other page you’d like people to go to.

d. Press the Submit URL button.

3. Try a local Yahoo! index.

Yahoo! Local, like Google Local, aims to document all the useful bricks-and-mortar businesses around the UK. If you have a physical shop, you can do worse than getting listed with the search service. Go to uk.local.yahoo.com and click the Help link at the top right-hand corner of the page. You’re asked to contact Yahoo! with your new business listing. Click the Contact Us link, and you’re presented with a form to fill in. When you’re finished, submit the form by simply clicking the Send button at the bottom of the page.

Getting listed with other search services

Search services can steer lots of business to a commercial Web site, based on how often the site appears in the list of Web pages that the user sees and how high the site appears in the list. Your goal is to maximise your site’s chances of being found by the search service.

Paying for search listings can pay off

Listing with search sites is growing more complex all the time. Many sites are owned by other sites. AltaVista is part of the Yahoo! Search Marketing network. You tell Yahoo! how much you’ll pay if someone clicks your listing when it appears in a list of search results. The higher you bid, the better your ranking in the results. In exchange for the fees you pay to Yahoo!, your search listings appear in multiple search sites. The same system applies with Google.

Some search services are part of the Overture network, but they still allow individuals to submit their sites for consideration. Here’s a quick example that shows how to submit your site (for consideration) to one of the search engines that still gives you the do-it-yourself option:

1. Connect to the Internet, start your Web browser, and go to AltaVista at uk.altavista.com.

The AltaVista home page appears.

2. Click the Submit A Site link.

The AltaVista Submit A Site page appears.

3. Click the Click Here link (under the heading Basic Submit).

The Yahoo! Search Sign In page appears. Confused? AltaVista gets its search results from Yahoo!. Therefore, you have to register Yahoo! in order to have people find you on AltaVista.

4. Enter your Yahoo! ID and password. (If you don’t have them yet, click the Sign Up Now link on the same page to obtain them.) Then click Sign In.

The Yahoo! Submit Your Site page appears.

5. In the box labelled Enter the URL, type the URL for your site’s home page and then click the Submit URL button.

Your page is added to the list of pages that Yahoo!’s ‘crawler’ program indexes. As the note on the Submit Your Site page says, you can expect the process to take several weeks.

Businesses on the Web can get obsessed with how high their sites appear on the list of search results pages. If a Web surfer enters the exact name of a site in the Excite search text box, for example, some people just can’t understand why that site doesn’t come back at the top – or even on the first page – of the list of returned sites. Of the millions of sites listed in a search service’s database, the chances are good that at least one has the same name as yours (or something close to it) or that a page contains a combination of the same words that make up your organisation’s name. Don’t be overly concerned with hitting the top of the search-hit charts. Concentrate on creating a top-notch Web site and making sales.

Adding keywords to key pages

Earlier in this chapter (see ‘Adding keywords to your HTML’), we show you how to add keywords to the HTML for your Web pages. Those keywords aren’t ones that visitors normally see, unless they view the source code for your Web page. Other keywords can be added to parts of your Web page that are visible – parts of the page that those programs called crawlers or spiders scan and index:

The title: Be sure to create a title for your page. The title appears in the title bar at the very top of the browser window. Many search engines index the contents of the title because it appears not only at the top of the browser window, but at the top of the HTML, too.

Headings: Your Web page’s headings should be specific about what you sell and what you do.

The first line of text: Sometimes, search services index every word on every page, but others limit the amount of text they index. So the first lines may be indexed, while others are not. Get your message across quickly; pack your first sentences with nouns that list what you have for sale.

The best way to ensure that your site gets indexed is to pack it with useful content. We’re talking about textual content: Search programs can’t view photos, animations, or sounds. Make sure that your pages contain a significant amount of text as well as these other types of content.

Web sites that specialise in search-engine optimisation talk about something called keyword density: the number of keywords on your page, multiplied by the number of times each one is used. Keyword density is seen as a way to gain a good search engine ranking. In other words, if you sell shoes and you use ten different terms once, you won’t get as good of a ranking compared to the use of six of seven words that appear twice, or a handful of well-chosen keywords used several times each.

Take the following passage taken from the home page of Startups.co.uk, which Dan used to be the editor of. It serves people who are considering starting a business, or those who have just taken the plunge. Notice how many times business, start, and entrepreneur are used. (The actual text is a lot longer, and the words are repeated several more times.)

‘Whether you are a budding entrepreneur ready to start a business for the first time or you are an established entrepreneur looking to do it a second time, we have all the news and information you need to get your business starting on the right foot.’

Don’t make your pages hard to index

Sometimes, the key to making things work is simply being certain that you aren’t putting roadblocks in the way of success. The way you format Web pages can prevent search services from recording your text and the keywords you want your customers to enter. Avoid these obvious hindrances:

Your text begins too far down the page. If you load the top of your page with images that can’t be indexed, your text will be indexed that much slower, and your rankings will suffer.

Your pages are loaded with Java applets, animations, and other objects that can’t be indexed. Content that slows down the automatic indexing programs will reduce your rankings, too.

Your pages don’t actually include the ideal keyword phrase you want your searchers to use. If you have a business converting LP records to CDs, you want the phrase ‘LP to CD’ or ‘convert LPs to CDs’ somewhere on your home page and on other pages as well.

Every image on your Web page can potentially be assigned a textual label (also known as ALT text because the ALT element in HTML enables it to be used). The immediate purpose of the label is to tell visitors what the image depicts in case it cannot be displayed in the browser window. As a trick to produce more keyword density, you can assign keywords or keyword phrases to these names instead.

Maximising links

Along with keywords, hyperlinks are what search engines use to index a site and include it in a database. By controlling two types of links, you can provide search services with that much more information about the contents of your site:

The hyperlinks contained in the bodies of your Web pages

The links that point to your site from other locations around the Web

The section ‘Links help searchers connect to you’, earlier in this chapter, mentions the links in the bodies of your own Web pages. One of the most effective tricks for increasing the number of links that point to your online shop is to create several different Web sites, each of which points to that shop. That’s just what Lars Hundley did with his main e-commerce site, Clean Air Gardening (www.cleanairgardening.com).

‘Creating my own network of gardening sites that provide quality information helps me rise to the top of the search engines in many categories,’ says Lars. ‘People find the content sites sometimes and click through to Clean Air Gardening to buy related products.’

It’s true: Do an Advanced Search on Google for sites that link to www.cleanairgardening.com, and you’ll find links in the following locations. First, the ones that are run by Lars:

Organic Pest Control (www.organicgardenpests.com)

Guide to Using a Reel Mower (www.reelmowerguide.com)

Organic Garden Tips (www.organicgardentips.com)

CompostGuide.com (www.compostguide.com)

Rain Barrel Guide (www.rainbarrelguide.com)

Next, just a sampling of the many sites that link to Clean Air Gardening and that aren’t run by Lars:

National Gardening Association (garden.garden.org)

GardenToolGuide.com (www.gardentoolguide.com)

Master Composter (www.mastercomposter.com)

Organic Gardening (www.organicgardening.com)

For the sites that Lars doesn’t run himself, he solicits links. ‘I also exchange links with other high-ranking related sites, both in order to improve my rankings, and to provide quality links for my visitors. If you stick with quality links, you can never go wrong.’

Some companies may offer you SEO in a box – in other words, search engine optimising software. This software has its benefits but is probably not worth the money. If you’re really serious about optimising your online visibility, try a consultancy like Oyster Web (www.oyster-web.co.uk). Either way, always make sure that you’ve exhausted all the free channels of search optimisation before you shell out any cash. A free online guide like the one at the following address is a good place to start: www.makemetop.co.uk/what_is_seo.

Monitoring Traffic: The Science of Webanalytics

How do you improve the number of times your site is found by search engines? One way is to analyse the traffic that comes to your site, a practice often called Webanalytics. When it comes to search engine placement, the type of research you need to perform is called log file analysis, which can tell you exactly what keywords already have been used to find your site. You can then combine those words into new keyword phrases, hopefully helping even more people find your site. You can get software that will do the analysis for you, or you can do it yourself:

Software options: Some software options are specifically designed to help improve search engine optimisation. OptiLink (www.optilinksoftware.com/download.html) counts the number of keywords on a Web page. It analyses the links that point at the page and helps you analyse what the best keywords are, where they need to be located, and what specific text will make the links rank higher in Google’s search results.

Do-it-yourself options: The other, more labour-intensive way to analyse what drives visitors to your Web site is through analysis of log files. A log file is an electronic document that a Web server compiles as a record of every visit made to a Web page, image, or other object on a site. Most Web-hosting services let you look at the log file for your Web site. The log file gives you a rough idea of where your visitors are from and which resources on your Web site are the most visited. By focusing on particular types of log file data, you can evaluate how visitors find your site and which search services are doing the best job of directing visitors to you.

If you look at log file information in its raw text form, you’re probably mystified by page after page of numbers and techie gibberish. Log files typically record information such as the IP address and the domain name of the computer that accesses a Web page. They don’t tell you the name and address of the person using the machine at the time. They give you an idea of where the computer is located geographically, based on the suffix at the end of a domain name (such as .de for Germany or .fr for France). You’ll probably need to make use of a log file analyser such as ClickTracks.com (www.clicktracks.com) or WebTrends (www.webtrends.co.uk), which present the data in a format that is easy to interpret.

When you’re viewing log files, one important thing to track is referrer reporting, which gives you the site the visitor was viewing just before coming to yours. This report tells you what sites are directing visitors to yours. Make note of the search engines that appear most frequently; these are the ones you need to work on when it comes to improving your placement in sets of search results.