« Crossing the Road, Very Slowly |
Main
| Reading List: To the End of the Solar System »
Monday, June 9, 2008
The Front Page
Search engines are an important driver of traffic to Web sites. On a typical day at Fourmilab, about 10,000 initial entries to the site are from people clicking on Google search results and 500 are from Yahoo searches. (These numbers are computed from the “Referrer” HTTP request field; if a user's browser does not report this item, or the request arrives via a proxy or is served from a cache which does not send this field to the server, it will not be counted.)
Measured this way, referrals from search engines account for about 1% of visits to Fourmilab. I suspect this number is lower here than for many sites, since this site contains many large documents which engender numerous subsequent navigation clicks, and dynamic content where a user many refresh a page many times after making incremental changes. These subsequent accesses will have intra-site referrer fields, and hence will not be counted as search engine hits even if the user initially arrived via a search engine. To a commercial site with relatively shallow content whose revenue depends upon referrals from search engines, the ranking of its pages in response to relevant queries is crucial to success.
Market research on Web commerce has shown how vital search ranking is. According to
this podcast interview (here's
the book), 80% of Web users never look beyond the first page of search results, and 80% of completed E-commerce transactions are generated by clicks on the
first five links on the first page of results. These statistics create a brutal imperative to come out on top in search results, and have resulted in myriad schemes of “search engine optimisation” ranging from the clever to the
crooked.
I suspect (but have no way of confirming) that visitors to an eclectic, non-commercial site like this are not as sharply focused on the top few search results as those looking to buy something online, but still it's interesting to see which common search terms return results on the magical first page of search results. The table at the right lists terms (each linked to the Fourmilab page the search engine cited) which appeared on the first page of Google U.S. Web search results in a test run on 2008-06-08 around 21:00 UTC. In listing the ranking of the page, I excluded any sponsored links which appeared before the results.
If you repeat this experiment, you will almost certainly get different results. When it comes to Google search rankings, Plonk's constant (the dimensionless quantity which measures the uncertainty in results from identical queries submitted at different times) appears to be quite large. If you immediately resubmit a query, you will usually get the same results, but if you try it again a few hours later or the next day, the results will usually differ, often substantially. There may be other user-dependent factors which affect search results as well: with Google you never know. It is clear that sponsored links are targeted based upon the requester's location.
In addition to monitoring the Google
PageRank of your site and major pages within it, you may find it an interesting exercise to see where various pages of your site rank in results for queries you might imagine users looking for such information might enter. Observing how a small qualification of the query can cause a page to appear much higher in search results may help you to refine the page title and keywords to rank better in searches looking for what you're providing.
For
Generation Y visitors, here is a “
tag cloud” of the top-ranked searches leading to Fourmilab from the table above, courtesy of the
Metamend Keyword Density Analyzer.
Posted at June 9, 2008 23:06