The Hilltop Algorithm

By Gord Collins
Posted Friday, September 17, 2004

Google.com has recently improved its dominant leadership in the search engine marketshare to 40%, and 89% of searchers are calling it their primary search engine. So why you may ask is Google making so many drastic changes such as using keyword synonyms and disregarding links from Yahoo and its own Google directory?

If Google thought its results were good enough, there wouldn’t be many changes, but it appears the company may feel their search results are not good enough, or even faltering. Is this because they need to filter out the effects of their close competitor, Yahoo? During the past 5 years, Yahoo has had a big role in Google’s search results. Yahoo was a huge human edited service that could validate the authenticity of Web sites.

It could be Google wants to undermine their big competitor by factoring it’s influence right out of Google’s search results. But that leaves only the Open Directory project for credibility checking and ODP has been having trouble with quality control and staying updated. Recently, Google dropped the ODP-based Google directory link from their home page. Many Yahoo listings are not visible in of backlinks of Web sites. We’re left to wonder whether Google is going to drop them completely or just de-emphasize them. If they’re given the heave-ho, what will replace them?

Google’s latest solution for credibility problem though could spell big trouble for corporate Web sites that don’t utilize search engine optimization internally. And it is proving big misery for small Web sites.

Emergency Alarm Has Sounded

Many SEOs have seen this coming and are advising their stodgy clients to begin buying text ads on topic-relevant, high PageRank sites to give their sites the necessary link reputation they lack. Those thousands of links from personal Web sites aren’t all that valuable anymore, and now they may actually need to get links from competitors! That doesn’t go over well in boardrooms and rather than adopt co-opetition, these companies will have to begin getting their rank support from links on authority sites (buying links) or building numerous new Web sites on company owned domains (domain spam). The only other place they can get the necessary link reputation is internally on their own primary company site (search engine optimization).

Artificial manipulation of search results is always an issue to Google yet the threats can come from many different directions: Blogs, link farms, link exchanges, paid links, web site optimization, domain spam and technical coding tricks. With all this taking place, Google is looking for a way to stabilize their index with an anchor they can be sure of.

Their solution is to create virtual anchors called authority sites. In the past, Yahoo and dmoz.com have been the traditional big anchors. They provided a quality control factor that kept search results accurate. The big directories may not be considered reliable by Google anymore. So, if Google outright drops or de-emphasizes these big directories, they will have to build another credibility establisher right into their search algorithm. Enter the Hilltop factor.

Hilltop is the patented algorithm provided for Google’s use by its creators, Krishna Bharat and George A. Mihaila of the University of Toronto. It is an algorithm that finds so-called expert documents that appear to have authoritative value with respect to particular keyword topics. These documents or Web sites then become “mini Yahoo directories” of sorts, and Google emphasizes the outbound links contained on them.

Hilltop emphasizes the voting power of these "authority" sites. They’re often sites/pages that have a high PageRank and a high link reputation for a particular set of keywords. Because they have big credibility, any links that point out of these sites can determine who else’s site gets a top ten ranking in the search results.

The boost you got from Yahoo or ODP may diminish to replaced by countless other “authority sites.” If this is happening, the Google index will be destabilized. That’s because these thousands of new authority sites are hard to keep an eye on. The site owners can be manipulated or outright bought. That means money determines relevancy, not the true topical value of a site. Oddly, these new sites that provide link reputation and PageRank will grow in stature not because of their inherent value, but because Web site owners want to support the hand that feeds them. Sites selling their links right now, stand to become some of the top sites on the Web in coming years. Can Google tell which sites are selling links and penalize them? No, Web site owners can work them into the pages so they look perfectly normal. Hilltop trudges onward despite the obvious, that the chessmen in the game are not who they appear to be!

How Hilltop Works

Hilltop is a snap-on factor that feeds the best topic-specific sites into the full Google algorithm equation. Some suggest that Hilltop is built right into PageRank, actually affecting the PR of sites in the Google index.

The revised PageRank calculation is suggested here: {(1-d)+a (RS)} * {(1-e)+b (PR * fb)} * {(1-f)+c (LS)}.

Hilltop would seem to reward a closed loop circuit among Web sites, all revolving around key “expert documents” much like the planets revolve around the Sun. This Web ring type system may be a dead end, since new sites would never appear in the rankings unless they were “approved” by the ringleader. That would make Google’s index less democratic, which ironically, was the reason it gained its popularity.

Although Hilltop may be a good solution to a faltering Google index quality, it focuses attention on Web sites making them the future target of index manipulators.

Will such a new algorithm dredge up all the really good sites that are currently down in the range of 30 to 100 and put them where users can get them? I’ve done some dredging myself in those dark regions and I didn’t find many gold nuggets. If Hilltop is supposed to improve the index quality, it seems much-a-do about nothing. I’ve noticed an increase in affiliate and link spam dominated search results lately. There’s other forms of spam getting in too (body tag redirects) and the only way this could be happening is that some “authority site” must be linking to them. Some of these spam pages have a zero PageRank too. That backs up the supposition that Hilltop creates a closed, self-affirming network of high ranking sites which cast their “votes” in an irresponsible way.

From a general perspective, it would also encourage sinister-type collaboration among corporations to dominate the Google index. If corporations don’t own the expert documents themselves already, they can simply purchase them. Domain spam by corporations is a huge problem for Google right now.

So in some ways, Hilltop may be good medicine for Google while at the same time be poison for its future.

Expert documents are nothing new. Google has always had an authority site element in its overall ranking algorithm (link reputation), but could Hilltop go further to affect a site’s PageRank? It’s hard to say, but it does look like PageRank is being correlated to the particular keywords used in searches. There are sites with a PR of 3, 4 or 5, that are in the top ten for popular searches. That means link reputation is now perhaps the biggest factor going and why it’s the hot topic in SEO forums.

Many corporations and mid sized businesses won’t allow search engine optimization of their site because it interferes with the expectations of company stakeholders. Webmasters don’t care about link reputation, and product managers only care about brand image, while copywriters don’t want keywords on the site.

To survive in this competitive era, search engine optimization appears to be the most inexpensive and reliable solution for big corporate Web sites. Hilltop will ensure this true.

About the Author
Gord Collins
SEO Specialist
Bay Street SEO
(http://www.bayst-search-engine-optimization.com)