But traveling through the forums recently, it appears that Google is trying to give off the impression that the supplemental index is nothing bad for your site. Whereas, all expert SEOs used to and some currently, call the supplemental index, the sign of death (in terms of your site ranking in Google).
A Search Engine Watch Forums thread is named "supplemental not bad according to matt cutts?" And you also have this huge 17 page thread for about two weeks ago that is still very active at WebmasterWorld linking to Adam Lasnik's Google Groups post named The skinny on the supplemental index.
1) Penalty? When your site has pages in our supplemental index, it does *not* indicate that your site has been penalized. In particular, we do not move a site's pages from our main to our supplemental index in response to any violations of our Webmaster Guidelines. 2) Freshness? You can expect to see a fresher supplemental index in the coming quarters. By the definition of "supplemental," however, I don't forsee it becoming as comprehensive or frequently updated as our main index.Ok, first two are great Google-like responses, IMO. But to call something a "cure," kind of implies that there is a major issue with being in the supplemental index.
3) Cure? Get more quality backlinks. This is a key way that our algorithms will view your pages as more valuable to retain in our main index.
At PubCon, Matt Cutts and Vanessa Fox both said that the supplemental index is not a bad thing. But is it a good thing? Not bad for who?
I would love to hear a definitive pro and con on the supplemental index from Google.
Forum discussion at Search Engine Watch Forums & WebmasterWorld.
Update: There is an other thread at WebmasterWorld named Supplemental Results: What exactly are they where Adam Lasnik of Google posted to yesterday where he said;
Pages are in the supplemental results because we still wanted to be able to show them to users, but the pages didn't have enough PageRank to make it into our main index (which is more extensive and updated with greater frequency).
Getting more *quality* backlinks is generally a good way to get more of your pages in the main index.Still, there is a lot of confusion on this.
The new system officially rolls out today for anyone using Google.com and searching in English. Not everyone will see it at first, but over the course of the next several days, Universal Search should be more, well, universal. A new navigational interface has also been unveiled for Google and is covered more in the companion piece to this article, Google’s New Navigational Links: An Illustrated Guide.
The move potentially should be a huge boon for searchers, while search marketers who have paid attention to the importance of specialized or vertical search will see new opportunities. To fully explain the importance to both groups, I’m going to work step-by-step through the concept of vertical search engines, how they’re often ignored by searchers and search marketers alike, then how Google is going to make this content more visible through Universal Search.
As this is a long article, here are jumps to specific sections, should you wish to skip ahead:
- What The Hell Is Vertical Search?
- Making Vertical Search Visible
- More Than A OneBox To Me & Book Search Blending
- News & Local Blending
- Single Listing Vs. Multiple Listing Replacement
- Video Blending & Video Meta Search
- Image Blending & Multiple Blends
- And Web Search Is….?
- Impact On Searchers & Search Marketers
I hate the term vertical search. I routinely ask audiences if they understand what it means, and significant numbers don’t. I prefer the term specialized search or specialty search, but the financial community in particular has popularized the use of "vertical" search. Let me explain the concept.
Regular search — when you go to Google, Yahoo, Microsoft, Ask or any general-purpose search engine — is a "horizontal" search in that you are searching across a wide spectrum of material. Information from sports sites, news sites, medical sites, shopping sites — the entire horizontal spectrum of topics is represented.
With vertical search, you slice down vertically through one topic area. You search only against the news sites or against the medical information, for example. This type of focus can make for more relevant results.
Need help on fixing windows in your house? Search on that horizontally, and information about the Windows operating system might dominate the listings, simply because there’s so much about Windows out there. Try using a vertical search engine that only has home improvement information, and information about repairing the type of windows you look out of should become much more visible and relevant.
Making Vertical Search Visible
Sadly, many people are unaware of the vertical search resources that are out there. Indeed, Google has long put links to its vertical search services above the search box on its home page. First they were in the form of "tabs," then later as regular links that currently promote that you can vertically search for images, or video, or news, or maps or in many more specialized areas.
To really understand all the vertical searches Google itself runs, consider this list:
- Blog Search
- Book Search
- Code Search
- Patent Search
- Product Search
- Web Search
Certainly the home page links haven’t worked to expose the vertical richness. Few searchers use those links. Test this yourself. Ask someone if they know what those links do. Quite often, they’ll tell you no – in fact, that they’ve not seen them before. I’ve shown people how do to an image search using them and literally gotten gasps of astonishment in response. These people had no idea you could search and get back actual images.
I’ve long written about this problem and described the solution as "invisible tabs." From my article on the topic back in December 2003:
This is what Google is doing with Universal Search. Google’s hitting several of its vertical search services all at once, then bringing back those results to blend in with "regular" results.Many people simply do not see or use tabs, just like they regularly ignore drop down boxes, radio buttons and any type of other option you put out. Search engines have told me this over the years, and I also see it first hand….The solution to tab blindness is clearly for me and an army of other search educators to head out and teach people how to use tabs! Naturally, that’s not going to happen. No, the solution really is for the search engines to make use of "invisible tabs," where they make the correct choice for the user, behind the scenes.
More Than A OneBox To Me & Book Search Blending
Hasn’t Google been already making vertical results available through those OneBox displays that appear at the top of some search results? Yes, but Universal Search is going beyond that. Let’s look at Google Book Search to illustrate this.
Google Book Search is Google’s vertical search engine that lets you find matching pages within books that it has scanned. Curious about that part in Tom Sawyer where he was whitewashing the fence? Enter tom sawyer fence, and Google Book Search tells you it’s on page 22 of one edition of Mark Twain’s classic novel.
So, Google Book Search lets you look within the content of books, but only if you specifically know to use that search engine. Most people just turn to Google’s regular search. That’s why Google tries to remind you there’s more than web search using OneBox displays like this:
See how the arrow is pointing at book search matches? This little section is called a OneBox, and it sits above and separate from the 10 "regular" search results that come from Google’s crawling of the web. This is effectively Google saying "Hey! We’ve got a book search engine with matches for this. You might want to check the results there!"
If you did click through, you’d then search just against the book content and get back matching book pages, like this:
Some people skip past the OneBox prompts because they look weird compared to the "real" results. However, Mayer said the bigger issue was that OneBoxes were "clunky" and often not as relevant as the results coming below them.
For example, Mayer said that the first result in Google might get clicked on more than a third of the time out of all the clicks on that page. In fact, Google’s search quality team makes it a goal to try and make that first result as relevant as possible. In contrast, the OneBox insertions get clicked on less despite getting prime placement on the page, since they aren’t as relevant.
Universal Search aims to fix this through blending. With Universal Search, Google will hit a range of its vertical search engines, then decide if the relevancy of a result from book search is higher than a match from web page search. (FYI, Infoseek got a patent related to this type of blending back in 1997; See also: Google’s Universal Search Patent Application & Assigned Patents from Infoseek from SEO By The Sea).
If so, a web page result might get dropped and a book search result inserted. That means instead of a OneBox, book results will begin showing up in Google search like this:
See how this is the same page listed at the top of Google Book Search, when I did a specific search there. This screenshot from Google illustrates how in the future, that "book page" will just get mixed in among the web pages of regular Google searches.
Of course, Google already does this with some of its other properties. For example:
FYI, the OneBoxes aren’t dead, not even for cases where Universal Search is in place. In other words, you might still see a news OneBox or a Books OneBox. Says Google:
While the OneBox hasn’t entirely gone away, it has been replaced in many cases by blended results in relevant positions. As we continue development, we’ll continue to evolve our use of both the OneBox and blended results to deliver the most relevant results.News & Local Blending
I’ve covered how book search results will be blended. Let’s turn next to news and local result blending, since they work in similar ways.
Here’s how news results currently get inserted:
As Universal Search rolls out this week, that OneBox display will get replaced, with news material blended like this:
Similarly, local search results are being integrated. Before Universal Search, a OneBox would display them like this:
With Universal Search, they’re blended like this:
FYI, news search results began being blended last month, for some people, as part of an official change. Local blending started getting spotted by other widely yesterday.
Single Listing Vs. Multiple Listings Replacement
(NOTE: I’ve updated and clarified this section since Universal Search went live)
With the news blending example above, Google dropped only one of its usual web page listings to insert either a news unit. That unit can move around, by the way. It might be at the top of the page, somewhere in the middle or even at the end, based on wherever Google determines it will be most relevant related to the other listings. But what it is NOT supposed to be is positioned outside the regular 10 results in the way the OneBoxes have historically been. Instead, these units take one of the 10 regular positions.
With the local blending example, you can see that three web page results have been removed to make room for the three local listings along with the map. But sometimes, only one local listing will show. Google told me:
For the time-being, three local results are taking the place of three web results for result sets that trigger a grouped local UI. This happens only to categorical use cases (e.g. restaurants mountain view, new york dental school), where a group is appropriate. When there’s just one navigational result, like siam royal palo alto, there’s only one local result displacing one web result. While there are no immediate plans to change this, it could happen over time.Google’s not taking the more dramatic step of simply replacing the bulk of the 10 web page listings with news or local material, even if a query might indicate this makes sense. For example, a search for new york hotels is pretty local oriented. Why not shift over to showing a full page of local results? Or if there’s a breaking news event, say with the Virginia shooting, most people searching under that term probably would be better served by getting many more actual news results than shown in a single unit.
Video Blending & Video Meta Search
There’s nothing new about Google listing links to video content in its regular search results, including links to pages in Google-owned Google Video and YouTube. However, Universal Search is changing things. Instead of considering that content as part of its web pages index, Universal Search will run a video search query across an enhanced version of Google Video.
What’s enhanced? Google Video is morphing into a meta video search site, similar to how Blinkx operates. Eventually, Google Video will list content that’s hosted at Google Video, YouTube or anywhere on the web.
Today, Google Video itself will remain listing only Google Video and YouTube content. The enhanced version will only be accessible through Universal Search. In the near future, Google Video itself will shift over.
As for that video content, unlike with news and local, more than one blended results might appear. For video content outside of Google Video and YouTube, the display will be like this:
Content on Google’s own video hosting services will look like this:
See the little + Watch Video part in the listing? That’s a video Plus Box, which have been spotted in the past. Now they are being formalized. Clicking on that will open up the video and let you watch it within the search results, like this:
After years of people ignoring tabs and links designed to get them to do specialized searches, Google is taking the plunge to push the right buttons behind the scenes and make specialized or vertical search results part of the "normal" experience.
Image Blending & Multiple Blends
For some searches, matches from Google Image Search will appear. Unlike the other blending, these will NOT take the place of one of the 10 regular listings. They were found to be too disruptive that way, Mayer said. Instead, those deemed to be of high relevancy will appear at the top of results, while those that are important but perhaps not as relevant will be shown at the bottom.
It’s also important to note that blending may happen from one or more of the vertical search engines. For example, Mayer talked about a search for Steve Jobs where pictures of him would be on top, news in the middle and a video clip at the bottom.
And Web Search Is….?
One thing that’s slightly confusing is that Universal Search will continue to be labeled as "Web" on the search results page. OK, most of the content is coming from web-based sources (Book Search is the key exception). But aside from these vertical search databases, there also remains a WEB PAGE database, what Google originally started with and what remains the core to everything else. I almost wish Google had labeled this "All" or "Everything" search, so that Web search could be a specific drill-down option for those specifically after web page content.
Impact On Searchers & Search Marketers
Overall, I think this is a great move for Google to make, assuming the actual implementation works. Mayer was at pains to explain this is the start of Universal Search, and that Google expect to refine and adjust things going forward. It should be helpful to searchers who have simply continued to ignore vertical search properties and miss out on good content within them.
As for search marketers, I’ve written for years that there were to two major changes coming for search engines: personalization of results and the growth of vertical. From 2003:
As a search engine marketer, the implementation of invisible tabs means that traffic from organic web page listings will diminish over time. The more specialty databases are implemented, the less traffic will fall to web search results.From last year:
That needn’t be a problem. The skilled search engine marketer’s most important asset is understanding how search engines get various types of listings, then helping their clients enter the appropriate databases. Think beyond web page databases, and you’ll be prepared for the future.
Vertical search is important because it’s one of the two major things I’ve long talked about as being how search will advance. First generation search analyzed words on a page to rank content. Second generation search tapped into link analysis. Third generation search to me is looking at both user input (what we visit; what we click on; personalized results) and making search go more vertical.Google’s now done both now within the space of weeks, moved forward on the personalized and vertical fronts.
Why the generational jumps at all? To improve relevancy. Each generation has fought the double challenge of being overwhelmed by increasing amounts of information to organize plus having ranking systems become more and more vulnerable to manipulation and outright spam.
Google Ramps Up Personalized Search and Google Search History Expands, Becomes Web History cover key things that both marketers and searchers need to understand, on the personalized from. As for verticals – search marketers must understand that the supremacy of web page search just took a major hit. Web pages will still draw traffic — still plenty of traffic. But vertical search databases offer what remains a largely unexplored and ignored frontier. Pay attention to them, at Google and elsewhere.
See further discussion on Universal Search via Techmeme here, and Gary Price at ResourceShelf has a nice look at "federated" or blended search from other players in Let’s Talk Metasearch, Federated Search, or Universal Search.
Postscript, January 2008: Be sure to read Google Universal Search: 2008 Edition on how Google Universal Search has continued to evolved.
SMX Seattle wrap-up
I had a really good time in Seattle. I got to meet many more members of the Google Kirkland office, go up in the Space Needle, visit the Science Fiction Museum, walk around Pike Place Market, and even see a little bit of West Seattle. West Seattle was especially fun. I ate delicious macaroni and cheese at West 5 and picked up the new album from The National. It’s called Boxer and it sounds like a little like Lou Reed and The Feelies had a love child. I like it.
Also, there was a search conference. This was the first SMX conference, and it was a blast. I think Danny did the right thing by pitching the first conference to an advanced audience. It made for a really friendly, laid-back atmosphere. I had a lot more people just walk up to say hello than I usually do, and that was wonderful. I met nice people from Austin, Isla Vista, and all over the world. If you look at the end-of-conference picture, you can just barely see me at the very top left (I’m wearing my green Ale-8 Tshirt). I talked to several Googlers during and after the conference, and we all got good feedback and ideas from the attendees.
As you may know, SMX joins two other major search conferences: Search Engine Strategies and PubCon. I don’t know how the competition between the industry conferences will turn out, but it’s good that each conference is ready to offer what they believe is the best experience for SEOs and webmasters. Adam Lasnik spoke on three panels at SES Toronto last week, and I’m looking forward to sitting down with him and going over the feedback from the last couple weeks. Please be patient though; I’m still digging out from my vacation too.
A few highlights of the conference:
- Seeing all the familiar SEO faces at the MSN party the first night (and getting to meet a few new folks!).
- Starting out the conference with a Q&A session with Danny, me, and the audience. Danny and I got to use these little headset mikes as if we were a boy band. I pretty much despise PowerPoint-heavy presentations, so it was fun to just talk search with a few hundred webmasters. We covered everything from supplemental results (and how they’re indexed) to Wikipedia to Stephen Colbert. You can read the write-ups in live-blogging style, a slightly more cleaned up “Susan Esparza” style, an abbreviated summary, or even a write-up in haiku format. Some write-ups don’t mention that to keep the conference fun and casual, Danny agreed to strip from his normal suit into shorts and red Vans.
- Patrick from feedthebot.com asked why the webmaster guidelines aren’t more detailed, and then Riona and Vanessa managed to launch more detailed webmaster guidelines in time for the second day of the conference.
- If you like video, Mike McDonald caught me for a video interview later that day to recap the topics that had come up at the conference so far. I also got Matt-jacked into a dark room by Rand for the “SMX Diaries” video interviews. And WebProNews had some snippets of me from the Q&A session.
- If that’s not enough Matt-video for you, I just realized while doing this write-up that someone posted the video of the keynote Q&A session that I did at SES London a few months ago. That’s pretty funny.
- A nice Yahoo mixer, complete with Yahoo ice cubes (they’re purple plastic and light up when you put them in liquid). I saved a couple as prizes for my team.
- A good dinner with other search folks.
- Meeting lots and lots of cool people at the Google party. I never made it into the dance floor until the party was winding down, but I did get to see the vintage arcade games that were set up and the cool Google ice sculpture. Thanks to the Googlers who helped organize the party or manned tables to answer questions there!
- Meeting Satya Nadella and hearing his Q&A with Danny Sullivan.
- Lots of other fun panels the second day, including the “Give it up!” session.
- The end-of-conference photo and the general niceness of the Third Door team to take care of speakers.
- Woohoo for wifi! I really don’t want to attend any conference without wifi at this point.
- Hitting the SEOmoz SMX party. It was a bit of an off-night in terms of my ability to represent Google well.
I bowled a 133, which is okay but not spectacular. Plus Rebecca “Bec” Kelley and Cameron Olthius beat me and a guy from Microsoft at eight-ball. I’d love to blame the MSFT fellow, but in truth I started the game trying to sink the wrong team’s balls. Of course, Cameron let me sink a ball before mentioning that. On the bright side, Danny and I got to rumble in pool. He brought his A-game, and I barely managed to eke out a win. Add in the chance to compare notes on how Google is doing with a couple blackhat spammers, and overall I’d call it a great party. Well done, SEOmozzers.
I know I’m missing a few things, but those are some of the things that stand out looking back a couple weeks later.
Google "Buffy" Update - June Google.com Update
Yes, Friday we reported a Google Webmaster Link Tool June 2007 Update but we now have a more detailed thread at WebmasterWorld describing major search results changes at Google.com.
WebmasterWorld Admin explains:
Ever since Update Jagger, which prepared the way for Google's new Big Daddy infrastructure, the changes and shifts to the SERPs come so frequently that we haven't called an Update in over a year. Instead, we've been trying to watch the SERP changes in an ongoing, monthly thread that shifts with the irregular roll, pitch and yaw of the good ship Google.But something quite major is now going on with the Google SERPs. Members are reporting major changes in the single word search results, most particularly, but lots of other things are stirred up as well. On one of my single keywords, I've just passed wikipedia (yeah!) and jdMorgan reported the same for one of his keywords.Forum discussion at WebmasterWorld.
So it's time -- we are officially declaring Update Buffy. We'll begin with recent posts from our June SERPs Watching thread. What do you see going on?
So why name it Buffy? Let's say it's in honor of someone who just left her job but knows a whole lot about Buffy the Vampire Slayer. We've never gone with someone's formal name for an update. Tip of the hat to reseller, goodroi, and jdmorgan for their input.
Investigate Legalese. The Language of the law society.
Investigate Black's Law Dictionary.
Words like Society, Public, Understand and many other words have completely different definitions than what you may think.
Learn how they created a fake you to impose their statutes on you. Learn their Language and documents they use to trick you into control.