Recently, Google released their Building Smartphone-Optimized Websites recommendations. Within these pages, Google briefly describes the different techniques for implementing mobile-optimized sites and, of course, how to ensure that the horde of Google robots recognizes the relationship between your desktop and mobile sites, and index them accordingly.
Responsive Web Design
Serving the same HTML to both desktop and mobile users, known as the responsive approach, is “Google’s recommended configuration” (their bolding, not mine). Google’s bots can determine that a site is responsive, and therefore has a built-in mobile component, and will “treat it accordingly.” Google recommends responsive design because of its utilization of a single URL, its avoidance of a reliance on mobile-site redirects, and the efficiency with which Googlebots can crawl your simplified site. Also, the developer doesn’t need to do anything special to boost the site’s bot-friendliness.
Different HTML, Same URL
Certain mobile sites display different content than their desktop counterparts but share the same URL. Googlebot-Mobile might have a hard time catching this one, so Google recommends implementing something called a Vary HTTP header. This is a bit of code that both prevents servers from delivering cached pages to the wrong version of the site, while also helping the robots discover your mobile content faster.
Different HTML, Different URL
This happens to be the technique that we utilized when building the Contiki mobile site. In this situation, Google encourages the use of special link tags, marked “canonical” and “alternate,” in the HTML of both the desktop and mobile versions, respectively. This unique annotation helps Google assess the relationship between the different implementations of the site and act accordingly. These tags tell the robots that the 2 versions “…should be treated as one entity instead of two entities. If they are treated separately, both desktop and mobile URLs are shown in desktop search results, and their positions may be lower than they would otherwise be.”
Redirects
If your mobile site has a different URL than your desktop site, a redirect is probably appropriate. There are a number of different says to achieve this, but the result should be the same: the mobile user navigates to your website and gets automatically redirected to the mobile-optimized version. Again, Google reminds us to set the Vary HTTP header when using automatic redirects- this helps with caching and “…is another signal for Googlebot and our algorithms to discover and understand your website’s configuration.”
In a blog post introducing the smartphone flavor of Googlebot-Mobile, Google tells us that its bots are now able to identify themselves as smartphones, and can therefore increase Google’s coverage of smartphone content. Google has also implemented something called Skip Redirect for Smartphone-Optimized Pages. In Google’s words: “When we discover a URL in our search results that redirects smartphone users to another URL serving smartphone-optimized content, we change the link target shown in the search results to point directly to the final destination URL. This removes the extra latency the redirect introduces leading to a saving of 0.5-1 seconds on average when visiting landing page for such search results.” Shaving any time off of that client-side redirect would be quite beneficial- yet another reason to follow Google’s recommendations.
The robots are friendly, if you give them what they want.
These particular bots are relatively innocuous- the worst they’ll do is ignore you. If, however, you want your mobile site to get some Googlebot attention, give them what they want: don’t disallow crawling using robots.txt, set the Vary HTTP header, and add the “alternate” and “canonical” link tags. And, lastly, hope for the best.