The evil that is link prefetching

While working on WP Super Cache integration today I stumbled upon a really puzzling problem… every time I reloaded my test page on my local server, with a basic shortcode variant on it (but no metric recorded on it), I was also getting a hit against the relevant goal page. I turned all sorts of things on and off, tried logging and checking for referrers, etc. until I finally noticed that the test page was printing the following code in its head:

<link rel='next' title='Goal!' href='http://shrimptest.local/goal/' />

This was occuring (and not occurring on any of my other test pages) because in this case the test page (/about/) was a WordPress page and Goal (/goal/) was the subsequent page… in no other cases had I ever had the variant page and the metric goal page as subsequent posts/pages.

Firefox was then prefetching this URL on page load. Link prefetching is A Feature, not A Bug, but as most ShrimpTest metrics record their values (in the simplest case, conversions) via PHP, not JS, this could be a major problem. If you happen to have your variant and metric-recording pages either as sequential pages or posts, your metric could get seriously messed up, giving you conversion rates in the 70%’s or 80%’s (depending on what browsers visit you) instead of a much more modest—and honest—value.

(Note, the HTML5 standard draft recommends the user of rel=”prefetch” instead of rel=”next” for this purpose.)

WordPress offers the filters previous_post_rel_link and next_post_rel_link which can be used to make it so that these links are not printed. However, doing so could mean a (very slightly) degraded experience for the user. Moreover, if that page involves a different experiment with some variants, the initial process of taking the user and randomly assigning them to a variant could be “offloaded” and computed first via prefetching… it’s just that we don’t want prefetching to record any metrics.

As such, the best thing to do would be to detect these incoming prefetch requests and do everything the same, except for the metric recording. Firefox supplies an HTTP header for exactly that: the X-Moz: prefetch. Safari’s Top Sites feature also does a sort of prefetching which can mess up analytics, and Safari adds X-Purpose: preview.

Unless I see a good reference soon, I’m going to have to do a little old-fashioned testing with some popular browsers to see whether (a) they do prefetching and (b) if they do, whether they send a special header like Firefox or Safari. If there are browsers which do absolutely nothing to distinguish between regular requests and prefetch requests, I could (i) use the filters above to hide the prefetching link tag for those browsers or perhaps (ii) offer an option for JavaScript-based metric recording.