There are two distinct groups of clicks being counted. For clicks to the search engine (from the micro-site) I can use referrer stuff.

For clicks to the micro-site itself, I disregarded referrer stuff (for the reason you said and from previous experience I know Twitter clients often omit them), and looked instead at the requests of the images on the page.

So what would click to the page but not request an image? Either they left immediately (though even then I'd expect the first image to be at least requested) or a lot of people have no images enabled (very unlikely for mainstream Twitter users) or it is some form of click fraud (e.g. bots, miscalculation?) or something else?