Sophos Cloud Optix
Facebook is testing data mining methods that would silently follow users’ mouse movements to see not only where we click but even where we pause, where we hover and for how long.
And holy mackerel, did somebody say something about there being the potential ability to track how long a user’s cursor hovers over an o-so-tasty, revenue-generating ad?
Why yes, and that somebody was Ken Rudin, Facebook’s analytics chief.
At the Strata and Hadoop World Conference in New York on Tuesday, Rudin told the Wall Street Journal that the already data-stuffed social network would have to purchase data pants with a stretchy waistband if it does decide to gorge itself on data about users’ cursor movements.
Rudin told the WSJ that the ongoing tests are part of a broader technology testing program.
Facebook should know in the coming months whether incorporating the new data collection makes sense for a slew of uses, be it product development or more precise targeting of ads, he said.
Facebook is looking at collecting data such as “did your cursor hover over that ad … and was the newsfeed in a viewable area,” he said.
You well might question whether cursor tracking isn’t in fact already a standard part of web analytics.
Back in 2011, Microsoft researchers looked at how to use cursor movement to understand and improve search results.
They came up with an easy way to track users’ gaze direction on a website using nothing but a standard web browser and a practically imperceptible Javascript of less than 1kb that could be run invisibly on any page without slowing its load time or a browser’s performance, as MIT Review described at the time.
It turns out that where we place our mouse cursor closely correlates with eye gaze – i.e., what we look at on pages – especially when looking at search results, the researchers found.
The researchers came up with (PDF) the ultralightweight gaze-tracking tool by examining mouse cursor behavior on search engine results pages, including not only clicks but also cursor movements and hovering over different page regions.
On page 5 of the Microsoft paper, images of heat maps of click positions vs. recorded cursor positions show that cursor movements provide far richer data about how frequently a user interacts with a given page.
Two years later, is Facebook ahead of the curve in planning cursor tracking, or is it playing catchup?
It turns out that Facebook well might be in the vanguard, given that advances in cursor tracking haven’t yet replaced, to any extensive degree, simple maps such as those for Google Analytics that merely show where we’ve clicked on a page.
In fact, such click maps, typical of most website analytics, don’t actually show where a user has clicked; rather, they show only which page the user ended up on and which links can go there.
Exceptions to the web analytics status quo of simple click maps include third-party services that do, in fact, offer cursor and hover tracking.
The WSJ reported on one such, Shutterstock, in March.
At the time, Shutterstock founder and CEO Jon Oringer said that his company – which is a marketplace for digital images – was looking at “every move a user makes,” including where site visitors place their cursors and how long they hover over an image before making a purchase.
Rudin, being Facebook’s data chief, is preparing the company’s infrastructure for the massive data binge that would come out of such cursor/hover tracking.
But as Rudin himself pointed out, the deluge of information isn’t going to help anybody unless Facebook can figure out how to make use of it:
Instead of a warehouse of data, you can end up with a junkyard of data.
He told the WSJ that he’s led a project to index the data in Facebook’s analytics warehouse, which is actually separate from its user data.
Javascript processing has relieved the strain on the browser for this type of tracking. Now, the only problem that remains is how to store and process all the resulting data.
What do you think: if Facebook does decide to collect the new behavioral data and actually does manage to to cinch its belt around its resulting bloated data belly, will users’ privacy be that much more pinched?
Or have we already been chewed up and digested to the point that it really doesn’t matter any more?
Let us know in the comments section below.
And not that we want to make you feel guilty or anything, but we think you should know that our feelings will be hurt unless you hover long and lovingly over everything posted on – where else? – Naked Security’s Facebook page.
Time to start surfing using only mobile devices with touch screens? Anyway, it won't affect me much, as I tend to use the mouse only when necessary and move the pointer out of the viewable area when it isn't. That's a habit I developed when using the computer too much was putting stress on my hands and fingers, and it's probably a good one for that reason alone, no matter who's tracking your mouse pointer.
With touch devices, you definitely don't get the kind of feedback you do from a computer and pointing device, since there's no 'mouse cursor' as such. Having said that, Facebook use a completely different, touch-optimised layout for such devices when they visit via the web.
Hmm, Samsung Galaxy devices with hover capabilities and eye tracking? Not quite the same, but they are tracking all sorts of things including the way you type/swype on a keyboard. Too much tracking of anything.
Using the proprietary Facebook app on a so called “Smart Phone” would be even worse – because then they’re no longer restricted by the limitations/boundaries of a web-browser. They will steal your address book, intercept your e-mail, grab your GPS coordinates when they wish, etc, etc, etc… One thing you can probably do for now is use the mobile page on a PC; until they decide to infect it with the same JS malware…
http://m.facebook.com/
It seems to me that this sort of activity is coming dangerously close to a privacy violation. What's next? Keystroke logging? What's to stop this script from remaining in memory and sending the user's activity even when they're logged out of Facebook?
There are limits to what information they can glean about what you're doing on Facebook's website through a standard desktop web browser. The only way they'd be able to do any more would be to somehow hook into the web browser via a toolbar or add-in, but even they that wouldn't allow actual mouse cursor tracking or keystroke logging – that would require hooking into the operating system itself. I doubt even Facebook is hungry enough for data to risk the potential financial and legal penalties for pulling a stunt like that! And in any case, this applies only to PCs – mobile devices would require a different route, such as logging what you do inside the Facebook iOS or Android app, which in turn would be inviting potential penalties if they were found out.
So if you’re running NoScript on Firefox and have Facebook untrusted, you’re OK, right? 😉
If Facebook were a person, you'd get a restraining order on him for his creepy, obsessive, personal-space violating stalking.
Please forgive if this is a very stupid question: does this mean that it is possible to track where cursor arrow is, anywhere on the page, even if nothing is clicked on? If so, this is so extremely creepy that I feel inclined to give up the internet. Track that.
Yes. Capturing the X and Y coordinates of where the cursor is on a page is a simple task in javascript and has been for a very, very, very long time.
So if it is javascript, disable the javascript when on fb.
Of course then fb will become almost useless. But that will make it much easier to decide to delete your account.
But this technology is not unique to Facebook and nor is it new. Facebook are simply saying in public that they might use this technology in future.
All of which is to say that taking specific measures against Facebook does not protect your privacy here. For all you know you have been using sites that track you like this for years.
Surely this is effectively the same as key logging; fb will know what is on the screen cos they put it there and then log what we do with the mouse, including clicking. What's the difference between clicking a mouse on pressing a key? Nothing.
If fb want to do this then I hope we have ability to opt out through the security settings – else I might just have to close my fb account.
If Facebook want to do key logging then they can – so long as you're browsing one of their pages they can capture everywhere your cursor goes and everything you type. I'm not saying they do, I've no idea, I'm just saying it's possible – any website can do it and it's very easy. The hardest part is storing and processing the data.
For example, if I had decided to stop writing this comment half way through and then deleted it without ever submitting it, it's entirely possible for the web page I'm typing into to have captured everything I typed even though I never 'sent' it.
The answer is simple – STOP USING FACEBOOK!
Quite. But not for this. Anyone can do this and you won't know if they are. So you either have to put up with it, stop using the web or use something like noscript to block the execution of scripts that you're unsure of.
The other problem is all the pages that facebook has their hooks in with link buttons and ad networks. Even without a facebook account, facebook and many other 3rd party sites are tracking users with "fingerprints" that include things like browser strings, loaded browser plugins, device profiles, IP addresses. And every day they come up with new ways to track us.
I think a lot of people are missing the big picture, and I mentioned this on Naked Securities’ Facebook page as well, but this is nothing new.
I work as a User Experience Designer/Analyst for a digital marketing company, and it’s my job to understand how people use web interfaces, understanding where users have issues, and improving their overall experience. Companies hire people like us to find out why there is a huge drop off rate at the checkout page, how to improve it and increase conversions.
Unfortunately in order to do things like this, test and studies need to be performed. One way is through A/B testing, another way is through mouse tracking, heat maps, click tracking, eye tracking, even in-person usability interviews.
Facebook is by no means the leader in this “new technology.” Amazon uses it, Microsoft uses it, Adobe uses it, and they’ve been doing so for years.
So while I understand where others may see it as an invasion of privacy, understand that the reason why Facebook wants to collect this information is to improve your overall experience with their site because they want you to keep using their product.
For me personally, I use ad blockers so I could really care less about the ads, my point is this kind of technology isn’t going away anytime soon. It’s been around for quite some time and will continue to do so.
People drop off at the checkout page because
a) Stunned at outrageous and unexpected shipping/handling charges.
b) Checkout page asks for coupon code, user goes off to find one, cannot find one, and shops elsewhere.
c) Price on checkout is higher than the price in the email advertisement the user received.
I've had all of these in the past week.
Getting fed up with Facebook., I am trying out other social medias. This mouse tracking is just the pebble adding onto my pile of frustrations of their antics. I usually use my pointer and then push it aside.
I use Firefox with AdBlock Plus, Disconnect, FB Purity, RIP, Social Fixer and TimeLineRemove.com. They all play well together and do a real good job of making Facebook more tolerable. Unfortunately Facebook doesn't see it that way; they've already threatened legal action against Social Fixer for violating their TOS but I don't see them having a leg to stand on considering it's an add-on that the user chooses to install. That's a discussion for another time. It would be nice for some developer to write an anti-keystroke add-on, but I'm not a programmer and wouldn't have the slightest idea how it could be accomplished.
Lisa, let's get the terminology right.
The on-screen element that moves with the mouse/trackpad/trackball is the "pointer."
The on-screen element that shows where data entry will next take place is the "cursor."
E.g., in MS-Word, the hollow arrow pointing northwest and following the mouse is the pointer. The I-beam element that shows the "insertion point" where letters will next be entered is the cursor.
On text fields on a web page, the cursor usually appears as a straight vertical bar instead of an I-beam but the pointer is still a northwest-pointing arrow.
I stand corrected. Thank you, Larry.
I wonder if someone could make the web browser (or something like a plugin) look for when the javascript is sending back the mouse data, and then feed bad data into that.
I use a trackball mouse on my desktop, so my mouse pointer is most often over at the side of the page for scrolling. I use "AdBlock Plus" so ads don't really show up on my FB newsfeed anyways. Good luck tracking me via mouse movement.
Just want to clarify, using definitions of cursor and pointer as specified by Larry Marks (above). Is it only the cursor that is being tracked and NOT the pointer arrow? My pointer arrow usually ends up on the scroll bar. I don't have a cursor line (The on-screen element that shows where data entry will next take place is the "cursor.") unless I actually click to start a comment, or similar entry.
This article is about tracking the pointer arrow but tracking either is trivial.
You can capture the pointer coordinates at any time so long as its over the web page. To capture it as a 'movie' you simply capture the coordinates as many times a second as you need.
The cursor, when used, triggers an event called 'focus'. You can track which element has focus, or has lost focus, at any given time. You've probably seen this in action with search fields that grow when you click in them or where default search text disappears. This is done using focus rather than clicks so it works if people use the tab key to navigate between elements.