A facial recognition system that’s been quietly used by an Oslo pizza shop has spilled its logs, revealing that a) the shop’s advertising app is keeping tabs on who looks at ads for how long and whether or not they’re smiling, and b) that there are very few women who eat pizza in Oslo.
A crashed advertisement reveals the code of the facial recognition system used by a pizza shop in Oslo… pic.twitter.com/4VJ64j0o1a
— Lee Gamble (@GambleLee) May 10, 2017
Redditor forsaken75 has claimed to be the one who took the original photo of the crashed app. In a thread whose Norwegian title translates to “This crispy screen on Peppes’Pizza shows a log of all their guests, with descriptions,” forsaken75 said that the photo in the Tweet above was taken in front of Peppes Pizza in South Oslo.
That screen normally shows advertisements. When the advertising app crashed, it revealed what was running beneath, forsaken75 explained two weeks ago. When he approached the screen to take a photo, it began describing him, including details such as that he’s a young male, wearing glasses, where he was looking, whether he was smiling, and how much he was smiling.
Forsaken75, whom the Norwegian news outlet Dinside identified as Jeff Newman, said he originally posted the photo on Facebook in order to point out that all of this information is being gathered surreptitiously:
People may not know that these sort of demographics are being collected about them merely by approaching and looking at an advertisement. The camera was not, at a glance, evident. [The original post] was merely meant as informational, maybe to point out what we all know or suspect anyway, but just to put it out in the open.
Newman theorized that the only intention of the advertising app is to engage customers and to collect demographic statistics for tuning targeted advertising. Another Redditor, vincent__h, said that sounded right. There’s no spying going on, they said – just plain old marketing:
I’ve worked with digital signage (not with Peppes) and that’s exactly what is happening here. No video or photo is recorded or sent anywhere. It’s just a facial recognition software running on a PC which logs impressions for the display. Set up correctly you’ll get decent stats for every message on the screen enabling the content producers to improve the overall effectiveness of the signage. No one at Peppes has time to spy on passerby’s :)
According to Dinside, the facial recognition technology in question comes from digital signage company ProntoTV. Jørn Olsen, ProntoTV’s marketing and communications manager, confirmed that the system dishes up gender-specific marketing: salad ads for women, and meaty ads for men.
If there’s a woman in front of the sign, it will display more healthy varieties, and less meat.
The ProntoTV system uses analytics from Kairos: an artificial intelligence (AI) company out of Miami that recognizes faces in terms of identity, emotions and demographics. Kairos says its Anonymous Video Analytics (AVA) uses sensors in display panels to scan the surrounding area and to collect anonymous information including total individual count, approximate age, gender, attention, duration of engagement, number of glances, about how far away from the sign a surveillance target – excuse me, that should be “an individual” – is, their posture, and their emotional expression.
It’s all anonymous, Kairos promises:
AVA has no ability to recognize or identify anyone. The software gathers purely numerical data, none of which is personally identifiable information. No images are ever saved.
In other words, it’s deja vu all over again. It’s the same old “It’s all anonymized” line that we hear over and over from companies whose surveillance has crept into every nook of public space and right up our high-contrast, pixelated nostrils.
If it’s not spying rubbish bins that ID and remember people’s smartphones, and thereby the movements and habits of their owners as they walk by (kind of like how web pages monitor site traffic), it’s facial recognition set up to thwart thieves by recognizing people who are exploiting the free public utility that is toilet paper.
At least when the FBI, sans warrants, dips its hands into its ocean of face images – the civil and criminal mugshot photos, the State Department’s visa and passport databases, the Defense Department’s biometric database, and the drivers’ license databases of 18 states – it doesn’t make any show of keeping it anonymous. Nope, it’s all about individual identity with that database of 30m likenesses.
The “it’s all anonymized” line that the marketers keep giving us may sound reassuring, but it’s really not.
Take the case of the spying trash bins: an array of MAC address detectors in a busy town or street could easily be used to harvest useful real-world information about an individual and their commuting and shopping habits. That’s information that marketing companies would love to get their hands on.
Simply knowing where somebody is and how long they spend there might be used to infer where they work, what shops they visit, where they eat lunch and all manner of other useful things that yes, could be used for individually targeted advertising… Or for surveillance purposes.
The pizza shop sign didn’t let viewers know they were being viewed and analyzed. Nor did it seek informed consent. The collected data may well be anonymized now, but who’s to say it will stay that way?
It doesn’t matter that Peppes employees are too busy flinging dough to spy on pizza eaters. They don’t have to do the spying. That’s what their employers pay advertising technology companies for.
More and more facial recognition is being foisted upon us, and it’s being done by companies that are far less accountable to laws stipulating how data can be stored and used than are law enforcement agencies such as the FBI.
In the US, lawmakers recently raked the FBI over the coals over its burgeoning facial recognition database.
But the FBI at least, is required, by law, to first publish a privacy impact assessment before it uses facial recognition technology. Granted, for years, it did no such thing. But at least there was a requirement, and an oversight committee to give the FBI hell when it flunked its privacy obligations.
Who’s taking Peppes Pizza to task? What requirements are there that your local pizza parlor assess the privacy impact of watching you drool over a Chicago Deep Dish?
ProntoTV claims to have gotten the thumbs-up on its spying signage from the Norwegian data inspector. But when Dinside contacted senior adviser Stian D. Kringlebotn, it suddenly didn’t sound all that kosher. He said he wasn’t sure the signage was legal. The rules for camera surveillance will largely apply, he said, meaning that the sign is probably not allowed under Norwegian privacy law.
Camera surveillance in the name of stopping crime? Yes, that’s OK, he said.
Camera surveillance for pizza advertising?
Not so much.