We had scores of still images, pored over by both official investigators and self-appointed ones on Reddit.
We had hours of video footage.
The suspects’ images were on file. The FBI had even previously targeted Tamerlan Tsarnaev as a person of interest.
Why, then, didn’t facial recognition technology help to winnow down the thousands of images and help to identify Tamerlan and his brother, Dzhokhar Tsarnaev, the two suspects behind the Boston Marathon bombings?
From the Saturday edition of The Washington Post, whose writers interviewed Boston Police Commissioner Edward Davis:
"The technology came up empty even though both Tsarnaevs’ images exist in official databases: Dzhokhar had a Massachusetts driver’s license; the brothers had legally immigrated; and Tamerlan had been the subject of some FBI investigation."
But as Ars Technica has noted, facial recognition systems are of limited use when fed grainy, low-resolution images, captured at a distance, from a cellphone camera or surveillance video, as with the images captured before, during and after the violence unleashed at the 2013 Boston Marathon.
People have high expectations for facial recognition, but those expectations might be premature.
Coincidentally, just two days ago, I told a client that it would be wise for us to kill a story on the growing use of facial recognition in airports, given that my initial research showed there is none.
Not that facial recognition technology doesn’t have great potential, mind you.
Peter Spatharis, CEO of ACAGI, which makes biometrics-based identification systems primarily for battlefield use, ticks off a number of reasons why facial recognition beats fingerprint or retinal scanning for use as a biometric identification tool to spot flight risks, terrorists and other criminals.
One of the best things about facial recognition technology, Spatharis pointed out, is that we have rich repositories of images, flowing from sources including CCTV networks strung up in airports and throughout urban areas like Boston’s Back Bay, where the bombs detonated.
That fact was underscored by the hundreds of hours of video sifted through by more than a dozen investigators following the Marathon Monday tragedy.
We are awash in images of dangerous or potentially dangerous people, held in voluminous databases: the US Department of Defense’s Biometrics Fusion Center, for one.
Another repository includes images retrieved from applications submitted by those who want to work for Al Qaeda, retrieved in a raid in Iraq, that’s now held at West Point’s Combating Terrorism Center, Spatharis notes.
Capturing images is easy and unobtrusive, being much less an impingement on a subject’s privacy and requiring none of the physical access required by retinal scans or fingerprinting.
As such, facial recognition technology would be, in many respects, a useful tool for airports, border crossings and police work, and it’s easy to see why many police departments are eager to get their hands on it.
The FBI, for its part, expects to create a facial recognition system in 2014 for a consortium of police agencies in California and eight other Western states in the US, as the San Jose Mercury News reported on Friday.
The Atlantic Wire reports that the FBI’s billion-dollar project, which now only consists of a few pilot programs, is called Next Generation Identification (NGI).
Beyond improved fingerprinting and iris scanning technologies, NGI also includes the building up of a facial recognition database—a core component of identifying criminals, of course, but also a privacy sore spot, as exemplified by Facebook’s use of facial recognition to identify your friends and potentially use such information in ways that many users find distinctly problematic.
But as the painstaking work of investigators in Boston clearly shows, the technology is far from perfect.
Beyond the difficulties presented by low-resolution images typical of CCTV and phone captures, it gets confused by differences in subjects’ posture and in varying illumination situations.
Given that high-quality images can make or break facial recognition, it’s understandable that when the technology does work, it’s typically riding on the back of monster bandwidth, Spartharis told me:
"One face, a [typical] picture, is a 2 megapixel picture. It's substantial. If you were to do it on a stream you'd need 1080p, which is, by definition, high-definition. Our technology doesn't need that kind of bandwidth. The processing is done at a different location. But there are systems out there, on iPhones, that shoot a picture and send it to a central processing unit wirelessly, but that's based on a single picture. And [even] that takes time."
Stitching together disparate facial image databases, as the FBI plans to do via NGI, should be one step closer to making facial recognition logistically practical.
But in the near term, facial recognition will likely continue to be carbon-based.
In Boston, that meant strings of connections, made via grueling, step by step manual processing of hours of video and thousands of images, bolstered by the simple, crucial human ability to recognize faces.
To wit: the eight words penned by Jeffrey Bauman, a man still groggy from anesthesia after having had both legs removed at the knee:
“Bag. Saw the guy, looked right at me.”
Image of Profile of man courtesy of Shutterstock.
14 comments on “Why facial recognition couldn’t identify Boston bombing suspects”
Zoom in! ENHANCE!.. no? obviously it's not CSI, but in the near future I think technology will improve to the point that even people on these low-res pictures can be identified
I think the more likely advancement will be elimination of these low-res cameras. Digital camera technology is advancing at a good rate and so is our ability to handle the larger res files.
On a side note, I do always love laughing at CSI and other shows though that start with a low-res picture and somehow zoom in and pull out details that just don't exist.
What happens when the bad guys are watching us?
Speaking as a guy who works in the security industry, shows like CSI and NCIS have completely and totally ruined people's expectations of what this stuff can do. I have access to some seriously cutting edge stuff, but at the end of the day it is nowhere close to what these boneheads show on TV.
It makes my job quite difficult sometimes!
Reminds me of http://xkcd.com/683/
The actors on those shows also fly through Windows 8 as if it were actually an intuitive GUI.
I have lost count of the number of times I have been asked to "enhance" a photo only to hear "that is the best you can do? I saw on CSI where they did better in less time with an ATM picture." My reply, "give me the budget of one CSI show and I too can send some out to take a better image with a better camera. TV is NOT real life."
What grinds on me especially is how they go about "hacking" websites. It's laughable really lol.
Not to worry, Comrades! Since the BFF was pulled off, we'll now be able to fly drones 24/7 over the Motherland, keeping an eye out for terrorists.
And if some of those drones are armed, well, that's the price you have to pay for liberty.
It's easy enough to identify someone from lo res CCTV footage… next to impossible to do the same via a still image taken from the same CCTV.
Just maybe the government doesn’t have the sharpest knives in the cabinet and rarely fires anyone for anything–especially incompetence.
Information on the first bomber’s trip to Russia in 2011 reportedly never entered the system due to a misspelling. Where was the phonetic analysis software? It would be useful to see just how much the government spent on FR software and development; if it’s like so many other bungled projects, the results wont be pretty. It would also be interesting to learn if the folks using software in casinos could have done a better job.
Here come the calls for more illegal surveillance of American citizens.
What if they where set up to be the fall guys? if you do your own research on the boston bombings you will find the evidence overwhelming in favor of a false flag operation. thats why facial recognition failed. Most of the photos you see are doctored and poorly at that.
Which is more likely:
1) Facial recognition failed because facial recognition systems aren’t very good with low quality images of faces
2) Facial recognition failed because facial recognition systems *are* good with low quality images of faces but every scrap of CCTV footage from the event has been edited