Why facial recognition couldn’t identify Boston bombing suspects

Boston Marathon bombersWe had scores of still images, pored over by both official investigators and self-appointed ones on Reddit.

We had hours of video footage.

The suspects’ images were on file. The FBI had even previously targeted Tamerlan Tsarnaev as a person of interest.

Why, then, didn’t facial recognition technology help to winnow down the thousands of images and help to identify Tamerlan and his brother, Dzhokhar Tsarnaev, the two suspects behind the Boston Marathon bombings?

From the Saturday edition of The Washington Post, whose writers interviewed Boston Police Commissioner Edward Davis:

"The technology came up empty even though both Tsarnaevs’ images exist in official databases: Dzhokhar had a Massachusetts driver’s license; the brothers had legally immigrated; and Tamerlan had been the subject of some FBI investigation."

But as Ars Technica has noted, facial recognition systems are of limited use when fed grainy, low-resolution images, captured at a distance, from a cellphone camera or surveillance video, as with the images captured before, during and after the violence unleashed at the 2013 Boston Marathon.

People have high expectations for facial recognition, but those expectations might be premature.

Coincidentally, just two days ago, I told a client that it would be wise for us to kill a story on the growing use of facial recognition in airports, given that my initial research showed there is none.

Not that facial recognition technology doesn’t have great potential, mind you.

Peter Spatharis, CEO of ACAGI, which makes biometrics-based identification systems primarily for battlefield use, ticks off a number of reasons why facial recognition beats fingerprint or retinal scanning for use as a biometric identification tool to spot flight risks, terrorists and other criminals.

One of the best things about facial recognition technology, Spatharis pointed out, is that we have rich repositories of images, flowing from sources including CCTV networks strung up in airports and throughout urban areas like Boston’s Back Bay, where the bombs detonated.

That fact was underscored by the hundreds of hours of video sifted through by more than a dozen investigators following the Marathon Monday tragedy.

Profile of man. Image from ShutterstockWe are awash in images of dangerous or potentially dangerous people, held in voluminous databases: the US Department of Defense’s Biometrics Fusion Center, for one.

Another repository includes images retrieved from applications submitted by those who want to work for Al Qaeda, retrieved in a raid in Iraq, that’s now held at West Point’s Combating Terrorism Center, Spatharis notes.

Capturing images is easy and unobtrusive, being much less an impingement on a subject’s privacy and requiring none of the physical access required by retinal scans or fingerprinting.

As such, facial recognition technology would be, in many respects, a useful tool for airports, border crossings and police work, and it’s easy to see why many police departments are eager to get their hands on it.

The FBI, for its part, expects to create a facial recognition system in 2014 for a consortium of police agencies in California and eight other Western states in the US, as the San Jose Mercury News reported on Friday.

The Atlantic Wire reports that the FBI’s billion-dollar project, which now only consists of a few pilot programs, is called Next Generation Identification (NGI).

Beyond improved fingerprinting and iris scanning technologies, NGI also includes the building up of a facial recognition database—a core component of identifying criminals, of course, but also a privacy sore spot, as exemplified by Facebook’s use of facial recognition to identify your friends and potentially use such information in ways that many users find distinctly problematic.

But as the painstaking work of investigators in Boston clearly shows, the technology is far from perfect.

Beyond the difficulties presented by low-resolution images typical of CCTV and phone captures, it gets confused by differences in subjects’ posture and in varying illumination situations.

Given that high-quality images can make or break facial recognition, it’s understandable that when the technology does work, it’s typically riding on the back of monster bandwidth, Spartharis told me:

"One face, a [typical] picture, is a 2 megapixel picture. It's substantial. If you were to do it on a stream you'd need 1080p, which is, by definition, high-definition. Our technology doesn't need that kind of bandwidth. The processing is done at a different location. But there are systems out there, on iPhones, that shoot a picture and send it to a central processing unit wirelessly, but that's based on a single picture. And [even] that takes time."

Stitching together disparate facial image databases, as the FBI plans to do via NGI, should be one step closer to making facial recognition logistically practical.

But in the near term, facial recognition will likely continue to be carbon-based.

In Boston, that meant strings of connections, made via grueling, step by step manual processing of hours of video and thousands of images, bolstered by the simple, crucial human ability to recognize faces.

To wit: the eight words penned by Jeffrey Bauman, a man still groggy from anesthesia after having had both legs removed at the knee:

“Bag. Saw the guy, looked right at me.”

Image of Profile of man courtesy of Shutterstock.