We’ve been thinking of the National Security Agency’s (NSA’s) XKeyscore search engine on the wrong scale.
Newly revealed documents from the Edward Snowden trove show that it’s much more than a Google-like search engine for the world’s private communications data.
Rather, it’s more like a gigantic sea-dwelling kimodo dragon with double-jointed jaws that snarfs up the data equivalent of Great White Sharks like they were hors d’oeuvres.
As The Intercept reports, besides suctioning off web traffic from the fiber-optic cables that form the backbone of the world’s communications network, including data on people’s internet searches, documents, usernames, passwords, emails, and chats, it also absorbs 700,000 voice, fax and video files every day.
That includes pictures, documents, voice calls, webcam photos, web searches, advertising analytics traffic, social media traffic, botnet traffic, logged keystrokes, computer network exploitation (CNE) targeting, intercepted username and password pairs, file uploads to online services, VOIP streams taken from Skype sessions, and more.
What’s more, XKeyscore users – which include not only NSA agents but also their counterparts in partnering countries that include Canada, New Zealand and the UK – can get the keys to every wireless network through the gathering of router information.
One of the slides in the new set of XKeyscore screen captures and other documents (the full list is at the end of The Intercept’s report) shows the incredible breadth of surveillance that can be done, showing a sample query of people based on their location, nationality and browsing histories.
The slide depicts the search “germansinpakistn”, showing an analyst querying XKeyscore for all individuals in Pakistan visiting specific German language message boards.
The “justification” for the search, as entered in the easy-to-navigate search form: “german ct targets in pak visiting web forums.”
Another slide shows how to search for CNEs, encouraging users to “go way back,” since “CNE Data doesn’t age off”.
Speaking of retention, the newly released documents show how long the XKeyscore servers have been holding this data, at least as of 2009.
From The Intercept:
[The XKeyscore system's] servers collect all of this data for up to five days, and store the metadata of this traffic for up to 45 days.
The system relies on more than 700 servers that, as of 2008, were located in 150 field sites in the US, Mexico, Brazil, UK, Spain, Russia, Nigeria, Somalia, Pakistan, Japan, Australia, and many other countries.
The servers reportedly store “full-take data” at the collection sites – meaning that they’ve captured all of the traffic collected.
One of the documents shows that Foreign Intelligence Surveillance Act (FISA) courts have authorized “full-take” collection of traffic from at least some US web forums.
XKeyscore is both brawny and resilient.
An NSA briefing says it’s “a fully distributed processing and query system that runs on machines around the world” that can run on multiple computers, giving it “the ability to scale in both processing power and storage.”
Other NSA documents have indicated that tens of billions of records are stored in its database.
As far as how the search engine prevents analysts from attempting to make overly broad searches that would break laws such as the US’s Fourth Amendment guarantee against unreasonable search and seizure, the answer is short: it doesn’t appear to.
The NSA has claimed that it doesn’t target US citizens’ communications without a warrant, but it’s also acknowledged that it “incidentally” collects and reads some communications without a warrant, minimizing the information that’s retained or shared.
Training documents for the system put the burden on the “user/auditor” to comply with rules such as USSID-18 – the US Signals Intelligence Directive 18 that governs “U.S. person minimization” – or, apparently, the British Human Rights Act (HRA).
But Kurt Opsahl, the Electronic Frontier Foundation’s (EFF’s) general counsel, believes that’s little more than a nod to CYA:
[It's] an attempt by the intelligence community to comply with the Fourth Amendment. But it doesn’t come from a court, it comes from the executive.
…and that there’s nothing built into XKeyscore that would serve as a technical roadblock to illegal searches taking place in the first place:
The document discusses whether auditors will be happy or unhappy. This indicates that compliance will be achieved by after-the-fact auditing, not by preventing the search.
As The Intercept notes, prior to Snowden having pulled back the curtain on surveillance, encryption wasn’t as common as it’s become, due in no small part to his revelations.
That means that XKeyscore could have gotten into far more unsecured networks, and it would have been a snap for any user to get significant returns.
The publication quotes security researcher Jonathan Brossard:
Anyone could be trained to do this in less than one day: they simply enter the name of the server they want to hack into XKEYSCORE, type enter, and are presented login and password pairs to connect to this machine. Done. Finito.
As The Intercept previously reported, the NSA has expressed a preference for targeting systems administrators, having this to say on the matter in a 2012 post in an internal discussion board:
Who better to target than the person that already has the ‘keys to the kingdom?’
In a statement, the NSA told The Intercept that all of this is perfectly legal:
The National Security Agency’s foreign intelligence operations are 1) authorized by law; 2) subject to multiple layers of stringent internal and external oversight; and 3) conducted in a manner that is designed to protect privacy and civil liberties. As provided for by Presidential Policy Directive 28 (PPD-28), all persons, regardless of their nationality, have legitimate privacy interests in the handling of their personal information. NSA goes to great lengths to narrowly tailor and focus its signals intelligence operations on the collection of communications that are most likely to contain foreign intelligence or counterintelligence information.