Flawed algorithms mark people for death

The National Security Administration’s Skynet system marks people for death based on algorithms and metadata—the same technology that Amazon uses to guess what books you’ll probably like.

I find that chilling.  I find the precedent it sets even more chilling.

Now an expert has come along who says the Skynet program is inherently flawed and has likely resulted in the deaths of thousands of innocent people.

TravelPatternsDocuments leaked to The Intercept indicate that the Skynet program collects data on people in Pakistan by monitoring their phone calls.  Supposedly terrorists can be identified within a certain margin of error by characteristics that, on average, differentiate them from non-terrorists.

Patrick Ball, director of research for the Human Rights Data Analysis Group and a frequent expert witness before human rights tribunals, told Ars Technica that the problem with this is that the terrorist sample is based on a very small number—seven individuals—and the innocent sample is based on a random sample of 100,000 people.

Since there is usually no independent way of verifying that the victim really was a terrorist, that means that there is no “learning” process, as would be the case with a commercial algorithm, such as Amazon’s, which is based on commercial responses.

One of the variables in setting the algorithms is that the fewer false negative (real terrorists who are not detected by the system), the more false positives there will be (innocent people who are marked as terrorists).

Bell said that if the algorithm is set at 50 percent false negatives, that means thousands of innocent people will be killed for every real terrorist who dies.

[Added later]  Martin Robbins wrote in The Guardian that Skynet is used to identify likely Al Qaeda couriers, who are not killed but tracked so as to reveal the locations of Al Qaeda camps and safe houses.   It is a fact that computer algorithms are used to target people for killing, but Skynet isn’t as clear an example as I originally thought.

Every American ought to be troubled by the fact that our government has a policy of (1) extrajudicial killings of (2) people who are not known enemies in (3) in countries with which the US is not at war and (4) have not officially invited the United States to operate in their countries, but especially that (5) responsibility has been shifted from human beings to computer programs.

The lives of people in remote areas of Pakistan and other countries are just as valuable in the cosmic scheme of things.  I don’t question that.  But my message to my fellow middle-class white Americans is that anything that can be done to a Muslim with a brown skin and a Arabic or Persian name can and sooner or later will be done to Christians with white skins and Anglo-Saxon names.

Incidentally, the Skynet system in the Terminator movie series was an artificial intelligence program created for fighting wars that eventually turned against the human race.  The fact that the NSA chose to name its system Skynet may say something about the mentality of its administrators.


The NSA’s SKYNET program may be killing thousands of innocent people by Christian Grothoff and J.M. Porup for Ars Technica.

U.S. Government Designated Prominent Al Jazeera Journalist as “Member of Al Qaeda” by Cara Currier, Glenn Greenwald and Andrew Fishman for The Intercept.

[added later] Has a rampaging AI algorithm really killed thousands in Pakistan? by Martin Robbins for The Guardian.

Tags: , , , , , ,

One Response to “Flawed algorithms mark people for death”

  1. thetinfoilhatsociety Says:

    It already is being done. Witness Finicum. What he was doing was no different than students taking over public buildings and doing sit ins. The difference is those students with rare exceptions weren’t called terrorists and gunned down. And they didn’t have land the Feds wanted.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: