The surveillance state has arrived. we are all winston smith now database for dummies

You don’t need a weatherman to know which way the wind blows and you don’t have to be paranoid to recognize that citizens of the industrialized world are now being watched, measured, rated, judged and analyzed daily to a degree that even Winston Smith, the hapless worker bee from George Orwell’s prescient 1944 novel 1984, might find ultra-invasive.

The average American appears on CCTV 75 times a day; the average Londoner more than 300. Our keystrokes and searches on computers and mobile devices are captured and analyzed. Google “ How to kill someone with an undetectable poison” and you can be certain that it will come up at your trial. Every time we fire up the old Dell, thousands of ransomware thugs are waiting out there in the Deep Net to freeze our screens and extort money. As tech investor Susan Guo of Greylock Partners said at a recent conference:


It’s inevitable that we will soon have cameras watching our every move. There’s a saying, ‘You are who you are when no one’s watching.” We’re going to be watched all the time and it will certainly alter our behavior. It’s not my fault what the future holds, so if that’s scary I apologize. Case in point

China is already building a vast surveillance network designed to eventually track all 1.4 billion citizens through a combination of facial-recognition technology, group-chat monitoring, and smartphone apps created by the state — giving people scores based on their “social credit.” People with good scores will be rewarded, and those with bad ones will be punished. Sixteen areas in China are already using facial-recognition technology. Writes People’s Daily:

The system is able to identify 40 facial features, regardless of angles and lighting, at an accuracy rate of 99.8 percent. It can also scan faces and compare them with its database of criminal suspects at large at a speed of 3 billion times a second, indicating that all Chinese people can be compared in the system within only one second.

The system is proof of outstanding performance in technology based assistance in cracking down various cases, such as drug trafficking, theft, robbery and abduction. In the past two years, more than 2,000 criminals at large were caught by public security organs with the aid of the system.

The system is part of Skynet, a nationwide monitoring program launched in 2005 to increase the use and capabilities of surveillance cameras. The system basically marries AI to CCTV surveillance and uses facial recognition and GPS tracking to spy on everybody. Currently, there are 170 million surveillance cameras in China and, by 2020, the country hopes to have 570 million. That’s roughly one camera for every two citizens.

The most frightening aspect of Skynet is its “social credit system,” also known as Citizen Score, which is linked to 1.3 billion Chinese citizens’ national ID cards, and scores each citizen on their behavior. The ACLU said the system leveraged “all the tools of the information age—electronic purchasing data, social networks, algorithmic sorting—to construct the ultimate tool of social control.”

For example, your Citizen Score, which is tied to credit scores, could go down if a “friend” in social media did or said something the government considered wrong, such as buying video games the state considered inappropriate, posting political comments without permission or posting anything that might annoy or embarrass the state.

In short, if you set out to build the most authoritarian country in the world in which every citizen’s behavior is monitored and directly controlled by the state, this is the type of system you would use. Transgressions covered range from tracking down fugitive officials spending stolen money abroad to the somewhat more mundane. In restrooms at Beijing’s Temple of Heaven Park, patrons have their faces scanned before a machine spits out a 24-inch strip of toilet paper. It will not dispense more to the same person until nine minutes have passed. It brings a whole new meaning to ‘taking a dump.’ Send in the drones

The scariest tech announcement of the past week is that drone manufacturer DJI has teamed up with Axon, the company that makes Taser stun guns and police body cameras, to sell drones to local police departments around the United States. Police departments will be able upload or stream footage to Axon’s digital cloud for police cameras, like the body cameras it currently sells, where it will be automatically analyzed by AI systems not disclosed to the public. The Axon announcement is short on details about how the system might work:

The Axon Air program allows law enforcement agencies to purchase drones from a trusted partner and links DJI’s drone technology with Axon’s connected data network and Evidence.com services – the same platform that more than 200,000 public safety professionals use today. Evidence.com provides the same rigorous data management system, chain-of-custody controls and security protocols that law enforcement agencies rely on to preserve and protect data from body cameras and in-car video systems. Within Evidence.com the integrity of drone videos is preserved for the protection of police agencies and the public they serve.

A few days earlier, Cambridge researchers in India and the UK published a paper with the unwieldy title– Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network—that showed how such a drone system might be used to detect “violent individuals” in real-time.

The system uses a simple drone to transmit video footage over a mobile internet connection for real-time analysis. An algorithm trained using deep learning estimates the poses of humans in the video and matches them to postures the researchers have designated as “violent.” For the purposes of the project, just five poses were included: strangling, punching, kicking, shooting, and stabbing.

To train the AI, the researchers flew a drone to snap 2,000 images of people pretending to hurt each other. But since the researchers didn’t use real data, and the photos are painfully and obviously staged, the likelihood of detecting real violence and normal human motion is considered suspect at best. David Sumpter, author of Outnumbered: Exploring the Algorithms that Control Our Lives, wrote on Medium:

Imagine, for example, that a western government used an AI-powered system to monitor and control the behavior of its citizens in the manner that China has openly embraced. Imagine it was used to monitor political enemies? What if the system was used to identify only members of a particular ethnic or minority or religious group? Should it ever be used to launch tear gas or smoke bombs? There are so many unanswered questions and the technology is outpacing the moral, ethical and legal choices.

The notion of local police departments having access to military-grade surveillance equipment makes about as much sense to me as the Army selling used tanks and tactical equipment to law enforcement groups or allowing individuals to own military-style assault weapons.

The complexity of building a drone-based AI “violence” detection system, even as simplified as the one from the Cambridge researchers, suggests that we are still some time away from a reliable public violence detection system. Axon, which has bought two AI companies and has access to petabytes of police camera footage, has not disclosed how its AI was trained or whether it’s accurate, unbiased and tested.

The public debate on ethics in AI related technologies has, so far, focused on commercial activities and especially on tackling unfair bias with commercial consultancies like Accenture entering the fray. While this may seem laudable, no-one has questioned the dichotomy that must exist between a profit-driven organization (like Accenture) and the problem it claims to be solving.

It seems to me that this kind of debate needs to be much broader with a heavy emphasis on the ethics in play. Our society is changing at breakneck speed and in radical ways. Ignoring the implications of advanced technology and turning a blind eye to the potential for creeping totalitarianism is dangerous for our collective future. Winston Smith never had that choice. We do.

banner