Who Watches the [Artificial Intelligence] Watchmen?

Open Question: What are the unintended consequences of artificial intelligence in the surveillance domain?

Recently, I came across a good basic guide for finding hidden cameras and bugs with a level of detail I’ve never seen online before. As with everything related to security, the first question to ask is: what is your threat model?

For most people, the need to look for hidden cameras and bugs is not something they need, or at least they don’t think they need it. But, there are situations where the kind of operational security used by spies could also be useful for everyone. The most obvious example is something like AirBnB, where there may be good reason to suspect the risk for hidden cameras and bugs might be higher for everyone than in other similar circumstances, such as a reserved hotel room at a reputable hotel chain or in our own homes.

So, it is useful information to know. If using a service like AirBnB is something you do regularly, it may be worth investigating these techniques in greater detail, or at least have it handily bookmarked.

The problem of recording devices in an AirBnB strikes me as similar to the case where you travel a lot and regularly use open wifi networks. The increased risk to your threat model might warrant the services of a virtual private network provider, if you don’t already use one. Again, it depends on your threat model.

This issue got me thinking about the larger pattern of surveillance, not just of online spaces but of physical space. Police departments are using overhead surveillance drones to monitor areas indefinitely (the level of which can be increased to monitor public spaces during large gatherings), registering private security cameras, automated license plate scanners, body and squad car cameras and so forth. These are being combined with online surveillance technologies to map social media to physical spaces. All of these technologies are being combined together:

“By combining drone, body-camera, police-car-camera, and closed-circuit-TV footage, Axon is clearly hoping to create a central hub for police to cross-reference and access surveillance data—a treasure chest of information that, according to Elizabeth Joh, a law professor at the University of California–Davis who studies civil liberties and police surveillance technology, police departments could find difficult to stop using once they start. “Not only is there no real competition from other vendors,” said Joh, “but once a police department has bought into a certain contract with a company, it’s very hard to drop it and move on. There’s a lot of investment in training the agency and the officers how to use it.”

April Glaser, “The Next Frontier of Police Surveillance Is Drones.” Slate.com. June 7, 2018

Companies like Palantir that cut their teeth on developing anti-terrorism surveillance and big analytic products that are now being rolled out to local police departments. All of this is happening with relatively little oversight.

Of course, big data means that artificial intelligence is being trained on all of this surveillance data. One task is to train artificial intelligence algorithms to recognize facial, gait, voice and other identifying characteristics of individuals. Another is to create a time series to be able to track those individuals in time and space. It will change the way police interact with their population, because they will have a good idea of who was in the area, so the software will offer them a list of possible perpetrators and witnesses, without necessarily good indication of which is which.

It reminds me of a quote:

“One of the major purposes of state simplifications, collectivization, assembly lines, plantations, and planned communities alike is to strip down reality to the bare bones so that the rules will in fact explain more of the situation and provide a better guide to behavior. To the extent that this simplification can be imposed, those who make the rules can actually supply crucial guidance and instruction. This, at any rate, is what I take to be the inner logic of social, economic, and productive de-skilling. If the environment can be simplified down to the point where the rules do explain a great deal, those who formulate the rules and techniques have also greatly expanded their power. They have, correspondingly, diminished the power of those who do not. To the degree that they do succeed, cultivators with a high degree of autonomy, skills, experience, self-confidence, and adaptability are replaced by cultivators following instructions. Such reduction in diversity, movement, and life, to recall Jacobs’s term, represents a kind of social ‘taxidermy’.”

― James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed

So, it isn’t hard to imagine a situation evolving where police work reduces to following up leads that are generated by artificial intelligence. The amount of data, the kind of data, the assumptions being employed, and so forth will all be a black box to the officer on the street. The simplified map will become the territory. The beat cop will become the instrument of the algorithm designers, who may or may not be getting feedback on failures of the system. Many of these problems of these tools will be subtle, such as how their use changes the culture of the police department. People won’t know what to look for, and by the time the problems are identified, they may already be baked into the culture. It will certainly be too late for the individuals effected by software bugs, with errors being miscarriages of justice against individuals and prison sentences.

It isn’t hard to imagine a mature industry progressing to Philip K. Dick concepts of “pre-crime”. Artificial intelligence systems will be expanded to look for larger patterns in the data that tend to lead to crime, and there will be compelling arguments to use this information to stage interventions.

Who will watch these artificial intelligence watchmen? Is it even possible? In the mad dash to implement these systems, what kind of oversight is there? Sadly, the answer is: none.

Rearranging Our Minds

Open Question: Should we make an effort to change our minds in some fundamental way? And if so, how?

There are a number of stories of people suffering a traumatic brain injury that results in the brain being rearranged in a way that gives them a new ability. Generally, this involves some skill with art, understanding music, improved memory or doing calculations in math. Although, a few also involve different kinds of experience, such as synesthesia.

It’s not limited to injuries. There is also the question of psychedelics. Scott Alexander makes this point in an article in his blog Slate Star Codex:

“The third possibility is the one that really intrigues me. A 2011 study found that a single dose of psilocybin could permanently increase the personality dimension of Openness To Experience. I’m emphasizing that because personality is otherwise pretty stable after adulthood; nothing should be able to do this. But magic mushrooms apparently have this effect, and not subtly either; participants who had a mystical experience on psilocybin had Openness increase up to half a standard deviation compared to placebo, and the change was stable sixteen months later. This is really scary. I mean, I like Openness To Experience, but something that can produce large, permanent personality changes is so far beyond anything else we have in psychiatry that it’s kind of terrifying.”

Scott Alexander, “Why Were Early Psychedelicists So Weird?” Slate Star Codex. April 28, 2016.

Anyone that has been around people that have taken a lot of LSD know that they are different. Often, they are different in ways that make it more difficult to function in society, not easier. But, the opposite can also be true.

There was also a lot of discussion a few years ago about how people in Silicon Valley were microdosing LSD in an effort to boost their creativity. Clearly, in this case, psychedelics were being used to improve performance in a particular context and probably without full consideration of the effects beyond creativity.

There has also been research done in using electrical impulses to change mental states in people. The U.S. military, for example, is using electrical brain stimulation to enhance skills. Of course, there has been a dark side to this as well, as any discussion of Electroconvulsive Therapy will invariably bring up.

Meditation is also said to have effects on our mental states. A meta-analysis into meditation research by the medical community described it as follows:

“Results indicate that meditation leads to activation in brain areas involved in processing self-relevant information, self-regulation, focused problem-solving, adaptive behavior, and interoception. Results also show that meditation practice induces functional and structural brain modifications in expert meditators, especially in areas involved in self-referential processes such as self-awareness and self-regulation. These results demonstrate that a biological substrate underlies the positive pervasive effect of meditation practice and suggest that meditation techniques could be adopted in clinical populations and to prevent disease.”

M. Boccia, L. Piccardi, P. Guariglia. “The meditative mind: a comprehensive meta-analysis of MRI studies.” Biomed. Res. Int. 2015:419808. 10.1155/2015/419808

It seems like meditation is a good idea and has many positive aspects, but it also fundamentally changes the biology and the functioning of our brains. Should we be doing it?

You could probably make arguments that music, creating art, exercise and many other activities have dramatic and important effects on the mind and likely change it on a biological level. But, should we be striving to reorganize our minds to achieve some goal or mental state? And what techniques should we be using and why? This strikes me as a fundamental unanswered question about human life that warrants investigation.

Reference: Might be useful to consult Michael Pollan’s “How to Change Your Mind: What the New Science of Psychedelics Teaches Us About Consciousness, Dying, Addiction, Depression, and Transcendence” to get a sense of how psychedelics are currently being used.

Changing Reality Tunnels

Open Question: How does one cultivate the skill of evaluating our world view, assessing its strengths and weaknesses, and changing it when our situation changes?

“Every kind of ignorance in the world all results from not realizing that our perceptions are gambles. We believe what we see and then we believe our interpretation of it, we don’t even know we are making an interpretation most of the time. We think this is reality.”

—Robert Anton Wilson

“Our job is to remind us that there are more contexts than the one that we’re in — the one that we think is reality.”

—Alan Kay

Yesterday’s The Cost of the Club discussion starting me thinking about reality tunnels. Reality tunnels can shape not only how we view the world but also how we view ourselves, and vice versa.

If you identify with a political party, then how you view the world is shaped by this identification. It may be impossible to see the limitation of the field of view because you do not have a point of comparison.

Like depth perception, you have to have a slightly different frame of reference in order to see into three dimensions. Without a second frame, it can be difficult to judge certain qualities in our environment, such as distance.

And, we can extend this analogy. Add in space and time as proxies for our geography and our historical moment and we can try to adopt different frames of reference to look at our own and other times. This gives us increased flexibility in outlook, and perhaps, we can cultivate a sense of their strengths and limitations.

For example, in the current moment, we like to imagine that the mind is like a computer, subject to programming. Before computers, minds were compared to locomotives, houses, gardens, sponges and so forth. All of them provide some insight into how to think about our minds, but none of them are true. All of them have limitations.

Whatever else it is like, the mind is like a filter, taking in all the overwhelming information of our sense experience and trying to narrow it down to some desirable essence that helps us to live. This essence can change depending on our circumstances. The needs of civilians living in a town torn apart by civil war are different from the needs of military prisoners of war living in captivity. Princesses need a different way of interpreting the world than does the cook preparing her meals, even though they both ostensibly live in the same environment.

The ability to adapt to our environment and cultivate mental models that help us to survive in them is a great gift. But, it is also a great gift to be aware of their limitations and learn to be able to change them at will, when circumstances change.

How does one improve this skill? It’s a good question. I’m thinking that a good place to start might be with Prometheus Rising by Robert Anton Wilson. It seems to be part of a larger question of: how do create an environment for yourself and others that is conducive to human flourishing? The ideas of Extropianism and their principles, particularly the other books in their recommended books near the bottom of the page might be a useful place to start.

Frauchiger-Renner Paradox Clarifies Where Our Views of Reality Go Wrong

“The experiment, designed by Daniela Frauchiger and Renato Renner, of the Swiss Federal Institute of Technology Zurich, involves a set of assumptions that on the face of it seem entirely reasonable. But the experiment leads to contradictions, suggesting that at least one of the assumptions is wrong. The choice of which assumption to give up has implications for our understanding of the quantum world and points to the possibility that quantum mechanics is not a universal theory, and so cannot be applied to complex systems such as humans.”

—Anil Ananthaswamy, “Frauchiger-Renner Paradox Clarifies Where Our Views of Reality Go Wrong.” Quanta Magazine. December 3, 2018.

Probably the clearest explainer you’ll find. The assumptions are that: quantum theory is universal, quantum theory is consistent, and opposite facts cannot both be true. This thought experiment suggests that at least one is false, and depending on which one either leads to positions that quantum theory collapses into classical physics at scale, observer perspective changes results, or the many worlds hypothesis.

Open Questions – Gwern.net

  • “Why do humans, pets, and even lab animals of many species kept in controlled lab conditions on standardized diets appear to be increasingly obese over the 20th century? What could possibly explain all of them simultaneously becoming obese?”
  • “Why does writing in the morning (anecdotally so far) seem to be so effective for writers, even ones who are not morning persons? While programmers, which seems like a similar occupation, are invariably owls?”

—Gwern Branwen. “Open Questions.” Gwern.net. Accessed: November 9, 2018.

I like the idea of open questions. Added a category to this blog to start collecting them.