Back in 2016, Medium published a sci-fi-esque question by a Canadian programmer grappling with emerging technological dilemmas. He later went on to publish a book, Proof of Stake, which posed the same question, and he’s also known for co-founding Ethereum. His name is Vitalik Buterin, and the full question reads as follows, “How can we regulate a very complex and very smart system with unpredictable emergent properties using a very simple and dumb system whose properties once created are inflexible?”1 To expand upon Buterin’s lengthy question, let’s travel back to a simpler time. In the 1980’s and earlier, programmers could be smarter than the computers they programmed. Sure, there were bugs and confusing outputs, but with enough analysis, these anomalies were often figure-outable or at least work-aroundable. Forty years later, we have computer system outputs that no human can understand. Not in a million years. Is the system doing what it’s supposed to be doing? Who knows? We can’t analyze it. We can’t break it down. It’s a complete mystery. The core of Buterin’s question asks how we can supervise a system like that.
To put it another way, how can we trust that a computer system is giving us the right answer? And how does it decide what’s right, anyway? We can’t really know what’s influencing its decisions. This problem exists in distributed systems that run cryptocurrency software, and also in artificial intelligence. In chess, a favorite game of Buterin’s, AI can make moves that grandmasters find baffling. They’re not based on any human strategy, but on a new, alien one. Cheaters are often identified by making moves that are characteristic of an AI rather than a human…no human would (or could) play in that way. What’s more, there’s no human that can beat the best chess AI. At least with chess, the game rules are rigid and simple enough for us to supervise. If an AI makes an illegal move, it would be spotted by humans instantly. Using AI and distributed systems in areas where the rules are unpredictable and complex is another matter; there’s no way to spot an error.
This is a new philosophical territory, one where humans are not the only super-intelligence on the block. Sci-fi movies often paint a bleak picture of worlds where computer systems have surpassed human intelligence. They do things that we can’t understand, and they can predict what we will do next. It would be smart of us to be on guard. Backpedaling isn’t really an option; there are few volunteers willing to turn back to a life without computers. It’s easy enough to shut down a personal computer, but it’s near impossible to avoid the systems that operate city infrastructure, commerce, and medical facilities. We’d best figure out good answers to questions like Vitalik’s, and figure them out fast.
Jared Leshin is an author, CMO at METADEV, and founder of hypersuade.co — a Web3-immersed creative marketing agency. Read his book, Advertising and the Nature of Reality, available on Amazon.
1https://medium.com/@VitalikButerin/why-cryptoeconomics-and-x-risk-researchers-should-listen-to-each-other-more-a2db72b3e86b
