Unfortunately, the most straightforward way for an AI to prove that a self-modification will not corrupt its goals is blocked by Löb’s theorem.

Yudkowsky (2013) surveys some promising attacks on this “Löbian obstacle,” but much work remains to be done.

Ray gordon chat-4

There, he wrote that despite his reputation as a technological optimist, “I often end up spending most of my time [in debates] defending [Joy’s] position on the feasibility of these dangers.” Now, documentary filmmaker James Barrat has written an engaging new book about the risks inherent in what is sure to be the most transformative technology of them all: machine superintelligence.

Although The rumor went like this: a lone genius had engaged in a series of high-stakes bets in a scenario he called the AI-Box Experiment. Barrat goes on to explain the risks of advanced AI by tracking down and interviewing the issue’s leading thinkers, one at a time — including the man behind the AI-Box Experiment, Eliezer Yudkowsky, my associate at MIRI.

To some degree, we’ll probably always have some uncertainty about our values. For decades, we’ve had a rich framework for talking about uncertainty about doing what we want even as it rewrites its core algorithms (to become smarter, to better achieve its goals)?

Since it will likely do this many, many times, we’d like to minimize the chance of goal corruption during each (major) modification, and the strongest assurance we know of is mathematical proof.

I quickly found that, with the exception of Amazon's kits, which you can get if you subscribe to Amazon Fresh, you can't just buy one meal.

You subscribe for a weekly drop off of three meals for two or, as Purple Carrot puts it, "Serves two or one with leftovers." Each individual meal costs around , sometimes a couple bucks more, occasionally a shade less.

Almost all of the ingredients for all of the meals come in the kits, though they might rely on you to have items such as salt or cooking oil on hand.

Large recipe cards with helpful photos guide you through the recipes.

In the experiment, the genius role-played the part of the AI. I generally open new books and articles about AI risk with some trepidation.

An assortment of dot-com millionaires each took a turn as the Gatekeeper — an AI maker confronted with the dilemma of guarding and containing smarter-than-human AI. Usually, people who write about these issues for a popular audience show little familiarity with the scholarly literature on the subject.

Home cooks can all get stuck in our own ruts of stir-fry, pastas, or taco night, but by providing everything you need for a recipe, these kits can get you to cook something you've never tried before or teach you a new technique in a way that leaves cookbooks or website recipes in the dust.