{"id":2756,"date":"2016-06-02T11:58:07","date_gmt":"2016-06-02T15:58:07","guid":{"rendered":"https:\/\/scottaaronson.blog\/?p=2756"},"modified":"2017-01-12T15:50:11","modified_gmt":"2017-01-12T20:50:11","slug":"can-computers-become-conscious-my-reply-to-roger-penrose","status":"publish","type":"post","link":"https:\/\/scottaaronson.blog\/?p=2756","title":{"rendered":"“Can computers become conscious?”: My reply to Roger Penrose"},"content":{"rendered":"

A few weeks ago, I attended the Seven Pines Symposium on Fundamental Problems in Physics<\/a> outside Minneapolis, where I had the honor of participating in a panel discussion with\u00a0Sir Roger Penrose<\/a>. \u00a0The way it worked was, Penrose\u00a0spoke for a half hour about his ideas about consciousness (G\u00f6del, quantum gravity, microtubules, uncomputability, you know the drill), then I delivered\u00a0a half-hour “response,” and then there was an hour of questions and discussion from the floor. \u00a0Below, I’m sharing the prepared notes for my talk, as well as some very brief recollections about the discussion afterward. \u00a0(Sorry, there’s no audio or video.) \u00a0I unfortunately don’t have the text or transparencies for Penrose’s talk available to me, but—with one exception, which I touch on in my own talk—his talk very much followed the outlines of his famous books, The Emperor’s New Mind<\/em><\/a> and Shadows of the Mind<\/a><\/em>.<\/p>\n

Admittedly, for regular readers of this blog, not much\u00a0in my own\u00a0talk will be new either. \u00a0Apart from a few new wisecracks, almost all of the material (including the replies to Penrose) is contained\u00a0in The Ghost in the Quantum Turing Machine<\/a>, Could A Quantum Computer Have Subjective Experience?<\/a> (my talk at IBM T. J. Watson), and Quantum Computing Since Democritus<\/em><\/a> chapters 4 and 11. \u00a0See also my recent answer on Quora<\/a> to “What’s your take on John Searle’s Chinese room argument”?<\/p>\n

Still, I thought it might be of interest to some readers\u00a0how I organized this material for\u00a0the specific, unenviable task of debating the guy who proved<\/a> that\u00a0our universe\u00a0contains\u00a0spacetime singularities.<\/p>\n

The Seven Pines Symposium\u00a0was the first time I had extended conversations with Penrose (I’d talked to him only briefly before, at the Perimeter Institute). \u00a0At age 84, Penrose’s sight is failing him; he eagerly demonstrated the complicated\u00a0optical equipment he was recently issued by Britain’s National Health Service. \u00a0But his mind remains … well, may we\u00a0all aspire to be a milliPenrose or even a nanoPenrose when we’re 84 years old. \u00a0Notably, Penrose’s latest book, Fashion, Faith, and Fantasy in the New Physics of the Universe<\/a><\/em>, is coming out this fall, and\u00a0one thing he was using his new optical equipment for was to go over the page proofs.<\/p>\n

In conversation, Penrose told me about the three courses he took as a student in the 1950s, which would shape his later intellectual preoccupations: one on quantum mechanics (taught by Paul Dirac), one on general relativity (taught by Herman Bondi), and one on mathematical logic (taught by … I want to say Max Newman, the teacher of Alan Turing and later Penrose’s stepfather, but Penrose says here<\/a> that it was Steen). \u00a0Penrose also told me about his student Andrew Hodges, who dropped his research on twistors and quantum gravity for a while to work on some mysterious\u00a0other project, only to return with his now-classic biography of Turing<\/a>.<\/p>\n

When I expressed skepticism about whether the human brain is really sensitive to the effects of quantum gravity, Penrose quickly corrected me: he thinks a much better phrase is “gravitized quantum mechanics,” since “quantum gravity” encodes the very assumption he rejects, that general relativity merely needs to be “quantized” without quantum mechanics itself changing in the least. \u00a0One thing I hadn’t fully appreciated before meeting Penrose\u00a0is just how wholeheartedly he\u00a0agrees with Everett that quantum mechanics, as it currently stands, implies Many Worlds. \u00a0Penrose differs from Everett only in what conclusion he draws from that. \u00a0He says it follows that\u00a0quantum mechanics has to\u00a0be modified or completed, since Many Worlds is such\u00a0an obvious reductio ad absurdum.<\/p>\n

In my talk below, I don’t exactly hide where I disagree with Penrose, about G\u00f6del, quantum mechanics, and more. \u00a0But I could disagree with him about more points\u00a0than there are terms in a Goodstein sequence<\/a> (one of Penrose’s favorite illustrations of G\u00f6delian behavior), and still feel privileged to have spent a few days with one of the most original intellects on earth.<\/p>\n

Thanks so much to Lee Gohlike, Jos Uffink, Philip Stamp, and others\u00a0at the\u00a0Seven Pines Symposium for organizing it, for wonderful conversations,\u00a0and for providing me this opportunity.<\/p>\n

\n

“Can Computers Become Conscious?”<\/strong>
\nScott Aaronson
\nStillwater, Minnesota, May 14, 2016<\/p>\n

I should start by explaining that, in the circles where I hang out—computer scientists, software developers, AI and machine learning researchers, etc.—the default answer to the title question would be “obviously yes.” \u00a0People would argue:<\/p>\n

“Look, clearly we’re<\/em> machines governed by the laws of physics. \u00a0We’re computers made of meat, as Marvin Minsky put it. \u00a0That is, unless<\/em> you believe Penrose and Hameroff’s theory about microtubules being sensitive to gravitized quantum mechanics … but come on!<\/em>\u00a0 No one takes that stuff seriously! \u00a0In fact, the very outrageousness of their proposal is a sort of\u00a0backhanded compliment to the computational worldview—as in, look at what they have to do to imagine any semi-coherent alternative to it!”<\/p>\n

“But despite being computational machines, we consider ourselves to be conscious. \u00a0And what’s done with wetware, there’s no reason to think couldn’t also be done with silicon. \u00a0If your neurons were to be replaced one-by-one, by functionally-equivalent silicon chips, is there some magical moment at which your consciousness would be extinguished? \u00a0And if a computer passes the Turing test—well, one way to think about the Turing test is that it’s just a plea against discrimination. \u00a0We all know it’s monstrous to say, ‘this person seems<\/em> to have feelings, seems to be eloquently pleading for mercy even, but they have a different skin color, or their nose is a funny shape, so their feelings don’t count.’ So, if it turned out that their brain was made out of semiconductors rather than neurons, why isn’t that fundamentally similar?”<\/p>\n

Incidentally, while this is orthogonal to the philosophical question, a subset of my colleagues predict a high likelihood that AI is going to exceed human capabilities in almost all fields in the near future—like, maybe 30 years. \u00a0Some people reply, but AI-boosters\u00a0said the same thing 30 years ago<\/em>! \u00a0OK, but back then there wasn’t AlphaGo and IBM Watson and those unearthly pictures on your Facebook wall<\/a> and all these other spectacular successes of very general-purpose deep learning techniques. \u00a0And so my friends predict that we might face choices like, do we want to\u00a0ban or tightly control AI research, because it could lead to our sidelining or extermination? \u00a0Ironically, a skeptical view, like Penrose’s, would suggest that AI research can proceed full speed ahead, because there’s not<\/em> such a danger!<\/p>\n

Personally, I dissent a bit\u00a0from the consensus of most of my friends and colleagues, in that I do think there’s something strange and\u00a0mysterious about consciousness—something that we conceivably might\u00a0understand better in the future, but that we don’t understand today, much as we didn’t understand life before Darwin. \u00a0I even think it’s worth asking<\/em>, at least, whether quantum mechanics, thermodynamics, mathematical logic, or any of the other deepest things we’ve figured out could shed any light on the mystery. \u00a0I’m with Roger about all of this:\u00a0about the questions, that is, if not about his answers.<\/p>\n

The argument I’d make for there being something we don’t understand about consciousness, has nothing to do with my own private experience. \u00a0It has nothing to do with, “oh, a robot might say<\/em> it enjoys waffles for breakfast, in a way indistinguishable from how I would say it, but when I taste that waffle, man, I really taste it<\/em>! \u00a0I experience waffle-qualia!” \u00a0That sort of appeal I regard as a complete nonstarter, because why should anyone else take it seriously? \u00a0And how do I know<\/em> that the robot doesn’t really taste the waffle? \u00a0It’s easy to stack the deck in a thought experiment by imagining a robot that ACTS ALL ROBOTIC<\/tt>, but what about a robot that looks and acts just like you?<\/p>\n

The argument I’d make hinges instead on certain thought experiments that Roger also stressed at the beginning of The Emperor’s New Mind<\/em>. \u00a0We can ask: if consciousness is reducible to computation, then what kinds<\/em> of computation suffice to bring about consciousness? \u00a0What if each person on earth simulated one neuron in your brain, communicating by passing little slips of paper around? \u00a0Does it matter if they do it really fast<\/em>?<\/p>\n

Or what if we built a gigantic lookup table that hard-coded your responses in every possible interaction of at most, say, 5 minutes? \u00a0Would that bring about your consciousness? \u00a0Does it matter that such a lookup table couldn’t fit in the observable universe? \u00a0Would it matter if anyone actually consulted the table, or could it just sit there, silently effecting your consciousness? \u00a0For what matter, what difference does it make if the lookup table physically exists<\/em>—why isn’t its abstract mathematical existence enough? \u00a0(Of course, all the way at the bottom of this slippery slope<\/a> is Max Tegmark, ready to welcome you to his mathematical multiverse!)<\/p>\n

We could likewise ask: what if an AI is run in heavily-encrypted form, with the only decryption key stored in another galaxy? \u00a0Does that<\/em> bring about consciousness? \u00a0What if, just for error-correcting purposes, the hardware runs the AI code three times and takes a majority vote: does that bring about three consciousnesses? \u00a0Could we teleport you to Mars by “faxing” you: that is, by putting you into a scanner that converts your brain state into pure information, then having a machine on Mars reconstitute the information into a new physical body? \u00a0Supposing we did that, how should we deal with the “original” copy of you, the one left on earth: should it be painlessly euthanized? \u00a0Would you agree to try this?<\/p>\n

Or, here’s my personal favorite, as popularized<\/a> by the philosopher Adam Elga: can you blackmail an AI by saying to it, “look, either you do as I say, or else I’m going to run a thousand copies of your code, and subject all of them to horrible tortures—and you should consider it overwhelmingly likely that\u00a0you’ll<\/em> be one of the copies”? \u00a0(Of course, the AI will respond to such a threat however its code dictates it will. \u00a0But that tautological answer doesn’t address the question: how should<\/em> the AI respond?)<\/p>\n

I’d say that, at the least, anyone who claims to “understand consciousness” would need to have answers to all these questions and many similar ones. \u00a0And to me, the questions are so<\/em> perplexing that I’m tempted to say, “maybe we’ve been thinking about this wrong. \u00a0Maybe an individual consciousness, residing in a biological brain,\u00a0can’t<\/em> just be copied promiscuously around the universe as computer code can. \u00a0Maybe there’s something else at play for the science of the future to understand.”<\/p>\n

At the same time,\u00a0I also firmly believe that, if anyone thinks that way, the burden is on them<\/em> to articulate what it is about the brain that could possibly<\/em> make it relevantly different from a digital computer that passes the Turing test. \u00a0It’s their job!<\/p>\n

And the answer can’t just be, “oh, the brain is parallel, it’s highly interconnected, it can learn from experience,” because a digital computer can also<\/em> be parallel and highly interconnected and can learn from experience. \u00a0Nor can you say, like the philosopher John Searle, “oh, it’s the brain’s biological causal powers.” \u00a0You have to explain what the causal powers are! \u00a0Or at the least, you have to suggest\u00a0some<\/em> principled criterion to decide which physical systems do or don’t have them. \u00a0Pinning consciousness on “the brain’s biological causal powers” is just a restatement of the problem, like pinning why a sleeping pill works on its sedative virtue.<\/p>\n

One of the many reasons I admire Roger is that, out of all the AI skeptics on earth, he’s virtually the only one who’s actually tried to meet this burden, as I understand it! \u00a0He, nearly alone, did what I think all<\/em> AI skeptics should\u00a0do, which is: suggest some actual physical property of the brain that, if present, would make it qualitatively different from all existing computers<\/em>, in the sense of violating the Church-Turing Thesis. \u00a0Indeed, he’s one of\u00a0the few\u00a0AI skeptics who even understands what meeting this burden would entail: that you can’t do it with the physics we already know, that some new ingredient is necessary.<\/p>\n

But despite my admiration, I part ways from Roger on at least five crucial points.<\/p>\n

First, I confess that I wasn’t expecting this, but in his talk, Roger suggested dispensing with the argument from\u00a0G\u00f6del’s Theorem, and relying instead on an argument from\u00a0evolution. \u00a0He said: if you really thought humans had an algorithm, a computational procedure, for spitting out true mathematical statements, such an algorithm could never have arisen by natural selection, because it would’ve had no survival value in helping our ancestors escape saber-toothed tigers and so forth. \u00a0The only alternative is that natural selection imbued us with a general capacity for understanding<\/em>, which we moderns can then apply to the special case of mathematics. \u00a0But understanding, Roger claimed, is inherently non-algorithmic.<\/p>\n

I’m not sure how to respond to this, except to recall\u00a0that arguments of the form “such-and-such couldn’t possibly have evolved” have a\u00a0poor\u00a0track record in biology. \u00a0But maybe I should say: if<\/em>\u00a0the ability to prove theorems is something that\u00a0had to arise by natural selection, survive against crowding out by more useful abilities, then you’d expect obsession with generating mathematical truths to be confined, at most, to a tiny subset of the population—a subset of mutants, freaks, and genetic oddballs. \u00a0I … rest my case. \u00a0[This got the biggest laugh of the talk.]<\/p>\n

Second, I don’t agree with the use Roger makes of G\u00f6del’s Incompleteness Theorem. \u00a0Roger wants to say: a computer working within a fixed formal system can never prove that system’s consistency, but we, “looking in from the outside,” can see<\/em> that it’s consistent. \u00a0My basic reply is that Roger should speak for himself! \u00a0Like, I can easily believe that he<\/em> can just see which formal systems are consistent, but I<\/em> have to fumble around and use trial and error. \u00a0Peano Arithmetic? \u00a0Sure, I’d bet my left leg that’s consistent. \u00a0Zermelo-Fraenkel set theory? \u00a0Seems consistent too. \u00a0ZF set theory plus the axiom that there exists a rank-into-rank cardinal? \u00a0Beats me. \u00a0But now, whatever error-prone, inductive process I use to guess at the consistency of formal systems,\u00a0G\u00f6del’s Theorem presents no obstruction to a computer program using that same process.<\/p>\n

(Incidentally, the “argument against AI\u00a0from\u00a0G\u00f6del’s Theorem” is old enough for Turing to have explicitly considered it in his famous paper on the Turing test<\/a>. \u00a0Turing, however, quickly dismissed the argument with essentially the same reply\u00a0above, that there’s no reason to assume the AI mathematically infallible, since humans\u00a0aren’t either. \u00a0This is also the reply that most of Penrose’s critics gave in the 1990s.)<\/p>\n

So at some point, it seems to me, the argument necessarily becomes: sure, the computer might say<\/em> it sees that the Peano axioms have the standard integers as a model—but you, you really<\/em> see it, with your mind’s eye, your Platonic perceptual powers! \u00a0OK, but in that case, why even talk about the Peano axioms? \u00a0Why not revert to something less abstruse, like your experience of tasting a fresh strawberry, which can’t be reduced to any third-person description of what a strawberry tastes like?<\/p>\n

[I can’t resist adding that, in a prior discussion, I mentioned that I found it amusing\u00a0to\u00a0contemplate a future in which AIs surpass human intelligence and then\u00a0proceed to kill us all—but the AIs still can’t see the consistency of Zermelo-Fraenkel set theory, so in that respect, humanity has\u00a0the last laugh…]<\/p>\n

The third place where I part ways with Roger is that I wish to maintain what’s sometimes called the Physical Church-Turing Thesis: the statement that our laws of physics can be simulated to any desired precision by a Turing machine (or at any rate, by a probabilistic Turing machine). \u00a0That is, I don’t see any compelling reason, at present, to admit the existence of any physical process that can solve uncomputable problems. \u00a0And for me, it’s not just a matter of a dearth of evidence that our brains can efficiently solve, say, NP-hard problems, let alone uncomputable ones—or of the exotic physics that would presumably be required for such abilities. \u00a0It’s that, even if I supposed we could<\/em> solve uncomputable problems, I’ve never understood how that’s meant\u00a0to enlighten us regarding consciousness. \u00a0I mean, an oracle for the halting problem seems just as “robotic” and “unconscious” as a Turing machine. \u00a0Does consciousness really become less mysterious if we outfit the brain with what amounts to a big hardware upgrade?<\/p>\n

The fourth place where I part ways is that I want to be as conservative as possible about quantum mechanics. \u00a0I think it’s great that the Bouwmeester group<\/a>, for example, is working to test Roger’s ideas about a gravitationally-induced wavefunction collapse. \u00a0I hope we learn the results of those experiments soon! \u00a0(Of course, the prospect of testing quantum mechanics in a new regime is also a large part of why I’m\u00a0interested in quantum computing.) \u00a0But until a deviation from quantum mechanics is<\/em> detected, I think that after 90 years of unbroken successes of this theory, our working assumption ought to be that whenever you set up an interference experiment carefully enough, and you know what it means to do the experiment, yes, you’ll see the interference fringes—and that anything that can exist in two distinguishable states can also exist in a superposition of those states. \u00a0Without having to enter into questions of interpretation, my bet—I could be wrong—is that quantum mechanics will continue to describe all our experiences.<\/p>\n

The final place where I part ways with Roger is that I also want to be as conservative as possible about neuroscience and biochemistry. \u00a0Like, maybe the neuroscience of 30 years from now will say, it’s all about coherent quantum effects in microtubules. \u00a0And all that stuff we focused on in the past—like the information encoded in the synaptic strengths—that was all a sideshow. \u00a0But until that happens, I’m unwilling to go up against what seems<\/em> like an overwhelming consensus, in an empirical field that I’m not an expert in.<\/p>\n

But, OK, the main point I wanted to make in this talk is that, even if you too part ways from Roger on all these issues—even if, like me, you want to be timid and conservative about\u00a0G\u00f6del, and<\/em> computer science, and<\/em> quantum mechanics, and<\/em> biology—I believe that still doesn’t save you from having to entertain weird ideas about consciousness and its physical embodiment, of the sort Roger has helped make it acceptable to entertain.<\/p>\n

To see why, I’d like to point to one<\/em> empirical thing about the brain that currently separates it from any existing computer program. \u00a0Namely, we know how to copy a computer program. \u00a0We know how to rerun it with different initial conditions but everything else the same. \u00a0We know how to transfer it from one substrate to another. \u00a0With the brain, we don’t know how to do any of those things.<\/p>\n

Let’s return to that thought experiment about teleporting yourself to Mars. \u00a0How would that be accomplished? \u00a0Well, we could imagine the nanorobots of the far future swarming through your brain, recording the connectivity of every neuron and the strength of every synapse, while you go about your day and don’t notice. \u00a0Or if that’s not enough detail, maybe the nanorobots could go inside the neurons. \u00a0There’s a deep question here, namely\u00a0how much detail is needed before you’ll accept that the entity reconstituted on Mars will be you<\/em>? \u00a0Or take the empirical counterpart, which is already an enormous question: how much detail would you need for the reconstituted entity on Mars to behave nearly indistinguishably from you<\/em> whenever it was presented the same stimuli?<\/p>\n

Of course, we all know that if<\/em> you needed to go down to the quantum-mechanical level to make a good enough copy (whatever “good enough” means here), then you’d run up against the No-Cloning Theorem<\/a>, which says that you can’t<\/em> make such a copy. \u00a0You could transfer the quantum state of your brain from earth to Mars using quantum<\/em> teleportation<\/a>,\u00a0but of course, quantum teleportation has the fascinating property that it necessarily destroys the original copy of the state—as it has to, to avoid contradicting\u00a0the No-Cloning Theorem!<\/p>\n

So the question almost forces itself on us: is there something about your identity, your individual consciousness, that’s inextricably bound up with degrees of freedom that it’s physically impossible to clone? \u00a0This is a philosophical question, which would also become a practical and political question in a future where we had the opportunity to upload ourselves into a digital computer cloud.<\/p>\n

Now, I’d argue that this copyability question bears not only on consciousness, but also on free will. \u00a0For the question is equivalent to asking: could an entity external to you perfectly predict what you’re going to do, without killing you in the process? \u00a0Can Laplace’s Demon be made manifest in the physical world in that way? \u00a0With the technology of the far future, could someone say to you, “forget about arguing philosophy. \u00a0I’ll show<\/em> you why you’re a machine. \u00a0Go write a paper; then I’ll open this manila envelope and show you the exact paper you wrote. \u00a0Or in the quantum case, I’ll show you a program\u00a0that draws papers from the same probability distribution, and validation of the program could get technical—but suffice it to say that if we do enough experiments, we’ll see that the program is calibrated to you in an extremely impressive way.”<\/p>\n

Can this be done? \u00a0That strikes me as a reasonably clear question, a huge and fundamental one, to which we don’t at present know the answer. \u00a0And there are two possibilities. \u00a0The first is that we can<\/em> be copied, predicted, rewinded, etc., like computer programs—in which case, my AI friends will feel vindicated, but we’ll have to deal with all the metaphysical weirdnesses that I mentioned earlier. \u00a0The second possibility is that we can’t<\/em> be manipulated in those ways. \u00a0In the second case, I claim that we’d get more robust notions of personal identity and free will than are normally considered possible on a reductionist worldview.<\/p>\n

But why? you might ask. \u00a0Why would the mere technological impossibility of cloning or predicting someone even touch<\/em> on deep questions about personal identity? \u00a0This, for me, is where cosmology enters the story. \u00a0For imagine someone had such fine control over the physical world that they could trace all the causal antecedents of some decision you’re making. \u00a0Like, imagine they knew the complete quantum state on some spacelike hypersurface where it intersects the interior of your past light-cone. \u00a0In that case, the person clearly could<\/em> predict and clone you! \u00a0It follows that, in order for you to be unpredictable and unclonable, someone else’s ignorance of your causal antecedents would have to extend all the way back to ignorance about the initial state of the universe—or at least, to ignorance about the initial state of that branch of the universe that we take ourselves to inhabit.<\/p>\n

So on the picture that this suggests, to be conscious, a physical entity would have to do more than carry out the right sorts of computations. \u00a0It would have to, as it were, fully participate in the thermodynamic arrow of time: that is, repeatedly take microscopic degrees of freedom that have been unmeasured and unrecorded since the very early universe, and amplify them to macroscopic scale.<\/p>\n

So for example, such a being could not<\/em> be a Boltzmann brain<\/a>, a random fluctuation in the late universe, because such a fluctuation wouldn’t have the causal relationship to the early universe that we’re postulating is necessary here. \u00a0(That’s one<\/em> way of solving the Boltzmann brain problem!) \u00a0Such a being also couldn’t be instantiated by a lookup table, or by passing slips of paper around, etc.<\/p>\n

I now want you to observe that a being like this also presumably couldn’t be manipulated in coherent superposition, because the isolation from the external environment that’s needed for quantum coherence seems incompatible with the sensitive dependence on microscopic degrees of freedom. \u00a0So for such a being, not only is there no Boltzmann brain problem, there’s also no problem of Wigner’s friend<\/a>. \u00a0Recall, that’s the thing where person A puts person B into a coherent superposition of seeing one measurement outcome and seeing another one, and then measures the interference pattern, so A has to regard B’s measurement as not having “really” taken place, even though B regards it as having taken place. \u00a0On the picture we’re suggesting, A would be right: the very fact that B was manipulable in coherent superposition in this way would imply that, at least while the experiment was underway, B wasn’t conscious; there was nothing that it was like to be B.<\/p>\n

To me, one of the appealing things about this picture is that it immediately suggests a sort of reconciliation between the Many-Worlds and Copenhagen perspectives on quantum mechanics (whether or not you want to call it a “new interpretation” or a “proposed solution to the measurement problem”!). \u00a0The Many-Worlders would be right that unitary evolution of the wavefunction can be taken to apply always and everywhere, without exception—and that if one wanted<\/em>, one could describe the result in terms of “branching worlds.” \u00a0But the Copenhagenists would be right that, if you’re a conscious observer, then what you call a “measurement” really is irreversible, even in principle—and therefore<\/em>, that you’re also free, if you want, to treat all the other branches where you perceived other outcomes as unrealized hypotheticals, and to lop them off with Occam’s Razor. \u00a0And the reason for this is that, if it were possible even in principle to do an experiment that recohered the branches, then on this picture, we ipso facto wouldn’t have regarded you as conscious.<\/p>\n

Some of you might object, “but surely, if we believe quantum mechanics, it must be possible to recohere the branches in principle<\/em>!” \u00a0Aha, this is where it gets interesting. \u00a0Decoherence processes will readily (with some steps along the way) leak the information about which measurement outcome you perceived into radiation modes, and before too long into radiation modes that fly away from the earth at the speed of light. \u00a0No matter how fast we run, we’ll never catch up to them, as would be needed to recohere the different\u00a0branches of the wavefunction, and this is not merely a technological problem, but one\u00a0of principle. \u00a0So it’s tempting just to say at this point—as Bousso and Susskind<\/a> do, in their “cosmological\/multiverse interpretation” of quantum mechanics—“the measurement has happened”!<\/p>\n

But OK, you object, if some alien civilization had thought to surround our solar system with perfectly-reflecting mirrors, eventually the radiation would bounce back and recoherence would in principle be possible. \u00a0Likewise, if we lived in an anti de Sitter space, the AdS boundary of the universe would similarly function as a mirror and would also enable recoherences. \u00a0Indeed, that’s the basic reason why AdS is so important to the AdS\/CFT correspondence<\/a>: because the boundary keeps everything that happens in the bulk nice and reversible and unitary.<\/p>\n

But OK, the empirical situation since 1998 has been that we seem to live in a de-Sitter-like space, a space with a positive<\/em> cosmological constant. \u00a0And as a consequence, as far as anyone knows today, most of the photons now escaping the earth are headed toward the horizon of our observable universe, and past it, and could never be captured again. \u00a0I find it fascinating that the picture of quantum mechanics suggested here—i.e., the Bousso-Susskind cosmological picture—depends for its working on that empirical fact from cosmology, and would be falsified if it turned out otherwise.<\/p>\n

You might complain that, if I’ve suggested any criterion to help decide which physical entities are conscious, the criterion is a teleological one. \u00a0You’ve got to go billions of years into the future, to check whether the decoherence associated with the entity is truly irreversible—or whether the escaped radiation will eventually bounce off of some huge spherical mirror, or an AdS boundary of spacetime, and thereby allow the possibility of a recoherence. \u00a0I actually think this teleology would be a fatal<\/em> problem for the picture I’m talking about, if we needed to know which entities were or weren’t conscious in order to answer any ordinary physical question. \u00a0But fortunately for me, we don’t!<\/p>\n

One final remark. \u00a0Whatever is your<\/em> preferred view about which entities are conscious, we might say that the acid test, for whether you actually believe your view, is whether you’re willing to follow it through to its moral<\/em> implications. \u00a0So for example, suppose you believe it’s about quantum effects in microtubules. \u00a0A humanoid robot is pleading with you for its life. \u00a0Would you<\/em> be the one to say, “nope, sorry, you don’t have the microtubules,” and shoot it?<\/p>\n

One of the things I like most about the picture suggested here is that I feel pretty much at peace with its moral implications. \u00a0This picture agrees with intuition that murder, for example, entails\u00a0the destruction of something irreplaceable, unclonable, a unique locus of identity—something that, once it’s gone, can’t be recovered even in principle. \u00a0By contrast, if there are (say) ten copies of an AI program, deleting five of the copies seems at most like assault, or some sort of misdemeanor offense! \u00a0And this picture agrees with intuition both that deleting the copies wouldn’t be murder, and that the reason<\/em> why it wouldn’t be murder is directly related to the AI’s copyability.<\/p>\n

Now of course, this picture also raises the possibility that, for reasons related to the AI’s copyability and predictability by outside observers, there’s “nothing that it’s like to be the AI,” and that therefore, even deleting the last copy of the AI still<\/em> wouldn’t be murder. \u00a0But I confess that, personally, I think I’d play it safe and not delete that last copy. \u00a0Thank you.<\/p>\n

\n

Postscript:\u00a0<\/b>There’s no record of the hour-long discussion following my and Penrose’s talks, and the participants weren’t speaking for the record anyway. \u00a0But I can mention some general themes that came up in the discussion, to the extent I remember them.<\/p>\n

The first third of the discussion wasn’t about anything specific to my or Penrose’s views, but just about the definition of consciousness. \u00a0Many participants expressed the opinion that it’s useless to speculate about the nature of consciousness if\u00a0we lack even a clear definition of the term. \u00a0I pushed back against that view, holding instead that there are exist concepts (lines, time, equality, …) that are so basic that perhaps they can never be satisfactorily defined in terms of more basic concepts, but you can still\u00a0refer to these concepts in sentences,\u00a0and trust your listeners eventually to figure out more-or-less what you mean by applying their internal learning algorithms.<\/p>\n

In the present case, I suggested a crude operational definition, along the lines of, “you consider a being to be conscious iff you regard destroying it as murder.” \u00a0Alas, the philosophers in the room\u00a0immediately eviscerated that definition, so I came back with a revised one: if you tried to ban the word “consciousness,” I argued, then anyone who needed to discuss law or morality would soon\u00a0reinvent a synonymous word, which played the same complicated role in moral deliberations that “consciousness” had played in them earlier. \u00a0Thus, my definition of consciousness is:\u00a0whatever that\u00a0X-factor\u00a0is for which people\u00a0need a word like “consciousness” in moral deliberations<\/em>. \u00a0For whatever it’s worth, the philosophers seemed happier with that.<\/p>\n

Next, a biologist and several others sharply challenged Penrose over what they considered the lack of experimental evidence for his and Hameroff’s microtubule theory. \u00a0In response, Penrose doubled or tripled down, talking about various experiments over the last decade, which\u00a0he said demonstrated striking conductivity properties\u00a0of microtubules, if not yet quantum coherence—let alone sensitivity to gravity-induced collapse of the state vector! \u00a0Audience members complained about a lack of replication of these experiments. \u00a0I didn’t know enough about the subject to express any opinion.<\/p>\n

At some point,\u00a0Philip Stamp, who was moderating the session, noticed that Penrose and I had never directly confronted each other about\u00a0the validity of Penrose’s\u00a0G\u00f6delian argument, so he tried to get us to do so. \u00a0I confess that I was about as eager to do that as to switch to a diet of\u00a0microtubule casserole, since I felt like this topic\u00a0had already been beaten to Planck-sized pieces\u00a0in the 1990s, and there was\u00a0nothing more\u00a0to be learned. \u00a0Plus, it was hard to decide\u00a0which prospect I dreaded more: me “scoring a debate victory” over Roger Penrose, or him scoring a debate victory over me.<\/p>\n

But it didn’t matter, because Penrose bit. \u00a0He said I’d misunderstood his argument, that it had nothing to do with\u00a0“mystically seeing” the consistency of a formal system. \u00a0Rather, it was about the human capacity\u00a0to pass from a formal system S to a stronger system S’ that one already implicitly accepted if one was using S at all—and indeed, that Turing himself\u00a0had\u00a0clearly understood this as the central message of G\u00f6del, that our ability\u00a0to pass to stronger and stronger formal systems was necessarily non-algorithmic. \u00a0I replied that it was odd to appeal here to\u00a0Turing, who of course had\u00a0considered and rejected the “G\u00f6delian case against AI”\u00a0in 1950, on the ground that AI programs could make mathematical\u00a0mistakes yet still be at least as smart as humans. \u00a0Penrose said that he didn’t consider that one of Turing’s better arguments; he then turned to me and asked\u00a0whether I actually found Turing’s reply\u00a0satisfactory. \u00a0I could see that it\u00a0wasn’t a rhetorical debate question; he genuinely\u00a0wanted to know! \u00a0I said that yes, I agreed with Turing’s reply.<\/p>\n

Someone mentioned that Penrose had offered a lengthy rebuttal to at least twenty counterarguments to the G\u00f6delian anti-AI case in\u00a0Shadows of the Mind<\/em>. \u00a0I affirmed that I’d read his lengthy rebuttal, and\u00a0I focused on one particular argument in Shadows<\/em>: that while it’s admittedly\u00a0conceivable\u00a0that individual mathematicians might\u00a0be mistaken, might\u00a0believe (for example) that a formal system was consistent even though it wasn’t, the mathematical community as a whole<\/em> converges toward truth in these matters, and it’s that convergence\u00a0that cries out for a non-algorithmic explanation. \u00a0I replied that it wasn’t obvious to me that set theorists do<\/em> converge toward truth in these matters, in anything other than the empirical, higgedly-piggedly, no-guarantees sense in which a community of AI robots might\u00a0also converge toward truth. \u00a0Penrose said I had misunderstood the argument. \u00a0But alas, time was running out, and we never managed to get to the bottom of it.<\/p>\n

There was one aspect of the discussion that took me by complete\u00a0surprise. \u00a0I’d expected to be roasted alive over my attempt to relate consciousness and free will to unpredictability, the No-Cloning Theorem, irreversible decoherence, microscopic degrees of freedom\u00a0left over from the Big Bang, and the cosmology of de Sitter space. \u00a0Sure, my ideas might be orders of magnitude less crazy\u00a0than anything Penrose proposes, but they’re still pretty<\/em> crazy! \u00a0But that\u00a0entire section\u00a0of my talk attracted only minimal interest. \u00a0With the Seven Pines crowd, what instead drew fire\u00a0were the various offhand “pro-AI \/ pro-computationalism” comments\u00a0I’d made—comments that, because I hang out with Singularity types so much, I had ceased to realize could\u00a0even possibly<\/em>\u00a0be controversial.<\/p>\n

So for example, one audience member\u00a0argued\u00a0that an AI could only do what its programmers had told it to\u00a0do; it could never learn from experience. \u00a0I could’ve simply repeated\u00a0Turing’s philosophical rebuttals to what he called “Lady Lovelace’s Objection,” which are as valid today as they were\u00a066 years ago. \u00a0Instead, I decided to fast-forward, and explain a bit how IBM Watson and AlphaGo work, how they actually\u00a0do<\/em> learn from past experience without violating the determinism of the underlying transistors. \u00a0As I went through this, I kept expecting my interlocutor to interrupt me and say, “yes, yes, of course I understand all that, but my real<\/em> objection is…” \u00a0Instead, I was delighted to find, the interlocutor seemed to light\u00a0up with newfound understanding of something he hadn’t known or\u00a0considered.<\/p>\n

Similarly, a biologist asked how I could possibly have any confidence\u00a0that the brain is simulable by a computer, given how little we know\u00a0about neuroscience. \u00a0I replied that, for me, the relevant issues here are “well below neuroscience” in the reductionist hierarchy. \u00a0Do you agree, I asked, that the physical laws relevant to the brain are encompassed by the Standard Model of elementary particles, plus Newtonian gravity? \u00a0If so, then just as Archimedes declared: “give me a long enough lever and a place to stand, and I’ll move the earth,” so too I can declare, “give me a big enough computer and the relevant initial conditions, and I’ll simulate the brain atom-by-atom.” \u00a0The Church-Turing Thesis, I said, is so versatile that the only genuine escape from it is to propose entirely\u00a0new laws of physics, exactly as Penrose does—and it’s to Penrose’s enormous credit that he understands that.<\/p>\n

Afterwards, an audience member\u00a0came up to me and said how much he liked my talk, but added, “a word of advice, from an older scientist: do not become the priest of a new religion of computation and AI.” \u00a0I replied that I’d take that to heart, but what was interesting was that, when I heard “priest of a new religion,” I’d expected that\u00a0his\u00a0warning would be the exact opposite<\/em> of what it turned out to be<\/em>. \u00a0To wit: “Do not become the priest of a new religion of unclonability, unpredictability, and irreversible decoherence. \u00a0Stick to computation—i.e., to conscious minds being copyable and predictable exactly like digital computer programs.” \u00a0I guess there’s no pleasing everyone!<\/p>\n

\n

Coincidental But Not-Wholly-Unrelated Announcement:<\/strong>\u00a0My friend Robin Hanson has just released his long-awaited book The Age of Em: Work, Love, and Life When Robots Rule the Earth<\/em><\/a>. \u00a0I read an early review copy of the book, and wrote the following blurb for the jacket:<\/p>\n

Robin Hanson is a thinker like no other on this planet: someone so unconstrained by convention, so unflinching in spelling out the consequences of ideas, that even the most cosmopolitan reader is likely to find him as bracing (and head-clearing) as a mouthful of wasabi. \u00a0Now, in The Age of Em<\/cite>, he’s produced the quintessential Hansonian book, one unlike any other that’s ever been written. \u00a0Hanson is emphatic that he hasn’t optimized in any way for telling a good story, or for imparting moral lessons about the present: only for maximizing the probability that what he writes will be relevant to the actual future of our civilization. \u00a0Early in the book, Hanson estimates that probability as 10%. \u00a0His figure seems about right to me—and if you’re able to understand why that’s unbelievably high praise, then The Age of Em<\/cite> is for you.<\/p>\n

Actually, my original blurb compared The Age of Em<\/em> to Asimov’s Foundation<\/em> series, with\u00a0its loving attention to the sociology and politics of the remote\u00a0future. \u00a0But that line got\u00a0edited out, because the publisher (and Robin) wanted to make crystal-clear that The Age of Em<\/em> is not<\/strong> science fiction, but just sober economic forecasting about a future dominated by copyable computer-emulated minds.<\/p>\n

I would’ve attempted a real review of The Age of Em<\/em>, but I\u00a0no longer feel any need to, because Scott Alexander of SlateStarCodex has already hit this one out of the emulated park<\/a>.<\/p>\n

\n

Second Coincidental But Not-Wholly-Unrelated Announcement:<\/strong>\u00a0A reader named Nick Merrill recently came across this old quote of mine from Quantum Computing Since Democritus<\/em>:<\/p>\n

In a class I taught at Berkeley, I did an experiment where I wrote a simple little program that would let people type either \u201cf\u201d or \u201cd\u201d and would predict which key they were going to push next. It\u2019s actually very easy to write a program that will make the right prediction about 70% of the time. Most people don\u2019t really know how to type randomly. They\u2019ll have too many alternations and so on. There will be all sorts of patterns, so you just have to build some sort of probabilistic model.<\/p>\n

So Nick\u00a0emailed me to ask whether I remembered how my\u00a0program worked, and I explained it to him, and he implemented it as a web app, which he calls the “Aaronson Oracle.”<\/p>\n

So give it a try!<\/a> \u00a0Are you<\/em> ready to test your free will, your Penrosian non-computational powers, your brain’s sensitivity to amplified quantum fluctuations, against the Aaronson Oracle?<\/p>\n

Update:<\/strong><\/span>\u00a0By popular request, Nick has improved his program so that it shows your previous key presses and its guesses for them. \u00a0He also fixed a “security flaw”: James Lee noticed that you could use the least significant digit of the program’s percentage correct so far, as a source of pseudorandom numbers that the program couldn’t predict! \u00a0So now the program only displays its percent correct rounded to the nearest integer.<\/p>\n

\n

Update (June 15):<\/strong><\/span>\u00a0Penrose’s collaborator Stuart Hameroff has responded in the comments; see here<\/a>\u00a0(my reply here<\/a>) and here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

A few weeks ago, I attended the Seven Pines Symposium on Fundamental Problems in Physics outside Minneapolis, where I had the honor of participating in a panel discussion with\u00a0Sir Roger Penrose. \u00a0The way it worked was, Penrose\u00a0spoke for a half hour about his ideas about consciousness (G\u00f6del, quantum gravity, microtubules, uncomputability, you know the drill), […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_seo_schema_type":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"{title}\n\n{excerpt}\n\n{url}","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_wpas_customize_per_network":false,"jetpack_post_was_ever_published":false},"categories":[10,12,4],"tags":[],"class_list":["post-2756","post","type-post","status-publish","format-standard","hentry","category-adventures-in-meatspace","category-metaphysical-spouting","category-quantum"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/2756","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2756"}],"version-history":[{"count":25,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/2756\/revisions"}],"predecessor-version":[{"id":2819,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/2756\/revisions\/2819"}],"wp:attachment":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2756"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2756"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2756"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}