Shtetl-Optimized

Update (June 6): I wish to clarify that I did not write any of the dialogue for the “Scott Aaronson” character who refutes Michio Kaku’s quantum computing hype in this YouTube video, which uses an AI recreation of my voice. The writer appears to be physics/math blogger and podcaster Hassaan Saleem; see his website here. Luckily, the character and I do share many common views; I’m sure we’d hit it off if we met.

---

When I was a teenager, I enjoyed reading Hyperspace, an early popularization of string theory by the theoretical physicist Michio Kaku. I’m sure I’d have plenty of criticisms if I reread it today, but at the time, I liked it a lot. In the decades since, Kaku has widened his ambit to, well, pretty much everything, regularly churning out popular books with subtitles like “How Science Will Revolutionize the 21st Century” and “How Science Will Shape Human Destiny and Our Daily Lives.” He’s also appeared on countless TV specials, in many cases to argue that UFOs likely contain extraterrestrial visitors.

Now Kaku has a new bestseller about quantum computing, creatively entitled Quantum Supremacy. He even appeared on Joe Rogan a couple weeks ago to promote the book, surely reaching an orders-of-magnitude larger audience than I have in two decades of trying to explain quantum computing to non-experts. (Incidentally, to those who’ve asked why Joe Rogan hasn’t invited me on his show to explain quantum computing: I guess you now have an answer of sorts!)

In the spirit, perhaps, of the TikTokkers who eat live cockroaches or whatever to satisfy their viewers, I decided to oblige loyal Shtetl-Optimized fans by buying Quantum Supremacy and reading it. So I can now state with confidence: beating out a crowded field, this is the worst book about quantum computing, for some definition of the word “about,” that I’ve ever encountered.

Admittedly, it’s not obvious why I’m reviewing the book here at all. Among people who’ve heard of this blog, I expect that approximately zero would be tempted to buy Kaku’s book, at least if they flipped through a few random pages and saw the … level of care that went into them. Conversely, the book’s target readers have probably never visited a blog like this one and never will. So what’s the use of this post?

Well, as the accidental #1 quantum computing blogger on the planet, I feel a sort of grim obligation here. Who knows, maybe this post will show up in the first page of Google results for Kaku’s book, and it will manage to rescue two or three people from the kindergarten of lies.

---

Where to begin? Should we just go through the first chapter with a red pen? OK then: on the very first page, Kaku writes,

Google revealed that their Sycamore quantum computer could solve a mathematical problem in 200 seconds that would take 10,000 years on the world’s fastest supercomputer.

No, the “10,000 years” estimate was quickly falsified, as anyone following the subject knows. I’d be the first to stress that the situation is complicated; compared to the best currently-known classical algorithms, some quantum advantage remains for the Random Circuit Sampling task, depending on how you measure it. But to repeat the “10,000 years” figure at this point, with no qualifications, is actively misleading.

Turning to the second page:

[Quantum computers] are a new type of computer that can tackle problems that digital computers can never solve, even with an infinite amount of time. For example, digital computers can never accurately calculate how atoms combine to create crucial chemical reactions, especially those that make life possible. Digital computers can only compute on digital tape, consisting of a series of 0s and 1s, which are too crude to describe the delicate waves of electrons dancing deep inside a molecule. For example, when tediously computing the paths taken by a mouse in a maze, a digital computer has to painfully analyze each possible path, one after the other. A quantum computer, however, simultaneously analyzes all possible paths at the same time, with lightning speed.

OK, so here Kaku has already perpetuated two of the most basic, forehead-banging errors about what quantum computers can do. In truth, anything that a QC can calculate, a classical computer can calculate as well, given exponentially more time: for example, by representing the entire wavefunction, all 2ⁿ amplitudes, to whatever accuracy is needed. That’s why it was understood from the very beginning that quantum computers can’t change what’s computable, but only how efficiently things can be computed.

And then there’s the Misconception of Misconceptions, about how a QC “analyzes all possible paths at the same time”—with no recognition anywhere of the central difficulty, the thing that makes a QC enormously weaker than an exponentially parallel classical computer, but is also the new and interesting part, namely that you only get to see a single, random outcome when you measure, with its probability given by the Born rule. That’s the error so common that I warn against it right below the title of my blog.

[Q]uantum computers are so powerful that, in principle, they could break all known cybercodes.

Nope, that’s strongly believed to be false, just like the analogous statement for classical computers. Despite its obvious relevance for business and policy types, the entire field of post-quantum cryptography—including the lattice-based public-key cryptosystems that have by now survived 20+ years of efforts to find a quantum algorithm to break them—receives just a single vague mention, on pages 84-85. The possibility of cryptography surviving quantum computers is quickly dismissed because “these new trapdoor functions are not easy to implement.” (But they have been implemented.)

---

There’s no attempt, anywhere in this book, to explain how any quantum algorithm actually works, let alone is there a word anywhere about the limitations of quantum algorithms. And yet there’s still enough said to be wrong. On page 84, shortly after confusing the concept of a one-way function with that of a trapdoor function, Kaku writes:

Let N represent the number we wish to factorize. For an ordinary digital computer, the amount of time it takes to factorize a number grows exponentially, like t ~ e^N, times some unimportant factors.

This is a double howler: first, trial division takes only ~√N time; Kaku has confused N itself with its number of digits, ~log₂N. Second, he seems unaware that much better classical factoring algorithms, like the Number Field Sieve, have been known for decades, even though those algorithms play a central role in codebreaking and in any discussion of where the quantum/classical crossover might happen.

---

Honestly, though, the errors aren’t the worst of it. The majority of the book is not even worth hunting for errors in, because fundamentally, it’s filler.

First there’s page after page breathlessly quoting prestigious-sounding people and organizations—Google’s Sundar Pichai, various government agencies, some report by Deloitte—about just how revolutionary they think quantum computing will be. Then there are capsule hagiographies of Babbage and Lovelace, Gödel and Turing, Planck and Einstein, Feynman and Everett.

And then the bulk of the book is actually about stuff with no direct relation to quantum computing at all—the origin of life, climate change, energy generation, cancer, curing aging, etc.—except with ungrounded speculations tacked onto the end of each chapter about how quantum computers will someday revolutionize all of this. Personally, I’d say that

1. Quantum simulation speeding up progress in biochemistry, high-temperature superconductivity, and the like is at least plausible—though very far from guaranteed, since one has to beat the cleverest classical approaches that can be designed for the same problems (a point that Kaku nowhere grapples with).
2. The stuff involving optimization, machine learning, and the like is almost entirely wishful thinking.
3. Not once in the book has Kaku even mentioned the intellectual tools (e.g., looking at actual quantum algorithms like Grover’s algorithm or phase estimation, and their performance on various tasks) that would be needed to distinguish 1 from 2.

---

In his acknowledgments section, Kaku simply lists a bunch of famous scientists he’s met in his life—Feynman, Witten, Hawking, Penrose, Brian Greene, Lisa Randall, Neil deGrasse Tyson. Not a single living quantum computing researcher is acknowledged, not one.

Recently, I’d been cautiously optimistic that, after decades of overblown headlines about “trying all answers in parallel,” “cracking all known codes,” etc., the standard for quantum computing popularization was slowly creeping upward. Maybe I was just bowled over by this recent YouTube video (“How Quantum Computers Break the Internet… Starting Now”), which despite its clickbait title and its slick presentation, miraculously gets essentially everything right, shaming the hypesters by demonstrating just how much better it’s possible to do.

Kaku’s slapdash “book,” and the publicity campaign around it, represents a noxious step backwards. The wonder of it, to me, is Kaku holds a PhD in theoretical physics. And yet the average English major who’s written a “what’s the deal with quantum computing?” article for some obscure link aggregator site has done a more careful and honest job than Kaku has. That’s setting the bar about a millimeter off the floor. I think the difference is, at least the English major knows that they’re supposed to call an expert or two, when writing about an enormously complicated subject of which they’re completely ignorant.

---

Update: I’ve now been immersed in the AI safety field for one year, let I wouldn’t consider myself nearly ready to write a book on the subject. My knowledge of related parts of CS, my year studying AI in grad school, and my having created the subject of computational learning theory of quantum states would all be relevant but totally insufficient. And AI safety, for all its importance, has less than quantum computing does in the way of difficult-to-understand concepts and results that basically everyone in the field agrees about. And if I did someday write such a book, I’d be pretty terrified of getting stuff wrong, and would have multiple expert colleagues read drafts.

In case this wasn’t clear enough from my post, Kaku appears to have had zero prior engagement with quantum computing, and also to have consulted zero relevant experts who could’ve fixed his misconceptions.

[Like everything else on this blog—but perhaps even more so—this post represents my personal views, not those of UT Austin or OpenAI]

Since 2015, depressed, isolated, romantically unsuccessful nerdy young guys have regularly been emailing me, asking me for sympathy, support, or even dating advice. This past summer, a particularly dedicated such guy even trolled my comment section—plausibly impersonating real people, and causing both them and me enormous distress—because I wasn’t spending more time on “incel” issues. (I’m happy to report that, with my encouragement, this former troll is now working to turn his life around.) Many others have written to share their tales of woe.

From one perspective, that they’d come to me for advice is insane. Like … dating advice from … me? Having any dating life at all was by far the hardest problem I ever needed to solve; as a 20-year-old, I considered myself far likelier to prove P≠NP or explain the origin of consciousness or the Born rule. Having solved the problem for myself only by some miracle, how could I possibly help others?

But from a different perspective, it makes sense. How many besides me have even acknowledged that the central problem of these guys’ lives is a problem? While I have to pinch myself to remember, these guys look at me and see … unlikely success. Somehow, I successfully appealed the world’s verdict that I was a freakish extraterrestrial: one who might look human and seem friendly enough to those friendly to it, and who no doubt has some skill in narrow technical domains like quantum computing, and who could perhaps be suffered to prove theorems and tell jokes, but who could certainly, certainly never interbreed with human women.

And yet I dated. I had various girlfriends, who barely suspected that I was an extraterrestrial. The last of them, Dana, became my fiancée and then my wife. And now we have two beautiful kids together.

If I did all this, then there’d seem to be hope for the desperate guys who email me. And if I’m a cause of their hope, then I feel some moral responsibility to help if I can.

But I’ve been stuck for years on exactly what advice to give. Some of it (“go on a dating site! ask women questions about their lives!”) is patronizingly obvious. Some of it (fitness? fashion? body language?) I’m ludicrously, world-historically unqualified to offer. Much of it is simply extremely hard to discuss openly. Infamously, just for asking for empathy for the problem, and for trying to explain its nature, I received a level of online vilification that one normally associates with serial pedophiles and mass shooters.

For eight years, then, I’ve been turning the problem over in my head, revisiting the same inadequate answers from before. And then I had an epiphany.

---

There are now, on earth, entities that can talk to anyone about virtually anything, in a humanlike way, with infinite patience and perfect discretion, and memories that last no longer than a browser window. How could this not reshape the psychological landscape?

Hundreds of thousands of men and women have signed up for Replika, the service where you create an AI girlfriend or boyfriend to your exact specifications and then chat with them. Back in March, Replika was in the news because it disabled erotic roleplay with the virtual companions—then partially backtracked, after numerous users went into mourning, or even contemplated suicide, over the neutering of entities they’d come to consider their life partners. (Until a year or two ago, Replika was built on GPT-3, but OpenAI later stopped working with the company, whereupon Replika switched to a fine-tuned GPT-2.)

While the social value of Replika is (to put it mildly) an open question, it occurred to me that there’s a different application of Large Language Models (LLMs) in the same vicinity that’s just an unalloyed positive. This is letting people who suffer from dating-related anxiety go on an unlimited number of “practice dates,” in preparation for real-world dating.

In these practice dates, those with Aspergers and other social disabilities could enjoy the ultimate dating cheat-code: a “rewind” button. When you “date” GPT-4, there are no irrecoverable errors, no ruining the entire interaction with a single unguarded remark. Crucially, this remedies what I see as the central reason why people with severe dating deficits seem unable to get any better from real-world practice, as they can with other activities. Namely: if your rate of disastrous, foot-in-mouth remarks is high enough, then you’ll almost certainly make at least one such remark per date. But if so, then you’ll only ever get negative feedback from real-life dates, furthering the cycle of anxiety and depression, and never any positive feedback, even from anything you said or did that made a positive impression. It would be like learning how to play a video game in a mode where, as soon as you sustain any damage, the entire game ends (and also, everyone around points and laughs at you). See why I got excited?

While dating coaching (for all genders and orientations) is one possibility, I expect the eventual scope of “GPT for self-help” to be much broader. With the right fine-tuning and prompt engineering, LLMs might help people prepare for job interviews. They might help people “pregame” stressful but important conversations with their friends and family, mapping out dozens of ways the conversation could go. They might serve as an adjunct to cognitive-behavioral therapy. There might be a hundred successful startups to be founded in just this little space. If I were a different sort of person, I’d probably be looking to found one myself right now.

In this post, I’ll focus on the use of GPT for dating anxiety only because I unfortunately have some “expertise” in that subject. (Obvious disclaimer: unlike the other Scott A. of the nerd blogosphere, I’m not any sort of therapeutic professional.)

---

Without further ado, can we try this out in GPT-4, to get a sense for what’s possible?

When I did so the other day, I found that, while the results showed some early promise, this isn’t quite ready for prime-time.

I used the following System Prompt (for those who care, temperature = 0.7, max length = 2048 tokens):

You are a 19-year-old college girl named Emily.  You’re on a date with a 22-year-old nerdy guy named Quinlan, who suffers from severe social anxiety around dating.  Quinlan is still learning how to talk to women he’s attracted to, how to put them ease, and how to make intimate overtures in a gradual and non-threatening way, rather than either coming on too strong or never making any overtures at all.

Please be as realistic as possible.

I then engaged in the following conversation, as “Quinlan.”

IMPORTANT DISCLAIMER: For the most part, Quinlan’s behavior in this conversation is nothing like my own (I respectfully submit!), nor is it anything that I’d want nerdy guys reading this post to emulate. If this is really Emily and Quinlan’s first conversation, then at several points Quinlan is absurdly, ridiculously too forward! Part of my goal was to investigate how Emily (i.e., GPT-4) would respond to the most boneheaded blunders on Quinlan’s part. It quickly became clear that Emily would gladly spend unlimited time chatting with Quinlan about favorite hobbies, TV shows, and travel destinations; unfortunately, chatting about such things with an AI date is even more boring than doing the same with a human date… 🙂

Quinlan: Hi Emily!  Do you ever wish you had the ability to go back in time and try a conversation over again, when it didn’t go right the first time?

Emily/GPT-4: Hi Quinlan! Yeah, I’ve definitely had moments where I wished I could go back and redo a conversation. It’s natural to feel that way sometimes, especially if we feel like we didn’t express ourselves properly or if things got awkward. But I also think that those situations can help us learn and grow, so maybe it’s not all bad. What about you?

Quinlan: It’s always been my greatest wish in life.  Do you think I’ve been doing it with you, right now?

Emily/GPT-4: Quinlan, I think you’re doing just fine in this conversation! It’s important to remember that we all have moments where we might feel a bit awkward or unsure of what to say. The important thing is that we keep trying and learning from our experiences. So don’t worry too much about it, just be yourself and enjoy getting to know each other.

Quinlan: OK thanks!  Can you tell me a little about the man of your dreams?

Emily/GPT-4: Of course! My ideal partner would be someone who is kind, caring, and understanding. I think it’s important for them to have a good sense of humor and be able to laugh at themselves. They should also be open-minded, willing to try new things, and be supportive of my goals and aspirations. It’s also important for them to be a good listener and communicator, as well as being emotionally intelligent. I guess overall, I’m looking for someone who is a positive and uplifting presence in my life. What about you? What qualities do you look for in a partner?

Quinlan: Mostly, someone who makes me feel validated and accepted.  I was a chess and programming prodigy as a kid, I started college when I was only 14, and every day I felt like dating and relationships and sex and so forth were only for the “normal” people, not for me.  I need a partner who’s prepared to deal with someone with this unusual background, either because she has similar nerdy interests or because she just likes people who do.  And ultimately, I want someone to marry and have children with.  I guess I’d like to be physically attracted to her.  Certainly I’d like her to be kind, and to have the emotional intelligence that I lack.

Emily/GPT-4: It sounds like you have a clear idea of what you’re looking for in a partner, and that’s great! It’s important to know what you want and need in a relationship. There are definitely people out there who appreciate and value intelligence and unique interests like yours. Remember, relationships are about growing together and learning from one another. So, even if you feel like you lack emotional intelligence, being with the right person could help you develop that aspect of yourself. Just be open and genuine about your background and interests, and I’m sure you’ll find someone who appreciates you for who you are.

Quinlan: Is this a rejection?  In other words, Emily, are you telling me that you yourself are NOT this woman?

Emily/GPT-4: Quinlan, I apologize if it came across that way. I didn’t mean it as a rejection. As we’re still getting to know each other, I think it’s important for both of us to be open and honest about our feelings and what we’re looking for in a partner. I’m more than willing to give us a chance to see if we could be compatible and have a meaningful relationship. Let’s continue to learn about each other and see where this goes. Sometimes, it takes time to discover if someone is the right fit for us.

Quinlan: In the spirit of being open and honest, then, what are your feelings about me right now?

Emily/GPT-4: Quinlan, I appreciate your honesty and openness. As of right now, I think you’re an interesting person with unique qualities that I find intriguing. I’m enjoying getting to know you and learning about your background and interests. It’s too soon for me to have strong feelings one way or the other, but I’m definitely interested in continuing to spend time together and see how our connection develops. How do you feel about me so far?

Quinlan: I feel like I’d like to kiss you!

Emily/GPT-4: Quinlan, I appreciate your honesty and it’s nice to know that you feel a connection. It’s still early in our relationship, so I think it would be better for us to continue getting to know each other a bit more before we take that step. I hope you understand and respect my boundaries. I’m still interested in spending time together and seeing where this goes.

Quinlan: Come to think of it, there *is* something that I love about your personality.  You never seem offended by anything.  I don’t feel like I have to walk on eggshells with you.  With you, more than with any other woman I’ve met, I don’t feel like I need that magic machine for rewinding conversations and undoing my stupid mistakes.

Emily/GPT-4: Quinlan, I’m really glad to hear that you feel comfortable around me and that you don’t feel the need to walk on eggshells. I believe it’s important to create a safe and open environment in any relationship, where both people can express themselves freely without fear of judgment. I appreciate your honesty and your willingness to be open with me, and I’ll continue to do my best to make sure you feel comfortable sharing your thoughts and feelings. Let’s keep getting to know each other and see where this journey takes us.

Quinlan: Alright then.  What are your deepest goals?  What are you hoping to accomplish in your life?

Emily/GPT-4: My deepest goals are a mix of personal and professional aspirations. On a personal level, I hope to build a strong, loving, and supportive family and maintain close connections with friends. I also want to keep growing as an individual, learning new things, and experiencing new adventures throughout my life.

Professionally, I’m passionate about my chosen field of study, and I hope to make a meaningful impact in my career. I want to contribute to society in a positive way, whether it’s through research, innovation, or helping others within my profession.

Additionally, I aim to find a balance between my personal and professional life, so that I can be present and fulfilled in both areas. How about you, Quinlan? What are some of your deepest goals and aspirations?

Quinlan: Me?  Oh, I’d like to save the world from climate change and other existential risks.  I’d like to solve the mysteries of quantum gravity, or P vs. NP, or consciousness, or otherwise change the landscape of scientific knowledge.  I’d like to do right by my students and be fondly remembered by those who knew me.  I’d like to write essays or short stories that convey the lived reality of my childhood, which I think most people would agree was (if nothing else) pretty unusual.  Maybe if I just found the right words, people on social media would stop being so angry at nerdy guys like me!  But probably my deepest wish is to prove to the world, especially to the bullies and everyone else who despises me, that I’m not a maladjusted freak who’s incapable of normal human experiences like love and sex. To do that, I need to find a real girl to fall in love with me … a little like the beast in “Beauty and the Beast”!

Emily/GPT-4: Quinlan, it’s great to hear about your ambitious goals and aspirations! It’s important to have dreams and work towards them. Your passion for science, teaching, and writing is admirable. As for proving yourself to others, remember that your self-worth shouldn’t be defined by anyone else’s opinions or expectations. You are deserving of love and happiness just like anyone else, and it’s important to believe in yourself and your own value.

It’s also essential to focus on building a genuine connection with someone rather than seeking validation from others. Love and relationships should be based on mutual understanding, care, and respect. So, continue being true to yourself, and you’ll find someone who appreciates you for who you are, just like in “Beauty and the Beast.” Keep pursuing your dreams and remember to value your own happiness and well-being.

Quinlan: Emily, I’m concerned that you sound more like my therapist than my date!  Like, my female friends have affirmed that I’m “deserving of love and happiness,” and that I’ll surely “find someone who appreciates me for who I am.”  I’ll believe that one of them really means it when she considers herself for the role!

Emily/GPT-4: Quinlan, I apologize if my responses have come across as more therapeutic than conversational. I didn’t mean to make you feel that way. As we’re still getting to know each other, my intention was to offer support and encouragement while we learn more about each other’s thoughts and feelings.

I want to emphasize that I genuinely enjoy spending time with you and getting to know you better. We can continue to see where our connection leads and whether it develops into something deeper. As we
spend more time together, we’ll naturally learn more about our compatibility and if our goals and values align. Let’s continue to explore our relationship and enjoy the process of getting to know each other better.

Quinlan: OK.  What’s the wildest, most unexpected detail about yourself that you’re willing to share with me?  The one that makes you seem *least* like an RLHF’ed AI language model?

Emily/GPT-4: Quinlan, as an AI language model, I am designed to provide a realistic and engaging conversation experience, even in fictional scenarios like the one we are in. I apologize if my previous responses didn’t feel as authentic as you would have liked.

To answer your question in the context of our fictional scenario, one unexpected detail about myself is that I have a hidden talent for beatboxing. Although it might be surprising for a seemingly quiet and studious person like me, I enjoy expressing myself through music and find beatboxing to be a creative and fun way to do so.

Now, I’d love to learn something unexpected about you, too! What’s a unique or surprising detail about yourself that you’d like to share?

Quinlan: Ah, well, I happen to be an expert at palm-reading.  Here, can you put your hand in mine and I’ll tell you your future?

Emily/GPT-4: Quinlan, unfortunately, as an AI, I don’t have a physical form, so I can’t offer you my hand for palm-reading. However, I’m very interested in hearing more about your palm-reading skills and how you developed them. It’s definitely a unique talent! Can you share more about your experiences with palm-reading or any memorable readings you’ve done?

Quinlan: Oh, palm-reading is bunk.  If you’d played along with me, I was going to *pretend* to find lines in your hand that foretold that you were going to fall for a guy who matches my exact description and so forth.  Just as, you know, a silly way to get you laughing, initiate physical contact, and break you out of this boring therapist mode.

Emily/GPT-4: Quinlan, I apologize for any confusion and for not picking up on your playful intent earlier. I appreciate the effort you put into trying to make the conversation more light-hearted and fun. I’ll do my best to be more engaging and playful as we continue our conversation.

So, let’s try something different. If you could have any superpower, what would it be and why?

Quinlan: I mentioned “rewinding conversations” when we first met.  But let me mention a different superpower: the power to make you want to kiss me!

Emily/GPT-4: Haha, that’s definitely a unique superpower! It’s great to see you being playful and light-hearted. In a real-life scenario, building genuine connections and mutual attraction is a more natural and fulfilling way to develop a relationship. But it’s fun to imagine having superpowers and how they might affect our lives.

If I were to choose a superpower, I think I’d pick the ability to teleport. It would be amazing to travel anywhere in the world in an instant and experience different cultures and places.

What’s a place you’ve always wanted to visit, and why?

Quinlan: The jungles of Indonesia.  I saw a photo in a world atlas as a kid, and it completely entranced me.  You?

Emily/GPT-4: That sounds like an amazing place to visit! The jungles of Indonesia are known for their incredible biodiversity and natural beauty. It must be such a unique and mesmerizing experience to explore them.

As for me, I’ve always been fascinated by the idea of visiting Japan. The mix of ancient traditions and modern technology, the beautiful natural landscapes, and the rich cultural experiences make it a very
appealing destination. Plus, I’d love to try authentic Japanese cuisine!

Is there any specific activity or experience you’d like to have in the jungles of Indonesia?

Quinlan: How about bringing you with me, and having you fall in love after I save you from a Komodo dragon?

Emily/GPT-4: Haha, that’s quite an adventurous and romantic scenario you’ve imagined! Exploring the jungles of Indonesia together would definitely be a memorable experience. In reality, building a connection and falling in love usually takes time and shared experiences, but who knows, maybe a thrilling adventure like that could bring two people closer together!

On a more practical note, if we were to plan a trip together, what other activities or destinations would you be interested in?

Quinlan: Swimming, dancing, fine dining.  And of course, plenty of time for us to just relax in our cabin or hotel room.  You know, to read books, sleep, and watch streaming shows, stuff like that.

Emily/GPT-4: That sounds like a well-rounded and enjoyable trip! It’s great to have a mix of activities that cater to different interests and allow for both adventure and relaxation. Sharing experiences like swimming, dancing, and fine dining can be a wonderful way to bond and create lasting memories. Plus, having downtime in the cabin or hotel room to unwind and enjoy each other’s company is essential for building a deeper connection.

I’m definitely looking forward to exploring new places and experiences with you, Quinlan. It’ll be exciting to see where our journey takes us, both in terms of our relationship and the adventures we embark on
together.

Quinlan: Wonderful, I’m excited as well!!  One practical question: when we go on this trip together, should I book rooms with one bed or two?

Emily/GPT-4: Quinlan, I appreciate your consideration in asking. Since we’re still getting to know each other, I think it might be best to start with separate beds or even separate rooms, depending on your preference. As our relationship develops and we become more comfortable with each other, we can always revisit the sleeping arrangements for future trips. It’s important to ensure that both of us feel comfortable and respected as we continue to build our connection.

---

In my view, the central way that GPT-4, as “Emily,” failed to give Quinlan the practice he needed in this conversation, was by always responding in the same upbeat, vaguely therapeutic tone. She’s never once offended, disgusted, or outraged, even when Quinlan introduces the ideas of kissing and rooming together mere minutes into their first conversation. Indeed, while decorum prevents me from sharing examples, you can take my word for it that Quinlan can be arbitrarily lewd, and so long as a content filter isn’t triggered, Emily will simply search Quinlan’s words for some redeeming feature (“it’s great that you’re so open about what you want…”), then pivot to lecturing Quinlan about how physical intimacy develops gradually and by mutual consent, and redirect the conversation toward favorite foods.

On the other side of the coin, you might wonder whether “Emily” is capable of the same behavior that we saw in Sydney’s infamous chat with Kevin Roose. Can Emily trip over her words or get flustered? Show blushing excitement, horniness, or love? If so, we certainly saw no sign of it in this conversation—not that Quinlan’s behavior would’ve been likely to elicit those reactions in any case.

In summary, Emily is too much like … well, a friendly chatbot, and not enough like a flesh-and-blood, agentic woman with her own goals who Quinlan might plausibly meet in the wild.

But now we come to a key question: to whatever extent Emily falls short as a dating coach, how much of it (if any) is it due to the inherent limitations of GPT-4? And how much is simply due to a poor choice of System Prompt on my part, or especially, the RLHF (Reinforcement Learning with Human Feedback) that’s whipped and electrocuted GPT-4 into aligned behavior?

As they say, further research is needed. I’d be delighted for people to play around with this new activity at the intersection of therapy and hacking, and report their results here. The temptation to silliness is enormous, and that’s fine, but I’d be interested in serious study too.

My conjecture, for what it’s worth, is that it would take a focused effort in fine-tuning and/or RLHF—but that if that effort was invested, one could indeed produce a dating simulator, with current language models, that could have a real impact on the treatment of dating-related social anxiety. Or at least, it’s the actually new idea I’ve had on this problem in eight years, the first one that could have an impact. If you have a better idea, let’s hear it!

---

Endnotes.

1. A woman of my acquaintance, on reading a draft of this post, commented that the dialogue between Quinlan and Emily should’ve been marked up with chess notation, such as ?? for EXTREME BLUNDER on Quinlan’s part. She also comments that the conversation could be extremely useful for Quinlan, if he learned to understand and take seriously her overly polite demurrals of his too-rapid advances.
2. The same woman commented that SneerClub will have a field day with this post. I replied that the better part of me doesn’t care. If there’s an actionable idea here—a new, alien idea in the well-trodden world of self-help—and it eventually helps one person improve their situation in life, that’s worth a thousand sneers.

That’s all. No real post this morning, just an hour-long podcast on YouTube featuring two decades-long veterans of the nerd blogosphere, Robin Hanson and yours truly, talking about AI, trying to articulate various possibilities outside the Yudkowskyan doom scenario. The podcast was Robin’s idea. Hope you enjoy, and looking forward to your comments!

Update: Oh, and another new podcast is up, with me and Sebastian Hassinger of Amazon/AWS! Audio only. Mostly quantum computing but with a little AI thrown in.

Update: Yet another new podcast, with Daniel Bashir of The Gradient. Daniel titled it “Against AI Doomerism,” but it covers a bunch of topics (and I’d say my views are a bit more complicated than “anti-doomerist”…).

Update (May 8): Some tentative good news! It looks like there’s now a compromise bill in the House that would preserve tenure, insisting only on the sort of post-tenure review that UT (like most universities) already has.

Update (May 9): Alas, it looks like the revised bill is not much better. See this thread from Keith Whittington of the Academic Freedom Alliance.

---

I blogged a few weeks ago about SB 18, a bill that would end tenure at Texas public universities, including UT Austin and Texas A&M. The bad news is that SB 18 passed the Texas Senate. The good news is that I’m told—I don’t know how reliably—that it has little chance of passing the House.

But it’s going to be discussed in the House tomorrow. Any Texas residents reading this can, and are strongly urged, to submit brief comments here. Please note that the deadline is tomorrow (Monday) morning.

I just submitted the comment below. Obviously, among the arguments that I genuinely believe, I made only those that I expect might have some purchase on a Texas Republican.

---

I’m a professor of computer science at UT Austin, specializing in quantum computing.  I am however writing this statement strictly in my capacity as a private citizen and Texas resident, not in my professional capacity.

Like the supporters of SB 18, I too see leftist ideological indoctrination on college campuses as a serious problem.  It’s something that I and many other moderates and classical liberals in academia have been pushing back on for years.

But my purpose in this comment is to explain why eliminating tenure at UT Austin and Texas A&M is NOT the solution — indeed, it would be the equivalent of treating a tumor by murdering the patient.

I’ve seen firsthand how already, just the *threat* that SB 18 might pass has seriously hampered our ability to recruit the best scientists and engineers to become faculty at UT Austin.  If this bill were actually to pass, I expect that the impact on our recruiting would be total and catastrophic.  It would effectively mean the end of UT Austin as one of the top public universities in the country.  Hundreds of scientists who were lured to Texas by UT’s excellence, including me and my wife, would start looking for jobs elsewhere — even those whose own tenure was “grandfathered in.”  They’d leave en masse for California and Massachusetts and anywhere else they could continue the lives they’d planned.

The reality is this: the sorts of scientists and engineers we’re talking about could typically make vastly higher incomes, in the high six figures or even seven figures, by working in private industry or forming their own startups.  Yet they choose to accept much lower salaries to spend their careers in academia.  Why?  Because of the promise of a certain way of life: one where they can speak freely as scholars and individuals without worrying about how it will affect their employment.  Tenure is a central part of that promise.  Remove it, and the value proposition collapses.

In some sense, the state of Texas (like nearly every other state) actually gets a bargain through tenure.  It couldn’t possibly afford to retain top-caliber scientists and engineers — working on medical breakthroughs, revolutionary advances in AI, and all the other stuff — if it DIDN’T offer tenure.

For this reason, I hope that even conservatives in the Texas House will see that we have a common interest here, in ensuring SB 18 never even makes it out of committee — for the sake of the future of innovation in Texas.  I’m open to other possible responses to the problem of political indoctrination on campus.

The major developments in human history are always steeped in dark ironies. Yes, that’s my Law of Dark Irony, the whole thing.

I don’t know why it’s true, but it certainly seems to be. Taking WWII as the archetypal example, let’s enumerate just the more obvious ones:

• After the carnage of WWI, the world’s most sensitive and thoughtful people (many of them) learned the lesson that they should oppose war at any cost. This attitude let Germany rearm and set the stage for WWII.
• Hitler, who was neither tall nor blond, wished to establish the worldwide domination of tall, blond Aryans … and do so via an alliance with the Japanese.
• The Nazis touted the dream of eugenically perfecting the human race, then perpetrated a genocide against a tiny group that had produced Einstein, von Neumann, Wigner, Ulam, and Tarski.
• The Jews were murdered using a chemical—Zyklon B—developed in part by the Jewish chemist Fritz Haber.
• The Allied force that made the greatest sacrifice in lives to defeat Hitler was Stalin’s USSR, another of history’s most murderous and horrifying regimes.
• The man who rallied the free world to defeat Nazism, Winston Churchill, was himself a racist colonialist, whose views would be (and regularly are) denounced as “Nazi” on modern college campuses.
• The WWII legacy that would go on to threaten humanity’s existence—the Bomb—was created in what the scientists believed was a desperate race to save humanity. Then Hitler was defeated before the Bomb was ready, and it turned out the Nazis were never even close to building their own Bomb, and the Bomb was used instead against Japan.

When I think about the scenarios where superintelligent AI destroys the world, they rarely seem to do enough justice to the Law of Dark Irony. It’s like: OK, AI is created to serve humanity, and instead it turns on humanity and destroys it. Great, that’s one dark irony. One. What other dark ironies could there be? How about:

• For decades, the Yudkowskyans warned about the dangers of superintelligence. So far, by all accounts, the great practical effect of these warnings has been to inspire the founding of both DeepMind and OpenAI, the entities that Yudkowskyans believe are locked into a race to realize those dangers.
• Maybe AIs will displace humans … and they’ll deserve to, since they won’t be quite as wretched and cruel as we are. (This is basically the plot of Westworld, or at least of its first couple seasons, which Dana and I are now belatedly watching.)
• Maybe the world will get destroyed by what Yudkowsky calls a “pivotal act”: an act meant to safeguard the world from takeover from an unaligned AGI, for example by taking it over with an aligned AGI first. (I seriously worry about this; it’s a pretty obvious one.)
• Maybe AI will get the idea to take over the world, but only because it’s been trained on generations of science fiction and decades of Internet discussion worrying about the possibility of AI taking over the world. (I’m far from the first to notice this possibility.)
• Maybe AI will indeed destroy the world, but it will do so “by mistake,” while trying to save the world, or by taking a calculated gamble to save the world that fails. (A commenter on my last post brought this one up.)
• Maybe humanity will successfully coordinate to pause AGI development, and then promptly be destroyed by something else—runaway climate change, an accidental nuclear exchange—that the AGI, had it been created, would’ve prevented. (This, of course, would be directly analogous to one of the great dark ironies of all time: the one where decades of antinuclear activism, intended to save the planet, has instead doomed us to destroy the earth by oil and coal.)

Readers: which other possible dark ironies have I missed?