Twenty years ago, when the idea of AI taking over the world in our lifetimes still struck most of us as the unconstrained fantasy of those who knew too much science fiction and too little science, many of us would say things like:<\/p>\n\n\n\n
Look, the part of the story that’s wildly implausible is that a recursively self-improving superintelligence will just explode from some hacker’s basement and take over the world without warning. If it’s going to happen, we’ll see many warning signs first. We’ll see, I dunno, AI agents breaking out of containment, conspiring with each other to hack websites, in fanatical pursuit of whatever strange goals they have. And then, of course, we’ll see major math problems getting solved by AIs—even the Clay Millennium Problems. That<\/em> will be the time to panic! Wake me up when that<\/em> happens!<\/i><\/p>\n\n\n\n Twenty years ago, the above was a take that even my most conservative, skeptical colleagues in academic CS would\u2019ve gladly endorsed.<\/p>\n\n\n\n If you want to know my current take, you simply start with the one above, then update on the fact that the wild prophecies have come true<\/strong>. The first rumblings, I\u2019d say, came a decade ago with AlphaGo, they got noticeably louder with LLMs and coding and reasoning agents, and they\u2019ve accelerated this summer and fall into a crescendo of wonders and terrors that one needs to be a particular kind of idiot to deny.<\/p>\n\n\n\n I recoil from the neverending shell game where you say \u201coh sure, of course<\/em> AI can now [escape from its sandbox \/ solve Millennium Problems \/ whichever dramatic thing it most recently did], no one ever denied that [I did<\/em> deny it], wake me up when AI does [thing AI hasn\u2019t yet done but is going to do next year], that\u2019s<\/em> when I\u2019ll reevaluate my whole worldview [no I won\u2019t].\u201d Where no matter how fast the rollercoaster accelerates, even after your whole familiar world has vanished behind you, you\u2019re still inventing reasons why it doesn\u2019t count.<\/p>\n\n\n\n My position on AI is merely the conservative, skeptical position of 2006, updated with intellectual honesty for the reality of late 2026. And that position, if you need me to spell it out, is as follows:<\/p>\n\n\n\n AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA It seems to me that the Singularity has already started<\/em>; it\u2019s just wildly unevenly distributed. Yes, I still unload the dishwasher and clip my toenails. On the other hand, in whatever years I have left, I don\u2019t expect that I\u2019ll ever again prove a theorem because I\u2019m actually needed to prove it. If I do, it will only be for my or others\u2019 enjoyment or edification.<\/p>\n\n\n\n The test is this: if we took the news of these past few weeks and sent it back in time twenty years, would I agree that it looked like the beginning of an AI Singularity? The intellectually honest answer is: yes, absolutely. But then that\u2019s all we need. No backsies.<\/p>\n\n\n\n I feel like it would be healthy for everyone to stop grinding their ideological axes, their sentiments about Dario Amodei or Sam Altman, for long enough simply to acknowledge that the wonders and terrors are here<\/em>. They couldn\u2019t be here more clearly if the sky had turned reddish-orange like in the Matrix<\/em> movies.<\/p>\n\n\n\n It\u2019s here clearly enough that, when I put my kids to sleep at night, I now feel it in the pit of my stomach: what sort of future can they possibly have? What could they learn today that could possibly be relevant to that future? (Yesterday, my 13-year-old daughter joked unprompted that, if she wants to become a mathematician, it now looks like she has maybe two more weeks.) Certainly when my grad students want to discuss what sort of careers might await them on graduation, I no longer have any clue what to tell them.<\/p>\n\n\n\n Maybe it will help if I briefly switch topics. Ever since my wife and I moved to Austin, I\u2019ve sometimes gotten some version of the following query: \u201cHow can you, as both a Jew and a skeptical scientist, possibly get along well with all those evangelical Christians down there in Texas? Sure, they might seem<\/em> super friendly to Jews, but don\u2019t you understand that that\u2019s only because of the special role Jews play in their eschatology\u2014when Christ will return in glory, and you\u2019ll either accept Him as Lord or else roast in hell for eternity?\u201d I stare at them and say: \u201cwait, so I get to accept Christ only after<\/em> He returns? What a great deal! How could I possibly have any objection to that?\u201d<\/p>\n\n\n\n For anyone who says AI doom sounds like an apocalyptic religion, that the rationalists\/Singulatarians seem like a Bay Area cult, that Eliezer Yudkowsky gives off the vibes of a messianic prophet: yes, yes, and yes. But crucially, today you\u2019re no longer being asked to believe in arguments and extrapolations, but only in the front-page news.<\/strong> Accepting the reality of the coming machine god after<\/em> it\u2019s solved Navier-Stokes and dozens of other longstanding open math problems (while dramatically ramping up in capability every month), is sort of like accepting Jesus after<\/em> he\u2019s returned to earth on the gleaming cloud. It\u2019s the epistemic bare minimum.<\/p>\n\n\n\n Yes, there\u2019s still enormous uncertainty about what the rest of our lives will look like, but as far as I can tell, there\u2019s no longer any real uncertainty that it\u2019ll all mostly revolve around AI, and the extent to which we succeed or fail at directing its power toward human flourishing.<\/p>\n\n\n\n By any accounting that doesn\u2019t stack the deck, Eliezer Yudkowsky was right<\/em> about what the greatest challenge facing civilization in our lifetimes was going to be, and you and I were wrong<\/em> about it. Why<\/em> I was wrong is a question I\u2019ll ask myself every day in whatever time remains. But, you know, at least I updated once the prophesied wonders and terrors actually started arriving! If you haven\u2019t done likewise, why haven\u2019t you?<\/p>\n\n\n\n As you presumably know by now—it was the talk of the nerd internet all week—the Navier-Stokes Millennium Problem<\/a> appears to be solved<\/a>, with crucial contributions from both humans and AI, albeit with a tangled dispute about exactly what happened and what ought to have happened. The answer, which an OpenAI model has apparently verified in Lean, is that (as many mathematicians suspected lately) there’s smooth initial data that leads to a singularity in finite time, at least if a smooth external force is applied (the case with no external force is still unresolved). This problem was supposed to carry a $1 million prize, except that OpenAI says they have no interest in collecting the prize and it’s unclear if any human is eligible to collect instead. OpenAI burned at least ~$15 million in compute to produce its 166-page solution<\/a>, which probably hasn’t yet been read and understood by any human.<\/p>\n\n\n\n See here for the Quanta<\/em> article<\/a>, and here for NYU mathematician Tristan Buckmaster’s account<\/a> of the role played by himself and Levent Alp\u00f6ge of Anthropic, which substantially differs from OpenAI’s account (you can read a response from OpenAI’s Sebastian Bubeck here<\/a>). It’s agreed that everything built on an approach pioneered in recent years by the human mathematicians Diego C\u00f3rdoba and Luis Mart\u00ednez-Zoroa.<\/p>\n\n\n\n My purpose here is not to adjudicate the dispute. Yes, in swooping in with vastly greater resources once it had gotten wind of progress on Navier-Stokes, OpenAI seems to have acted in a way that some might describe as “unsportsmanlike.” No, I don’t find it plausible that OpenAI’s models meaningfully benefitted from being trained on Buckmaster and Alp\u00f6ge’s chat logs. But this leaves a crucial question unanswered: what exactly did OpenAI know about Buckmaster and Alp\u00f6ge\u2018s work and when did it know it?<\/p>\n\n\n <\/p>\n <\/p>\n Anyway, as Zvi points out<\/a>, it’s easy to get hung up on the details and lose sight of the high-order bit: namely, that it seems safe to say that human mathematicians are forevermore dethroned as the main theorem-proving entities on planet earth. I feel privileged to have had the traditional kind of career in theoretical computer science in the last decades when that was possible.<\/p>\n <\/p>\n <\/p>\n <\/p>\n If we were just<\/em> talking about Navier-Stokes, you might accuse me of jumping to conclusions here. But we’re not. In the areas I know best (such as quantum complexity theory), and presumably other areas as well, there’s now a deluge, with longstanding open problems both major and minor falling by the day.<\/p>\n <\/p>\n <\/p>\n Go to the arXiv<\/a> or ECCC<\/a>. Pretty much all<\/em> the papers that I’d be interested in now include “AI statements” near the acknowledgments (as this is often the central thing I want to know, I wish I didn’t need to scroll to the end of the paper to find it!). These statements can range from “our main result came entirely from GPT-6, but we understood it and take responsibility for it,” to “the results came from an interaction between the human authors and AI” to “we used AI, but only for proofreading and other incidental things” to (mad props!) “the author did not use AI for anything<\/a>.”<\/p>\n <\/p>\n <\/p>\n If you talk right now to editors or program committee chairs, it’ll remind you of those ominous scenes from the Lord of the Rings<\/em> movies where the men of Gondor or Rohan or whatever are grimly fortifying their walled city against the expected onslaught of 50,000 orcs. Reviewing will have<\/em> to be done partly by AI, because otherwise there’s no way to handle the orc army: the reviewers can’t unilaterally disarm.<\/p>\n <\/p>\n <\/p>\n Anyway, here’s a small sampling of the significant AI-proved or -assisted results from, like, the last month<\/strong>, besides Navier-Stokes—restricting myself to those that solved longstanding open problems I had previously known or cared about.<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n Feel free to remind me of anything I left out.<\/p>\n <\/p>\n <\/p>\n <\/p>\n Let me try to convey the mood in the mathematical community right now, at least as far as my experience reaches. Nearly every conversation is about the AI tsunami, or eventually circles around to the tsunami even if it’s originally about something else. Often, though, the focus is less on the unknowable future—for how much longer will mathematical research as a human enterprise even exist?—than on immediate questions of how to respond<\/em>.<\/p>\n <\/p>\n <\/p>\n What are the new rules for when you get to write a paper with your name on it, and, y’know, get credit for it? That you fully understand the proof, can give talks about the proof, can answer questions about it, take responsibility for its correctness? Do you need to have played any role in finding<\/em> the proof?<\/p>\n <\/p>\n <\/p>\n In the cases, likely to become more and more numerous, where all of those conditions are not<\/em> satisfied, how do you share AI-generated math, if at all? Do you tweet it, like Alp\u00f6ge hilariously did with Fable’s disproof of the Jacobian Conjecture? Do you post to the arXiv or GitHub? Do you publish a paper that lists “GPT-6 Astra” or “Claude Fable” as the author—but then let the AI profusely thank you<\/em> in the acknowledgments for suggesting such a wonderful problem to it?<\/p>\n <\/p>\n <\/p>\n <\/p>\n Of course, how one responds to the immediate problems ultimately does<\/em> depend on one\u2019s broader beliefs about what mathematical research is for and about. Are we just trying to decide whether various conjectures are true or false? Or are we trying to maintain a human community, across the generations, that understands the conjectures and cares about whether they’re true or false and why? If the latter, how do we incentivize people to join that community, to undergo the years of intense training required, if their role will now be reduced to verifiers and explicators (if even that) of gargantuan arguments dumped into their laps by the AI companies?<\/p>\n <\/p>\n <\/p>\n As many of you will have seen, twenty-five Fields Medalists, including Terence Tao, released an open letter entitled A Severe Misalignment of AI in Mathematics<\/a>, which articulates some of these concerns in the wake of the Navier-Stokes announcement. As many critics have pointed out, the open letter doesn’t really have a clear ask: mostly, it just eloquently sets out the values of the human mathematical community that the authors consider worth preserving in the age of AI. After reflection, I decided to endorse the statement, because I want to preserve those values as well.<\/p>\n <\/p>\n <\/p>\n I don’t think any of the signatories are na\u00efve enough to imagine that AI won’t permanently change the way mathematical research is done—indeed, that it isn’t already doing so. There’s surely at most a tiny market for “certified organic theorems.” That isn’t the question. The question is, do we incorporate AI in a way that still puts human understanding, of what either humans or AIs are producing, at the center of the whole enterprise? Maybe someday, it becomes unsustainable to do that. Maybe someday we say: “human math had a great 4,000-year run, but today we close up shop and turn everything over to the machines, continuing to apply our own brains to math, when we do, at most for exercise, recreation, or competition, like chess.”<\/p>\n <\/p>\n <\/p>\n But, partly because of my worries about AI misalignment, I’m not ready to throw in the towel just yet. I still do<\/em> want to keep insight and understanding at the center of what mathematicians, computer scientists, and physicists do, for as long as we can keep it there, even as the human race now cedes its supremacy at the task of proving or disproving conjectures.<\/p>\n <\/p>\n <\/p>\n <\/p>\n Speaking of alignment: if you’re any kind of mathematical researcher, and the present age of wonders and terrors has inspired you to want to spend your remaining time confronting the tsunami head-on, rather than pretending it doesn’t exist or is still far away, please join your dozens of colleagues who’ve arrived at the same place!<\/p>\n <\/p>\n <\/p>\n My friend and colleague Mike Winer<\/a> was trained as a theoretical physicist, did a postdoc with Juan Maldacena at the Institute for Advanced Study in Princeton, but then got AGI-pilled and decided to switch to full-time work at the Alignment Research Center in Berkeley (founded by Paul Christiano<\/a>, who moved to AI alignment a decade ago after doing quantum computing theory with me). Mike recently wrote a Substack post entitled From Academia to Alignment<\/a>, which I enjoyed and which I’d commend to anyone currently considering this transition.\u00a0 In a similar vein, see this<\/a> from Xiaoyu He.\u00a0 And, one more: a meditation on mathematicians’ possible future as priests or monks<\/a>, by Stanford math undergrad Logan Graves.<\/p>\n <\/p>","protected":false},"excerpt":{"rendered":" Twenty years ago, when the idea of AI taking over the world in our lifetimes still struck most of us as the unconstrained fantasy of those who knew too much science fiction and too little science, many of us would say things like: Look, the part of the story that’s wildly implausible is that a […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_seo_schema_type":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"{title}\n\n{excerpt}\n\n{url}","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_wpas_customize_per_network":false,"jetpack_post_was_ever_published":false},"categories":[31,5,4,8],"tags":[],"class_list":["post-10062","post","type-post","status-publish","format-standard","hentry","category-announcements","category-complexity","category-quantum","category-the-fate-of-humanity"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/10062","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10062"}],"version-history":[{"count":25,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/10062\/revisions"}],"predecessor-version":[{"id":10113,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/10062\/revisions\/10113"}],"wp:attachment":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10062"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10062"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10062"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA<\/p>\n\n\n\n
\n\n\n\n
\n\n
\n
\n
\n