{"id":1820,"date":"2014-06-18T21:27:14","date_gmt":"2014-06-19T01:27:14","guid":{"rendered":"https:\/\/scottaaronson.blog\/?p=1820"},"modified":"2017-01-12T16:24:49","modified_gmt":"2017-01-12T21:24:49","slug":"eigenmorality","status":"publish","type":"post","link":"https:\/\/scottaaronson.blog\/?p=1820","title":{"rendered":"Eigenmorality"},"content":{"rendered":"

This post is about an idea I had around 1997, when I was 16 years old and a freshman computer-science major at Cornell. \u00a0Back then, I was extremely impressed by a research project called CLEVER<\/a>, which one of my professors, Jon Kleinberg<\/a>, had led\u00a0while working at IBM Almaden. \u00a0The idea was to use the link structure of the web itself to rank which web pages were most important, and therefore which ones should be returned first in a search query. \u00a0Specifically, Kleinberg defined “hubs” as pages that linked to lots of “authorities,” and “authorities” as pages that were linked to by lots of “hubs.” \u00a0At first glance, this definition seems hopelessly circular, but Kleinberg observed that one can break the circularity by just\u00a0treating the World Wide Web as a giant directed graph, and doing some linear algebra on its adjacency matrix. \u00a0Equivalently, you can imagine an iterative process where each web page starts out with the same hub\/authority “starting credits,” but\u00a0then in each round, the pages distribute their credits among their neighbors, so that the most popular pages get more credits, which they can then, in turn, distribute to their neighbors by linking to them.<\/p>\n

I was\u00a0also\u00a0impressed by a\u00a0similar\u00a0research project\u00a0called PageRank, which was\u00a0proposed later by two guys at Stanford named Sergey Brin and Larry Page. \u00a0Brin and Page dispensed with Kleinberg’s bipartite\u00a0hubs-and-authorities structure in favor of a more uniform structure, and made some other changes, but otherwise their idea was very\u00a0similar. \u00a0At the time, of course, I didn’t know\u00a0that CLEVER was going to languish at IBM, while PageRank (renamed\u00a0Google) was going\u00a0to expand to roughly the size of the entire world’s economy.<\/p>\n

In any case, the question I asked myself about CLEVER\/PageRank was not the one that, maybe in retrospect, I\u00a0should have<\/em> asked: namely,\u00a0“how can I leverage the fact that I know\u00a0the importance of this idea before most people do, in order to make millions of dollars?”<\/p>\n

Instead I asked myself:\u00a0“what other ‘vicious circles’ in science and philosophy could one unravel using the same linear-algebra trick that CLEVER and PageRank exploit?” \u00a0After all, CLEVER and PageRank\u00a0were both founded on\u00a0what looked like a hopelessly circular intuition: “a web page is important if other important web pages link to it.” \u00a0Yet\u00a0they both\u00a0managed to use math\u00a0to defeat\u00a0the circularity. \u00a0All you had to do was find\u00a0an “importance equilibrium,” in which your assignment of “importance” to each web page was stable under a certain linear map. \u00a0And such an equilibrium could be shown to exist—indeed, to exist uniquely.<\/p>\n

Searching for other circular notions\u00a0to elucidate using linear algebra, I hit on morality<\/em>. \u00a0Philosophers from Socrates on, I was vaguely aware, had struggled to define what makes a person\u00a0“moral” or “virtuous,” without tacitly presupposing the answer. \u00a0Well, it seemed to me that, as a first attempt, one could do a lot worse than the following:<\/p>\n

A moral person is someone who cooperates with other moral people, and who\u00a0refuses to cooperate with immoral people.<\/strong><\/p>\n

Obviously one can\u00a0quibble with\u00a0this definition on numerous\u00a0grounds: for example, what exactly does it mean to “cooperate,” and which other people are relevant here? \u00a0If you don’t donate money to starving children in Africa, have you implicitly “refused to cooperate” with them? \u00a0What’s the relative importance of cooperating with good people and withholding cooperation with bad people, of kindness and justice? \u00a0Is there a duty not<\/em> to cooperate with bad people, or merely the lack<\/em> of a duty to cooperate with them? \u00a0Should we consider intent, or only outcomes? \u00a0Surely we shouldn’t hold someone accountable for sheltering a burglar, if they didn’t know about the burgling? \u00a0Also, should\u00a0we compute your “total morality” by simply summing over your interactions with everyone else in your\u00a0community? \u00a0If so, then can a career’s worth\u00a0of lifesaving surgeries\u00a0numerically overwhelm the badness of murdering\u00a0a single child?<\/p>\n

For now, I want you to set all of these important questions aside, and just focus on the fact that the definition doesn’t even seem to work on its own terms<\/em>, because of circularity. \u00a0How can we possibly know\u00a0which people are moral (and hence worthy of our cooperation), and which ones immoral (and hence unworthy), without presupposing the very thing that we seek to define?<\/p>\n

Ah, I thought—this is precisely where linear algebra can come to the rescue! \u00a0Just like in CLEVER or PageRank, we can begin\u00a0by giving everyone in the community an equal number of “morality starting credits.” \u00a0Then we can apply an iterative update rule, where each person A can gain morality credits by cooperating with each other person B, and A gains more<\/em> credits the more credits B has already. \u00a0We apply the rule over and over, until the number of morality credits per person converges to an equilibrium. \u00a0(Or, of course, we can shortcut the process by simply finding the principal eigenvector of the\u00a0“cooperation matrix,” using whatever algorithm we like.) \u00a0We then have our\u00a0objective measure of morality for each individual, solving a 2400-year-old open problem in philosophy.<\/p>\n

The next step, I figured, would be to hack together\u00a0some code that computed this “eigenmorality” metric, and then see what happened when I ran the code to measure\u00a0the morality of each participant in a simulated society. \u00a0What would happen? \u00a0Would the results conform to my pre-theoretic intuitions about what sort of behavior was moral and what wasn’t? \u00a0If not, then\u00a0would watching the simulation give me new ideas about how to improve the morality metric? \u00a0Or would it be my intuitions themselves<\/em> that would change?<\/p>\n

Unfortunately, I never got around to the “coding it up” part—there’s a reason why I became a theorist! \u00a0The eigenmorality idea went onto my back burner, where it stayed\u00a0for the next 16 years: 16 years in which our\u00a0world descended ever further into darkness, lacking a principled way to quantify morality. \u00a0But finally, this year, just two separate things have happened on the eigenmorality front, and that’s why I’m blogging about it now.<\/p>\n

Eigenjesus and Eigenmoses<\/strong><\/p>\n

The first thing that’s happened is that Tyler Singer-Clark, my superb former undergraduate advisee, did<\/em> code up eigenmorality metrics and test them out on a simulated society, for his MIT senior thesis project. \u00a0You can read Tyler’s\u00a012-page\u00a0report here<\/a>—it’s a fun, enjoyable, thought-provoking first\u00a0research paper, one that I wholeheartedly recommend. \u00a0Or, if you’d like to experiment yourself with the Python code, you can download it here from github<\/a>. \u00a0(Of course, all opinions expressed in this post are mine alone, not necessarily Tyler’s.)<\/p>\n

Briefly, Tyler examined\u00a0what eigenmorality has to say in the setting of an Iterated Prisoner’s Dilemma<\/a>\u00a0(IPD)\u00a0tournament. \u00a0The Iterated Prisoner’s Dilemma is the famous game in which two players meet repeatedly, and in each turn can either “Cooperate” or “Defect.” \u00a0The absolute best thing, from your perspective, is if you defect while your partner\u00a0cooperates. \u00a0But you’re also pretty happy if you both cooperate. \u00a0You’re less happy if you both defect, while the worst<\/em> (from your standpoint) is if you cooperate while your partner defects. \u00a0At each turn, when contemplating\u00a0what to do, you have the entire previous history of your interaction with this partner available to you. \u00a0And thus, for example, you can decide to “punish” your partner for past defections, “reward” her for past cooperations, or “try to take advantage” by unilaterally defecting and seeing what happens. \u00a0At each turn, the game has some small constant probability of ending—so you know approximately<\/em>\u00a0how many times\u00a0you’ll meet this partner in the future, but you don’t know exactly when the last turn will be. \u00a0Your score, in the game, is then the sum-total of your score over all turns and all partners (where each player meets each other player once).<\/p>\n

In the late 1970s, as recounted in his classic work\u00a0The Evolution of Cooperation<\/a>, Robert Axelrod invited people all over the world to submit computer programs for playing this game, which were then pit against each other in the world’s first serious\u00a0IPD tournament. \u00a0And, in a tale\u00a0that’s been retold in hundreds of popular books, while many people submitted complicated programs that used machine learning, etc. to try to suss out their opponents, the program that won—hands-down, repeatedly—was TIT_FOR_TAT, a few lines of code submitted by the psychologist Anatol Rapaport to implement an ancient moral maxim. \u00a0TIT_FOR_TAT starts out by cooperating; thereafter, it simply does whatever its opponent did in the last move<\/em>, swiftly\u00a0rewarding every cooperation and punishing every defection, and ignoring the entire previous history. \u00a0In the decades since Axelrod, running Iterated Prisoners’ Dilemma tournaments has become a minor industry, with countless variations explored (for example, “evolutionary” versions, and versions allowing side-communication between the players), countless new strategies invented, and countless papers published. \u00a0To make a long story short, TIT_FOR_TAT continues to do quite well across a wide range of environments, but depending on the mix of players present, other strategies can sometimes beat TIT_FOR_TAT. \u00a0(As one example, if there’s a sizable\u00a0minority of colluding players, who recognize each other by cooperating and defecting in a prearranged sequence, then those players can destroy\u00a0TIT_FOR_TAT and other “simple” strategies, by cooperating with one\u00a0another while defecting against everyone else.)<\/p>\n

Anyway, Tyler sets up and runs a fairly\u00a0standard IPD tournament, with a mix of strategies that includes TIT_FOR_TAT, TIT_FOR_TWO_TATS, other TIT_FOR_TAT variations, PAVLOV, FRIEDMAN, EATHERLY, CHAMPION (see the paper for details), and degenerate\u00a0strategies like always defecting, always cooperating, and playing randomly. \u00a0However, Tyler\u00a0then asks an unusual question about the IPD tournament: namely, purely on the basis of the cooperate\/defect sequences, which players should we judge to have acted morally<\/em>\u00a0toward their partners?<\/p>\n

It might be objected that the players didn’t “know” they were going to be graded on morality: as far as they knew, they were just trying to maximize their individual utilities. \u00a0The trouble\u00a0with that objection is that the players didn’t “know” they were trying to maximize their utilities either!<\/em> \u00a0The players are bots, which do whatever their code tells them to do. \u00a0So in some sense, utility—no less than morality—is “merely\u00a0an interpretation” that we\u00a0impose on the raw cooperate\/defect sequences!\u00a0\u00a0There’s nothing to stop\u00a0us from imposing some other interpretation (say, one that explicitly tries to measure morality) and seeing what happens.<\/p>\n

In an attempt to measure the players’ morality, Tyler uses\u00a0the eigenmorality idea from before. \u00a0The extent\u00a0to which player A “cooperates” with player B is simply measured by the percentage\u00a0of times A cooperates. \u00a0(One acknowledged limitation of this work is that, when two players both defect, there’s no attempt to take into account “who started it,” and to judge the aggressor more harshly than the retaliator—or to incorporate time in any other way.) \u00a0This then gives us a “cooperation matrix,” whose (i,j) entry records the total amount of niceness that player i displayed to player j. \u00a0Diagonalizing that matrix, and taking its largest eigenvector, then gives us our morality scores.<\/p>\n

Now, there’s a very interesting ambiguity in what I said above. \u00a0Namely, should we define the “niceness scores” to lie in [0,1] (so that the lowest, meanest possible score is 0), or in [-1,1] (so that it’s possible to have negative<\/em> niceness)? \u00a0This might sound like a triviality, but in our setting, it’s precisely the mathematical reflection of one of the\u00a0philosophical\u00a0conundrums I mentioned earlier. \u00a0The conundrum can be stated as follows: is your morality a monotone function of your niceness?<\/em> \u00a0We all agree, presumably, that it’s better to be nice to Gandhi than to be nice to Hitler. \u00a0But do you have a\u00a0positive\u00a0obligation to be not-nice<\/em> to Hitler: to make him suffer because he made others suffer? \u00a0Or, OK, how about not Hitler, but someone who’s\u00a0somewhat<\/em> bad? \u00a0Consider, for example, a woman who falls in love with, and marries, an unrepentant armed robber\u00a0(with full knowledge of who he is, and with other options available to her). \u00a0Is the woman\u00a0morally praiseworthy for loving her husband despite<\/em> his bad\u00a0behavior? \u00a0Or is she blameworthy because, by rewarding his behavior with her love, she helps to enable it?<\/p>\n

To capture two possible extremes of opinion about such questions, Tyler and I defined two different morality metrics, which we called …\u00a0wait for it\u00a0…\u00a0eigenmoses<\/strong> and eigenjesus<\/strong>. \u00a0Eigenmoses has the niceness scores in [-1,1], which means that you’re actively rewarded<\/em> for punishing evildoers: that is, for defecting against those who defect against many moral players. \u00a0Eigenjesus, by contrast, has the niceness scores in [0,1], which means that you always do at least as well by “turning the other cheek” and cooperating. \u00a0(Though note that, even with eigenjesus, you get more<\/em> morality credits by cooperating with moral players than by cooperating with immoral ones.)<\/p>\n

This is probably a good place to mention a second limitation of Tyler’s current study. \u00a0Namely, with the current system, there’s no direct way for a player to find out<\/em> how its partner has been behaving toward third parties. \u00a0The only information that A gets about the goodness or evilness of player B, comes from A and B’s direct interaction. \u00a0Ideally, one would like to design\u00a0bots that take into account, not only the other bots’ behavior toward them<\/em>, but the other bots’ behavior toward each other. \u00a0So for example, even if someone is unfailingly nice to you<\/em>, if that person is an asshole to everyone else<\/em>, then the eigenmoses moral code would demand that you return the person’s cooperation with icy defection. \u00a0Conversely, even if Gandhi is mean and hateful to you<\/em>, you would still be morally obliged (interestingly, on both the eigenmoses and<\/em> eigenjesus codes) to be nice to him, because of the amount of good he does for everyone else.<\/p>\n

Anyway, you can read Tyler’s paper if you want to see the results of computing the eigenmoses and eigenjesus scores for a diverse population of bots. \u00a0Briefly, the results accord pretty well with intuition. \u00a0When we look at eigenjesus scores, the all-cooperate bot comes out on top and the all-defect bot on the bottom (as is mathematically necessary), with TIT_FOR_TAT somewhere in the middle, and generous versions of TIT_FOR_TAT higher up. \u00a0When we look at eigenmoses, by contrast, TIT_FOR_TWO_TATS comes out on top, with TIT_FOR_TAT in sixth place, and the all-cooperate bot scoring below the median. \u00a0Interestingly, once again, the all-defect bot gets the lowest score (though in this case, it wasn’t mathematically necessary).<\/p>\n

Even though the measures acquit themselves well in this particular tournament, it’s admittedly easy to construct scenarios where the prescriptions of eigenjesus and eigenmoses alike violently diverge from most people’s moral intuitions. \u00a0We’ve already touched on a few\u00a0such scenarios above (for example, are you really<\/em> morally obligated to lick the boots of someone who kicks you, just because that person is a saint\u00a0to everyone other than you?). \u00a0Another type of scenario involves minorities. \u00a0Imagine, for instance, that 98% of the players are unfailingly nice to each other, but unfailingly\u00a0cruel to the remaining\u00a02% (who they can recognize, let’s say, by their long noses or darker skin—some trivial feature like that). \u00a0Meanwhile, the put-upon 2% return the favor by being nice to each other and mean to the 98%. \u00a0Who, in this scenario, is moral, and who’s immoral? \u00a0The mathematical verdict of both eigenmoses and eigenjesus is unequivocal: the 98% are almost perfectly good, while the 2% are almost perfectly evil.<\/em> \u00a0After all, the 98% are nice to almost everyone, while the 2% are mean to those\u00a0who are nice to almost everyone, and nice only to a tiny minority who are mean to almost everyone. \u00a0Of course, for much of human history, this is precisely<\/em>\u00a0how morality worked, in many people’s minds. \u00a0But I dare say it’s a result that would make moderns uncomfortable.<\/p>\n

In summary, it seems clear to me that neither eigenmoses nor eigenjesus correctly captures our intuitions about morality, any more than \u03a6\u00a0captures our intuitions about consciousness<\/a>. \u00a0But as they say, I think there’s plenty\u00a0of scope here for further research: for coming up with new mathematical measures that sharpen our intuitive judgments about morality, and (if we like) testing those measures out using IPD tournaments. \u00a0It also seems to me that there’s something<\/em>\u00a0fundamentally right about the eigenvector\u00a0idea: all else being equal, we’d like to say, being nice to others is good, except that aiding and abetting evildoers is not<\/em> good, and the way we can recognize the evildoers in our midst is that they’re not nice to others—except that, if the people who someone isn’t nice to are themselves<\/em> evildoers, then the person might again be good, and so on. \u00a0The only way to cut off\u00a0the infinite regress, it seems, is to demand some sort of “reflective equilibrium”<\/a>\u00a0in our moral judgments, and that’s precisely what eigenmorality tries to capture. \u00a0On the other hand, no such idea\u00a0can\u00a0ever make moral debate obsolete—if for no other reason than that we still need to decide\u00a0which specific<\/em> eigenmorality metric to use, and that choice is itself a moral judgment.<\/p>\n

Scooped by Plato<\/strong><\/p>\n

Which brings me, finally, to the second<\/em> new thing that’s happened this year on the eigenmorality front. \u00a0Namely, Rebecca Newberger Goldstein<\/a>—who’s far and away my favorite contemporary novelist—published a\u00a0charming new\u00a0book entitled\u00a0Plato at the Googleplex: Why Philosophy Won’t Go Away<\/a>. \u00a0Here she imagines that Plato has reappeared in present-day\u00a0America (she doesn’t bother to explain how), where he’s taught himself English and the basics\u00a0of modern science, learned how to use the Internet, and otherwise gotten himself up to speed. \u00a0The book recounts Plato’s dialogues with various modern interlocutors, as he volunteers to have his brain scanned, guest-writes a relationship advice column, participates in a panel discussion on child-rearing, and gets interviewed on cable news by “Roy McCoy” (a thinly veiled Bill O’Reilly). \u00a0Often, Goldstein has\u00a0Plato answer the moderns’ questions\u00a0using\u00a0direct quotes from the Timaeus<\/em>, the Gorgias<\/em>, the Meno<\/em>, etc., which makes her Plato into a\u00a0very\u00a0intelligent sort of chatbot. \u00a0This is a genre that’s not often seriously attempted, and that I’d love to read more of (possible subjects: Shakespeare, Galileo, Jefferson, Lincoln, Einstein, Turing…).<\/p>\n

Anyway, my favorite\u00a0episode in the book is the first, eponymous one, where Plato visits the Googleplex in Mountain View. \u00a0While eating lunch in one of the many free cafeterias, Plato is cornered by a somewhat self-important, dreadlocked coder named Marcus, who tries to convince Plato that Google PageRank has finally solved the problem agonized over in the Republic<\/em>, of how to define justice. \u00a0By using the Internet, we can simply crowd-source the answer, Marcus declares: get millions\u00a0of people to render moral judgments on every conceivable question, and also moral judgments on each other’s judgments. \u00a0Then declare those judgments the most morally reliable, that are judged the most reliable by the people who are themselves the most morally reliable. \u00a0The circularity, as usual, is broken by taking the principal eigenvector of the graph of moral judgments (Goldstein doesn’t have Marcus put it that way, but it’s what she means).<\/p>\n

Not surprisingly, Plato is skeptical. \u00a0Through Socratic questioning—the\u00a0method\u00a0he\u00a0learned from the horse’s mouth—Plato\u00a0manages to make\u00a0Marcus\u00a0realize that, in the very act of choosing which of several variants of PageRank to use<\/em>\u00a0in our\u00a0crowd-sourced justice engine, we’ll implicitly be making moral choices already. \u00a0And therefore, we can’t use PageRank, or anything like it, as the ultimate ground of morality.<\/p>\n

Whereas I imagined that the raw data for an “eigenmorality” metric would consist of numerical measures of how nice people had been to each other<\/em>, Goldstein imagines the raw data to consist of abstract moral judgments, and of judgments about judgments. \u00a0Also, whereas the output of my kind of metric would be\u00a0a measure of the “goodness” of each individual\u00a0person, the outputs of hers would presumably be verdicts about general<\/em> moral and political questions. \u00a0But, much like with CLEVER versus PageRank, it’s obvious\u00a0that the ideas are similar—and that I should credit Goldstein with independently discovering my nerdy 16-year-old vision, in order to put it in the mouth of a nerdy\u00a0character in her story.<\/p>\n

As I said before, I agree with Goldstein’s Plato that eigenmorality can’t serve as the ultimate ground of morality. \u00a0But that’s a bit like saying that Google rank can’t serve as the ultimate ground of importance, because even just to design and evaluate\u00a0their ranking algorithms, Google’s engineers must have some prior notion of “importance” to serve as a standard. \u00a0That’s true, of course, but it omits to mention\u00a0that\u00a0Google rank is still\u00a0useful<\/em>—useful enough to have changed civilization in the space of a few years. \u00a0Goldstein’s book has the wonderful property that even the ideas she gives to her\u00a0secondary characters,\u00a0the ones who serve as foils to Plato,\u00a0are sometimes\u00a0interesting enough to deserve book-length treatments of their own, and crowd-sourced morality strikes me as a perfect example.<\/p>\n

In the two previous comment threads, we got into a discussion of anthropogenic climate change, and of my own preferred way to address it and related threats to our civilization’s survival,\u00a0which is simply to\u00a0tax every economic activity at a rate commensurate with the environmental damage that it does, and use the funds collected for cleanup, mitigation, and research into alternatives. \u00a0(Obviously, such ideas are nonstarters in the current political climate of the US, but I’m not talking here about what’s feasible<\/em>, only about what’s necessary<\/em>.) \u00a0As several commenters pointed out, my view raises an obvious question: who is to\u00a0decide<\/em> how much “damage” each activity causes, and thus how much it should be taxed? \u00a0Of course, this is merely a special case of the\u00a0more general question: who is to decide\u00a0on any question of public policy whatsoever?<\/p>\n

For the past few\u00a0centuries, our main method for answering such questions—in those parts of the world where a\u00a0king or dictator or Politburo doesn’t decree the answer—has been representative democracy. \u00a0Democracy is, arguably, the best decision-making method that our sorry species has ever managed to put into practice, at least outside the hard sciences. \u00a0But in my view, representative democracy is now failing spectacularly\u00a0at possibly the single most important\u00a0problem it’s ever faced: namely, that of leaving our descendants\u00a0a livable planet. \u00a0Even though, by and large, reasonable\u00a0people mostly agree about\u00a0what needs to be done—weaning ourselves off fossil fuels (especially the dirtier ones), switching to solar, wind, and nuclear, planting forests and stopping deforestation, etc.—after decades of debate we’re still taking only limping, token steps toward those goals, and in many cases we’re moving rapidly in the opposite\u00a0direction. \u00a0Those\u00a0who, for financial, theological, or ideological reasons, deny the very existence<\/em> of a\u00a0problem,\u00a0have proved that despite being a minority, they can\u00a0push hard enough on the levers of democracy to prevent anything meaningful from happening.<\/p>\n

So what’s the solution? \u00a0To put the world under the thumb\u00a0of an environmentalist dictator? \u00a0Absolutely not. \u00a0In all of history,\u00a0I don’t think any dictatorial\u00a0system\u00a0has ever<\/em> shown\u00a0itself robust against takeover by murderous tyrants (people who probably aren’t too keen on alternative energy either). \u00a0The problem, I think, is epistemological<\/em>. \u00a0Within physics and chemistry and climatology, the people\u00a0who think anthropogenic climate change exists and is a serious problem have\u00a0won the argument<\/em>—but the news of their intellectual victory hasn’t yet spread to the opinion page of the Wall Street Journal<\/em>, or cable news, or the US Congress, or the minds of enough people to tip the scales of history. \u00a0Because our\u00a0domination of the earth’s climate and biosphere is new and\u00a0unfamiliar;\u00a0because the evidence for rapid\u00a0climate change\u00a0is\u00a0complicated and statistical; because the\u00a0worst effects are still remote from us in time, space, or both; because the sacrifices needed to address the problem are real—for all of these reasons, the deniers have learned\u00a0that they can subvert the Popperian process by which bad explanations\u00a0are discarded\u00a0and good explanations\u00a0win. \u00a0If you just repeat\u00a0debunked\u00a0ideas through a loud enough megaphone, it turns out, many\u00a0onlookers\u00a0won’t be able to tell the difference<\/em>\u00a0between you and the\u00a0people\u00a0who have genuine\u00a0knowledge—or they will eventually, but not until it’s too late. \u00a0If you have a few million dollars, you can even set up your own parody of the scientific process: your own phony experts, in their own phony think tanks, with their own phony publications, giving each other legitimacy by citing each other. \u00a0(Of course, all this is a problem for many fields, not just climate change. \u00a0Climate is special\u00a0only because there, the future of life on earth might literally<\/em> hinge on our ability to get epistemology right.)<\/p>\n

Yet for all that, I’m an optimist—sort of. \u00a0For it seems to me that the Internet has given us new tools with which to try to fix our collective epistemology, without giving up on a democratic society. \u00a0Google, Wikipedia, Quora, and so forth\u00a0have already<\/em> improved our situation, if only by a little. \u00a0We could improve it a lot more. \u00a0Consider, for example, the following attempted definitions:<\/p>\n

A trustworthy source of information is one that’s considered trustworthy by many sources who are themselves trustworthy (on the same topic or on closely related topics). \u00a0The current scientific consensus, on any given issue, is what the trustworthy\u00a0sources consider to be the consensus. \u00a0A good decision-maker is someone who’s considered to be a good decision-maker by many other good decision-makers.<\/p>\n

At first glance, the above\u00a0definitions sound ludicrously\u00a0circular—even Orwellian—but we now know that all that’s needed to unravel the circularity is a principal eigenvector computation on the matrix of trust. \u00a0And the computation of such an\u00a0eigenvector need be no more “Orwellian” than … well, Google. \u00a0If enough people want it, then we have the tools today to put flesh on these definitions, to give them agency: to build\u00a0a crowd-sourced deliberative\u00a0democracy, one that “usually just works” in much the same way Google usually just works.<\/p>\n

Now, would those with axes to grind\u00a0try to subvert such a system\u00a0the instant it went\u00a0online? \u00a0Certainly. \u00a0For example, I assume\u00a0that millions of people would rate Conservapedia<\/a> as a more trustworthy source than Wikipedia—and would rate other people who had done so as, themselves, trustworthy sources, while rating as untrustworthy anyone\u00a0who called Conservapedia untrustworthy. \u00a0So\u00a0there would arise a parallel world\u00a0of trust and consensus and “expertise,”\u00a0mutually-reinforcing yet\u00a0nearly\u00a0disjoint from the world of the real. \u00a0But here’s the thing: anyone would be able to see, with the click of a mouse, the extent to which this parallel world had diverged\u00a0from the real one.<\/em> \u00a0They’d see that there was a huge, central\u00a0connected component in the trust graph—including almost all of the Nobel laureates, physicists from the US nuclear weapons labs, military\u00a0planners, actuaries, other hardheaded people—who all accepted the reality of humans warming the planet, and only tiny, isolated tendrils of trust reaching from that component into the component of Rush Limbaugh and James Inhofe<\/a>. \u00a0The deniers and their think-tanks would be exposed to the sun; they’d lose their thin\u00a0cover of legitimacy. \u00a0It should go without saying that the same would happen to various charlatans on the left, and should go without saying that I’d cheer that outcome as well.<\/p>\n

Some will object: but people who believe in pseudosciences—whether creationists or anti-vaxxers or climate change deniers—already know<\/em>\u00a0they’re in a\u00a0minority! \u00a0And far from being worried about it, they treat it as a badge of honor.<\/em>\u00a0 They think they’re Galileo, that their belief in spite<\/em> of a\u00a0scientific\u00a0consensus makes them heroes, while those in the giant central component of the trust graph are merely slavish followers.<\/p>\n

I admit all this. \u00a0But the point of an eigentrust system wouldn’t be to convince everyone. \u00a0As long as I’m fantasizing, the point would be that, once people’s individual decisions did<\/em> give rise to a giant connected trust component, the recommendations of that component could acquire the force of law. \u00a0The\u00a0formation of the giant component would be the\u00a0signal that there’s now enough of a\u00a0consensus to warrant action, despite the continuing existence of a vocal dissenting minority<\/em>—that the minority has, in effect, withdrawn\u00a0itself from the main conversation and retreated\u00a0into a different discourse. \u00a0Conversely, it’s essential to note, if there were a dissenting minority, but that minority had strong trunks of topic-relevant\u00a0trust pointing toward it from the main component\u00a0(for example, because the minority contained a large fraction of the experts in the relevant field), then the minority’s objections might be enough to veto action, even if it was numerically small. \u00a0This is still democracy; it’s just democracy enhanced by linear algebra.<\/p>\n

Other people will object that, while we should<\/em> use the Internet to improve\u00a0the democratic process, the idea we’re looking for is not eigentrust or eigenmorality but rather prediction markets<\/a>. \u00a0Such markets would allow us to, as my friend Robin Hanson<\/a> advocates, “vote on values but bet on beliefs.” \u00a0For example, a country\u00a0could vote for the conditional\u00a0policy that, if<\/em> business-as-usual is predicted to\u00a0cause sea levels to rise at least 4 meters by the year 2200, then<\/em>\u00a0an aggressive emissions\u00a0reduction plan\u00a0will be triggered, but not otherwise. \u00a0But as for the prediction itself, that would be left to a\u00a0futures market: a place where, unlike with voting, there’s a serious penalty for being wrong, namely losing your shirt. \u00a0If the futures market assigned the prediction at least\u00a0such-and-such a probability, then the policy tied to that prediction would become law.<\/p>\n

I actually like the idea of prediction markets—I have ever since I heard about them—but I consider them limited in scope. \u00a0My example above, involving the year 2200, gives a hint as to why. \u00a0Prediction markets are great whenever our disagreements are over something that will<\/em> be settled one way or the other, to everyone’s assent, in the near future (e.g., who will win the World Cup, or next year’s GDP). \u00a0But most of our important disagreements aren’t like that: they’re over which direction society should move in, which issues to care about, which statistical indicators are even the right ones to measure a country’s health. \u00a0Now, those broader questions\u00a0can<\/em>\u00a0sometimes be settled empirically, in a sense: they can be settled by the overwhelming judgment of history, as the slavery, women’s suffrage, and fascism debates were. \u00a0But that kind of empirical confirmation typically takes way<\/em> too long to set up a decent betting market around it. \u00a0And for the non-bettable questions, a carefully-crafted eigendemocracy really is the best system I can think of.<\/p>\n

Again, I think Rebecca Goldstein’s Plato is completely\u00a0right\u00a0that such a system, were it implemented,\u00a0couldn’t\u00a0possibly solve the philosophical problem of finding the “ultimate ground of justice,” just like Google can’t provide us with the “ultimate ground of importance.” \u00a0If nothing else, we’d still need to decide which<\/em> of the many possible eigentrust metrics\u00a0to use,\u00a0and we couldn’t use eigentrust for that<\/em> without risking an infinite regress. \u00a0But just like Google, whatever\u00a0its flaws, works well enough\u00a0for you to use it dozens of times per\u00a0day, so a crowd-sourced eigendemocracy might—just might—work well\u00a0enough to save civilization.<\/p>\n

\n

Update (6\/20):<\/strong><\/span>\u00a0If you haven’t been following, there’s an excellent discussion in the comments, with, as I’d hoped, many commenters raising strong and pertinent objections to the eigenmorality and eigendemocracy concepts, while also proposing possible fixes. \u00a0Let me now mention what I think are the most important<\/em> problems\u00a0with eigenmorality and eigendemocracy respectively—both of them things that had occurred to me also, but that the commenters have brought out very clearly and explicitly.<\/p>\n

With eigenmorality, perhaps the most glaring problem is that, as I mentioned\u00a0before, there’s no notion\u00a0of time-ordering<\/em>, or of “who started it,” in the definition that Tyler and I were using. \u00a0As Luca Trevisan aptly points out<\/a> in the comments, this has the consequence that eigenmorality, as it stands, is completely unable to distinguish between a crime syndicate that’s hated by the majority because of its crimes, and an equally-large ethnic minority that’s hated by the majority solely because it’s different, and that therefore<\/em> hates the majority. \u00a0However,\u00a0unlike with mathematical theories of consciousness—where I used counterexamples to try to show\u00a0that no<\/em> mathematical definition of a certain kind could possibly capture our intuitions about consciousness—here the problem strikes me as much more circumscribed and bounded. \u00a0It’s\u00a0far from obvious to me that we can’t easily improve\u00a0the\u00a0definition of eigenmorality so that it does<\/em>\u00a0agree with most people’s moral\u00a0intuition, whenever intuition renders a clear verdict, at least in the limited setting of Iterated Prisoners’ Dilemma tournaments.<\/p>\n

Let’s see, in particular, how to solve the problem that Luca stressed. \u00a0As a first pass, we could do so as follows:<\/p>\n

A moral agent\u00a0is one who only initiates<\/em> defection against agents who it has good reason to believe are immoral (where, as usual, linear algebra is used to unravel the definition’s apparent circularity).<\/strong><\/p>\n

Notice that I’ve added two<\/em> elements to the setup: not only time but also knowledge. \u00a0If you shun someone solely because you don’t like how they look, then we’d like to say that reflects poorly on you, even if (unbeknownst to you) it turns out that the person really is<\/em> an asshole. \u00a0Now, several\u00a0more clauses would need to be added to the above definition to flesh it out: for example, if you’ve initiated defection against an immoral person, but\u00a0then the person stops<\/em> being immoral, at what point do you have a moral duty to “forgive and forget”? \u00a0Also, just like with the eigenmoses\/eigenjesus distinction, do you have a positive duty to initiate defection against an agent who you learn is immoral, or merely no duty not<\/em> to do so?<\/p>\n

OK, so after we handle the above issues, will there still<\/em> be examples\u00a0that our time-sensitive, knowledge-sensitive eigenmorality definition gets badly, egregiously wrong? \u00a0Maybe—I don’t know! \u00a0Please let me know in the comments.<\/p>\n

Moving on to eigendemocracy, here I think the biggest problem is one pointed out<\/a> by commenter Rahul. \u00a0Namely, an essential aspect\u00a0of how\u00a0Google is able to\u00a0work so well is that people have reasons for linking to webpages other than boosting those pages’ Google rank.<\/em>\u00a0 In other words, Google takes a link structure that already exists, independently of its ranking algorithm, and that (as the economists would put it) encodes people’s “revealed preferences,” and exploits\u00a0that structure for its own purposes. \u00a0Of course, now that Google is the main way many\u00a0of us navigate the web, increasing Google rank has become<\/em> a major reason for linking to a webpage, and an entire SEO industry<\/a>\u00a0has arisen to try to game the rankings. \u00a0But Google\u00a0still isn’t the only<\/em> reason for linking, so the link structure still contains real information.<\/p>\n

By contrast, consider an eigendemocracy, with a giant network\u00a0encoding who trusts whom on what subject. \u00a0If the only reason why this trust network existed was to help make political decisions, then gaming the system would probably be rampant: people could simply decide first which political outcome they wanted, then choose the “experts” such that claiming to “trust” them would do the most for their favored outcome. \u00a0It follows that this system can only improve on ordinary democracy if the trust network has some other<\/em> purpose, so that the participants\u00a0have an actual incentive to reveal the truth about who they trust. \u00a0So, how would an eigendemocracy suss out the truth about who trusts whom on which subject? \u00a0I don’t have a very good answer to this, and am open to suggestions. \u00a0The best idea so far is to use Facebook for this purpose, but I don’t know exactly how.<\/p>\n

\n

Update (6\/22):<\/strong><\/span>\u00a0Many commenters, both here and on Hacker News<\/a>, interpreted me to be saying something obviously stupid: namely, that any belief identified as “the consensus” by an eigenvector analysis is therefore<\/i> the morally right one. They then energetically knocked down this strawman, with the standard examples (Hitler, slavery, discrimination against gays).<\/p>\n

Admittedly, I probably contributed to this confusion by my ill-advised decision to discuss eigenmorality and eigendemocracy in the same blog post—solely because of their mathematical similarity, and the ease with which thinking about one leads to thinking about the other. But the two are different, as are my claims about them. For the record:<\/p>\n