The Goodness Cluster
The blog-commenters come at me one by one, a seemingly infinite supply of them, like masked henchmen in an action movie throwing karate chops at Jackie Chan.
“Seriously Scott, do better,” says each henchman when his turn comes, ignoring all the ones before him who said the same. “If you’d have supported American-imposed regime change in Venezuela, like just installing María Machado as the president, then surely you must also support Trump’s cockamamie plan to invade Greenland! For that matter, you logically must also support Putin’s invasion of Ukraine, and China’s probable future invasion of Taiwan!”
“No,” I reply to each henchman, “you’re operating on a wildly mistaken model of me. For starters, I’ve just consistently honored the actual democratic choices of the Venezuelans, the Greenlanders, the Ukrainians, and the Taiwanese, regardless of coalitions and power. Those choices are, respectively, to be rid of Maduro, to stay part of Denmark, and to be left alone by Russia and China—in all four cases, as it happens, the choices most consistent with liberalism, common sense, and what nearly any 5-year-old would say was right and good.”
“My preference,” I continue, “is simply that the more pro-Enlightenment, pluralist, liberal-democratic side triumph, and that the more repressive, authoritarian side feel the sting of defeat—always, in every conflict, in every corner of the earth. Sure, if authoritarians win an election fair and square, I might clench my teeth and watch them take power, for the sake of the long-term survival of the ideals those authoritarians seek to destroy. But if authoritarians lose an election and then arrogate power anyway, what’s there even to feel torn about? So, you can correctly predict my reaction to countless international events by predicting this. It’s like predicting what Tit-for-Tat will do on a given move in the Iterated Prisoners’ Dilemma.”
“Even more broadly,” I say, “my rule is simply that I’m in favor of good things, and against bad things. I’m in favor of truth, and against falsehood. And if anyone says to me: because you supported this country when it did good thing X, you must also support it when does evil thing Y? (Either as a reductio ad absurdum, or because the person actually wants evil thing Y?) Or if they say: because you agreed with this person when she said this true thing, you must also endorse this false thing she said? I reply: good over evil and truth over lies in every instance—if need be, down to the individual subatomic particles of morality and logic.”
The henchmen snarl, “so now it’s laid bare! Now everyone can see just how naive and simplistic Aaronson’s so-called ‘political philosophy’ really is! Do us all a favor, Scott, and stick to quantum physics! Stick to computer science! Do you not know that philosophers and political scientists have filled libraries debating these weighty matters? Are you an act-utilitarian? A Kantian? A neocon or neoliberal? An America-First interventionist? Pick some package of values, then answer to us for all the commitments that come with that package!”
I say: “No, I don’t subcontract out my soul to any package of values that I can define via any succinct rule. Instead, given any moral dilemma, I simply query my internal Morality Oracle and follow whatever it tells me to do, unless of course my weakness prevents me. Some would simply call the ‘Morality Oracle’ my conscience. But others would hold that, to whatever extent people’s consciences have given similar answers across vast gulfs of time and space and culture, it’s because they tapped into an underlying logic that humans haven’t fully explained, but that they no more invented than the rules of arithmetic. The world’s prophets and sages have tried again and again over the millennia to articulate that logic, with varying admixtures of error and self-interest and culture-dependent cruft. But just like with math and science, the clearest available statements seem to me to have gotten clearer over time.”
The Jackie Chan henchman smirks at this. “So basically, you know the right answers to moral questions because of a magical, private Morality Oracle—like, you know, the burning bush, or Mount Sinai? And yet you dare to call yourself a scientific rationalist, a foe of obscurantism and myticism? Do you have any idea how pathetic this all sounds, as an attempted moral theory?”
“But I’m not pretending to articulate a moral theory,” I reply. “I’m merely describing what I do. I mean, I can gesture toward moral theories and ideas that capture more of my conscience’s judgments than others, like liberalism, the Enlightenment, the Golden Rule, or utilitarianism. But if a rule ever appears to disagree with the verdict of my conscience—if someone says, oh, you like utilitarianism, so you must value the lives of these trillion amoebas above this one human child’s, even torture and kill the child to save the amoebas—I will always go with my conscience and damn the rule.”
“So the meaning of goodness is just ‘whatever seems good to you’?” asks the henchman, between swings of his nunchuk. “Do you not see how tautological your criterion is, how worthless?”
“It might be tautological, but I find it far from worthless!” I offer. “If nothing else, my Oracle lets me assess the morality of people, philosophies, institutions, and movements, by simply asking to what extent their words and deeds seem guided by the same Oracle, or one that’s close enough! And if I find a cluster of millions of people whose consciences agree with mine and each others’ in 95% of cases, then I can point to that cluster, and say, here. This cluster’s collective moral judgment is close to what I mean by goodness. Which is probably the best we can do with countless questions of philosophy.”
“Just like, in the famous Wittgenstein riff, we define ‘game’ not by giving an if-and-only-if, but by starting with poker, basketball, Monopoly, and other paradigm-cases and then counting things as ‘games’ to whatever extent they’re similar—so too we can define ‘morality’ by starting with a cluster of Benjamin Franklin, Frederick Douglass, MLK, Vasily Arkhipov, Alan Turing, Katalin Karikó, those who hid Jews during the Holocaust, those who sit in Chinese or Russian or Iranian or Venezuelan torture-prisons for advocating democracy, etc, and then working outward from those paradigm-cases, and whenever in doubt, by seeking reflective equilibrium between that cluster and our own consciences. At any rate, that’s what I do, and it’s what I’ll continue doing even if half the world sneers at me for it, because I don’t know a better approach.”
Applications to the AI alignment problem are left as exercises for the reader.
Announcement: I’m currently on my way to Seattle, to speak in the CS department at the University of Washington—a place that I love but haven’t visited, I don’t think, since 2011 (!). If you’re around, come say hi. Meanwhile, feel free to karate-chop this post all you want in the comment section, but I’ll probably be slow in replying!
Follow
Comment #1 January 7th, 2026 at 9:12 pm
Seems like you should read up on https://en.wikipedia.org/wiki/Sealioning
Comment #2 January 7th, 2026 at 9:31 pm
David Karger #1: I know the concept of “sealioning” but never found it helpful — it suffers from the fatal flaw that, in the original cartoon, the sea lion is clearly the more sympathetic character and is right. (Or at least, that’s what my Morality Oracle tells me. 🙂 )
Comment #3 January 7th, 2026 at 9:58 pm
Scott: my overwhelming feeling on reading these kinds of posts is that there’s been a massive failure in communication. Your parodic characterizations of the “henchmen” blog-commenters are just that. They’re actually worse than caricatures in that they often don’t even rhyme with reality, at least not to my ear. I’m not sure what to do about it tbh.
Re Maduro, the point I think a lot of people were trying to make, over and over in different ways, was that whether what happened was good or bad is more complicated than whether *all else being equal* a liberal Venezuela is preferable to an autocratic Venezuela. Process, laws, norms are important as well — they’re key to the enlightenment liberalism you love and of course also to many of the ideals we associate with America — and it’s reasonable to worry about the longer term effects of breaking them in order to achieve various short term goods. On one hand, I’d be surprised if you didn’t understand this simple point. On the other, I don’t remember you so much as acknowledging that it’s a legitimate thing to worry about.
Comment #4 January 7th, 2026 at 10:32 pm
John #3: I agreed that there could be all sorts of bad second-order or third-order effects, as there were with previous US interventions. Especially if Trump is going to leave the gangster regime in place, I expect the effects to be more likely bad than good. I also would’ve been happier if US law had been followed, which it obviously wasn’t. The part I deny is that there’s any “norm” of the form “election-overturning autocrats like Maduro get permanent immunity from being invaded and overthrown” that’s worthy of upholding. The norms I care about are the ones that say that Gonzalez is the rightful president of Venezuela.
Comment #5 January 7th, 2026 at 11:50 pm
Scott #4: But I don’t think anyone in that comment thread said that election-overturning autocrats like Maduro should get permanent immunity from being invaded and overthrown. Perhaps my eyes skipped a few crucial lines, but I can’t find any comment that expresses this particular opinion. It seemed most people simply thought that even though this may seem good now (removing a callous dictator) and has certainly generated much jubilation amongst Venezuelans, there is a significant risk (and this because the administration doesn’t seem to have thought much of this through, partly because they ignored the law) that things could turn very sour in the long run. Of course many people displayed poor reading comprehension, and that one person attempted to psychoanalyze you, but I don’t think too many people liked Maduro.
One of my main criticisms in that thread was in response to your claim that international law does more harm than good. I think that’s fairly incorrect regardless of the beliefs one has about this particular scenario; even if one disagrees with my assessment, I still think it’s a strong enough claim that requires really strong justifications. In response, you said that too many governments in the UN with voting power are still illiberal, and at some point it became clear (from some of your rather vivid language in the comment thread, both here and in your “Deep Zionism” post which I only read later) that this antipathy of yours towards international law and norms isn’t really from a belief that “might makes right” or that “the law was and forever will be a farce” or even that “certain illiberal nations must be removed from the UN,” but instead that the UN found Israel to be guilty of a genocide. At that point it became clear that there’s no real convincing to be done here and that we probably just disagree on a few facts and ways of organizing the world even though we’re both liberals (I think); I’m not sure I can convince you on any of those facts or dispositions, and I’m not even sure I have the right to do so.
Comment #6 January 7th, 2026 at 11:55 pm
I have nothing against Karikó, but how did she end up as an example for morality?
Comment #7 January 8th, 2026 at 3:53 am
Some people just like talking smack to you. Maybe they’re not smart enough to comment on the CS posts, I don’t know but don’t let it bother you too much, you’re doing fine. The vast majority of people see you as reasonable even when they disagree with you…
Comment #8 January 8th, 2026 at 7:20 am
domotorp #6: While there are many other examples that would’ve worked equally well, she saved millions of lives by standing up to blankfaces and courageously pursuing scientific truth, which seems pretty good in my book?
Comment #9 January 8th, 2026 at 9:58 am
Here’s my mental model for this discussion.
The moral state of the world, M, is a sum of many, many contributions. For this discussion, there are a few relevant ones. R for removing Maduro. F_US for the future state of Venezeula as determined by the US. O for other implications of the US capturing Maduro.
M = R + F_US + O + …
I take Scott’s main point is that R > 0 (where “good” means that M is positive or at least bigger).
I also think Scott is saying F_US is probably negative, given that the US has not allowed Machado to take over. But, F_US has a larger uncertainty than R because it depends on future events.
I think probably everyone reasonable agrees on those statements.
My main point — I am not sure if I am really disagreeing with Scott or not — would be that I think we can have reasonable Bayesian priors that the sum R + F_US + O is negative, meaning that M is smaller than it would be had Trump not invaded.
I say that because Trump has articulated no plan to give the Venezuelans a democratically elected government, and has only expressed interest in exploiting them by extracting their oil. And, we can’t neglect O, because statements by Trump’s administration indicate that they are at least talking about going after Greenland, Cuba, Colombia, and Mexico. There are other things that I think could also appear in O, like the fact China may take from all this that they have a rationale to invade Tibet, but those are more uncertain.
So what do you do with that? Is the relevant quantity R? Is it R+F_US+O? Are there other terms in M that balance out this incident?
My personal feeling is that the net effect of this incident on M is very likely negative, and that should be the headline. But, I don’t disagree that R is positive. And I don’t disagree that the net change in M is uncertain because it depends on events that have not yet occured.
Comment #10 January 8th, 2026 at 10:07 am
Somehow I had a massive brainfart when writing the previous comment. Of course, I meant O should probably include an increased risk of China invading *TAIWAN*, not Tibet.
Also in retrospect I probably should have used notation that included Venezuela, like F^V_US, instead of F_US.
Comment #11 January 8th, 2026 at 10:11 am
Didier Drogba’s headband #5: OK. If someone opened by saying, “yes, of course Maduro should’ve been overthrown so that legitimately elected leaders could take charge — but not like this; here’s how it should’ve happened instead,” I’d be open to that discussion and would likely even agree with them.
Regarding “international law,” I’m open to arguments that the concept does more good than harm.
But yes, the more people insist to me that “international law” (or the norms of all decent educated people, or the consensus of all academic humanities fields, etc etc) mean that dictators get impunity as long as they’re anti-Western, that Israel needs to be dismantled, and that Jews once again need to be at the mercy of those who would exterminate them — the more I’m going to respond like G. K. Chesterton did, in the famous passage where he says that if the whole logic of the modern world implies that little girls can no longer have long hair, then I’d rather set fire to the entire modern world. (The hair length of little girls is not the hill that I personally choose to die on. 🙂 )
Comment #12 January 8th, 2026 at 10:31 am
Gavin #9: Thank you; that’s very well expressed! You had me on your side as soon as you wrote the value of the action as a sum of multiple components. 😀
Comment #13 January 8th, 2026 at 10:41 am
So here you embrace the fact that you have no coherent philosophy that can be treated rationally between different situations, that your sense of morality is tautological and based on your gut intuition of what a 5-year-old would do (what his parents told him of course), and the main advantage is that you can judge people who “aren’t in your cluster” as immoral. You figure this is the best way to be, you don’t know a better way and aren’t going to look for one no matter what the internet says.
I’m all for dismissing slippery slopes and considering things case-by-case, but this is a big oof. I thought it was a good thing to reject groupthink, not go for it whole hog? Isn’t the role of a public intellectual, professor writing commentary on moral sociopolitical issues, to apply some sort of framework to a question and see where it leads – not to start with their immediate preformed, emotional opinion, declare that basically correct, and work backwards to figure out “who’s with me?”
“I will always go with my conscience and damn the rule” – this is dismissing logic in favor of stubborness, outright. Just say you’re completely closeminded and not open to any discussion on certain issues, or maybe this is your way of doing it. Obviously we have our biases but the logical and moral puzzle is to figure out WHY to damn the rule, and of course, to examine one’s conscience. Which I guess you are outright refusing to do
Comment #14 January 8th, 2026 at 10:57 am
Didier Drogba’s headband #5:
“I think that’s fairly incorrect regardless of the beliefs one has about this particular scenario; even if one disagrees with my assessment, I still think it’s a strong enough claim that requires really strong justifications. […] antipathy of yours towards international law and norms isn’t really from a belief that “might makes right” or that “the law was and forever will be a farce” or even that “certain illiberal nations must be removed from the UN,” but instead that the UN found Israel to be guilty of a genocide.”
I think that the UN falsely accusing Israel of genocide is downstream of the other bad stuff that Scott and others mentioned when talking about international law. I can’t speak for Scott but personally, I think that conflict-related international law is bad not because of its idealistic content, but rather because of the international organs charged with defining and enforcing that law. Those institutions have been hijacked by third-world dictators and anti-Enlightenment/anti-liberal movements like the far-left that support those dictatorships. Just about all of the unfairness of the UN comes from that, and the invective campaign against Israel (which long predates the genocide accusations of 2023) is just one symptom of the rot. It is a particularly big symptom, though, when one looks at the share of UN resolutions targeting Israel versus UN resolutions targeting all other countries combined. Other symptoms are the appointment of autocrats as goodwill ambassadors and chairmen of UN human rights commissions, the fact that just about every sanctions regime against the world’s dictatorships originates from outside the UN, the petty corruption and mismanagement that resulted in UN peacekeepers introducing cholera to Haiti, and so on.
I don’t think its unfair to say that if a good law is implemented badly, then the law becomes a farce. If the intent of international law is the Universal Declaration of Human Rights, then we can judge organizations like the UN based on how their resolutions etc., agree with this declaration. And what do we see? We see a blankfaced refusal to deal with the autocracies and their torture prisons and the numerous wars and genocides and oppression going on across the world. It is no mystery why: the world’s dictatorships and hybrid regimes hold a solid majority in the UN, and they collectively enforce an omerta protecting one another from accountability. So yes, international law is a farce, and this is a structural problem at the UN and at the other various human rights/international law agencies around the world. Change the structure, and international law might no longer be a farce. Except, you cannot actually change the structure, since the UN derives its legitimacy from having every government being a member.
There is another, deeper reason why international law is a farce: it is all bark and no bite. There is no world police agency capable of enforcing international law, as the planet’s military capabilities are controlled by the various national governments. So, to enforce “international law”, the UN, Red Cross, ICC, etc. can only issue strongly-worded statements and toothless warrants. This is actually a good thing, when considering that dictators and far-left ideologues control these organizations. It also means that international law is a hopeless farce, both badly implemented by corrupt and rotten bodies, but also not actually enforceable the way that laws within nations are enforceable.
As for the question, does international law produce more harm than good? I would say yes. Since international law is not actually enforceable, the entire thing can best be seen as a large propaganda apparatus. It has a halo effect as a result of its founders’ idealistic intentions, and its grandiose mandate to rid the world of aggressive war, tyranny, discrimination, etc. What does that apparatus do with all of that legitimacy and moral power? Take a look at where the General Assembly and Human Rights Council focus most of their resolutions over the past three decades. Anyone who actually cares about human rights, and does not just hate Israel, should be appalled at the sheer waste of time and resources if nothing else. And there are many idealistic people out there, who can feel their eyes filling with tears when reading old documents like the UDHR, who decide to dedicate their lives to making the world more peaceful and tolerant and free, and therefore join the organizations who have that written on their mission statements. What do those idealists find themselves doing? Rubber stamping “Israel is evil resolution #123456”. International law, as currently practiced, does more harm than good since it is a large global propaganda apparatus dedicated towards spreading antisemitism in its incarnation as far-left “antizionism”. If it didn’t do that, and instead applied the law fairly and evenly, then I would say that for all of its uselessness it at least does more good than harm. I cannot say that about actually existing international law.
https://unwatch.org/database/#dictatorships-at-un
Comment #15 January 8th, 2026 at 11:06 am
Just wanted to say that this is a much better conversation so far. I’m not sure how much else there is to be learned or said here re Venezuela specifically, but at least people aren’t talking past one another. Thanks, Scott! And Scott #4: sorry about missing your agreement regarding second or third-order effects.
Comment #16 January 8th, 2026 at 11:50 am
Scott #11: International law does not require that the state of Israel be abolished. Where did you get that from? Israel is a member of the United Nations. It is a sovereign state, with just as much legitimacty as the United States, France, Italy, Japan, or any other sovereign state. The so-called Palestinian right of return is also not guaranteed by international law, see https://ir.lawnet.fordham.edu/faculty_scholarship/499/ (full paper at the link https://www.law.upenn.edu/live/files/1949-kent34upajintll1492012pdf ).
Even Noam Chomsky https://www.thenation.com/article/archive/israel-palestine-and-bds/ writes “Nor is [the Palestinian ‘right of return’] dictated by international law. The text of UN General Assembly Resolution 194 is conditional, and in any event it is a recommendation, without the legal force of the Security Council resolutions that Israel regularly violates. Insistence on [the Palestinian ‘right of return’] is a virtual guarantee of failure.”
Comment #17 January 8th, 2026 at 11:52 am
AI alignment is a hard problem. Even human alignment is a hard problem – that’s called politics. The fact that human alignment is the same thing as politics is something I’ve noticed before.
Comment #18 January 8th, 2026 at 12:14 pm
Gavin #9 Scott #12
How could we obtain a reasonable estimate of the moral state of the world prior to the arrest of Maduro? One method would be to weight UN General Assembly votes by the proportion of the global population that each country represents. I took a quick look at a recent vote condemning the “unlawful” presence of Israel in Gaza-124 For/14 Against/43 Abstentions and so far more than a majority of global population For when considering population by country.
I question the overall rectitude of global morality. I agree democracy with well developed democratic institutions likely better than the alternative and that one must individually make a judgement as to the specific facts. Some of the worst dictators in history started their political journeys with electoral success and so in fact democracies do make mistakes.
My point is not that democracy isn’t inherently superior but just that in fact the world has not migrated to liberal democracy post the optimism of post WW 2 that has long been claimed to be inevitable. The post post WW 2 strategies of the world’s liberal democracies have been unexpectedly ineffective.
My view is that in fact the liberal democracies themselves are now facing increasing existential risk while largely continuing with the status quo path or even internally moving away from liberal democratic principles. It will be difficult to develop new strategies without accepting that the old ones have failed. One position could be that no other potentially effective strategy is possible that wouldn’t compromise our principles. In this case, considering foes, liberal democracies may become and interesting experiment that failed. Another might be-things are great and so let’s continue-that leads to the same ultimate conclusion it seems to me.
Comment #19 January 8th, 2026 at 1:08 pm
I just saw Iran shut down internet access countrywide.
Comment #20 January 8th, 2026 at 1:43 pm
Approving Trump for anything he does is a bit like approving ChatGPT in a game of chess for making some good initial moves. It can only lead to regret down the line:
Comment #21 January 8th, 2026 at 2:14 pm
Much like John #15, I think this has gone much better than the other comment thread and I’m pretty satisfied with my contributions to this discussion, so I probably won’t add too much more beyond this comment. I’m also going to disregard all the parts of AF #14’s comment because it’s all poorly reasoned gibberish that I’m not especially interested in engaging with; except to mention that if some think the left (or the “far-left,” whatever that means these days) has overreacted to Israel’s actions in the aftermath of October 7th, I fear they may be very surprised by what many young people on even the moderate right believe these days (and I say so as someone who (1) disagrees with the vast majority of hard leftist positions on most topics (2) is frequently in conversation with people on the left and right regardless of how often I agree with them). I have no right to speak decisively on the matter, but I really worry that “antizionism is antisemitism” has ultimately led to a significant resurgence of antisemitic instincts especially among young (non-Jewish) people throughout the political spectrum. Even though I think of myself as a Zionist (in the sense that I think Israel must have a right to exist as itself, not in the “Deep Zionism” sense, I frequently hear of my (especially right-leaning) friends suddenly finding an avenue to express antizionist sentiments among antisemites, and the longer they spend time among these types the more antisemitic they become. But I’m sure you’ve heard some form of this in the past, so I won’t press the issue (especially, since, like I already said before, I have no chance of convincing you on this particular matter since it pertains to your identity).
On international law: I will never claim international law has done no harm, as it certainly has done at least some harm in practice. I also won’t attempt to claim that it has so much power; if it did, the worst of Scott’s fears about the UN’s reactions to Israel would’ve been borne out by now. International law doesn’t mean that dictators get impunity as long as they’re anti-Western, that Israel needs to be dismantled, and that Jews once again need to be at the mercy of those who would exterminate them. I agree that perhaps a few reforms to the UN Security Council are worth considering, but even simple things like the Universal Declaration of Human Rights, the Geneva Conventions and later protocols, the many trade agreements and alliances, the law of the sea — I don’t think a counterfactual world in which all of these things do not exist is a better world than the world we have today. If you agree with that, I don’t know how it’s then possible to claim that they have done more harm than good (a different claim — that I still disagree with but that I can at least tolerate — is that they have done about as much harm as would have happened without them).
Final word on Maduro: If the US wanted to actually do some good for the Venezuelan people, much effort should have first been invested in actually dismantling the Maduro regime and dramatically reducing the influence of the goons there. It would have only required a little more tact — certainly nowhere near the level of Operation Grim Beeper, certainly achievable according to the commentary of many experts on the issue — and would have actually freed Venezuela from their socialist dictatorship; we’d see Venezuelans celebrating in Caracas (which hasn’t happened), not in Miami. After that, Machado should’ve been placed as the president, and had that happened I wouldn’t even be too fussed about the resource extraction that is going on now and was always going to happen. Either way, I hope things end well for them in the end; all I have insisted throughout is that although it is better for Venezuela to have no socialist dictators, the situation right now could easily devolve into something worse than it currently is or something slightly better but still bad enough, which could itself have been very easily prevented with just a bit of tact.
With that, I think I’ve said my last words here. Thanks for approving my comments!
Comment #22 January 8th, 2026 at 3:29 pm
MT #13:
So here you embrace the fact that you have no coherent philosophy that can be treated rationally between different situations…
Do you know anyone who successfully reduced morality to an articulable list of if-then rules, without any horrifying implications? If not, then why blame me for being honest about it?
As far as I know, people who performed heroic deeds most often did so not because they followed from some moral theory, but because doing otherwise violated their consciences. It’s interesting after the fact to construct and debate theories that explain as much as we can about what these people were doing, but we shouldn’t confuse the cart and the horse.
Comment #23 January 8th, 2026 at 4:39 pm
Just passing by to say Cuba is next guys. Keep safe, because the world will only get more dangerous. Hopefully for dictators and not democratically elected leaders, but i am not holding my breath
Comment #24 January 8th, 2026 at 6:28 pm
John,
“worse than caricatures in that they often don’t even rhyme with reality”
Seemed a pretty good accounting of the conversation in last thread to me!
Comment #25 January 8th, 2026 at 6:43 pm
Allow me to go ahead and quote one of those people that argues about how to ground morality in somethingother than people’s individual consciences:
“I hold that moral intuitions are nothing but learned prejudices. Historic examples from slavery to the divine right of kings to tortured confessions of witchcraft or Judaism to the subjugation of women to genocide all point to the fallibility of these ‘moral intuitions’. There is absolutely no sense to the claim that its conclusions are to be adopted before those of a reasoned argument.”
– Alonzo Fyfe
Source: https://atheistethicist.blogspot.com/2009/06/objection-to-desire-utilitarianism.html?m=1
Speaking for myself, to some extent querying a “morality oracle” shaped by one’s upbringing and society can in fact be dangerous, because there have been a lot of moral questions throughout history that the consciences of “good people” have gotten horribly wrong, but in practice I don’t know if we as humans actually can do much better.