{"id":1615,"date":"2013-12-11T17:42:06","date_gmt":"2013-12-11T22:42:06","guid":{"rendered":"https:\/\/scottaaronson.blog\/?p=1615"},"modified":"2016-12-10T04:49:34","modified_gmt":"2016-12-10T09:49:34","slug":"23-me-and-the-right-to-misinterpret-probabilities","status":"publish","type":"post","link":"https:\/\/scottaaronson.blog\/?p=1615","title":{"rendered":"23, Me, and the Right to Misinterpret Probabilities"},"content":{"rendered":"

If you’re the sort of person who reads this blog, you may have heard that 23andMe<\/a>—the company that (until recently) let anyone spit into a capsule, send it away to a DNA lab, and then learn basic information about their ancestry, disease risks, etc.—has suspended much of its service, on orders from the US Food and Drug Administration. \u00a0As I understand it, on Nov. 25, the FDA ordered 23andMe to stop marketing to new customers (though it can still serve existing customers), and on Dec. 5, the company stopped offering new health-related information to any<\/em> customers (though you can still access the health information you had before, and ancestry and other non-health information is unaffected).<\/p>\n

Of course, the impact of these developments is broader: within a couple weeks, “do-it-yourself genomics” has gone from an industry whose explosive growth lots of commentators took as a given, to one whose future looks severely in doubt (at least in the US).<\/p>\n

The FDA gave the reasons for its order in a letter<\/a>\u00a0to Ann Wojcicki,\u00a023andMe’s CEO. \u00a0Excerpts:<\/p>\n

For instance, if the BRCA-related risk assessment for breast or ovarian cancer reports a false positive, it could lead a patient to undergo prophylactic surgery, chemoprevention, intensive screening, or other morbidity-inducing actions, while a false negative could result in a failure to recognize an actual risk that may exist. \u00a0Assessments for drug responses carry the risks that patients relying on such tests may begin to self-manage their treatments through dose changes or even abandon certain therapies depending on the outcome of the assessment. \u00a0For example, false genotype results for your warfarin drug response test could have significant unreasonable risk of illness, injury, or death to the patient due to thrombosis or bleeding events that occur from treatment with a drug at a dose that does not provide the appropriately calibrated anticoagulant effect …\u00a0\u00a0The risk of serious injury or death is known to be high when patients are either non-compliant or not properly dosed; combined with the risk that a direct-to-consumer test result may be used by a patient to self-manage, serious concerns are raised if test results are not adequately understood by patients or if incorrect test results are reported.<\/p>\n

To clarify, the DNA labs that 23andMe uses are already<\/em> government-regulated. \u00a0Thus, the question at issue here is not whether, if 23andMe claims (say) that you have CG instead of CC at some particular locus, the information is reliable. \u00a0Rather, the question is whether 23andMe should be allowed to tell you that fact, while also<\/em> telling you that a recent research paper found that people with CG have a 10.4% probability of developing Alzheimer’s disease, as compared to a 7.2% base rate. \u00a0More bluntly, the question is whether ordinary schmoes ought to be trusted<\/em>\u00a0to learn such facts about themselves, without a doctor as an intermediary to interpret the results for them, or perhaps to decide that there’s no good reason for the patient to know at all.<\/p>\n

Among medical experts, a common attitude seems to be something like this: sure, getting access to your own genetic data is harmless fun, as long as you’re an overeducated nerd who just wants to satisfy his or her intellectual curiosity (or perhaps narcissism). \u00a0But 23andMe crossed a crucial line when it started marketing its service to the hoi polloi<\/em>, as something that could genuinely tell them about health risks. \u00a0Most people don’t understand probability, and are incapable of parsing “based on certain gene variants we found, your chances of developing diabetes are about 6 times higher than the baseline” as anything other than “you will<\/em> develop diabetes.” \u00a0Nor, just as worryingly, are they able to parse “your chances are lower than the baseline” as anything other than “you won’t<\/em> develop diabetes.”<\/p>\n

I understand this argument. \u00a0Nevertheless, I find it completely inconsistent with a free society. \u00a0Moreover, I predict that in the future, the FDA’s current stance will be looked back upon as an outrage, with the subtleties in the FDA’s position mattering about as much as the subtleties in the Church’s position toward Galileo (“look, Mr. G., it’s fine to discuss heliocentrism among your fellow astronomers, as a hypothesis or a calculational tool—just don’t write books telling the general public that heliocentrism is\u00a0literally true<\/em>, and that they should change their worldviews as a result!”). \u00a0That’s why I signed this petition<\/a>\u00a0asking the FDA to reconsider its decision, and I encourage you to sign it too.<\/p>\n

Here are some comments that might help clarify my views:<\/p>\n

(1) I signed up for 23andMe a few years ago, as did the rest of my family. \u00a0The information I gained from it wasn’t exactly earth-shattering: I learned, for example, that my eyes are probably blue, that my ancestry is mostly Ashkenazi, that there’s a risk my eyesight will further deteriorate as I age (the same thing a succession of ophthalmologists told me), that I can’t taste the bitter flavor in brussels sprouts, and that I’m an “unlikely sprinter.” \u00a0On the other hand, seeing exactly which gene variants correlate with these things, and how they compare to the variants my parents and brother have, was … cool<\/em>. \u00a0It felt like I imagine it must have felt to buy a personal computer in 1975. \u00a0In addition, I found nothing the slightest bit dishonest about the way the results were reported. \u00a0Each result was stated explicitly<\/em> in terms of probabilities—giving both the baseline rate for each condition, and the rate conditioned on having such-and-such gene variant—and there were even links to the original research papers if I wanted to read them myself. \u00a0I only wish that I got half<\/em> as much context and detail from conventional doctor visits—or for that matter, from most materials I’ve read from the FDA itself. \u00a0(When Dana was pregnant, I was pleasantly surprised when some of the tests she underwent came back with explicit probabilities and base rates. \u00a0I remember wishing doctors would give me that kind of information more often.)<\/p>\n

(2) From my limited reading and experience, I think it’s entirely possible that do-it-yourself genetic testing is overhyped; that it won’t live up to its most fervent advocates’ promises; that for most interesting traits there are just too many genes involved, via too many labyrinthine pathways, to make terribly useful predictions about individuals, etc. \u00a0So it’s important to me that, in deciding whether what 23andMe does should be legal, we’re not being asked to decide any<\/em> of these complicated questions! \u00a0We’re only being asked whether the FDA should get to decide the answers in advance.<\/p>\n

(3) As regular readers will know, I’m far from a doctrinaire libertarian. \u00a0Thus, my opposition to shutting down 23andMe is not at all<\/em> a corollary of reflexive opposition to any government regulation of anything. \u00a0In fact, I’d be fine if the FDA wanted to insert a warning message on 23andMe (in addition to the warnings 23andMe already\u00a0provides), emphasizing that genetic tests only provide crude statistical information, that they need to be interpreted with care, consult your doctor before doing anything<\/em> based on these results, etc. \u00a0But when it comes to banning access to the results, I have trouble with some of the obvious slippery slopes. \u00a0E.g., what happens when some Chinese or Russian company launches a competing service? \u00a0Do we ban Americans from mailing their saliva overseas? \u00a0What happens when individuals become able just to sequence their entire genomes, and store and analyze them on their laptops? \u00a0Do we ban the sequencing technology? \u00a0Or do we just ban software that makes it easy enough to analyze<\/em> the results? \u00a0If the software is hard enough to use, so only professional biologists use it, does that make it OK again? \u00a0Also, if the FDA will be in the business of banning genomic data analysis tools, then what about medical books<\/em>? \u00a0For that matter, what about any<\/em> books or websites, of any kind, that might cause someone to make a poor medical decision? \u00a0What would such a policy, if applied consistently, do to the multibillion-dollar alternative medicine industry?<\/p>\n

(4) I don’t understand the history of 23andMe’s interactions with the FDA. \u00a0From what I’ve read, though, they have<\/em> been communicating for five years, with everything 23andMe has said in public sounding conciliatory rather than defiant (though the FDA has accused 23andMe of being tardy with its responses). \u00a0Apparently, the key problem is simply that the FDA hasn’t yet developed a regulatory policy specifically for direct-to-consumer genetic tests. \u00a0It’s been considering<\/em> such a policy for years—but in the meantime, it believes no one should be marketing such tests for health purposes before a policy exists. \u00a0Alas, there are very few cases where I’d feel inclined to support a government in saying: “X is a new technology that lots of people are excited about. \u00a0However, our regulatory policies haven’t yet caught up to X. \u00a0Therefore, our decision is that X is banned<\/em>, until and unless we figure out how to regulate it.” \u00a0Maybe I could support such a policy, if X had the potential to level cities and kill millions. \u00a0But when it comes to consumer DNA tests, this sort of preemptive banning seems purposefully designed to give wet dreams to Ayn Rand fans.<\/p>\n

(5) I confess that, despite everything I’ve said, my moral intuitions might be different if dead bodies were piling up because of terrible 23andMe-inspired medical decisions. \u00a0But as far as I know, there’s no evidence so far that even a single person was harmed. \u00a0Which isn’t so surprising: after all, people might run to their doctor terrified about something they learned on 23onMe, but no sane doctor would ever make a decision solely on that basis, without ordering further tests.<\/p>\n","protected":false},"excerpt":{"rendered":"

If you’re the sort of person who reads this blog, you may have heard that 23andMe—the company that (until recently) let anyone spit into a capsule, send it away to a DNA lab, and then learn basic information about their ancestry, disease risks, etc.—has suspended much of its service, on orders from the US Food […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_seo_schema_type":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"{title}\n\n{excerpt}\n\n{url}","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_wpas_customize_per_network":false,"jetpack_post_was_ever_published":false},"categories":[11,16],"tags":[],"class_list":["post-1615","post","type-post","status-publish","format-standard","hentry","category-nerd-interest","category-rage-against-doofosity"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/1615","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1615"}],"version-history":[{"count":6,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/1615\/revisions"}],"predecessor-version":[{"id":1621,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=\/wp\/v2\/posts\/1615\/revisions\/1621"}],"wp:attachment":[{"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scottaaronson.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}