This is the sixth and final part of my summary of the Science-Based Medicine conference at TAM7, which will be followed by a summary of TAM7 itself. Part one, Dr. Steven Novella’s introduction, is here. Part two, Dr. David Gorski on cancer quackery, is here. Part three, Dr. Harriet Hall on chiropractic, is here. Part four, Dr. Kimball Atwood on evidence-based medicine and homeopathy, is here. Part five, Dr. Mark Crislip on chronic Lyme disease, is here.
The sixth session speaker was Dr. Val Jones, CEO of BetterHealth, on “Online Health and Social Media: The Good, The Bad, and The Ugly.” In this last post of my SBM conference summary, I’ll cover her talk as well as the Q&A panel that concluded that day’s events.
Personal Story
Dr. Jones began her talk with her personal history–she was raised in Nova Scotia by hippie parents from New York City and grew up on a farm with cows. She said her parents were “moderately weird”–they would ascribe magical powers to yogurt, but they vaccinated their children. She called herself a “shruggie” with respect to complementary and alternative medicine (CAM)–ambivalent about whether there could be anything to it.
She worked with the Ontario March of Dimes, became a doctor, did biophysics and vaccination research at the Mayo Clinic, and earned her M.D. in physical medicine at Columbia. She then worked at Medscape with George Lundberg, editor of JAMA, and created the Clinical Nutrition & Obesity journal there (at Medscape). She was then recruited by RevolutionHealth, an online provider of health information started by Steve Case of America Online. She described it as an “OnStar system for navigating the health care system.” She moved to Washington D.C. to take the job, and, as she put it, “entered the Twilight Zone."
Revolution Health
She served as an editorial director and medical reviewer with a staff of 100 doctors across the country, and “couldn’t believe the crap that came across [her] desk. Who are these people and why are they so into their colons?"
At one point, an article was submitted from a writer for Alternative Medicine magazine that claimed olive oil cures breast cancer. The study behind the story showed that breast cancer cells in a petri dish, exposed to a chemical found in olive oil, made some kind of genetic change–that was transformed into an alleged cure for breast cancer.
The company developed a health tracker tool, and developers kept adding trackers based on what they thought would be cool, such as a “hot flash tracker.” She asked, “Why?” The developers answered, “so they can tell the doctor if a hot flash occurred at 2 or at 3 o’clock!” There was no clinical review of the tracking tools.
Medicine Chest
Another product was developed called Medicine Chest, which allowed people to vote their medicines up or down for how much they like them. “It’s not going to be misinformation, it’s the wisdom of crowds,” the developers said. Not only could users vote on their medicines that were listed, they could add suggestions of their own in free-format text fields. The display of the results on the site didn’t distinguish FDA-approved treatments from what people entered in on their own.
The result was that the best treatment for headaches, back pain, strains, etc. was narcotics, followed by marijuana. The best treatment for diabetes (without distinguishing type 1 from type 2) was dog walking.
Dr. Jones compared this to the Citizens’ Briefing Book on Obama’s change.gov website, where the general public could vote on what they considered to be the most important issues, with the resulting winner being the legalization of marijuana.
Other recommended treatments from the Medicine Chest feature included yogurt for colon cancer, acupuncture for ovarian cysts, herbal treatments for hip fracture, and steroids for cellulitis (which she observed is “very bad and dangerous”). Other similar sites took things to a further extreme, such as Patientslikeme.com, which allowed patients to conduct and report their own clinical trials online. This led to promotion of fish oil to slow the progression of multiple sclerosis and ALS. And beer and dogs as a treatment for lack of motivation.
She cited a quote from Poincare: “Science is facts; just as houses are made of stones, so is science made of facts; but a pile of stones is not a house and a collection of facts is not necessarily science."
She searched the Internet for help understanding the craziness, and came across Orac’s Respectful Insolence blog, which she followed for several months. The last straw for her at Revolution Health was when it promoted chelation as a treatment for autism, which could kill a child, and she felt violated her Hippocratic Oath.
Dr. Jones listed a set of psychological factors which lead people to wrong conclusions, of the sort you might find in Kahnemann & Tversky’s Judgment Under Uncertainty. On the list was the Hawthorne Effect, which purportedly showed that any change made in a business environment temporarily improves productivity. This effect was named after a study of worker productivity based on data collected after changes in lighting and other conditions at the Hawthorne Works between 1924 and 1932, but subsequent studies have failed to replicate the effect. The original data was recently rediscovered and reanalyzed by Steven Levitt (author of Freakonomics) and John List, with the result that “we find that existing descriptions of supposedly remarkable data patterns prove to be entirely fictional. There are, however, hints of more subtle manifestations of a Hawthorne effect in the original data."
Miscellaneous Slides
She concluded her talk with a few slides with various observations, such as The Onion’s “NSF: Science Hard” article and a quote from Surgeon General Rich Carmona that the average American understands medicine at the 4th to 6th grade level. She pointed out that there’s a cottage industry of quack cancer treatment providers around the M.D. Anderson cancer center, taking advantage of cancer patients. She criticized the 1994 passage of DSHEA and its signing into law by Bill Clinton, which exempted dietary supplements from FDA approval requirements unless they’re found to be harmful. She quoted lots of examples of harm from whatstheharm.net. She recommended Memorial Sloan Kettering’s herbal guide, noting that “doesn’t work” is the conclusion for most descriptions, and recommended ClinicalTrials.gov for accurate information. And she closed with a quote from Hippocrates: “There are two things, science and opinion; the former begets knowledge, the latter ignorance."
Q&A
There was then a Q&A panel with all speakers. The first questioner came up to note that the CDC of Maine recently sent pediatricians copies of Paul Offit’s book, Autism’s False Prophets. He also noted that humor of the sort in the “That Mitchell and Webb Look” clip about the homeopathic emergency room was effective, and we need more things like that. Steven Novella responded that we need lots of different things, not any one thing, because AltMed has its marketing down pat.
Another questioner asked if there was a way for social media to work effectively in medicine, to which Dr. Jones responded that MedHealth “has lifeguards in the pool–physicians to moderate.” In a later comment, she pointed out that MedHealth has 200-300 doctors who answer questions for free, because the referrals they get as a result more than make up for the [opportunity cost].
Someone else said that the book Snake Oil Science needs to be turned into an easily usable website, and complained that Quackwatch is hard to use and too polemical. Dr. Novella agreed that SBM needs to provide more, better, and more usable information. It would be good to have a place where you can find overviews on topics, allow you to dig as deep into technical detail as you want, and provides a list of sentinel references. (This is essentially what the TalkOrigins Archive provides for the creation/evolution debate, in particular with Mark Isaak’s Index of Creationist Claims; the power of providing these kinds of broad and deep archives of reliable material was one of the key points of the talk I gave in June to the American Humanist Association.)
Another questioner asked whether there is anything we can do to get rid of the National Center for Complementary and Alternative Medicine. Dr. Novella said that it is becoming more widely known that NCCAM’s Trial to Assess Chelation Therapy, its largest and most expensive trial to date, is loaded with “quacks and criminals” (guilty of insurance fraud and worse) and “totally corrupt,” as has been reported by the Associated Press in several good stories. Bioethicist Art Caplan has pointed out that these are unethical experiments on human subjects that would (should?) never be tolerated by NIH. (NCCAM is part of NIH.) NCCAM has spent $1.2 billion $2.5 billion of taxpayer money to date, and produced zero new treatments.
Someone raised the question of what kinds of questions to ask your own doctors to make sure they’re giving good advice. Dr. Jones suggested asking, “Do you use UpToDate?”, which is a service that searches the world medical literature regularly and provides current data reviewed by 300 full-time reviewers. Dr. Gorski suggested asking whether a doctor follows the NCCN guidelines, which are evidence-based cancer treatment recommendations. Dr. Novella observed that just using Google, a “pull approach,” how most people look for medical information online, is highly unreliable because “bad sites are good at looking like good sites.” (I’d suggest that a more specialized search engine is a better way–Tim Farley suggested some ways of creating such capabilities at TAM6 last year.) Dr. Hall said that Stephen Barrett’s rule of thumb for distinguishing good from bad sites is that “if it’s selling something, it’s a bad site.” I’m not sure how effective that rule is, since even good sites are typically selling something.
Someone raised a problem for use of prior probability, noting that it could have made us miss out on the discovery of lithium as a treatment for bipolar disorder, since it was originally postulated on rather shaky grounds. He gave a second example as SSRIs, which are effective in treating depression, but the original MAOI hypothesis of their operation has been refuted. Dr. Novella responded by saying that first of all, no known mechanism should imply a neutral prior probability (i.e., 0.5). Second, in deciding what to research, it’s better to err on the side of the implausible–but not for treatment. He further suggested that lack of mechanism should not be equated with implausibility. Dr. Atwood seconded that there is a difference between lack of mechanism and contravention of a physical law, and made reference to the discussion that he and I had during the break. He gave aspirin as another example of a substance where the mechanism was discovered later than its effectiveness, and expressed doubt about the questioner’s story of the discovery of lithium’s usefulness.
David Whitlock raised the question of framing, asking why we don’t draw the distinction as science-based vs. faith-based medicine. Dr. Novella responded that this would cause more problems than it would solve, at least in the United States, because of the immunities granted to the free exercise of religion. A questioner wondered whether it might at least stop the government from paying for “faith-based” medicine under single payer. (I don’t think we’re likely to get single payer, and I note that we still have an Office of Faith-Based Programs, so I think this is not a good suggestion.) Dr. Gorski noted that very little CAM is actually religion-based.
A questioner asked how corrupt the Cochrane data is, to which Dr. Atwood replied that the contributions on CAM subjects are unreliable, but reviews of substances are good and Cochrane in general is good. Dr. Novella said that he uses Cochrane to get studies and results, but ignores their conclusions, and pays close attention to authorship. Dr. Hall said that when it comes to meta-analysis, if the result is negative, you should believe it, but if the result is positive, you should look further. Dr. Novella noted that the systematic reviews in Cochrane aren’t actually meta-analyses.
And that pretty much wrapped up the day for the Science-based Medicine conference.
If you’d like to continue on to my summary of The Amazing Meeting 7, it begins here.
UPDATE (July 19, 2009): I’ve been reminded that I neglected to mention one of the more interesting questioners, a massage therapist who stood up and said that he was probably the only “woo practitioner” in the room (though the doctors disagreed that massage therapy really counts as “woo”–and see Dr. Atwood’s talk, where he classified massage therapy as having high prior plausibility), who regularly attends CAM conferences. He complimented the speakers and the audience for having a level of displayed intelligence, sophistication, and scientific knowledge that is not seen at those CAM conferences.
...