I listen to several podcasts each work during my daily commute to and from the office. One podcast, the Skeptics Guide to the Universe (SGU), has several segments each week, with a few discussing scientific findings. My favorite is a game for the panel where three scientific research findings are offered and the panelists guess which one is false. That is, a finding is made up by the host; often, it is in the opposite direction of a recent study. The other two findings are "science;" a result that has been published in a recent journal.
Two episodes ago (#330, 11/12/2011), the podcasters were sent an email from a listener that linked to a peer-reviewed study that offered the following result:
Remote, retroactive intercessory prayer said for a group is associated with a shorter stay in hospital and shorter duration of fever in patients with a bloodstream infection and should be considered for use in clinical practice.
That is, the BMJ study had people, in the year 2000, pray for patients with bloodstream infections during the years 1990-1996. The study shows a statistically significant reduction in both the duration and level of the fevers the patients had. Naturally, this confounded both the guests and the panelists, which lead to a lengthy discussion about the obvious flaws of the study.
Within a week of that podcast airing, a listener informed the hosts that the BMJ's December issue tends to contain whimsical studies for the sake of humor. That is, the study should not be taken seriously or touted as coming from a peer-reviewed source. In the printed version of the journal, these articles would be grouped together and the farcical nature of the study would be obvious. For example, the 2010 issue contains the following studies:
- Effect on gastric function and symptoms of drinking wine, black tea, or schnapps with a Swiss cheese fondue: randomised controlled crossover trial
- Testing the validity of the Danish urban myth that alcohol can be absorbed through feet: open labelled self experimental study
- Beauty sleep: experimental study on the perceived health and attractiveness of sleep deprived people
- Can he fix it? Yes, he can!– A case of a woman wearing Bob the Builder goggles to correct her idiopathic ocular neuromyotonia.
The lesson learned, of course, is that the context of an article can mean everything. Taking any of the above articles as serious, peer-reviewed work would be erroneous. While we in political science have special issues that can contextualize studies, I have not come across anything that demanded the context of the journal for an article to be interpretted correctly. I am amused by the prospect of having our own fake studies. This, perhaps, could be the proper forum for an article on the daily temperature high in Washington D.C. and Senate votes on force (with forecasting for global warming).
