A normal day in the OSC office, pondering about science, and two things came up:
- Is hypothesis testing using p values still way forward in science?
- Do red unicorns exist?
Yes, we did say that. Allow us to elaborate…
In March 2019 the American Statistician published a special issue on ‘Statistical Inference in the 21st Century: A World Beyond p < 0.05,’ which included 43 new papers by leading statisticians. The objective of the publication was to ‘end the practice of using a probability value (p value) of less than 0.05 as strong evidence against a null hypothesis or a value greater than 0.05 as strong evidence favouring a null hypothesis’. Yes. Strong stuff indeed. This arbitrary cut-off value has been treated traditionally for decades as the ‘holy grail’ by inter alia biologists. A p value of 0.049 is only marginally different (numerically, at least) to a value of 0.051, yet the former is classed as statistically significant while the latter is not. We all know that p values are engrained in the practice and teaching of statistics, so their abandonment is challenging to implement (that’s a politically-correct way of saying ‘no way, José!). So, we thought we’d put this out to vote. This editorial presents useful suggestions about how to move away from p values (do try, we’d love to see you get it through peer-review).
Today’s discovery led to some ‘fairly interesting’ discussions within our group comprising: n =1 astrophysicist and n = 4 biologists (collective IQ of…well, a LOT), about sample sizes. For example, if only one event/item is observed once, or an event happens only once (but is recorded in a scientifically-robust and quantifiable manner), surely that ‘event’ is valid scientifically, i.e. no further observations are required to substantiate its existence? Let us give an example provided by one of our Directors (who happens to be a biologist, don’t judge).
A scientist sees a pink unicorn (Equus improbablus va. rocacea), takes a photograph, and manages to get a DNA sample before it vanishes in a cloud of rainbow-coloured mist (no, the scientist didn’t get a sample of the mist, so let’s not get finicky about this). That sighting ‘event’ (assuming said scientist wasn’t imbibing anything illegal) was real, and has been proven by a photo and a DNA sample (no, the sample wasn’t unicorn poo, which as we all know, comprises Smarties, which is why one always strives to collect unicorn blood, because smarties have a very short ½ life… we digress). The DNA substantiates its existence as ‘real’; however, with a sample size of one (n =1), the scientist is unable to extrapolate about other unicorns, or colours of unicorn, such as potential existence of red (as opposed to pink) unicorns. More pink unicorns might exist. Even red unicorns might exist. They could be salsa dancing every Wednesday night with vampires in waterfalls, for all one knows (we digress again, ahem). In fact, while we’re on the topic, pink unicorns might, in fact, be a melanistic form of red unicorns, but this is mere supposition, drawn from real-life examples, such as jaguars and panthers (Panthera once). Annnyyyywwaaaay. To cut a long story short: at this point in the discussion, the astrophysicist’s and the biologists’ opinions actually aligned. n =1 wins. Pink unicorns exist. We could solve world peace.
In environmental monitoring, hypothesis testing is moot, something academic peer-reviewers seem to struggle with…but that’s another story – or battle- we won’t get into here.
Editorial (not including unicorns) here.
Full issue (still not including unicorns) here.
Comments welcome on Facebook (or by post, but remember to include unicorn poo in all parcels).
Have a nice weekend!
