The blog is one in a series of blogs around paranormal research. It is one of my more “dry” blogs on my views of statistics in paranormal research. It is an important topic because so many decisions in parapsychology research are dependent on statistical testing.

Statistical testing is the mainstay of the paranormal questionnaire and experimental research. Statistical tests are an aid to allow consistent testing of data to decide on two hypotheses. There is no statistical effect, or there is evidence of a statistical effect. The statistical testing is based on various factors that give us confidence in the data, such as sample size, a variation of results, distribution of results and type of scale used, i.e., whether yes or no or a continuous scale (1-5).

So what do statistics tell us? There are two approaches; (1) some researchers suggest that it is whether the results are due to chance or (2) it gives us a value on how confident we are in the data. All statistics provide a probability value of the result. The accepted criterion is 0.05, which is a suggestion that the probability of this occurring at random is a one in a twenty.

So a paranormal researcher has a statistically significant result. Happy Days! The researcher has proved the existence of paranormal phenomena. Or have they? The likelihood is, they haven’t. All the researcher has found is the results are unlikely to be caused by a chance occurrence. Therefore, there could be other reasons for creating the effect. One could be poor quality questionnaire design or methodological flaws in the experiment. We shouldn’t assume because a result is statistically significant it has any meaning.

There are two crucial questions to ask about the research; (1) is the research valid, and (2) how reliable is the research? The validity is whether the questionnaire or experiment is measuring what it should measure. If it is a questionnaire study around paranormal, is it actually measuring paranormal, i.e., if it is asking about life on other planets, then we can say the questionnaire is lacking validity as that isn’t included in the definition of paranormal. Or the experimental procedures are flawed. Therefore, researchers have delicately called this “garbage in and garbage out principle.” In short, there is a discrepancy between what the researchers are claiming to the actual evidence they are presenting.

The second important principle is around reliability is if we accept the experiment results are valid, then the next question is how reliable are the results. Can the experiments be re-tested, run by different experimenters who get the same results and are they consistent with other theories and experiment results. If they pass all these tests, then there might be actual evidence to support the researcher.

One of the difficulties for paranormal researchers is that they are using negative evidence to form conclusions. I go into more depth in my book on this approach. But paranormal researchers tend to rule out all standard explanations before calling it paranormal. Most of the researchers aren’t testing a theoretical theory, just hoping they can rule out conventional explanations with proper research techniques. The problem with this approach, if there is a genuine phenomenon it doesn’t know what is causing the phenomena.

Let’s apply this to a typical parapsychology experiment around telepathy. Two participants are assigned two different roles, one is called the sender and the other the receiver (both are in separate rooms). The aim of the experiment is whether the sender can send an image to the receiver via telepathic means. The receiver has a forced-choice to pick an image that they think the sender is viewing. The researcher can now, statistically test whether there are statistical differences in what the sender chooses.

There are a few issues with this approach to telepathic research. Due to the negative evidence approach, the researcher isn’t testing how the mechanisms work. We can compare this to an e-mail analogy. For a telepathic experiment to be successful these things need to happen; (1) the sender sends the image in their mind (someone writes an e-mail), it is transferred through the ether (e-mail is transmitted from one server to another), and it arrives at the receiver (the e-mail arrives). In all three parts in the telepathic experiment, all three could fail, and we won’t know at what point it fails.

There is more to successful research than just significant statistical results. At times they can be meaningless if they are not supported by robust methodology. To evaluate the meaning of research, these things need to be considered; validaty of research (measuring what it should measure), reliability (it can be repeated in different settings), and it tells us something about how the phenomena happen. Paranormal research is sometimes too focused on whether the results are statistically significant. What researchers should be focused on is how meaningful the research is into understanding the paranormal phenomena.

I will be presenting a series of blogs around the paranormal: The next topic is “Life after Physical Death- is there scientific evidence?”

If your interested, you can buy my book atAmazon
Thanks for reading, Ian