Pseudoscience
Pseudoscience
Commentary on Information and Uncertainty in Remote Perception Research, Dunne and Jahn, PEAR
Abstract
Princeton Engineering Anomalies Research, PEAR, is one of the pre-eminent paranormal phenomena research groups. A PEAR document, "Information and Uncertainty in Remote Perception Research" (PEAR, 2002), describes a summation of approximately 25 years of PEAR data gathering and research into remote viewing (RV), or clairvoyance. A considerable amount of statistical mathematics in the paper precedes a conclusion that there is ultimately very little, if anything, in the PEAR data that supports RV. Following this there is an unusual and contorted attempt to explain away these results and to re-assert preconceived results using reference to mysticism, ancient divination techniques and Jungian psychology.
This critique provides an overview of the re-analysis of the mathematics employed that reveal statistical anomalies, an outline of how the data and results have been presented, and a commentary as to the possible motives behind the presentation method.
Who is PEAR?
From their website, this is a short description of PEAR:
"The Princeton Engineering Anomalies Research (PEAR) program was established at Princeton University in 1979 by Robert G. Jahn, then Dean of the School of Engineering and Applied Science, to pursue rigorous scientific study of the interaction of human consciousness with sensitive physical devices, systems, and processes common to contemporary engineering practice. Since that time, an interdisciplinary staff of engineers, physicists, psychologists, and humanists has been conducting a comprehensive agenda of experiments and developing complementary theoretical models to enable better understanding of the role of consciousness in the establishment of physical reality."PEAR and its founders have impeccable scientific pedigrees, and the aims of their research are certainly admirable. However when we look at the financial support situation, it gets a little murkier, as we shall see later.
Review of the Statistics Situation
Already there have been criticisms of the PEAR data and data-gathering methods used to accumulate the database on which this paper is based. A notable and relevant document is "Critique Of The Pear Remote-Viewing Experiments" by George Hansen, Jessica Utts, and Betty Markwick, in the Journal of Parapsychology, 1992. These are some of their critical points to be considered with regard to the quality of the initial PEAR data:
"Dunne et al. (1989) report that 211 of 336 formal trials (63%) were in the "volitional" mode. For these, there was no random selection of the target whatsoever. This is virtually unique in modern parapsychology. For the remaining 125 trials, only in the most recent report is there any information as to how the random selection was made. The information provided is very limited and does not meet the reporting guidelines recommended by Hyman and Honorton (1986)."Further statistical issues became apparent. The controversial PEAR "Subject 10" seemed to have a much higher hit-rate than the other subjects, greatly improving the overall "success rate" of the data. But there were questions raised as to the independence of that subject’s data and mode of participation, particularly that the subject had repeatedly been both a percipient (remote viewer) and an agent for a number of other percipients. When adjustment was made for just that one anomaly, the "amazing RV result" was reduced to a much more evenly spread and therefore possibly random set of data. The recalculation of the PEAR data puts the result barely on the border of any significance at all.
(Hansen Utts Markwick, p101, Methodological Problems - Randomization)"Virtually no precautions are described that would preclude the agent’s and percipient’s coming in contact, or a third party coming into contact with each of them, and innocently conveying information about the target. Needless to say, this is not usual procedure in parapsychology."
(Hansen Utts Markwick, p102, Methodological Problems - Shielding of Agent from Percipient)"PEAR’s methods made it easy to cheat. Without the use of randomly selected targets and adequate shielding of the agent from the percipient, it is virtually impossible to detect even simple trickery. Parapsychologists have long been aware of such problems and have issued strong warnings…We should point out that we have no reason to think cheating actually took place in the PEAR research."
(Hansen Utts Markwick, p103, Methodological Problems - Potential Cheating by Subjects)"As Kennedy (1979b) pointed out, the percipient might systematically avoid describing features of previous targets. This can introduce a severe artifact... This is a serious problem in the PEAR analysis because the mismatch distribution is used to assess the significance of the correct matches. If a percipient avoids descriptors in one trial based on feedback from previous trials, then the mismatch scores could be artifactually deflated."
(Hansen Utts Markwick, p104, Statistical Issues – Dependence Due to Target Selection Method)"Thus, in the analyses, responses are compared with some targets that were not available for actual random selection for that trial. This can introduce an artifact in the baseline."
(Hansen Utts Markwick, p104, Statistical Issues –Target Pool Definition)"
"However, according to their Tables E and F, Subject 10 contributed 77 trials as percipient and 167 as agent, for a total of 244 trials (i.e., over 70% of the formal trials). Because the procedures allow deception by either percipient or agent acting alone, the contribution of that subject should be considered. If we remove Subject 10’s trials from the set, the z score drops from 6.355 ( p = 1.04 x 10-10 ) to 2.17 ( p = .015, one-tailed)"Quoting the Hansen Utts Marwick conclusions entire best puts the statistical summary of this particular critique.
(Hansen Utts Markwick, p103, Methodological Problems - Potential Cheating by Subjects)
"The PEAR remote-viewing experiments depart from commonly accepted criteria for formal research in science. In fact, they are undoubtedly some of the poorest quality ESP experiments published in many years. The defects provide plausible alternative explanations. There do not appear to be any methods available for proper statistical evaluation of these experiments because of the way in which they were conducted."Naturally, PEAR has disagreed with this critique extensively, and has responded in force with a statistical battle de luxe that is too complex to include here. However, their most recent paper, which is the subject of this critique, is the latest in their responses to the above criticisms. And deep within that PEAR paper, right after the final and most searching analysis results were posted, is the following admission that the results simply did not appear:
(Hansen Utts Markwick, p107, Conclusions)
"Once again, there was reasonably good agreement among the six scoring recipes, but the overall results were now completely indistinguishable from chance."So what had gone wrong?
(PEAR, p227, Distributive Scoring)
Shapes in the Clouds
It is now well documented that the human mind is capable of finding patterns in a huge variety of situations. In fact, it is actively seeking recognisable patterns in the environment, in an attempt to make sense of it, to take advantage of it. This particular subject is far larger than can be dealt with in this critique, and in any case it is has a lot of supporting evidence to support it. So I will concentrate on that subject with specific reference to PEAR’s analyses of their data. The point being made here is that the human mind’s search for patterns is innate to us as humans – we have this ability in us, and it is relentless – it is almost impossible for us to avoid the process.
With this in mind, consider these two actual and unretouched pictures of clouds, courtesy of CloudClub. Our human minds, in search of patterns, will almost inevitably find sufficient pattern information to define "a bunny" and "a dragon". (Although it would probably be more accurate to say that our Westernised minds would allow these definitions to come to the fore – other cultures do not have rabbits or dragons.)
But consider the actuality of these cloud formations – what is really so about them – what is their data? Do they contain something actually physically different from any other clouds that make them form shapes recognisable to us? In all sensibility, any factors affecting cloud formation create clouds that are no different to the innumerable others that form all the time in our atmosphere. It is just that these particular ones look like a bunny and a dragon to us because the shapes are familiar to us. There is no reason or evidence to show that the formation of a "bunny cloud" or a "dragon cloud" is any more or less likely than the appearance any other shape of cloud. In reality, the shape we perceive is purely the result of a desire, a search-for-a-pattern, in the human mind, not any inherent property of the cloud formation itself.
So it is possible to say that in almost all cases, a pattern of some sort that we recognise can possibly be seen be seen in just about any ambiguous data situation. It would then be reasonable to conclude that such pattern seeking in the face of the ambiguity of the data will be an artifact and very highly subjective.
And this is what has happened with the PEAR data, and seems to fully explain the results they obtained. The initial "positive" effects were obtained as the result of some highly subjective and high-level judging of the data sets. This is the equivalent of seeing a shape in the clouds. But as the analyses were refined, the positive effects tended to be less visible, in the same way that the closer you look at any cloud, the less it looks like any recognisable shape at all. The ultimate result for PEAR was that the "final analyses" revealed no evidence in support of the initial positive results at all, the equivalent of getting down to droplet level in a cloud where its total shape becomes meaningless. In effect, the PEAR analyses changed from being subjective, with good results, to objective, with none.
In most situations, improved analysis of data will tend to magnify an effect, if it is there to be found. Refinement of the testing and analyses throws the focuses more and more on the reasons and/or components that create the positive effect, and they become magnified. However, if the effects tend to disappear with such refinement, it can be reasonably assumed that either the refined testing did not take all available data into account, or that the effects were imaginary or an artifact. PEAR sieved the whole 25 years’ worth of data in this exercise, and used a number of methods of analysis to try to capture the intended effects, so all the available data was in play. Therefore it becomes a bit difficult to escape the conclusion, given their stated results, that PEAR were actually chasing something subjective - imaginary - from the outset.
In other words, there was never any RV effect at all to be found. They just saw a bunny in the clouds!
But Why Overlook the Obvious Answers?
As a person with some scientific training myself, one of the first things I noted after reading this PEAR paper and seeing the results was that there seemed to be some obvious gaps in it. To me, there seemed to be a leap from analysis and results straight to some fairly esoteric and, at first glance, unrelated rationalisations for the results. Two issues here stand out for me.
The first issue has already been described above: bad experimental method and poor data. It is long beholden on all experimenters to ensure that they actually do try to obtain the best data they can by designing the best process they can. But what is more, when no measurable results materialise, they must consider seriously in their discussions if the process and the data really were the best they could obtain. That is, to be highly self-critical of their own process and thus the data obtained. Certainly the peer review process will provide that criticism of the methodologies employed if the research is released publicly, as we can see above.
However, I would describe the results in this particular paper far more succinctly as "garbage in, garbage out". The failure to critically evaluate the data gathering process, and thus to evaluate the quality of the data to be examined, rendered any results pretty much moot right from the outset. But what is worse, failure to even allow that the data were poor, to push ahead with it anyway, and then to not discuss possible problems in this area in this paper, seems to be a deliberate reluctance to address the point. One starts to wonder why.
For example, on page 237 of the paper, the following startling admission is made, seemingly inadvertently:
"Recall, for example, that the early exploratory trials, where percipients did not know the identity of the agent or the time of target visitation, produced completely null results…"This suggests straight out that all trials that produced "results" were conducted either by percipients and agents that knew or were in contact with each other, and/or where the trial times were known in advance. As mentioned above, without safeguards against interactions that might affect the outcomes, including possible connivance, the trials must surely have been highly suspect. However this was not raised or discussed in the analysis phase, even though it was plainly admitted. We are left to wonder to what extent the basic rules of "properly independent" scientific testing of phenomena had actually been avoided.
(PEAR, p237, Section X – From Analysis to Analogy)
The second issue is an even simpler one, especially if the experimenter is still sure that the data are reasonable and fairly obtained, and properly processed. That is, that there is nothing there to be discovered in the data anyway – no patterns, no correlations, no startling discoveries. Failure by the experimenters to consider this as an acceptable result indicates that the PEAR team had a predetermined result in mind instead. The intent of their "experiment" seemed not to examine the hypotheses critically but simply to find some support for this predetermined result. In effect, it was doing the science backwards, which is hardly likely to inspire confidence in any results so presented. It is also ethically questionable. The subsequent section of the paper lends support to this possibility.
Falling Back on Mysticism, Anecdotes and Jung
I note something else quite interesting about the overall shape of this paper. It starts out in a highly mathematical and statistical mode – tables, statistical procedures and the minutiae of analysis abounds. The improved search for statistical significance even reads like a mystery novel! And then the denouement is reached, and alas, the analyses reveal nothing to support RV at all. And suddenly the whole tenor of the paper changes in a paragraph – it is as though someone else entirely had written the remainder without reference to the first part at all. Or that they had read it, and wanted a different ending to the story.
Suddenly we are in the world of Jungian psychology, the mysticism of the ancient Chinese, the anecdotes of startling psychic phenomena during the testing processes years ago, and so on. And this new discussion is trying to draw the conclusions that RV really does exist in some fashion, despite the results to the contrary from only a few paragraphs prior. Here are some samples of this switch – note the underlying assumption that RV does indeed exist, and that the explanation is all that is required:
"Having exhausted the search for the source of the remote perception signal deterioration in the analytical techniques themselves, we were driven to look further afield for a satisfactory explanation."The more I read this section of the paper, the more I was drawn to ask why this line of analysis was being used by way of "explanation". Here was a reputedly science-based body conducting reputedly valid statistical analyses on what, at first, I took to be acceptable data sets, and yet here was this plethora of unscientific and irrelevant excuses being generated to explain away the results obtained at the front of the paper. It was most puzzling in its intent.
(PEAR, Opening sentence, p231-232, Section X – From Analysis to Analogy)"Insights can also be derived from a quite different realm of human experience, namely, the practice of certain mystical divination traditions where anomalous relationships between signal and noise are also evident…
One such example is the renowned Oracle of Apollo at Delphi in ancient Greece, a highly respected source of wisdom that long played a central role in Greek culture and politics. Consultation of the oracle involved a priestess called the Pythia who, crowned in laurel and in an altered state of consciousness stimulated by vapors arising from a cleft in the earth over which she sat on a tripod, produced a "free response" utterance, which was then interpreted by the attending priest in response to the seeker’s query.
Notwithstanding the subjective nature of the interpretation of the texts, a vast body of evidence accumulated over many millennia testifies to the efficacy of the I Ching in producing accurate and consequential results. Despite the claim of many rationalists that such oracles are nothing more than bizarre combinations of wishful thinking and "mere chance," this is the same "irrational" formula that seems to underlie the remote perception phenomena that have now been demonstrated, by rigorous analytical quantification, to convey more meaningful information than can be attributed to "mere chance." Hence the principles invoked by the ancient sages in developing the I Ching may shed some light on these more contemporary anomalies.
Psychologist Carl Jung, who devoted more than 30 years to the study of the I Ching, pointed out … that "we know now that what we term natural laws are merely statistical truths and thus must necessarily allow for exceptions… Yet, Jung’s model, the ancient divinatory traditions, evolutionary theory, contemporary signal processing research, and human/machine anomalies all suggest that noise may be a requisite component of the process of signal generation, and that objective linear causality may not prevail under these circumstances.
(PEAR, p233-235, Section X – From Analysis to Analogy)
Excreta Tauri Cerebrum Vincit?
Then there was another factor that I found hard to explain. While I do understand that each branch of science generates their own jargon, and that some of this is probably rather obscure to the uninitiated, this paper seemed to be burying itself and its conclusions neck-deep in utter polysyllabic terminological obscurity. In fact, I felt very much in need of a decoder based on the complete version of Roget’s Thesaurus. I am quite capable of scientific reading comprehension at a high level. However this last section was so densely packed with obscurities and jargon, that, by the end, the impression was that an original, clearer draft was prepared and then put through some sort of buzzword-generating program in order to deliberately obscure what was really being said. But significantly to my mind, this was done only in the opening precis, the sections where the results are revealed, and in the subsequent discussion section.
So the question that came to mind was: Why? And: Why those sections only? Why so comprehensively obscure such a paper more than it needs to be? Surely the intention of any such scientific paper is to convey clearly and concisely to its audience the work done, results obtained, and the subsequent discussion? The alternative is that there was some other driving force besides pure scientific publication at work here, dictating that obscurity actually is a necessity, the order of the day…
One such reason could be that it is used to simply skate around the tough bits that require explanation, to touch only lightly on them, or to simply avoid saying anything at all about them. It is the sophisticated way of the old "padding your essay" with superfluous words when you have nothing to say - to simply bury the reader in trying to decipher the words, while, in total, the plot is really thin and weak. Again, the question is: Why?
Another reason is to use buzzwords to impress the reader – the venerable excreta tauri cerebrum vincit approach. But, on the basis that scientists could decipher the verbiage, this would only apply successfully to non-scientific readers. Which would suggest that scientists were not the actual target audience in mind at all for this paper. If not, then who?
So Who Dunnit?
So let us look finally at PEAR again, and perhaps more specifically at their listed supporters: "Institut für Grenzgebiete der Psychologie und Psychohygiene, Lifebridge Foundation, Richard Adams, George Ohrstrom, Laurance S. Rockefeller, and other private donors."
Why try to impress them in this way? Let me posit a simple answer: the money. Consider the alternative. If PEAR had indeed published a summary of 25 years of remote viewing data and allowed the null conclusion to be clearly visible then their stream of support funding from their supporters becomes in jeopardy. However if this result could be buried in gobbledegook and made up to look like research was ongoing and yielding results then the income stream is protected. The impression on the rest of the scientific world is really secondary – the Hansen Utts Markwick paper is not really a major issue. The sinecure of retaining PEAR’s continued existence and funding, not to mention prestige, would probably count more highly.
This premise seems to fit all the available data quite well – the thorough analysis with a hidden result, the failure to examine the data sources and processes, the elaborate diversion towards metaphysics, the hiding of simplicity in the over-elaboration of words. Of course, this is my own highly speculative analysis of the situation. And I expect to be roundly criticised for it too!
References
