Our Crude Scientific Method

The process of scientific experimentation and replication is the foundation of our scientific culture. It’s the method used to help decide everything from what pharmaceuticals we should take to determining what is occurring in the environment. It has been used to help show how humans think and how humans are genetically designed. And yet there is one variable that is seldom mentioned in publication- how those results could be influenced by what researchers, and those who closely follow their work, want to see. This is explored in a recent New Yorker article by Jonah Lehrer titled “The Truth Wears Off.”

He shows how several scientific studies had initial dramatic results and how the published replication studies initially supported those findings. Then, over time, further studies showed diminishing results. The so-called decline effect could have many reasons behind it, according to Lehrer. Part of the problem may lie in what studies get published. Nil results are not very exciting. Results that replicate dramatic findings are more interesting and thus more likely to get published. There may also be a problem with the interpretation of testing results by scientists and the very human tendency to seek out the interesting, rather than report findings that are not significant.

Lehrer points to the Kuhnian concept that scientific paradigms grow in stature and then finally get to a point where they are so established they are actively questioned and enough data builds up to dispute the initial finding. At this juncture the results disproving the original thesis become more interesting. If the new experimentation tears down the old theories, a paradigm shift occurs and a new paradigm is developed.

I find these ideas interesting because they call into question our basic way of building knowledge. And it raises an important point. Debate and disagreement are often as important in science as the replication studies. This can be seen clearly in a recent announcement made by NASA scientists, and reported on this blog last week. The scientists reported that they had trained bacteria to grow on a diet of arsenic providing the possibility that organisms could exist in the universe using entirely different biological structures than what we have seen thus far on Earth. This week there has been a firestorm of controversy over the study, with criticism coming from microbiologists at several institutions. The debate is critical. These complaining researchers have examined the study and found flaws in the design, flaws that could call into question the results.

Perhaps our scientific method is too crude for us to even begin to understand extraterrestrial science? People often worry that extraterrestrial visitors would not be able to speak our languages. This seems absurd, considering the technological prowess that would be needed to travel between the stars. One would imagine that advanced technology, combined with some time for research, would allow extraterrestrial visitors to both understand human languages and develop a way to communicate with us. More likely we would have a very hard time understanding their scientific language and thus their technology. It would be like trying to build a skyscraper of knowledge without any understanding of the blueprints.

Perhaps After First Contact we would need to learn a new way of examining the world around us- building a new scientific method? Part of human progress could be the development of artificial intelligence that could conduct scientific experiments, without all of the human flaws of expectation and the need for drama. We could be getting in the way of science, instead of nurturing it.

The way science works here on Earth is a critical issue when it comes to considering the impact of extraterrestrial contact. The sharing of scientific knowledge and technological information is something many people would expect to occur if we ever have a relationship with an extraterrestrial civilization. The question is whether or not we could handle that information. If the extraterrestrials are willing to share, we will need to put together a carefully structured process for how we will receive such information and how it will fit into our system of research and scientific knowledge. It makes one wish that Thomas Kuhn was still alive and that he could lead such an effort in the wake of First Contact. His way of viewing science and the process of discovery is important for this discussion.