I watched a documentary called Mr Tornado - a scientist, active after WWII through the 80's, who made contributions to the science of understanding things like tornados and microbursts (responsible for a number of airline crashes).

I'm wondering, what does it say about peer scientists who reject a new theory or hypothesis without basis, and it turns out to be true. As a scientist, seems like one should keep an open mind until data is suggestive. How much of a scientist is one to reject a thought that is later proven true?

I wonder how many advancements are lost because of naysayers who are wrong....

Physics and astronomy are full of theories that were initially immediately dismissed by other scientists and later found to have merit years later.

An interesting question that surfaces, or should surface in any examination of the history of science. I’m not certain how journalists discovered the term “peer reviewed” in reference to a scientific theory, study, or report; perhaps it’s something that the scientists themselves take pains to ensure is mentioned one way or the other to reporters, but I always tighten up a little inside when I see it because I believe it confers a presumption of reliability and validity that is unwarranted. It has almost become a sort of shibboleth that is intended to prove without further details or even thought that the results should be accepted as being valid. On the other hand, if the product has not been peer reviewed, then it should be considered suspect, or at least viewed with caution.

In fact, though, the peer review process hardly guarantees that the claims are valid. As an article by the University of California Berkely points out, “Peer-reviewed work isn't necessarily correct or conclusive, but it does meet the standards of science.” And even the claim that it meets the standards of science is an opinion rendered by other scientists.

In fact, peer-reviewed or not, countless articles and other reports advance ideas and claims that are not accepted by the science community as a whole but are hardly proved to be invalid merely because of that skepticism. I think back to my junior high school days when the theory of plate tectonics was being advanced and considered. It was known as “continental drift” at the time and although hardly any geologist questions the theory now, at that time it was still controversial, and was not accepted by everyone. As a contrary example, some years ago scientists claimed to have found evidence of life in a meteorite from Mars. That claim was strongly challenged and I don’t see it even mentioned in discussions about the subject any longer.

In reading science journals in recent times, I’ve found that it’s almost universal for any controversial scientific theory or other claim to be strongly challenged by other scientists. Recently I watched a video about the origin of the disease syphilis and the theory it was introduced to Europe from the Americas by early explorers such as members of Columbus’ crew. One scientist claimed to have found evidence that it existed in England before Columbus’ voyage of discovery, but when it was presented to another researcher who has focused his investigations on the disease and concluded that it originated in the New World, he was highly skeptical and opined that the other claim was simply flawed in some way: Two “experts” with totally different conclusions about an either/or question.

The point of all this is that scientists are people and therefore prey to the same sorts of things that affect everyone’s thinking. They may be more resistant to certain types of diseased thinking than most of the rest of us, but they are hardly totally immune. There have been any number of examples of even well-respected scientists’ rejecting good ideas or accepting bad ideas due to poor reasoning.

But none of that means the science baby should be thrown out with the bath water. It should be obvious to anyone that Western science (and there’s hardly any other kind) has changed the world in ways that would have been literally unimaginable even a century ago, much less further back in time. That has happened because of the validity and robustness of the scientific process, and to reject or even question it as a whole is far more senseless and harmful to progress than science’s weaknesses or occasional failings.

“Peer review” of a study or paper is actually very simple.

It just means that a few people reasonably competent in the field (“peers” of the author) have looked at it and don’t see any obvious problems with the starting point (mathematical axioms, data set and method of analysis, experiment design, whatever, it depends hugely on the specific topic) or with the chain of reasoning that leads to the conclusion.

It very specifically does NOT mean that the reviewers have redone any of the work/experiment/analysis themselves to try to reproduce or verify the result, and in fact, in papers/studies based on existing data or data produced by experiment for the paper/study, the reviewers would not generally have access to the raw data.

It’s basically the minimum possible process for catching most of the papers/studies that are steaming piles of crap.

A paper/study that passes peer review can still be wrong for any number of reasons.

“Peer review” is the standard for getting a paper/study published in any even semi-reputable scientific journal or conference, and the peer review is organized by the journal or conference.

The scientific community doesn’t take a peer reviewed publication as absolute truth, because they know there are a million reasons a peer reviewed publication can still be wrong.

It’s kind of the opposite (as you mention) - they tend to see NOT-peer-reviewed publications as not worth even looking at. The scientific community is very publication-driven. If you have a publication that says anything remotely significant and it isn’t peer reviewed, well, it must be because it’s garbage that couldn’t possibly pass peer review, because otherwise, it is unthinkable that you wouldn’t have published it in a peer reviewed journal or conference.

Having reviewed a few papers prior to publication, and having papers reviewed, I understand a bit more about the efforts of technical paper authors.

In general, publishing papers leads to greater job security and potential for advancement.

Restating the known in a new way cannot be accepted, so a new notion or concept is required.

Referencing the known, and adding to the subject knowledge is not easy. The subject is often already well-understood, and creating new insights require research and funds and time.

Reviewers normally are charged with evaluating the paper, and giving anonymous feedback via the journal’s portal. In my experience, there was no easy way to reject the foundational premise, but there were ample opportunities to challenge the data presented, the relationships suggested between cause and effect, etc.

In our industry, reviewers are most often helpful and engaging, to allow the submitter opportunity to correct or clarify the info.

“ It’s basically the minimum possible process for catching most of the papers/studies that are steaming piles of crap.‘

^^^^^^^^^^^^^*

Examples of this happening and two Nobel prizes awarded.
I remember the first one in the news in the 80's. Two doctors that went to extraordinary lengths to prove their theory.


The second one was a little closer to home for me. In my teen years, I mowed the lawn at a small summer house in my town. The owner was the recipient of the longest delayed, time of discovery to award, of the Nobel prize. 1910 to the 60's. Until the others of his era died off or retired, many wouldn't admit what he discovered was true. Viruses causing cancers.

Cause of common ulcers. They were treated as being caused by stomach acid for decades when a common antibiotic was able to easily cure them. Before that, drugs and eventually surgery was the treatment.

https://www.jyi.org/2005-febru...acteria-cause-ulcers

Virus causing cancers.

https://www.jyi.org/2005-febru...acteria-cause-ulcers

Then, my all time favorite, the incorrect interpretation of data. How could that be wrong when the answer seems so obvious?

https://www.trevorbragdon.com/...-the-wrong-solution/

quote:

Originally posted by 220-9er:
Then, my all time favorite, the incorrect interpretation of data. How could that be wrong when the answer seems so obvious?

I ran across that report just recently and was also mystified that something so obvious would not have been recognized immediately.
But then I also just listened to a video about how people can be highly educated and reach very influential positions, but can still be stupid. I have seen that so often, but it was interesting to hear it be discussed.
https://www.youtube.com/watch?v=3O9FFrLpinQ

quote:

Originally posted by sigfreund:

quote:

Originally posted by 220-9er:
Then, my all time favorite, the incorrect interpretation of data. How could that be wrong when the answer seems so obvious?

I ran across that report just recently and was also mystified that something so obvious would not have been recognized immediately...

Probably because of the fog of war.

sigfreund's mention of plate tectonics highlights what is, I think, a common misperception. Scientists might reject a hypothesis, but that doesn't mean they ignore it or reject all of it. A hypothesis can have validity and still be wrong. If the hypothesis is tested and found to be lacking, the theory must be rejected and a better explanation developed. It may sound like it's rejected without basis, but it's more commonly a case of rejecting it because the theory is incomplete.

For example, continental drift (which is not the same as plate tectonics) was and is rightly rejected as a theory because it is demonstrably wrong. Alfred Wegener proposed the theory based on observational evidence that the continents have moved over time. That part is correct, and with GPS we can now measure that movement with considerable accuracy. What was not correct, and what got the theory rejected, was his proposed mechanisms for the physics behind it. Wegener never could come up with a plausible mechanism for the movement because he was tied to the idea of the continental crust breaking through the oceanic crust like an icebreaker cutting through sea ice. That would take a tremendous amount of force, and no satisfactory explanation for this force was offered.

The geologists of Wegener's time did not dismiss all of his observations, but they did reject the theory of continental drift that he proposed. They took his observations and essentially said, "He's on to something here, even if he can't explain it." The modern theory of plate tectonics was developed with Wegener's rejected work as a foundation. Some of the "rejected" theories that we read about today follow the same pattern. They are not accepted because there are parts that do not pass the testing phase of peer review. They need to be refined.

Geologic pontificating aside, theory refinement doesn't address every situation that is discussed in this thread. Scientists are people and people have egos. Especially successful people who know they are at the top of the cognitive pyramid. Ego and tradition have undoubtedly caused a lot of good theories to be dust-binned, to our loss.

Good examples and info.

From the ulcer example: Seems like the idiots and NIH have been around for decades.

This seems especially rich, even for academics; essentially, it seems, changing incorrect documentation is a barrier to truth:


Reclassification, according to Thagard, "involves moving a concept from one branch in the tree of concepts to another branch" - steep barriers to reclassification inhibit such changes.

quote:

Originally posted by sigfreund:
But then I also just listened to a video about how people can be highly educated and reach very influential positions, but can still be stupid. I have seen that so often, but it was interesting to hear it be discussed.
https://www.youtube.com/watch?v=3O9FFrLpinQ

Democrats can be officially classified as Stupid People by this theory.

I've become a big fan of Bret Weinstein and Heather Heying. Both are scientists (biologists). (Their youtube is DarkHorse)

They state that peer review is not how science is done, peer review is how academia is done. They don't have much nice to say about peer review.

quote:

Originally posted by jaaron11:
sigfreund's mention of plate tectonics highlights what is, I think, a common misperception.

Thanks for that explanation.

At the time I referred to I wasn’t aware of the specific reasons for the skepticism of some geologists, but at the level I was being taught, it was clear that there was skepticism of even the basic contention that land masses moved over time, regardless of why. I still clearly recall one teacher’s pointing out how well the coasts of eastern South American and western Africa seemed to fit together. He believed that just that bit of evidence indicated that the two had been joined at one time, but he also admitted that some authorities did not feel it was compelling; i.e., the shapes of the two coastlines could have been coincidence.

It is good to understand, though, why skepticism and resistance to accepting new ideas that challenge old ones exist. It’s easy to scoff later at someone for having originally adhered to a cherished theory when it’s demonstrated to be wrong much later, but those examples are the ones that make it into the popular consciousness. The unsuccessful challenges to accepted theories aren’t so well known. How many people for example remember or were even ever aware of Peter Duesberg’s claim that the human immunodeficiency virus (HIV) didn’t cause AIDS? How about the earlier rejection by some physicians of the theory that cholera was caused by bacteria? How many younger people today have ever heard of the “steady state” theory of the universe? That was still being debated when I first heard of continental drift, but it’s been a long time since I’ve even seen it mentioned, much less advanced.

One of my favorite science-related quotations is attributed to James Clerk Maxwell:
“There are two theories of the nature of light, the corpuscle theory and the wave theory; we used to believe in the corpuscle theory; now we believe in the wave theory because all those who believed in the corpuscle theory have died.”

I had difficulty reconciling light as photons that had the wave-like properties. It was easy to understand the behavior of light as EM waves. I can see how light might be confusing since it exhibits both particle and wave like properties, depending on the experiment.

Established science rejected germ theory for years, decades even, clinging to the idea that miasma caused disease. It happens, and then the evidence becomes overwhelming.

This doesn't mean science is bad, it means that it is done by humans with all the flaws that humans have.

Especially on the topic of climate change, etc, where they know they are absolutely correct and agree with the prevailing scientific community, I like to remind people that the best minds of the time thought that Galileo was wrong when he proposed that it's the earth that revolves around the sun and not the other way around. I offer that as chum and when they bite, I point out that the best minds of the world including Thomas Edison thought that heavier than air flight was impossible even as near as months before the Wright brothers' first successful flight.

quote:

... heavier than air flight was impossible even as near as months before the Wright brothers' first successful flight.

How heavy is air? It's amazing to see a turkey fly. Watching them waddle around so clumsily, it seems like they would never be able to leave ground. It's amazing to see them fly up into trees and glide a good distance.

quote:

Originally posted by konata88:

quote:

... heavier than air flight was impossible even as near as months before the Wright brothers' first successful flight.

How heavy is air? It's amazing to see a turkey fly. Watching them waddle around so clumsily, it seems like they would never be able to leave ground. It's amazing to see them fly up into trees and glide a good distance.

Queue the WKRP clip ...

quote:

Originally posted by Fly-Sig:
I've become a big fan of Bret Weinstein and Heather Heying. Both are scientists (biologists). (Their youtube is DarkHorse)

They state that peer review is not how science is done, peer review is how academia is done. They don't have much nice to say about peer review.

Peer review has its problems, but it serves a useful purpose.

To offer an example, there's an important open question in computer science, "does P=NP?" The exact definition is very technical, but a simplified way to think about it is "are problems where you can easily check if an answer is correct also always problems you can easily solve?"

The reason the question is so important isn't obvious if you aren't pretty familiar with computer science, but it's an important enough question that it's one of the Millennium Prize problems with a $1 million prize for proving the answer.

It's honestly more of a math question than a science question, but the role peer review plays is fundamentally the same.

There are literally hundreds of self-published papers claiming to answer the question. None of them have gotten published in a peer-reviewed journal because they're all garbage.

The peer review process lets the computer science community as a whole say, "We don't know whether P=NP, and until a peer-reviewed paper shows up, I don't have to worry about it or look at hundreds of shitty papers about it. When a peer-reviewed paper shows up, it might still be wrong, but at least I'll know that one is actually worth looking at."

As I mentioned in my earlier post in this thread, the peer review process is not about establishing truth, it is about screening out most of the crap.

As far as the "how science is done" part...

I think most scientists would agree that we can never really know "THE TRUTH" with absolute certainty.

To the extent that we can be pretty confident that something is pretty much true, subject to other stuff that we haven't figured out yet, that isn't the result of the peer review process. It's the result of many independent people repeating the work and reproducing the result.

At its most fundamental, "how science is done" is the scientific method (at the individual scientist/individual experiment level) and reproducibility (at the community/accepted scientific knowledge level).

Peer review is absolutely not an essential part of that, but it sure cuts down the amount of crap that scientists have to wade through.