Making a Mess of Good Research

You may have read my post last month about a research paper that found that degradable polymers in landfills generated methane, a potent greenhouse gas. While I did have a few quibbles about the research, I generally thought it was a good paper, one in which the researchers did not overgeneralize their results or have a bias towards a certain outcome.

Boy was I wrong.

One of the researchers has an opinion piece in this week's Plastic News [1] in which he completely overstates his research results, does not understand the reaction of others to his research, and isn't cognizant of his own biases. Let's start with one of the most innocuous statements

"The foundation of this research is a life-cycle accounting of the greenhouse gas (GHG) emissions associated with discarding waste in both national-average and state-of-the-art landfills. A state-of-the-art landfill collects the generated methane and beneficially uses it. Only an estimated 35 percent of waste is discarded in state-of-the-art landfills, while about 31 percent of waste is in landfills without any gas collection. The rest is in landfills that collect and flare the gas.

The results of this research show that there are significant benefits to collecting and beneficially using landfill gas. Disposing of mixed municipal solid waste in a state-of-the-art landfill is carbon negative, but disposing of similar waste in a national-average landfill leads to positive GHG emissions. The results of this analysis also show that the more degradable a material is, the greater the GHG emissions it generates when disposed in a landfill. The best material to have in a landfill, from a GHG emissions standpoint, is one that does not degrade at all."

I'm fine with every line in that except the second to last one: "The results of this analysis also show that the more degradable a material is, the greater the GHG emissions it generates when disposed in a landfill."

While this certainly was true as a trend in what they found in their research, the data and analysis is nowhere even hinting that such a result is universal for all materials. A trend gathered from the few select materials that they studied is not what is used to make predictions - the data they have is far, far short of what is needed to suggest that it is a universal law. It certainly can be used to make a hypothesis and then further test that hypothesis, but it is not a result, a demonstrable truth, and for the author to make that statement now is unnerving.

"Numerous articles and comments written by anti-environmentalists have tried to use the results to portray environmentalists and environmentalism as naive and/or misguided. This argument is nonsensical when made by those who deny anthropogenic climate change. This research is meaningless if one does not first accept basic climate science."

The last 2 sentences have me completely floored. The results of the research are meaningless depending on what you think about global warming? So is he saying that, if you believe in global warming (as the author does) you get one result, but if you don’t, then you get another result? Or is it more along the lines that evolutionary research results are meaningless to creationists? Either way, I don't understand it.

"The purpose of the research is to allow us to more effectively mitigate GHG emissions by making informed decisions."

There you have it, an admission that this research was undertaken to prove a certain point rather than explore an unknown or test a hypothesis. The author has a bias that he was looking to support. No wonder he calls the headline "Biodegradable products are often worse for the planet" as "reckless". I don't see that headline as such.

Why the author decides to swing so widely from research supported conclusions to wildly broad assertions is beyond me. And for him to somehow try to say that he is the final arbiter of what the research means is even worse. As I stated at the beginning, I read the paper and thought that it was a nice piece of research. I still do. I just can't believe the way it is being misrepresented by its own author, all while attempting to state that others are misrepresenting it.

[1] Don Loepp first raised this issue in his Plastics News blog last week. I didn't realize that the opinion piece he quoted was being published on paper.

[2] I'm reminded of an inventor filing for a patent with overly broad claims that end up getting greatly narrowed at the patent office. As soon as the patent issues however, the inventor starts making the over broad claims again