by Josh Kearns
* * *
Hi – Your erstwhile correspondent Josh Kearns here! As a Chemist Without Borders doing work of on-the-ground relevance, you may enjoy this critical look at the continuing struggle to bring academia into accord with the real world….
* * *
OK, so a paper came out in PLOS-ONE a couple months back where the authors used statistical analysis to predict that the area in the southeastern US covered by suburban sprawl would double or triple by 2050 if we continue development by business-as-usual (defined as a forward linear extrapolation of the past few decades).
“BLECH – more suburban sprawl!!!!” was my knee-jerk first response, typical of most enviro-types like me I reckon.
But then I was like, “Hold on – two to three times the area? That’s a huge area. It would take a vast amount of energy, resources, and capital to build that!”
“And hold on – business-as-usual continuing several more decades? No way! Not happenin’! Resource constraints are already seriously hobbling our economy – how can they assume growth like we experienced during the 1980’s and 1990’s? Fuelled by what? Shale oil? Ha ha good luck with that….”
I’m not in a position to judge the internal details of the authors’ statistical and modeling methods. But I can recognize that the baseline assumption upon which their model is predicated is false. And absurd.
You would think, then, that it wouldn’t make it through peer-review. You would think.
Anyway, this thing was enough of a burr under my saddle that I decided to write the authors, and leave comments on the article’s web page, and even get in touch with the editor. To wit, the exchange is reproduced below.
In a society as afflicted with such poverty of imagination regarding any economic M.O. other than “grow or die” as ours is, hokey baseline assumptions for future scenario modeling made by tame academics is a factor we’ll have to contend with if we hope to meaningfully address our current economic and environmental predicaments.
The first step is rooting out learned stupidity.
Disclaimer: It’s not that I think this particular paper represents the most egregious instance of functional stupidity in academia. Not at all. Not by a long shot, in fact. (See, for example, our Reinvent the Toilet Project. That project was arguably significantly more stupid because it wasted a lot of money and physical resources and was not limited to simply inconveniencing some electrons in a futile computer modeling exercise.)
I pick on this PLOS-ONE paper because it’s pretty straightforward to argue that the conclusions are totally meaningless since the premises upon which the modeling was based were false (notwithstanding however technically rigorous the subsequent computations were). If a dumb hillbilly like me can see that much, then it should’ve raised some red flags among the reviewers, if not the authors themselves. Therefore, carping on this paper serves as an effective foil to underscore the point that absurdities can sail through the peer review process without a hitch as long as they conform to prevailing prejudices.
And of course this particular kind of blindness isn’t limited to academia, but is pervasive across all sectors of our society.
* * *
Hi XXXXXXX (editor’s name redacted) –
I am writing in response to an article recently published in PLOS-ONE entitled “The Southern Megalopolis: Using the Past to Predict the Future of Urban Sprawl in the Southeast U.S” by Terando et al. for which you are listed as the editor.
This paper has some methodological flaws that deserve highlighting. In fact, a fundamental premise of the study is false. I have raised these concerns with the corresponding author as well as in the paper’s comments online section at the PLOS-ONE website, but – perhaps predictably – have not gotten any traction in so doing. This is an interesting case to consider however, as it provides an illustration of how study premises that are in reality absurd or impossible are often fairly likely to pass peer review if they conform to prevailing prejudices.
The paper is based on the stated assumption that, “For fast growing regions such as the Southeast US, the most relevant scenario for conservation and adaptation planning is the “business-as-usual” (BAU) scenario in which the net effect of growth is in line with that which has occurred in the past.” However, here the author’s have committed a version of Bertrand Russell’s “turkey fallacy.” Being fed by the farmer every morning consistently for several months, the turkey contentedly but wrongly extrapolates the happy circumstances of the recent past far into the future. The turkey is correct, until Thanksgiving.
The authors conclude that, “Our simulations point to a future in which the extent of urbanization in the Southeast is projected to increase by 101% to 192% [over the next 50 years].” However, they appear not to have tallied the (affordable) energy, material, and financial resources that would be required to deploy suburban sprawl over 2-3 times the current area throughout the region, or inquired whether it is realistic to expect that the required growth in these resources will materialize over the time period in question.
To illustrate with one example: the extensi犀利士
ve deployment of the suburban armature over the preceding few decades has been keyed to dependence upon an increasing supply of affordable, high-energy-profit-ratio (or energy return on investment, “EROI”) oil. The last major discoveries of such high EROI oil – the North Sea, and the north slope of AK – were brought online in the late 70’s/early 80’s, and their production is now winding down. Today’s “tight oil” (e.g. shale deposits) does not qualify as high EROI since it is so much more expensive in both energy and monetary terms – and thus will not support the same rate of growth in infrastructure and finance as yesteryear’s high-EROI conventional “light, sweet” crude. In addition to the declining energy profit ratio of oil (the “master resource”), other factors of resource depletion, climate destabilization, unsustainable debt accumulation, and the impingements of bio-physical limitations to continual economic growth make projections of growth “in line with that which has occurred in the past” (i.e., the “turkey fallacy”) increasingly implausible.
ve deployment of the suburban armature over the preceding few decades has been keyed to dependence upon an increasing supply of affordable, high-energy-profit-ratio (or energy return on investment, “EROI”) oil. The last major discoveries of such high EROI oil – the North Sea, and the north slope of AK – were brought online in the late 70’s/early 80’s, and their production is now winding down. Today’s “tight oil” (e.g. shale deposits) does not qualify as high EROI since it is so much more expensive in both energy and monetary terms – and thus will not support the same rate of growth in infrastructure and finance as yesteryear’s high-EROI conventional “light, sweet” crude. In addition to the declining energy profit ratio of oil (the “master resource”), other factors of resource depletion, climate destabilization, unsustainable debt accumulation, and the impingements of bio-physical limitations to continual economic growth make projections of growth “in line with that which has occurred in the past” (i.e., the “turkey fallacy”) increasingly implausible.
I suggested this to the corresponding author in a personal communication. His response included the statement, “we believe it’s reasonable to first give folks an idea of what patterns of growth may look like if we continue our current policies and preferences.” Implicit here is that continuation of “our current policies and preferences” is a possibility, when it is in fact not a possibility. In a society where there is widespread lack of understanding of and the political will to deal with bio-physical limits to continued expansion of the human economy (a lack that extends right up to the highest levels of government, business, and academia), this suggestion by the authors sends an incorrect and unhelpful message.
Realistically, if planners and officials believe that BAU is possible, they will pursue it. The authors’ agenda of modeling based on BAU and then pointing out the dreadful ecological implications of the resulting sprawl is an emotional/moral appeal. The environmental/climate movement has long used this strategy with little success. Could a more compelling reality-based argument be made by demonstrating that that, due to high costs, insufficient EROI, accumulation of un-repayable debt, Ponzi finance dynamics, etc., BAU and the linear extrapolation of the suburban sprawl economy decades into the future is a physical and economic impossibility – and, therefore, we don’t actually have a choice whether to develop along a path that’s substantially different from BAU?
Rhetorical concerns aside, if the baseline condition (BAU based on the past few decades linearly projected forward 50 years) used by the authors in their modeling exercise is physically and financially impossible, then the output of their model is not meaningful. Consideration of the vast quantities of rapidly depleting energy and resources, as well as finance capital (itself dependent upon net energy and surplus wealth), that would be required to deploy suburban sprawl at 2-3 times the current expanse in the SE US should have instigated a “gut-check” regarding the plausibility of this modeling exercise – if not by the authors themselves, then by reviewers and PLOS-ONE editors. That – apparently – no “gut-check” was performed by any of the parties involved signifies the extent to which the idea of infinite growth on a finite planet is taken for granted, even among members of the intellectual class ostensibly concerned with “sustainability.”
In this analysts’ opinion, in light of these issues and oversights the paper should be retracted, and an explanation for the retraction published in PLOS-ONE. This would help to shift the conversation towards the reality-based domain and away from the unhelpful mythology of perpetual “progress” and growth.
In this analysts’ opinion, in light of these issues and oversights the paper should be retracted, and an explanation for the retraction published in PLOS-ONE. This would help to shift the conversation towards the reality-based domain and away from the unhelpful mythology of perpetual “progress” and growth.
Regards –
Josh
I received the following response from the editor:
Hi Josh,
Thanks for the thoughtful email.
The crux of accepting this paper revolved around this explicit criterion at PLoS One: “Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.” I feel that the technical side of the paper is sound. Your criticism is of an assumption, but that (to me) does not affect the technical standards of the model. As such, I found this manuscript met criteria for publication in PLoS One.
Wow! I just couldn’t leave that one hanging out there… So I responded:
Fascinating!
Indeed my comment does not pertain to the technical standards of model application, but only to the relevance and actuality of the conditions upon which the model is predicated.
I have a friend who is seeking to publish a statistical analysis of superb technical standards and described in ample detail regarding the metabolism of alfalfa by unicorns (based upon extrapolation from zebra data). I will let them know that they should submit the study to PLOS-ONE!
To which the editor replied:
It is a strange standard – and why you see such high variance in paper quality there.
Well, that’s quite an admission.
So, basically, the editor agrees with me but cites the journal’s editorial policies as the root of the problem and – what’s more – as constituting a block to any real corrective action.
This seems to be an instance of functional stupidity at the level of the academic enterprise (or at least at the level of one of the top journals). This concept was introduced by Alvesson and Spicer in a 2012 paper in the Journal of Management Studies entitled “A stupidity-based theory of organizations.”
According to Alevsson and Spicer
Functional stupidity refers to an absence of reflexivity, a refusal to use intellectual capacities in other than myopic ways, and avoidance of justifications.
It’s functional in the sense that it contributes to the stability of the organization or enterprise. It’s stupid in the sense that it involves willful or unconscious intellectual impairment – for example, that stemming from “group-think.”
What’s a contemporary example of group-think? How about the pervasive unwillingness to question the pursuit of infinite economic growth on a finite planet, for starters?
Alvesson and Spicer’s concept of functional stupidity has most obvious relevance to the corporate world. But they readily acknowledge is applicability in academia as well:
Functional stupidity as a general condition that pervades many spheres of social life, including academia. Contemporary academia could be seen as a hothouse for functional stupidity. In academia, huge amounts of time and energy are expended on writing papers for publication in top ranked journals, in our bid to ‘play the game’. These papers may be read or used by very few, and mainly by those eager to pad out the reference lists attached to their own papers. Rarely is there any serious discourse around the meaningfulness of this enterprise, apart from occasional debates about ‘relevance’. Perhaps this is because publications are not only a measure of our ‘market value’ but also are seen as an expression of our intelligence and knowledge. The result of an article being accepted for publication can be a deep sense of satisfaction and strong identity-confirmation, simply because it ‘proves’ how smart we are. Of course there are material rewards, but these are often less important than the symbolic ones. One could say that functional stupidity is a key resource for any institution eager to maximize careerism. This can make researchers into willing journal paper technicians who focus on writing papers for leading journals within a narrow subfield. This may detract from broader scholarship with slower and less predictable results and, perhaps, with a greater likelihood of saying something really interesting and/or socially useful.
Yeah – what they said.
I agree with your points, Josh. Having recently seen the movie Cowspiracy (http://cowspiracy.com, a must see), it seems that many of our efforts regarding the sustainability of our planet are a waste of time owing to our ignoring the principal contributor to climate change and environmental damage. (I was so disturbed by the environmental impact of what I saw that I decided there and then to abandon eating all animal products henceforth, something I would never have imagined doing beforehand.) This is what statisticians refer to as Errors of the Third Kind, i.e., getting the right answer to the wrong problem (http://en.wikipedia.org/wiki/Type_III_error#Kimball).
Keep up the great work, Josh.
I can't wait to read the unicorn alfalfa article! As a journal author, I always try to suggest the most knowledgeable and competent reviewers. Sometimes that work out well and we get excellent feedback and make significant improvements on the paper. Other times, the reviewer is a competitor and wants to quash our work out of spite. Anyways, fascinating reading Josh!
Steve C.