About Rationally Speaking


Rationally Speaking is a blog maintained by Prof. Massimo Pigliucci, a philosopher at the City University of New York. The blog reflects the Enlightenment figure Marquis de Condorcet's idea of what a public intellectual (yes, we know, that's such a bad word) ought to be: someone who devotes himself to "the tracking down of prejudices in the hiding places where priests, the schools, the government, and all long-established institutions had gathered and protected them." You're welcome. Please notice that the contents of this blog can be reprinted under the standard Creative Commons license.

Monday, May 27, 2013

Do Republicans have a point in questioning NSF guidelines?

by Massimo Pigliucci

You’ve probably heard of the uproar that has recently been caused by a bill introduced by Texas Representative Lamar Smith, the chair of the US House of Representatives’ Science Committee. Lawrence Krauss has commented on it on behalf of the Center for Inquiry, and so has the Chronicle of Higher Education, the American Physical Society, The Scientist magazine, and Science Insider, among others.

At issue is draft legislation that would require the director of the National Science Foundation to declare that each grant approved for funding is:

i) “... in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;
ii) ... [of] the finest quality, is groundbreaking, and answers questions or solves problems that are of utmost importance to society at large; and
iii) ... not duplicative of other research projects being funded by the Foundation or other Federal science agencies."

Like almost every Republican move in a while (like, years, if not decades), it sounds superficially reasonable, until one digs just a little deeper to uncover the deeply ideological and highly disturbing motives behind the proposal. That Smith’s intentions are anything but benign was immediately betrayed by his request to Cora Marrett, the acting NSF director, asking for specific information about five particular grants awarded recently by the agency. According to Science Insider, Smith wanted comments of the copies of each reviewer, together with the notes of the NSF officers managing the process of peer review. Why? He said that “I have concerns regarding some grants approved by the Foundation and how closely they adhere to NSF’s ‘intellectual merit’ guideline ... the proposals about which I have requested further information do not seem to meet the high standards of most NSF funded projects.”

Needless to say, this is a highly unusual — perhaps unprecedented — move by a US Congressman, and one wonders exactly why Smith feels better qualified than the NSF officers and scientists who have actually reviewed the proposals in question (Science Insider does not identify the research topics with which the five grants are concerned, but one can make highly educated guesses...)? If you look up his bio, you will find that Smith graduated in 1965 from the the Episcopal School of Texas, in 1969 from Yale University, and in 1975 from Southern Methodist University’s Law School. That is, his technical background is as a lawyer, which makes him just as unqualified to be concerned about NSF’s standards of peer review as it makes me unqualified to represent anyone in a court of law.

It should therefore be clear that this is just another attempted Republican hatchet job to undermine the nation’s standards of reason and evidence in favor of their narrow minded ideological agendas. That said, let us take a closer look at the three points above, and see how unreasonable they are when considered on their own merits, as opposed to as thinly disguised trojan horses.

Quite frankly, all three standards seem pretty reasonable to me. The third one simply asks that NSF certifies that the money isn’t going to duplicate efforts. This is already standard NSF procedure, as Principal Investigators (i.e., the scientists submitting grant proposals) cannot submit the same proposal to different agencies except under very specific circumstances (e.g., if they are young investigators at their first attempt at funding), and even then they will have to withdraw their proposals from all agencies but one if they happen to be multiply funded. (A possibility that, given recent funding rates at federal agencies, especially NSF, has ridiculously low priors.)

The second one should be qualified because it contains the clause “of utmost importance to society at large.” As long as we agree that scientific knowledge is in itself important to society, we are good to go. After all, NSF’s very mission is not directed toward practical applications (for which we have NIH, the USDA, the DOE, etc.), but toward basic research. Still, not all basic research is actually worthwhile, or at the very least not worth the money, so setting standards is reasonable.

The first requirement, “to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science” also seems to withstand scrutiny as long as one changes the “and” to another “or” (otherwise one could read the sentence as saying that all research has to specifically aid national defense, which would definitely be too restrictive). Again, this assumes the idea that a nation’s health, prosperity etc. also depends on the accumulation of basic scientific knowledge, an assumption without which the very existence of NSF wouldn’t make any sense.

So why shouldn’t NSF support such standards, discounting the specific and pernicious intentions of Representative Smith? Indeed, NSF — to some extent — already does address these issues. The process of peer review that examines each grant proposal is, at its ideal, designed precisely to guarantee that the funded science is worthwhile by the highest intellectual standards of the relevant community (i.e., the community of scientists in a given discipline). And NSF has for years required Principal Investigators to write a “statement of broader impact” in which they explain why and how their proposed research will benefit society at large. What, then, is the big deal?

The thing is: a lot of funded basic scientific research doesn’t really have a social impact, broad or narrow. Which is why most grant proposals’ statements of such impact are boiler plate stuff copied and pasted from one grant to another (trust me, I’ve done it), and usually can be summarized into a few rather unconvincing points: (1) the grant will fund some student (undergrad, grad or both), who will learn “valuable skills” of a sort; (2) well, it is well known that basic research often leads to unexpected and unplanned important applications “down the line”; and (3) the one usually informally given by scientists when pressed to explain how, exactly, spending hundreds of thousands of dollars and years of research time investigating the sexual habits of an obscure species of butterflies in the middle of Panama will help anything at all: “it’s intrinsically interesting, of course.”

Let’s start with (3) first: I’m not sure how to cash in on the concept of intrinsic intellectual interest, though I suspect there may be some more or less reasonable ways of doing so. But I can tell you that most of my colleagues (myself included, of course!) use that nice phrase as a synonym for whatever we happen to find interesting or challenging. There is, of course, nothing wrong in having idiosyncratic, even arbitrary, taste in intellectual matters — just like I cannot really fault you for preferring milk to dark chocolate, however strongly I feel you are actually wrong in your bizarre preference... But when one is asking a federal agency funded by taxpayers to shell out significant amounts of money one needs a bit more than just “I find it interesting” as a justification.

Which brings me to number (1) above: it is surely of some social value to train a few students in the ways of scientific research (though all too often for undergraduates this amounts to little more than pipette washing). But it also likely isn’t worth the hundreds of thousands or even millions of dollars (depending on the field) of your average NSF grant. Indeed, good small liberal arts colleges manage to give their students a nice taste of what it means to do research with comparatively small budgets, for the simple reason that learning how to do science doesn’t require being involved with cutting edge and/or expensive projects.

We are then left with (2): the contention that basic research is necessary because it leads eventually to applications of direct human interest. This is certainly true, as far as it goes, but you will notice that for a crowd that prides itself in abiding by the weight of evidence, most scientists are exceedingly recalcitrant to provide actual quantitative evidence of such connections. Yes, you do get a number of anecdotal stories, usually the same select few, recycled over and over. Or you get the argument that pretty much any complex example of applied science would simply not have been possible without some basic research preceding it.

While both the anecdotes and the generic basic applied argument are reasonable, they are a bit of a red herring, since they do not actually answer the fundamental question. The issue is not whether basic science can lead to human applications, but how often, and whether NSF’s funding strategy is the most efficient way to get us there. [1] As far as I know (and by all means, let’s crowdsource this), neither NSF as a federal agency, nor certainly individual Principal Investigators, have been making that argument in any detailed and verifiable way.

All of which does not, I hasten to say, justify the latest Republican assault on basic science. Nor does it mean that we shouldn’t have a National Science Foundation. But researchers themselves have for years called for a more rational approach to scientific funding and peer reviewing, which certainly has to include transparent ways of allocating money, and certainly more than just generic, boiler plate statements of “social impact.”

This is a complex conversation to have, and it is not one at which only scientists ought to be invited either. NSF, NIH, the USDA, the DOE etc. are all funded at taxpayers’ expense, and they are all under Congressional oversight — as they ought to be. It is therefore up to scientists to make a good case to the public, and yes to policy makers, that funding basic research is not a luxury, but rather a practical and cultural necessity for an open society. So far, they haven’t been doing a particularly good job, which I think only helps demagogues like Lamar Smith. We really ought to be able to do better than this.

———

[1] Interestingly, there is now a whole field of “translational research” that attempts to deal with this sort of question in a systematic way.

21 comments:

  1. "they haven’t been doing a particularly good job, which I think only helps demagogues like Lamar Smith"

    As much as I agree that the motives behind Mr. Smith's proposal are suspect, maybe it does take a periodic existential threat to provide a good kick-in-the-pants review of what really matters.

    ReplyDelete
  2. If we are sincere about not wanting to "duplicate effort", how about revising patent and copyright laws? Instead of funding research with monopoly rents through patents and c.r., we could *expand* NSF grants and funding. The results from the research would be openly available to all as a condition of the grant. This availability of information would promote greater understanding of the issues involved. This would also eliminate the expensive, Herculean, and ultimately futile task of policing patent/c.r. violations.

    This would also cut down on corruption (e.g. promoting profitable drugs for inappropriate situations), and relatively useless research (e.g. researching yet another new variant of a profitable drug so as to extend the patent, in lieu of research on new drugs for which there is a greater need).

    ReplyDelete
  3. "It should therefore be clear that this is just another attempted Republican hatchet job"

    The opening language often signals a weak inference, and I think that's the case here. Without knowing which specific grants Smith objects to, the argument is an appeal to authority:

    Smith's expertise is the law. The NSF is run by scientists. Therefore, Smith should defer to the NSF on questions of science funding.

    Politicians are supposed to have expertise beyond just the law. No one suggests that the President should defer to experts on all matters related to health care, or the military, or the budget, simply because his formal education focused on the law.

    I'm reluctant to defend Lamar Smith, and it's entirely plausible that this campaign is empty politicking, but without knowing the specifics, it's also possible that he has a valid concern. The NSF receives a lot of taxpayer dollars, and it should be willing to put up with some oversight from elected representatives.

    ReplyDelete
  4. Lamar Smith and Republicans' intrusion into NSF grants is not about saving money. It is about their annoyance over the funding of research, mostly in the behavioral, political, and social sciences, that could come to conclusions that would run counter to their Tea Party doctrine. It's as simple as that.

    ReplyDelete
  5. contrarian,

    these are the five grants Smith singled out:

    1. Award Abstract #1247824: “Picturing Animals in National Geographic, 1888-2008,” March 15, 2013, ($227,437);

    2. Award Abstract #1230911: “Comparative Histories of Scientific Conservation: Nature, Science, and Society in Patagonian and Amazonian South America,” September 1, 2012 ($195,761);

    3. Award Abstract #1230365: “The International Criminal Court and the Pursuit of Justice,” August 15, 2012 ($260,001);

    4. Award Abstract #1226483, “Comparative Network Analysis: Mapping Global Social Interactions,” August 15, 2012, ($435,000); and

    5. Award Abstract #1157551: “Regulating Accountability and Transparency in China’s Dairy Industry,” June 1, 2012 ($152,464).

    > Without knowing which specific grants Smith objects to, the argument is an appeal to authority: Smith's expertise is the law. The NSF is run by scientists. Therefore, Smith should defer to the NSF on questions of science funding. <

    That's exactly right. It is no more fallacious than pointing out that if you have a toothache and you are not a dentist, you call the dentist. Yes, lawmakers have broad overview of NSF funding, as it should be. But this means that they need to be in dialogue with NSF officers about the general scope of the agency, not meddling with specific grants about which they very likely know next to nothing.

    ReplyDelete
  6. But this means that they need to be in dialogue with NSF officers about the general scope of the agency, not meddling with specific grants about which they very likely know next to nothing

    Broad dialogue without any knowledge of the specifics tends to be empty dialogue.

    My problem is with ii) ... [of] the finest quality, is groundbreaking, and answers questions or solves problems that are of utmost importance to society at large;

    Gosh, what researcher would dare claim that his work will be of finest quality...groundbreaking...of utmost importance to society That kind of judgement is usually retrospective.

    Being forced to commit beforehand to extravagant claims like this can only have a chilling influence.

    ReplyDelete
  7. You noted the most problematic aspect of the standards, the use of the word "and" in the first standard, i.e. "and to secure the national defense." As Mr. Smith presumably learned before he went to Washington (I couldn't resist), in the law "and" is conjunctive, not disjunctive. You rightly note this would require all funded projects to be in the interest of securing the national defense."

    ReplyDelete
  8. The criteria are ALL wrong. First of all, as other have noticed, the NSF is the founding agency of last resort. It founds research that is not related to particular applications: the NIH, DOE, DOD, DARPA, NASA, etc. found those. And they are much more generous with their funds. Stuff that the DOD routinely funds would never make even the first cut at NSF.

    Second, NSF founds research in fundamental Physics and Astronomy that could conceivably have applications in the distant future, but clearly not anytime soon. The physics research founded by NSF clearly fails i). It also fails iii), because such borderline research needs to be independently confirmed. There are dozens of (partially) NSF-funded experiments looking for dark matter.

    The CDMS-II dark-matter experiment, for instance, was founded by 17 NSF grants (plus DOE, Canadian and Swiss grants.) Of course, the incredibly sensitive detectors developed by the CDMS collaboration could have short-term applications.

    Clearly the good congressman is only interested in stopping projects in the social/environmental sciences. I am not competent to say anything about the first Award, but about the others, they have obvious relevance.

    Conservation in Amazonian and Patagonian South America is very important: Amazonian South America is one of the battlegrounds where the future of the planet will be decided.

    Conservatives dislike the ICC, but its existence is an important historical fact: understanding it is important to American diplomatic and even military efforts.

    Network Analysis is, at the present, one of the most active fields in complexity research: it is important in Math, Sociology, Economy, etc.

    The regulation of agriculture in foreign countries is obviously of great importance to the number one exporter of agricultural products in the world (the USA.)

    ReplyDelete
  9. Filippo,

    while I agree with most of what you say, I'm not sure about this:

    > First of all, as other have noticed, the NSF is the founding agency of last resort. <

    It was always my first (and often only!) go to when my lab was active (before I turned full time to philosophy). NSF simply has a different mission (to fund basic research) from all the other federal agencies, so I'm not sure in what sense it is the "last resort."

    That said, I find it interesting that most people here are focusing on Lamar Smith and his clearly politically motivated actions while ignoring the major trust of my post: that scientists really ought to come up with better rationales for why the public should finance very expensive research that very often yields no social benefit whatsoever.

    ReplyDelete
    Replies
    1. On "scientists really ought to come up with better rationales": Haven't great rationales for this already been presented, and the actual problem is that a significant fraction (close to half?) of political power lies in those for whom no good rationale would matter (e.g., the Tea Party and creationism believing members of the Republican caucus). Until those numbers in Congress go down, or more people are convinced not to vote for them, the best rationale will fall on deaf ears.

      Delete
    2. > Haven't great rationales for this already been presented? <

      Where? I missed them... Here is the problem, based on a true example. A colleague of mine spends most of his life (and - over decades - millions of taxpayers' money) to study the mating habits of an obscure species of butterflies. When asked, he can't think about any possible application for human welfare at all. Indeed, it is even hard to see what his research may contribute that is worth the effort to the understanding of insect mating systems. When it comes down to it, his best justification is "it is intrinsically interesting," by which he means, as I noted in the main post, that it is of interest to *him*. Is that sufficient, though? I think not.

      Delete
    3. I would have given a better answer: "When Mendel was experimenting with peas, it wasn't realized that his research would lead to modern genetics, the discovery of DNA, and the modern synthesis of evolutionary biology. My butterfly research may not seem important now, but my discoveries could someday be a critical component of biological theory."

      Delete
    4. So you are making an argument from a very weak inference: since doing X *very occasionally* leads to Y, then it is worth spending a lot of time and money doing X? Doesn't seem compelling to me...

      Delete
    5. I don't know how to calculate how much money should go the the "obscure butterfly" scientist (the fact that it is obscure seems interesting right there) vs. how much money should go to a hadron collider (which also needs a rationale) vs. something else. Maybe the butterfly guy is getting the right amount.

      Delete
    6. Massimo,

      I tend to agree with Philip about the “obscure butterfly” guy. Also, you did not point-out the obvious fact that “butterfly man” is not the only one that finds his research “intrinsically interesting.” Because he receives founding from the NSF, the specialists that review his grants also must think that his research is sufficiently interesting. In such cases I tend to defer to the opinion of the experts, since I was trained in a different field.

      What you are asking is to reject the peer-review system that has served science reasonably well until now. Do you think that the system should be abandoned? If so, how is your position different from that of representative Smith?

      Delete
    7. Is it possible that “obscure butterfly” guy research could be important to other research regarding the impacts of climate change on butterfly populations or the impacts of hormone mimicking compounds in the environment? It is rare that a single study or even a small group of studies leads to significant social benefits. Rather whole suits of research over many years can possibly lead to significant benefits. One study by itself (even a pricey one) may not have any obvious societal benefits. But it might stimulate other fruitful lines of inquiry that lead to important discoveries and perhaps beneficial applications. Expanding our knowledge of nature is a worthy endeavor for society. But teasing out nature's secrets is big undertaking and each study only adds a small piece to the puzzle.

      Delete
  10. Massimo,

    I don't think that it would be a good thing if scientists had to justify their projects individually – except for passing the peer-review process of organizations like the NSF. What is needed is a global justification for spending money on pure research, and also to explain why the present system of peer review cannot be abandoned for a more “democratic” system, in which “the people” or their representatives would make the choices at the single-grant level. At a global level, the people's representatives set the total funding for scientific research, of course.

    Clearly, research projects in frontier fields, like dark matter searches, are not justified by practical application, but they should be easy to justify – especially if they are reasonably inexpensive – because of the fundamental importance to our understanding of the universe. More expensive projects – like big particle accelerators - are harder to justify and, in fact, no large accelerator will be built in the USA in the foreseeable future.

    ReplyDelete
  11. Filippo,

    > I don't think that it would be a good thing if scientists had to justify their projects individually – except for passing the peer-review process of organizations like the NSF. What is needed is a global justification for spending money on pure research <

    That is precisely what I’ve been arguing. My example of “the butterfly guy” wasn’t meant to call for individual justifications from scientists to politicians. But I don’t see much along the lines of what you say actually happening. I also think, however, that individual scientists ought to ask themselves why is it worth to do what they are doing on an everyday basis. In distressingly many cases, frankly, I don’t think it is. We need more big thinking and less spending time and money on what amounts to very expensive hobbies.

    > “butterfly man” is not the only one that finds his research “intrinsically interesting.” Because he receives founding from the NSF, the specialists that review his grants also must think that his research is sufficiently interesting <

    Ah yes, but I’ve been on NSF panels, and I know what happens in those cases: people fund projects similar to theirs because they want to reinforce the idea that what *they* do is worthwhile. But that comes pretty darn close to a circular justification.

    > What you are asking is to reject the peer-review system that has served science reasonably well until now. Do you think that the system should be abandoned? <

    No, though it can certainly be improved, and there have been a variety of proposals out there in recent years. But I am asking for both individual scientists and the scientific community at large to come down to earth and think really hard about why society ought to finance their research. At the moment, most of them are simply taking for granted that it is interesting (to them) and therefore valuable (to society). But the link between the two is tenuous at best.

    ReplyDelete
  12. Smith is expressing the layman's suspicion of the closed circle - when he has no choice but to accept that what science needs is what science says it needs - the layman is in no position to determine the issue comprehensively. So he wants to open up the responsibility of science to society - the layman - and to ensure its not a closed circularity of unnecessary funding of the same old same old, or incremental unimportance. There is probably a time and place for that view, and now might be as good as any. At the end of the day scientists will still decide what science needs, so let's hope the public are not bugged by too much propaganda either way to sway them. It's their vote at the end of a very long day and a very winding road. Politics.

    ReplyDelete
  13. The reality of politics is that democracy requires an equal vote for all, and possibly, equal standing under the law for all as well. An equal vote is the last resort, or bedrock, for practical decision making via representatives. Utilitarians, including utilitarian scientists, are motivated by the last resort of simple counting - greatest good as determined by the majority. Does the majority or do individuals, or both, make breakthroughs for the greater good? Utilitarianism is mindless counting, and gambling on whether the conforming majority is best placed. Just concentrate on making breakthroughs away from the status quo of the majority.

    Leave the simple counting & gambling as the last resort of the voting booth, after everyone has had a good old open barney to get all the options on the table, from all sources. Smith is suspicious, and the only way to allay his suspicions is for science to put its options on the table in a way that the public can understand. We simply need more lobbying, and rigorous sorting of options by lay politicians (with adequate resources) in an open lobbying process giving a fair hearing to all. I'm with Smith to the extent of saying get on your bikes, scientists & philosophers, question your own work and the work of the majority, and put some breakthroughs on the table.

    ReplyDelete

Note: Only a member of this blog may post a comment.