* Building a theory of assholism...
* Nate Silver is apparently dangerously wrong on at least a couple of things.
* If Marx had used emoticons.
* The art of living well requires constant practice.
* Apparently, studying economics turns you into a liar.
* Why do so many people so relish the thought of the Apocalypse?
* Has science replaced philosophy? Nah.
* Are there two Higgs bosons?
* The folly of scientism, as described by a scientist.
* On reading weird books in public.
* Begging to differ: when disagreement is reasonable.
* Michael Shermer defends himself from accusations of anti-feminism.
* Is there a (good) moral case for free markets?
* The disreputable state of (much) contemporary art.
From the MBTI article: "Everyone, they posited, fits one of the 16 possible combinations."
ReplyDeleteThis is a tautology, since the combinations are defined by the outcomes of the tests.
Yes, but that isn't problematic, since the authors *posited* that, i.e., they assumed it in order to build the test.
DeleteWhat I mean is that this is a useless fact because it is content-free. The tests will never identify anyone outside of the 16 types. It's like saying that every time I roll a standard six-sided die, I will get a number from 1 to 6. What else could I expect?
DeleteAny usefulness would come from finding correlations between membership in the 16 classes and behavior, or performance in a given area. For example, I think it's true that many engineers (my profession) are INTP (my MB type, 3 out of 4 times I took the test, although once I came out INTJ). But I decided to become an engineer long before I'd heard of the MBTI, and I don't think that information would have had a useful influence on my career. You go to college and study what interests you most, and if you bring a certain personality type along, that seems to be incidental. I can understand the point that companies that use this test might not get measurable benefits (and very likely don't bother to try to measure them). You aren't going to have many ESFJs applying for an engineering position if ESFJs, by and large, don't find engineering interesting.
Well, I'm actually no fun of the MBTI test, precisely because there doesn't seem to be evidence backing up the existence of the 16 types empirically. But I still think you are missing a point: when someone *posits* something that means it is an assumption. There is no charge of circularity there, but there is still a question of, well, did you check your assumptions empirically?
DeleteWhat would be the point of checking a tautology empirically?
DeleteI did not miss the fact that it was "posited". It may be the fault of imprecision in the article. If the context of the assumption is, "If everyone fits into one of these 16 types, then we can design a test for it", then the assumption isn't tautological at all. If the context is, "If we have a test that assigns each individual to one of 16 categories, then those categories must mean something", then the assumption is a tautology. I guess you read the sentence the first way and I read it the second. Going back to the relevant paragraphs of the article, I'm not sure which reading is correct. The article doesn't say at what stage of development they "posited" the statement. Earlier would be better.
Kudos on Cathy O'Neil's article on Nate Silver.
ReplyDeletePrime complaint of mine: things are touted as "scientifically demonstrated" without looking at the political and monetary motivations for bogus science. Math works on a "garbage in garbage out" principle. Your conclusions are only as accurate as the assumptions you make in the beginning and the data you choose to include or exclude.
Without having read Silver's book (it's on my prospective reading list), I'm not sure I see a point in her complaint.
DeleteIf I understood correctly, she is taking Silver to task for identifying methodological errors without correctly (in her opinion) identifying the sinister motives behind those errors. But isn't the book about methodology, not about motive? Surely the talk about motives is a worthwhile topic for a book. It just happens to be a different book from this one, that's all.
The "political and monetary motivations for bogus science" argument is almost always wrong when it is applied from necessity, and never necessary when it is valid. Refutation of any true pseudoscience (there's an oxymoron for you) never requires the "political and monetary motivations" argument, as the empirical evidence is already sufficient. Conversely, it raises my suspicions when I hear that argument being used as an ad hominem argument against a theory. It's a likely indication that the evidence doesn't support the deniers' viewpoint, and this is their last refuge. Think of AGW deniers -- they do this constantly.
I don't allow that noting bias constitutes an "ad hominem" attack. I've had this argument before --
Deletesee:http://rationallyspeaking.blogspot.com/2012/08/michaels-picks-drone-edition.html
I do.
DeleteTo refute a theory, one needs to assess the evidence and the rational trail from the evidence to the conclusion, solely on their merits. If the facts are verified, and the reasoning is sound, then the ad hominem (which it indeed is) is irrelevant. If they are not, then you have your refutation already, and the ad hominem is unnecessary.
In cases where the evidence is unavailable for examination, or the rational basis for the conclusion is obscure, then you have a justification for the ad hominem. At that point, the whole thing becomes argumentative rather than scientific. The debate reduces to your ad hominem argument against the other's argument from authority. These are the rare cases where accusations of bias have merit.
Richard,
Deleteyou seem to be treating evidence as if it were independent of assumptions, theories, biases etc. It isn't. Which means that taking into consideration the possible biases of a source is a way to assess, indirectly, the quality of the evidence. Not at all a fallacy.
Massimo,
DeleteI did say, "One needs to assess the evidence". In the physical sciences, that assessment needs to look at experimental or observational methods, instrumentation technique, accuracy of data reporting, and similar measures. If these direct assessments do not find significant errors, then the motives and biases of the researcher aren't relevant. If they do find significant errors, then we can consider whether the fault lies in those motives and biases. Indeed, if we start with an assumption of bias in our assessment, then we might predispose ourselves to the same type of error we are accusing the researcher of. Better to start with neutral assumptions.
In the thread that Tom linked to above, you wrote: "The ad hominem is a fallacy only if one relies *solely* on character information to reject someone's claim. But clearly character, or connections, are absolutely relevant to establishing someone's credibility, as in the case under discussion. That said, one still needs to do the separate work of actually debunking the claims themselves."
Following this point logically: If you *don't* do the actual debunking (or if you try and fail) then you haven't refuted the argument after all (which is what I was saying). Otherwise, I see an inconsistency: Why say the debunking is still necessary once the job is done?
Too often, I hear the bias argument used as a last resort by apologists for views that can't be supported by evidence. Rarely do I hear it used by persons who have an abundance of evidence on their side, except to point out the blindness of those who persist in ignoring the evidence. So forgive me if my suspicions perk up when I hear the bias argument.
Put another way, I hope that the assumptions of researchers who are perceived as unbiased receive as much scrutiny as those of researchers with known biases. People make mistakes that aren't caused by bias, and biased persons are still capable of getting it right.
DeleteThanks for these links. I hadn't seen a lot of it before, and it was very informative.
ReplyDeleteThe article about reading weird books in public is definitely true. I live in Knoxville and any of my regular reading materials receives those kinds of reactions. The general reaction is subtle finger pointing and whispered conversations. Fascinating really.
ReplyDeleteHaven't read Nate Silver's book, but the following thing she said doesn't give me a good impression of her. She says,
ReplyDeleteSilver has an unswerving assumption, which he repeats several times, that the only goal of a modeler is to produce an accurate model. (Actually, he made an exception for stock analysts.)
This assumption generally holds in his experience: poker, baseball, and polling are all arenas in which one’s incentive is to be as accurate as possible.
What? Pollsters don't have any incentive to be anything other than accurate? Does that mean Rasmussen is not a pollster?
That's total nonsense. Some polls were clearly politically motivated and Silver took that into account in his model. That's why he succeeded.
Having said that, he could very well be wrong about finance stuff.
Richard,
ReplyDelete> she is taking Silver to task for identifying methodological errors without correctly (in her opinion) identifying the sinister motives behind those errors. But isn't the book about methodology, not about motive? <
Yes, but her point is that there wasn’t anything wrong with the models *other than* willful fraud by bankers and analysts. Silver, by focusing narrowly on the models, is missing the big picture, which is dangerous because it may mislead people to believe that better models would have averted the crises (a claim he does make in the book). That’s naive at best.
> What would be the point of checking a tautology empirically? <
Obviously Im not explaining myself. The 16 categories were defined a priori, so it makes no sense to accuse the authors of engaging in circular reasoning. But it does make perfect sense to ask whether they then went out into the real world and found empirical confirmation for the assumption that people do fit into the 16 categories they had started with.
brainoil,
> What? Pollsters don't have any incentive to be anything other than accurate? <
That is *not* what she is saying. She was simply giving Silver the best case scenario, granting him the assumption that polls in the fields she mentioned (poker, baseball, etc.) are not subjected to willful distortions by the modeler.
Okay, what am I missing here? She says,
DeleteThis assumption generally holds in his experience: poker, baseball, and polling are all arenas in which one’s incentive is to be as accurate as possible.
Doesn't the "and" here mean that she is talking about three arenas and the three are distinct from each other?
On the other hand, if she's only talking about baseball and poker, it's even worse. She's then ignoring politics just to maker her point that Silver has no experience in areas where modelers are motivated by things other than accuracy.
The secterism of skeptical / atheist / rationalist organizations is very disheartening, but perhaps understandable - especially in societies where these ideals are, or are viewed as, fringe positions. These organizations need to do everything possible to avoid exclusivity and to promote public outreach and active work, rather than fostering a sense of righteous indignation.
ReplyDelete"it may mislead people to believe that better models would have averted the crises (a claim he does make in the book)." Maybe. I don't know how strongly he phrased the claim and won't know its full context until I read the book. Certainly, if he claimed that more accurate models would have helped, even if the bankers ignored those models, then the claim doesn't make sense. Perhaps Silver is naive in that he is more concerned with means (the technical details) than ends (ulterior motives of the bankers). But "naivete" is not what Cathy O'Neil is accusing him of -- just look at her statement that "I can’t emphasize this too strongly: this is not just wrong, it’s maliciously wrong".
ReplyDelete"The 16 categories were defined a priori, so it makes no sense to accuse the authors of engaging in circular reasoning." Understood -- but initially you had agreed that it was a tautology, which isn't the case if it was an a priori hypothesis.
Richard,
Delete> But "naivete" is not what Cathy O'Neil is accusing him of -- just look at her statement <
Agreed, she went a bit too strong there.
> but initially you had agreed that it was a tautology, which isn't the case if it was an a priori hypothesis. <
Yeah, I see the confusion. The distinction is that if one wishes to stick with postulates, then yes, they are tautological, but not necessarily uninteresting (I mean, all of math and logic is a giant tautology, still...). What is problematic here is that to be useful the postulate of the 16 categories needs, at some point, to make contact with empirical reality. Which has apparently failed to do.
"Agreed, she went a bit too strong there."
DeleteAnd that is really the core of my complaint.
"What is problematic here is that to be useful the postulate of the 16 categories needs, at some point, to make contact with empirical reality. Which has apparently failed to do."
I agree with that.
I am actually reading Silver's book now. In the section on the ratings agencies, he BOTH discusses their potential ulterior motives, and goes on to criticize their models (the criticism being, in essence, that they treated the probability of any individual mortgage default as being statistically independent of any other - not true since an outside causal factor (the housing bubble) was making them all dependent.
DeleteAs for the ulterior motives:
"Instead their equation was simple. The ratings agencies were paid by the issuer of the CDO every time they rated one: the more CDOs, the more profit..."
"The CEO of Moody's, Raymond McDaniel, explicitly told his board that ratings quality was the least important factor driving the company's profits."
This doesn't sound like a whitewash to me, although maybe he places less *emphasis* on malicious intent than Cathy O'Neill. Ultimately, Silver seems to believe that it was a complicated combination of financially motivated self-deception & bad reasoning.
Thanks for the insight, Ian. It sounds like O'Neill might be experiencing a bit of sour grapes syndrome. I'm sure a volume could be written about the bad motivations behind the faulty ratings, but I don't think such details are relevant to what Silver's book is about, so as I said, it's a subject for a different book.
DeleteI look forward to reading it, although I have a bit of a backlog of unread books right now.
Thanks for great reads, Massimo. Again!
ReplyDeleteYou may like to know that I plan to use quite a lot of your blog posts, writings, videos, etc. in my teaching next semester. A colleague and I are creating a new course for Swedish high-schools: "Science +", which aims at combining Liberal Arts and Science education. We want to confront our 17-18 year-old science students with contemporary issues in psychology, philosophy and civics at a relatively early stage - before they decide where to go for college / university studies. We're inspired by, among others, Brockman's www.edge.org and a similar initiative at Gothenburg University, www.utbildning.gu.se/program/program_detalj/?programid=H1LIB (Swedish only).
Please feel free to take a look at:
http://natvetplus.blogspot.se
http://www.facebook.com/natvetplus
I just realized that the main links are in Swedish. The modules are: Evolution, Psychology (Cognitive, Social), Logic (including Probability and Statistics), Science (including Philosophy of Science), Philosophy (with emphasis on Ethics), and Society (e.g., Political Science, Economics, Environmental Science).
DeleteBjörn,
Deletesplendid idea! I've actually been toying with the idea of a similar course myself. Maybe I should stop toying and just do it.
Perhaps in lieu of any purpose you should acquire a reason to write and teach about how evolution lacks one.
DeleteGive me one good reason why I should read an article from The New Atlantis chaired by that delightful ass Leon Kass?
ReplyDeleteBecause it's perfectly possible to evaluate the article's claims on their own merits, no matter how much of an ass the outlet's chair may be.
Deletebrainoil,
ReplyDelete> if she's only talking about baseball and poker, it's even worse. <
Or you can read that as allowing Silver more benefit of the doubt in cases were there is no clear evidence of malicious actions, as opposed to the financial crisis, where such evidence is abundant. I try to read people charitably, but either way, Silver's case is undermined at least in some instances.
Louis,
> why I should read an article from The New Atlantis chaired by that delightful ass Leon Kass? <
Because it's an interesting article?
"Has science replaced philosophy?"
ReplyDeleteScience is the theater. Philosophers are its critics. They are the Addison DeWitts ("All About Eve", 1950) of science: "My name is Addison DeWitt. My native habitat is the theater. In it, I toil not, neither do I spin. I am a critic and commentator. I am essential to the theater."
Relevant update on criticisms of Nate Silver: he responded to questions on reddit last week, one of which was:
ReplyDelete>Q: Last month, the quant-blogger mathbabe took your book to task for confusing cause and effect. She said, “We didn’t have a financial crisis because of a bad model or a few bad models. We had bad models because of a corrupt and criminally fraudulent financial system … this is not just wrong, it’s maliciously wrong.” She then claimed you were “a man who deeply believes in experts,” which is where your book went wrong.
Could you address this criticism and defend your conclusions?
(full post: http://mathbabe.org/2012/12/20/nate-silver-confuses-cause-and-effect-ends-up-defending-corruption/)
— stickycinnamon
>A: I'd encourage you to read my book and ask whether she fairly interprets my hypothesis. I don't think she does. The financial crisis chapter is quite explicit about asserting that the credit ratings agencies were not just stupid, but also a bunch of dirty rotten scoundrels, so to speak. And the book is generally quite skeptical about the role played by "experts".