From: Mike Godwin <mnemonic@eff.org>
 Subject: Re: More PEDagogy (was Re: TIME Cover on Cyberporn)
 To: kieran@interport.net (Aaron Dickey)
 Date: Fri, 28 Jul 1995 13:16:54 -0400 (EDT)
 Cc: mnemonic@well.com, mnemonic@eff.org, prof@well.com, ped@well.com,
         ped@panix.com
 
  
 Aaron, please post this response to the relevant forums. I'm also cc'ing
 it to Philip and to Donna.
 
 Philip Elmer-DeWitt writes [in alt.internet.media-criticism and related
 newsgroups]:
 
 > To reiterate. The story was elaborately fact-checked by two researchers
 > against the primary text (Rimm's study) and the correspondents' files. It
 > was also played back to the correspondents who did the reporting, and they
 > called for various changes in their CAC's (comments and corrections). That
 > is TIME's standard checking procedure. In the course of this, the
 > researchers did find some inconsistencies within Rimm's text.
 > (Generalizations in the summary that didn't seem to be supported by the
 > text, for example. Or the obvious problem with the title of the study, in
 > which the 8.5 million downloads are made to seem to apply to all 900,000
 > plus files, which is far from the truth.) These made us uncomfortable, but
 > not enough to stop the presses.
 
 And not even enough to check the study wih any independent researchers.
 The irreducible fact is that Philip was wholly opposed to risking 
 the discovery of facts or questions that might force him to hold the
 story, or to change it in a fundamental way. Philip's story as constructed
 would not have worked if it had been pegged on a dubious, and possibly
 even fraudulent, study.
 
 > We dealt with them taking out of the TIME
 > story the stuff in Rimm's study that we knew was problematic (the 8.5
 > million figure, for example). 
  
 This suggests, astonishingly, that Time did not or could not have known
 that certain things they *did* print were problematic: to wit, that the
 same study that purported to be the first of its kind also purported to
 show a _trend_ toward the marketing of more "extreme" sexual images
 online. That is a _logical_ inconsistency -- not one that requires
 sociological expertise to recognize.
 
 Similarly, but even more inexplicably, given what Philip writes here,
 Time uncritically accepted Rimm's claims that Robert Thomas's Amateur
 Action BBS was "the market leader," and Thomas's idiosyncratic "marketing"
 strategies (some of which represented, in reality, little more Thomas's
 practical jokes on his users) somehow marked the leading edge of an entire
 industry. Rimm adduced no data to support this claim.
 
 Rimm's claim that Thomas was the "market leader" was also mendacious in
 another,  subtler way -- one that requires requires a tiny bit of critical
 thinking of the sort that Philip was not prediposed to engage in.
 Specifically, what is the essence of a claim that a businessman is a
 "market leader"? Is it the volume of his business? If so, Thomas wasn't
 the leader. One may anticipate Rimm's likely response, when confronted
 with the fact that Thomas, while successful, could not be said to be the
 most successful player in his market: "I never said that Thomas made the
 most money -- I said he was the market leader. His strategies show which
 way the marketing trends are going." 
 
 There's a conceptual problem with this argument, though -- one that is
 entirely distinct from the evidentiary problem (as is *normally* the case
 in his article, Rimm doesn't provide the evidence for his
 generalizations). The conceptual problem is this: you can never know who
 *is* the market leader in innovative strategies -- you can only tell who
 *was* the leader at some point in the past. The test for whether a player
 turns out to be a "market leader" in innovation will *always* be whether 
 his innovations have found their way into the mainstream and are commonly
 used. In short, you can't do what Rimm tries to do -- that is, he tries to
 imply in his discussion of Thomas's business that Thomas tells us
 something about the industry as a whole -- if not where it is now, then at
 least where it is going. *Even if Rimm has collected the data he purports
 to have collected, Rimm cannot predicate either conclusion on that data.*
 
 Philip writes:
 
 > I think the crux of it is that we made the mistake of
 > treating Rimm's non-peer reviewed Georgetown Law Journal article as if it
 > were a peer-reviewed study in Science or Nature.
 
 Philip omits to mention here that he knew prior to publication of the 
 Time "Cyberporn" story that the Georgetown Law Journal isn't
 peer-reviewed. Donna Hoffman and I discussed this fact beforehand, and we
 made sure that Philip knew about it beforehand.
 
 > In this case we went a bit further, because Hoffman and Godwin had raised
 > questions about the study. So we took those questions back to Rimm and
 > Sirbu and tried to get some answers. Rimm, as I say, did a pretty good job
 > fielding the main criticism of the study (that he didn't know the
 > difference between an adult BBS and a Usenet newsgroup; in fact, he did).
 
 To characterize this as the essence of Hoffman's and my questions about
 the study is deeply false. I won't speculate whether Philip is
 deliberately misrepresenting us here or simply engaging in conveniently
 exculpatory memory revision.
 
 One of the principal issues that Hoffman and I each raised was the
 invalidity of generalizing about the general availability and
 (purportedly) evolving character of sexual material *in cyberspace
 generally* from a sample drawn primarily from commercial "adult"
 BBSs. One *possible* explanation for Rimm's engaging in such a clearly
 invalid type of inference is that Rimm didn't know the difference between
 Usenet newsgroups and BBSs, but I never seriously advanced that view, and
 I doubt Hoffman did either.
 
 (If one were charitable toward Rimm, one might mention that at Carnegie
 Mellon the students and faculty routinely refer to the Usenet newsgroups
 as "bboards" -- this is CMU idiom, and while it's not unique to that
 school, it's also not the most commonly used expression, either. So it's
 not *wholly* ridiculous that a CMU student might have slipped into
 equating the two types of forums.)
 
 What seemed far more likely prior to publication of the story and the
 study, and what turned out to be true, as we discovered when we got
 our hands on the article, is that Rimm's article relies on *his readers'*
 confusion, or at least conflation, of BBSs, the Internet, and "the
 Information Superhighway" in general. What you discover when you actually
 *read* the study and its footnotes is that Rimm *defines* "Information
 Superhighway" in a way that implies an essential interchangeability.
 What Hoffman and I kept trying to tell Philip, who really wasn't
 interested in hearing us, was that Rimm's study "confuses BBSs and Usenet
 newsgroups" in a *causative* way -- that is, that it draws inferences
 based upon an implicit equation of the two types of forum, and that it
 certainly would give rise to the same confusion on the part of its
 readers.
 
 > The other main criticism was that he was an undergraduate EE major and
 > not qualified to write about sex on the net."
 
 Here Philip transmutes Hoffman's and my questions about Rimm's academic 
 background into manifestations of a sort of credentialism. In reality,
 neither Hoffman nor I was claiming that an EE undergraduate *couldn't*
 have produced a valid study. Instead, we were attempting to underscore
 a *red flag* for Philip -- trying to explain why he needed to have
 *some independent qualified individuals* review the article before
 Time took the leap of hyping the Rimm study as the truth about sex on the
 Net.
 
 I had had a small hope, after reading some of Philip's implicit and
 explicit mea culpas on the WELL, that Philip had finally gotten a clue,
 as a result of the painful experience of having been conned by a glib
 hustler, about what he'd done wrong as a science journalist.
 
 Reading this, however, convinces me that Philip is still making excuses
 for himself, trying to persuade himself, his bosses, and the world that
 his mistakes, while regrettable, were still *reasonable*. He does this by
 misrepresenting what Hoffman and I told him (this isn't the first time
 he's misrepresented us on that score, but I've been trying to overlook
 the more recent instances), by fudging a bit on the chronology of what
 Philip knew and when he knew it, and by deemphasizing all the red flags
 that, based on my own experience as a journalist and on my knowledge of
 the work and methods of a number of science journalists, I must insist
 that no reasonable, competent person in Philip's position could have
 ignored.
 
 For years we shall be paying for the education of Time in how easily a
 self-promoting pseudo-academic conman could wrap a nation's leading newsweekly
 (one is tempted to call Time "the market leader") around his little finger.
 Given how big the "tuition bill" will continue to be, it doesn't seem
 too much to ask that Philip *learn the damned lesson*.
 
 
 --Mike Godwin