Extra Ordinary Ideas

Dr. Lott Replies

Dr. John Lott kindly sent several of us a personal email in response to our recent posts about the allegations surrounding his references to a 1997 survey (see Tangled Webs, below). I replied, asking permission to reprint some portions of the email here, but haven't heard back from Dr. Lott yet. However, Jim Henley has posted an edited version of the communique here. It goes without saying that anyone interested in this issue should read what Dr. Lott has to say.

Also, Julian has turned investigative reporter on us, making additional phone calls and sending emails to fill in some of the details.

UPDATE: Dr. Lott has given me permission to post the following comments, which I quote:

"Dear Everyone:

Here is a response to some of what has been going on over the web. I have
already sent much of this information to people who have already contacted
me in person. If Eugene would like to post this on his web site, I must ask
that all the e-mail addresses and telephone numbers be removed. If you all
don't trust the leg work done by Dan Polsby on this issue since Christmas,
you can nominate someone else to go and do it, but I don't think that it is
appropriate for everyone from Lambert on to go and harass these people.

Regnery (the publisher of my new book due the middle or end of March) wants
me not to release the results from the poll last year. They want me to keep
quite about the book until it comes out. As has been reported previously,
the survey was done with similar questions in a very similar way to what was
done earlier and the results were essentially the same. I will check with
the publisher about releasing this data early, but it is still two months
before the book is due to be published. In the interim, I am sure that I
could arrange it so that interested parties could question the person who
keep the survey results as they came in to confirm that we only got one
person who said that they had actually fired a gun and that the rest were
brandishings.

Here are some of the things that I have done to try to establish a record of
events. 1) My wife contacted the bank that we had in Chicago and tried to
get copies of bank statements and checks from the period of time.
Unfortunately, the bank does not keep copies of statements or checks longer
than five years. (If you would like to verify, we talked to Yvonne Macias
in the book keeping department at University National Bank.) Lindgren does
not accurately report my conversation with him about how I paid people (in
that I said that I possibly paid by check), but this information makes that
point irrelevant. 2) I asked Sam Peltzman last year about whether the
Alumni Association has the e-mail of past students. Sam, who seems to know
virtually everything that is going on at the University, told me that they
have the e-mail addresses for at most 10 percent of the former students. 3)
I had a former alumni and several time co-author, John Whitley, placed in an
ad in the Alumni magazine in the December issue to track down the students.
I don't know if the ad has appeared but thus far I have gotten no response.

I have given out massive amounts of data to people on the guns and other
issues, and I will be happy to do so on the new survey. Data has been given
to critics as well as people who have been unwilling to share their own data
on other projects. I have given out county, state, and city level crime
data to academics at dozens of universities, with data sets ranging from
36MB to over 300MB. I have given out data on multiple victim public
shootings as well as safe storage laws. These different data have often
been given out before the research is published and sometimes even before it
has been accepted for publications. We are not talking about recent events
or conversations and there is a question about what is a reasonable time
period for people to keep records. There is also a question as to why
people have waited so long to ask for this additional information when
people have known about the lost data for years.

As to the claims about 'apparently changing positions,' I disagree. I have
told people directly (including Otis Duncan) from the beginning that the
data were lost. Op-ed pieces and other public statements where I mention
these numbers briefly usually do not lend themselves to discussions of the
sources of numbers. The fact that David Mustard does not remember exactly
when we discussed the survey 6+ years ago does not surprise me given how
long ago this was.

Unfortunately, there are many problems with Lindgren's write up. He gives
essentially uncritical acceptance of Otis Duncan's discussion of events in
1999. Yet, while Lindgren writes that 'Otis Dudley Duncan raised questions
about the 98% figure . . . after exchanges between Lott and Duncan,'
Duncan's write-up in the Criminologist news letter failed to mention any
such possible discussions. In fact his newsletter piece leaves the opposite
impression as he endlessly speculates about what I may have meant about
certain statements. My response in the Criminologist also discussed other
incorrect claims by Duncan.

As to the attribution of sources, look at the complete context of the quote
Lindgren mentions:

Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates
show that there are at least 760,000, and possibly as many as 3.6 million,
defensive uses of guns per year. In 98 percent of the cases, such polls
show, people simply brandish the weapon to stop an attack. -- August 6,
1998, Chicago Tribune and August 14, 1998, Washington Times

References by Lindgren to things like the Linnet Myers piece in the Chicago
Tribune to provide evidence that I didn't do a survey or that I have changed
my statements over time are simply bizarre. Attached below is an edited
down version of the letter that was published by me in the Tribune. Myers
used her article to refloat claims such as my Olin Funding, inaccurately
reported exactly what the concealed handgun research covered, and claimed
that 'others haven't confirmed (my) findings.' I no longer have the
original letter to the editor, but as I recall this is just a partial
listing of her inaccurate statements. The Tribune was not willing to run a
longer letter, though the letter that they ran was quite long.

As to so-called technical problems, I am have always acknowledged that theseare small samples, especially when one breaks down the composition of those who use guns defensively. Even the largest of the surveys have few
observations in this category. The attached e-mail that I sent to Glenn
Reynolds goes into this more in depth.

'No direct evidence of survey' discussing Lindgren's point-by-point
discussion of our conversation

1) 'No funding for the project'
I regularly have paid for research myself. Sometimes large amounts of
money have been spent, but it is not uncommon for me to spend several
thousand dollars. On the paper on multiple victim public shootings, I know
that one payment that I made to Kevin, a research assistant to Landes and
Posner, was $750. I paid for the special issue of the JLE in 1999 on
sentencing myself, and the special issue and part of the conference cost me
around $30,000. I have not applied for funds from outside sources over the
years.

2) 'No financial employee records'
This is not unrelated to the first point. Incidentally, I told Jim that
there were 'two' Chicago students. Those students had also gotten others
that they knew from other campuses from places such as I think the
University of Illinois at Chicago circle (but I am not sure that I remember
this accurately). What I told him was that I remembered that one of the two
University of Chicago students was a senior.

3) 'calling was done by the undergraduates from their own phones.'
most of this next statement is correct except the point about the 'possible'
use of checks. But as noted earlier this point is irrelevant in terms of
evidence.

4) 'does not remember names'
I have had 12 interns and RAs just since I arrived at AEI. This excludes
people whose only work was on the survey. I am horrible at names and I
couldn't even give you the names for all of these folks let alone people who
did something six years ago. All my names and addresses for everything were
on my computer when the hard disk crashed.

5) 'no discussions with any samplers'

I had lunch Tom Smith during the fall of 1996. However, while I asked him
many questions about surveys, I did not tell him what I was planning on
doing because Tom works very closely with gun control organizations.

6) weighting the sample

I did not weight the sample by household size but used the state level age,
race, and sex data that I had used in the rest of my book. There where 36
categories by state. Lindgren hypotheses why you can get such small weights
for some people and I think that this fine of a breakdown easily explains
it. I don¹t remember who answered what after all these years, but suppose
someone who fired a gun was a elderly black in Utah or Vermont.

7) 'commercially available CD-ROM with names on it. He does not remember
where he got it from.'

It is true that I don't have the original CD-ROM. I have a telephone number
CD from the end of 1997, but it is not the one that we used. I only picked
up the other one on the off chance that I was going to have the time and
resources to redo the lost data. The CD did have the features that the
earlier one had and was not very useable. I was so rapped up in trying to
replace my lost data on so many other projects that I had no thought of
going back to what I regarded as a minor project. I had revise and
resubmits at the JPE and other journals that had much greater importance and
the data for the book had to be replaced.

8) 'Lott does not remember how he drew his sample from the CD-ROM'

Not true. I told Jim that one of the students had a program to randomly
sample the telephone numbers by state. My guess is that it was part of the
CD, but on that point I can¹t be sure.

9) 'doesn't remember the wording of the questions.'

It is also not quite correct to say that 'doesn't remember the wording of
the questions.' I told Jim that I don't remember the 'exact wording' of the
questions, but I gave him the general outline of the questions.

10) more on weighting

See point 6 above.

11) 'A chapter he had not yet written'

This is not correct. What I had done is write up the section, but I only
had a computer file of it. When the hard disk crashed, I only had a hard
copy of the book and I had to spend considerable time scanning in the book
and correcting the new file. I was unable to replace the lost polling
section that I had recently added. I didn't think that it was worthwhile
relying solely on memory for different things and I had too much else to do
to concern myself with something that wasn't central to the book.

12) 'did not retain any of the tally sheets'

I have looked through some things but I haven't found anything. As Lindgren
correctly notes, I have moved three times in the last six years.

13) Sheets versus entry of data into computers

Lindgren has the 'impression' that the students entered the data on sheets.
I do not directly recall this part of our conversation, but I would have
said that both were done.

I sent Lindgren two e-mails on December 26th. Just so no one accuses me of
adding new things in now, one of my e-mails to Lindgren noted: 'I did not
take the time to correct or respond to all the issues raised, but I wanted
to mention a few points.' Recent e-mails to Lindgren have also already
responded to some of these points beyond the e-mail that he apparently
posted.

I have not participated in the firearms discussion group nor in the apparent
online newsgroup discussions, but what I have done is respond to e-mails.
(The one exception are those from Lambert whose e-mail address was placed on
my blocked list.) If you all have questions, I will be happy to discuss
them, but I am not going be involved in these online groups. My response to
Glenn below goes through some of the history of what I heard on this and
when I heard it. The bottom line is that you all should not assume that
everyone participates in these discussions."

Please Note: Dr. Lott's email also included a clipped letter to the editor of the Chicago Tribune published on June 20, 1999, which should be accessible through public sources, and shortened version of another, slightly older and mostly duplicative email also reviewing his account of events and discussing his new 2002 study. I'd post it all, but this is a heck of a long blog entry. Go ahead and email me at marie@mariegryphon.com if you must have the balance. I'll supply a link if I find any additional comments elsewhere.

Posted by Marie Gryphon on January 14, 2003