I'm looking at Miller et al right now. A couple observations:
There's nothing automatically invalid about a mail survey. The one reported here appears to have used
good methodology and the reported response rate of 52% is very good. They describe a lottery with cash prize for participating in the survey, but they don't disclose the amount of the prize. A very large prize might raise some concerns, but would probably not have been allowed by their human subjects committee.
Correlational research is not automatically invalid. There are lots of useful things you can learn from studying correlations.
Not all "social epidemiology" research is bad science, and there's no law that says epidemiological research has to use correlational designs.
The major problems with the survey results are that there was no consideration of whether guns were lawfully owned, no consideration of permitted carry, and no context for the question about having been threatened with a gun. For example if you own a gun and keep it at home, and are mugged on the street while unarmed, it is hard to see how the the firearm at home can be considered a risk factor.
As is common with this type of survey, there are some difficulties with the numbers of respondents in critical parts of the analysis.
For example, 4.3% of students had a firearm, and 5.9% of these reported having been threatened. 1.4% of those not owning guns reported having been threatened. That implies 430 gun owners in the survey, and 25 of these had been threatened with a gun (vs 140 of those who didn't own guns). Yet these variables are included in multivariate analysis with something like 15 predictors, and compared across 9 regions. This means that conclusions about any given predictor are likely to be based on very small numbers of individual respondents. This is especially true when considering relationships between being threatened by a gun and crack use, which also has very low base rates. If 47% of the gun owners had a gun for protection and 1.8% of these used crack, that's two individuals. Only .05% of those owning guns for other reasons reported crack use. That's one individual. It is absurd for the authors to comment on this difference as if it were meaningful (but they do).
It appears to me based on the above considerations that some of the reported confidence intervals and significance tests may have been derived on the basis of rather heroic assumptions. There might be a clue buried somewhere in the article, or perhaps a detail that is telling through its absence.
The authors do note that the cross sectional analysis cannot show causation, and that the data "do not show whether guns at college confer a net benefit, impose a net cost, or have an indifferent effect on college communities or on individual gun owners." They also sort of comment on the built-in ambiguity of the relationship between gun ownership and victimization: "Additional research is also needed to explore whether and if so,under what conditions, gun possession itself emboldens students to put themselves at risk for victimization, is a response to past victimization, is a response to accurate or systematically flawed perceptions of risk, or reflects an attitude towards risk that predisposes to both gun posession and risk-seeking behavior." I find these particular comments to be reasonable enough, although a little silly in that there was no consideration of lawful vs unlawful gun ownership in the article. Some criminals do go to college...
Oh, and Jim H, the data were actually collected in early 2001 for the this report.
