The "Most of Us" survey about drugs and alcohol projects a positive message backed up by data, but it suffers from numerous flaws in analysis, by both professionals at Montana State University, where it is analyzed, and students at Paly.

One error by the professionals is to assume that "Seventy-nine percent think most do something" is the same as "Most think 79 percent do that thing." Another flaw is to complain that people overestimate alcohol use despite "most" having used it at least once. Students, however, forget that some people reply "not sure" to some questions about behavior.

The "Key Findings" report, written by program manager Valerie Roche at "Most of Us" in Montana, states on its front page that 76 percent of students report never having used tobacco, while only 21 percent think most have never used tobacco. From this, the analysts seem to conclude that "this disparity shows that Palo Alto High School students think that tobacco use is far more prevalent among their peers than it actually is — three and a half times more prevalent, to be precise." However, the data does not seem to necessarily support this claim. It is possible that the remaining 79 percent of the school believes that 50.1 percent have used tobacco. In that case, they only think that twice as many people use tobacco as actually do. Having 79 percent think most students use tobacco is not the same as most students thinking 79 percent use tobacco.

The analysts still claim that the students’ perception of their fellows is worse than it should be even when most Paly students do not take the healthiest option. For example, "Thirty-seven percent of Palo Alto High School students reported that they never use alcohol. While this is far from a majority norm, even this level of non-use has significant misperceptions attached to it: Only 6 percent of Palo Alto High School students think that most of their peers never use alcohol." The phrasing makes it sound as though a large portion of the student body have shown that they overestimate the amount of alcohol use. Unfortunately for the analysts, however, this data shows very little difference between actual and perceived use. Most Paly students (63 percent) have used alcohol at least once, and the vast majority (94 percent) accurately recognized this norm. Those mere 6 percent were wrong, not right.

However, bad analysis is not solely the province of the professionals. Some students (including me, last year) see statistics like "Most (58 percent) of us would rather NOT drink alcohol when hanging out with friends" and think to themselves, "100 percent minus 58 percent, or 42 percent, of students would rather drink alcohol." Looking closer at the data reveals that, in fact, 26 percent are "not sure" and only 16 percent would rather drink when with friends. Still uncomfortably high, perhaps, but nowhere near two fifths.

Students complain that the survey is inaccurate due to rampant lying. However, given the lying that I have heard about and the message, this strikes me as strange. Most cases of lying that I have heard of are students who claim that they do more drugs than they actually do, which would only inflate the portion of students doing drugs. Since the claim is that far fewer people do drugs than students realize, this lying does not change the accuracy of campaign’s core claim.

There are several other examples where the analysts conclude something that the survey data seems insufficient to support.

Overall, given the doubt that the surveyors face (only a little over a third of students find "Most of Us" believable, according to the survey), they must be particularly careful to ensure that any claims they make are fully backed up by data.