Dartblog

On Monday, Dartmouth’s Office of Institutional Research (where eager dirt-diggers can find many a fascinating document, by the way) released its report of Dartmouth’s results from the 2006 “Senior Survey.” The Survey is a detailed questionnaire administered annually by an outside group, the Higher Education Research Institute, to graduating seniors at participating American colleges and universities. The questionnaire examines student opinion on a wide variety of aspects of instruction, administrative services, student life, etc. Dartmouth participates in the Survey every other year. This is the first time that one of these reports, which Dartmouth’s bureaucrats presumably prepare biannually each year Dartmouth participates in the Survey, has been made public.

The report—there it is in all its 19-page pdf glory—contains a surprisingly detailed and technical analysis of the collected data. I was mildly impressed—the bureaucrats put a lot of time and effort into this.

The report compares Dartmouth on a variety of dimensions to three “peer groups.” Peer 1 is “highly selective, co-ed liberal arts colleges,” so Amherst, Swarthmore, Williams, Rice, Oberlin, etc. Peer 2 is “highly selective, private institutions in Northeast,” which would seem to overlap greatly with Peer 1, unless you assume that by the word “institutions” the bureaucrats mean “research universities,” in which case we have the rest of the Ivy League, MIT, NYU, etc. Peer 3 is “highly selective, private institutions beyond Northeast,” which also seems to overlap with Peer 1, but I’m assuming the bureaucrats mean Stanford, Caltech, Wash. U, U Chicago, Northwestern, etc.

The overall sense one gets from the comparative section of the report is that Peer 1 students are happier than Dartmouth students, who are happier than the students in Peers 2 and 3. (Figures for the peer groups are not given school by school, but as aggregate numbers—presumably averages across the schools in each group, but possibly medians or some other central tendency measure.) That finding doesn’t surprise me at all—overall student satisfaction appears negatively correlated with the size of the school.

There are two particular variables of interest. First, in “pre-major advising,” Dartmouth stinks. We are far worse than all three of the peer groups, and on a four-point scale of “Very dissatisfied,” “Generally dissatisfied,” “Generally satisfied,” and “Very satisfied,” fully 65 percent were “very” or “generally” dissatisfied. Only six percent were “very satisfied.” And “satisfied” is a far cry from pleased.

Second, and I feel like a broken record here, Dartmouth is worse than all three peer groups in “Administration’s responsiveness to students.” Dartmouth students rated that item as the third-worst aspect of Dartmouth. One has to wonder how the bureaucrats preparing the report felt when they wrote that.

The end of the report contains some interesting summary graphs that examine which aspects of Dartmouth are important “drivers” of student satisfaction. These graphs plot the “importance” to students of each item on the vertical axis, against student satisfaction with that item on the horizontal axis. Thus, the upper right-hand sector contains “key strengths,” aspects of Dartmouth students like and find important; the upper left-hand sector “key weaknesses,” aspects of Dartmouth students dislike but find important; the lower right-hand sector peripheral strengths, and the lower left-hand sector peripheral weaknesses. Here are two of these graphs, the first for academics and instruction, the second for campus life.

There are a few points of interest here, particularly in the campus life graph. First, notice the high rating students give to “Social life.” With this in mind, the administration’s simmering distaste with Greek-letter organizations and the recent campus obsession with finding “alternative social spaces” look a lot less useful, and a lot more like ideologically-driven social engineering.

Second, notice the position of the item “Diversity of campus.” It’s in the peripheral weakness sector. Students apparently find the level of diversity on campus unsatisfying, but don’t particularly care. Is that not the exact opposite of the standard line from the propaganda apparatus at 7 Lebanon Street?

As a final note, one has to wonder to what extent, if at all, the figures in the report were spun. As the report was not initially prepared for public viewing, but rather by Dartmouth bureaucrats for Dartmouth bureaucrats, there was presumably not much of an incentive to smear away from the truth. And as I have discussed, there are elements of the report that paint an unflattering picture. But it is a question nonetheless.

You might wonder how a number can be spun—a number is a number. With a little clever presentation, it’s not hard. Take for instance the graphs above. The impression a reader takes away isn’t based on the numbers behind the graphs, but on the graphs themselves—where the data points are located with respect to the four strength/weakness sectors. But realize that if you were constructing one of those graphs, you could place a given data point anywhere you pleased—it’s just a matter of choosing intelligent axis scaling. For example, if the horizontal axis scale extended up to 4.0 instead of 3.9, all the data points would shift to the left—and students would appear less satisfied.

With this in mind, notice that there is not a single data point in the “key weaknesses” sector of the academics and instruction graph, but that “Course availability” is extremely close. I would not be surprised if the bureaucrat who created that graph fiddled with axis scaling for half an hour to find just the right numbers.