What do political scientists think of the poli-sci experiment in the Montana state Supreme Court election?

While I'm not a political science faculty member, my degrees are from the University of Montana's political science department so I feel at least a slight desire to weigh in here. It's pretty obvious that this rumpled more than a few feathers, including those of sitting Montana senator Jon Tester who sent a letter to Stanford and Dartmouth (http://billingsgazette.com/tester-s-letter-to-stanford-and-dartmouth/pdf_79c534eb-75ed-50f8-9c67-883e7beb6fa4.html). Based on my reading of the "information guide" it's pretty apparent that this looks more like it serves a political purpose than a research purpose. There are no measurable outcomes to be gained (unless they are monitoring media backlash or the actual outcome of the election...which they couldn't know was going to be one side or the other until the election day, thus not knowing the impact of their guide) and it is not asking for respondent opinions. From the article linked to in the question: As it turns out, the experiment was intended to measure whether receiving additional information about the candidates affected voter participation. In this case, the additional information was about the ideological leanings of the candidates for Montana Supreme Court, based on their past political contributions and other publicly available information. The study was funded as part of a $250,000 grant from the Hewlett Foundation, which Stanford matched with $100,000, according to the university. According to a description provided by Stanford, the research was intended "to compare voter participation levels in precincts that receive the additional information with voter participation in precincts that do not." Now with the increased media attention the initial mailing's impact will be blown out of proportion and the experiment is compromised because it is even less controlled (if it ever was) than it was before. It's odd that they used the Montana state seal in the document if they did intend this only to be a political science experiment. That is a gross misrepresentation to those who received the guide. It's a pretty clear violation of Montana statutes to misrepresent  oneself as being an official state serviceman or document. Overall, I think this is improper and unethical. I'll finish my answer with a quote from the initial document provided again: "It's basically political science malpractice. That's what I'd call it," Jennifer Lawless, professor of government at American University in Washington, D.C., told TPM. "When you're going to engage in an experiment as a political scientist, I think you have a responsibility not to affect election outcomes, let alone break the law."

I , like Luke, graduated from the University of Montana with both a BA and MA in Political Science. While I have since moved on to greener field  in Political Rhetoric and Communication, I feel as someone who has previously studied judicial elections that my commentary would be appropriate.  I borrowed (previous comment) Antonymous's organization just because it is  the most obvious example of a sympathetic interpretation of the events.  (1) "A lot of criticism exists simply because this is coming from Stanford." I would raise to points to this assertion. The first is that Stanford is an external institution whose placement outside of the state has gotten the ire of native Montanans. I reject the notion that any institution would have been forgiven had it acted like Stanford and Dartmouth. Regardless of less or more impressive zip codes, the question of outside influences in Montana elections has been hotly contested. (1a) My second response deals with the nature of the study, which may give some insight into the reaction by Montana citizens. By admission of all parties involved, the study was an attempted pseudo experiment of some variety. This means that a certain variable was manipulated to ascertain the potential influence on voting behavior. For Montanans, it is the manipulation that is key. Through the choice to include certain items of information (a sloppily constructed ideology scale in this case), the researchers attempted to manipulate Montanan voters, measure the effects of the scientific manipulation, and presumably profit it from academically or financially given one researchers connection to a start-up. Whenever you are measuring experimental effects on behavior, it has been tradition/regulation to gain consent of those participating. In this case, people were not even made aware of their participation robbing them of the opportunity to consent or not.  Living in a state that constitutionally honors an independent judiciary, does not mean that proxy consent should be inferred on behalf of Montana citizens.  As I have suggested previously, this line of reasoning suggests it is manipulation not ill conceived malcontent for our intellectual betters that has concerned so many citizens. (2 and 3) "Usable information" It is impossible to tell what usable information the researchers were attempting to obtain. From several readings of their previous work and their involvement with a start-up, I have observed their attempt to measure voters reaction not just to partisan information but their constructed ideology scale. The last line is probably the most important. Without any exception, we know that increased information in judicial races (regardless of partisan, non-partisan, or retention elections) leads to increased voter turnout.  This suggests two things. First, that the mailer intervention would/should have been known to affect voter behavior. A survey of the relevant research leaves this phenomenon in little doubt.  Second, this information is already available to relevant and interested parties.  Because we are not privy to the research design, we can not say for certain what considerations were made when choosing their sample. Given this performance and lack of consideration for Montana laws, I am not inclined to suggest they gave it much thought. From the list of Montana cities to whom the mailers were sent, it is obvious the were targeting the larger population centers. Without a clear idea of which districts within Montana the mailers were sent to, it is impossible to assume the study would have produced relevant information. I'll make a brief list of confounding variables that may have lead to an increase in voter turnout that deal nothing with the ideological scale being tested in question: (1) Weather and location- though the outside researcher may not realize, districts that were not chosen to receive the "experimental" mailers tend to be outside city centers making their voting behavior a question of proximity not information availability. (2) Alternative information- what research designers of the mailers did not account for is the hotly contested race where other organizations are attempting to spread similar or contradictory information about the judicial candidates. Without accounting for the other mailers in the manipulated and non-manipulated districts, it would be difficult for them to isolate the effects of their information over others or absence of mailers. (3) We cannot say for certain whether or not they accounted for it, but at this point the use of the state seal would be a massive confounding variable. Where voters may have ignored similar information from other mailers, the state seal acted as an attention grabber that impacted the way voters dealt with the information. In the end, optimally the study would have measured the influences of physical location to voting centers, the role official state insignia in guiding behavior, and the impact of partisan ideology continuum based on assessment of campaign contributors.  There are others, but these seem the most relevant to me. (4) Malpractice "too aggressive"--I would disagree fundamentally in the idea that this issue is anything but a violation of ethics and malpractice. First, I would point out that for states like Montana and Oregon who maintain nonpartisan elections it is essential to set a precedent whereby political scientists and interested sociologists feel unwelcome to manipulate populations according to their determination that it would be helpful to do so. At this point, it isn't possible to determine the effect of these manipulated mailing might be. For political advocates in the future, it might mean that expensive mailers attempting to enter into deliberative political action might be disregarded out right as experimental.  Study-desirable populations should not and do not provide consent based on their physical locations. Finally, I'll end on noting, and agreeing, how the most charitable aims of the study can be achieved through more ethical means that did not require the manipulation of un-consenting participants. Though the discourse of post-positivism is privileged, alternative studies could pair researchers  with local organizations obtaining even more valuable information. Other studies have profitably demonstrated this research design without the controversy brought on by the Stanford and Dartmouth researchers. Researchers like these should be punished and openly critiqued for opting for a method that impoverishes their results and the discipline's reputation.