FeaturedPrecarious Work

Making the Grade: Rating Professors

Created by Silicon Valley’s John Swapceinski[1] during the 1999 dot-com boom, Rate My Professors (RMP) is the most popular professor ratings site today, having survived competitors such as PassCollege, ProfessorPerformance, Reviewum, and RatingsOnline and featuring far more reviews and monthly visits than current rivals, including MyEdu (formerly Pick-A-Prof), Koofers, Uloop, and RateMyTeachers. With over 15 million reviews of 1.4 million professors, the site allows students to research prospective instructors and rate their current teachers on 1 to 5 scales measuring “overall quality” and “difficulty” with an accompanying happy face representing “Good Quality” (3.5-5), an emotionless face representing “Average Quality” (2.5-3.4), and  a  sad  face  representing  “Poor  Quality” (0-2.4).[2] Students can also write 350-character descriptions of their instructors and courses and click on a “chili pepper” indicating whether they found their teachers physically attractive.

Discussions of RMP frequently question that site’s reliability in view of its alleged selection bias, that is, the probability that students with either very positive or very negative experiences would be most likely to take the time to review an instructor.[3] However, my research, based on both interviews and surveys, found that RMP users appear to enact a shared code that defies the individualist orientation of competitive consumer identities and relations, and that reviewers were for the most part focused less on punishing or rewarding individual instructors than on “helping”—or “paying it forward” to—fellow students.

In fact, the existence of a community of explicitly pro-student and conscientious RMP reviewers evokes the historic origins of student evaluations, which developed as a tactic for advancing popular political demands for student empowerment during the 1960s and 1970s radical student movements. Yet, a system originally based on challenging the inherently hierarchical and sometimes abusive teacher–student relationship has, as with so much else, evolved into something quite different during the forty- year consolidation of the neoliberal university. The evolution from student-as-political-activist to student-as-consumer-activist was neither inevitable nor organic, but instead reflects, among other things, the effectiveness of top- down efforts to fully corporatize the university while redefining students in its image.

The evolution from student-as- political-activist to student-as- consumer-activist was neither inevitable nor organic…

Demonstrating the success of the corporate university’s ideological reconfiguration, RMP highlights the contingent nature and limitations of consumer power both in student life and beyond. On the one hand, the ability of student reviewers to publicly share their thoughts is contingent upon the online platform provided by RMP, whose business model is based on increasing user traffic—and thereby advertising revenue—through amassing reviews via users’ unpaid labor. RMP circumscribes criticism of not only instructors but also the university itself, restricting reviewers’ ratings of their schools (via an RMP blog)  to factors  such  as  “Internet,” “Clubs,” “Social,” and “Food” with no space to think about, for instance, the cost of tuition or the size of classrooms. On the other hand, student reviewers are highly cognizant of their ultimate role as consumers, and as such express their desire to select instructors who are easy graders—a self- defeating effort insofar as it accelerates grade inflation,[4] thereby diminishing the value and utility of high grades. That is, RMP provides students with essentially illusory power that ultimately does little more than reduce the power of not the university per se but its most vulnerable faculty: adjuncts. Adjunct instructors, motivated both by their desire to improve their teaching and by their fear of under-enrollment, frequently read their own reviews on RMP and can alter their teaching habits in response to reviews.

Students as Consumers

In 2015, I joined two dozen college and university Facebook groups to recruit respondents to two surveys on the website SurveyMonkey concerning students’ use of and attitudes toward RMP.[5] Out of forty-one respondents to my general survey, forty had consulted RMP at some point during their college careers, with thirty- two—or nearly 80 percent—answering unreservedly, writing “Yes,” “Yes always,” “Of course,” or “Definitely.” Moreover, thirty-six respondents—or 88 percent—have written at least one review, and several noted that they have written reviews of every course they have taken. A total of fourteen respondents wrote that they were motivated to write reviews by their desire to “help” other students or “give back” to a service that they have benefited from, while a smaller number noted that they were interested in helping their instructors or in correcting perceived unfairness in existing reviews of a particular instructor.

To be sure, several respondents indicated that they write reviews because they either loved or hated an instructor: “I only write them for two reasons. I love the professor, or I dread them.” Yet, far more respondents wrote that they were motivated by a form of altruism. Indeed, even when students vent about “bad” instructors, it is often inseparable from altruistic concerns, as students who do write harsh reviews frequently say that they do so not to enact personal revenge but to provide a “heads up” or, as several students noted, to “warn other students of bad professors.” Another student stated that he writes reviews of “all of my instructors because good or bad it could help future students when picking out classes like it always does for me.”

RMP provides students with essentially illusory power that ultimately does little more than reduce the power of . . . its most vulnerable faculty: adjuncts.

For many reviewers, this sense of duty to fellow students is coupled with a commitment to fairness. They were generally taken aback by questions concerning possible RMP impropriety, all forty-one answering “no” to my question on whether a respondent has ever threatened an instructor with writing a negative review. Several respondents scoffed at the suggestion, writing,

I would think they either wouldn’t care or would write me up if I suggested such a preposterous thing. If there is a REAL problem, go to the administration, not some dumb website where you can write anything you want without verification.

What is notable is that although numerous respondents explicitly use RMP to find easy instructors, they simultaneously bridle at the suggestion that they would use the threat of a negative RMP review to induce an instructor to provide a good grade. That is, indirect coercion through the marketplace can be consistent with maintaining students’ integrity and idealism, while direct, interpersonal coercion is dismissed as either a practical absurdity or an affront to one’s dignity.

It is significant that RMP reviewers, unlike, for instance, Yelp reviewers, do not have screen names or other identifiable site presences and therefore do not win compliments, let alone “friends,” for being “funny” or “cool.” At most, reviewers receive points for being “helpful,” although these points accrue to the anonymous review itself rather than the reviewer. The fact that RMP reviewers do not build reputations or receive the various immaterial rewards associated with them supports the claim that reviewers are indeed sincere when they assert that they are motivated to help others.

 . . . .[A]lthough numerous respondents explicitly use RMP to find easy instructors, they simultaneously bridle at the suggestion that they would use the threat of a negative RMP review to induce an instructor to provide a good grade.

But what is it that RMP reviewers are alerting other students to? Many referred to the ease of the course, including “toughness,” “work- load,” or “grading.” Some were more overt: “Honestly, how easy the professor is. Why would I take the same class with a harder professor???” Often this was reflected in their comments about fairness, as in the comment, “Ease of class and fairness of grading.” In view of the fact that students are not generally privy to everyone’s grades in the course, “fairness” most likely implies “reasonableness” which implies not being too difficult, a demand that numerous students made explicitly. Only one respondent explicitly referred to rigor as a positive trait.

While reviewers were split on whether they believed reviews directly affected instructors, it is notable that of those who believe that reviews do affect instructors, several believe that they do so only indirectly via altering class size: “I think it may affect what classes students choose,” and “I think it affects them if the student could choose another class.” Regardless of the reality, approximately half of the reviewers believe that they are unable to directly change instructors’ behavior, but they can affect instructors’ course sizes and thereby potentially affect the instructors themselves. Accordingly, most respondents agree that RMP has empowered them in their efforts to secure better grades. As one reviewer notes, RMP “certainly has empowered me in getting the most for my money at my university.”

Impact on Adjuncts

RMP does not merely encourage students to think of themselves as consumers; it shows that they already do, a fact that is explicable not ultimately to bad or entitled attitudes but to the material economic realities of higher education in which students are making an investment— often a massive one that they will be unable to ever pay back—that they seek to maximize.

RMP has affected the way some adjuncts teach, a point expressed not only by some of my respondents but also by recent research demonstrating  the  relationship  between RMP and grade inflation.[6] Due to their awareness of RMP, respondents report feeling pressure to “play to the crowd,” returning assignments more quickly, and an understanding that “poor grades lead to poor evals.” These adjuncts expressed particular concern that poor reviews would lead to under- enrollment, leading them to lose courses and, with them, income. Although some respondents shared their belief that their department chairs monitor RMP, RMP affects adjuncts regardless of whether their employers ever look at the site.

Due to their awareness of RMP, [adjunct] respondents report feeling pressure to “play to the crowd” . . .

I surveyed forty-seven adjunct instructors recruited from Facebook adjunct groups throughout the country (an arguably politicized sample) and my own department’s student list- serv. Of these, forty wrote that they check their RMP reviews, an unsurprisingly high number given that RMP reviews are by definition devoted to instructors per se, and that the stakes are both personal and high. Instructors who check their RMP reviews are making sure that nothing libelous or embarrassing is being writ- ten about them, or, conversely, seeing if they have received any compliments. That many instructors tend to self-monitor their reviews on RMP is quite explicable but also indicates that instructors must pay for the relative prestige associated with being a college instructor (vs., say, a restaurant server) by proactively checking and to a degree internalizing their own consumerist surveillance. If they do not check the site, instructors can conceivably be at a disadvantage vis-à-vis students (and possibly colleagues and administrators) who are “in the know” about instructors’ popularity, tendencies, foibles, and quirks if not more harmful information.

Answering whether, and if so why, they check their reviews, respondents wrote comments, including, “Yes. Always read them. They have been consistently very positive. But, a change in that would make me reevaluate my approach”; “Yes, I want to know how I am perceived by students”; and “Yes. I care about student concerns.” Two respondents referred to the psychological lift provided by positive reviews, with one asking whether the compliments made up for the job’s paltry wages: “Yes. I usually got good reviews, and it helped boost my ego. Was that compensation for being underpaid? It actually helped, since it said that the people I was really working for appreciated my efforts.”

Rampant Sexism

Describing the inevitable flip side of student compliments, numerous respondents recounted the emotional difficulty of reading insulting and sometimes malicious anonymous criticisms: “They’ve been harmful in so far as some students have said things that have hurt my feelings and made me self-conscious”; “Sometimes they are just hurtful rather than being productive”; “I read evaluations once, they were mostly good but the bad ones were hard to for- get”; and “They are often harmful because they are anonymous and can be quite unfair. The last thing any poorly paid adjunct needs is to be dumped on with negativity.”

One instructor wrote,

I used to read my evals on Rate My Professor but they weren’t helpful. My first reviewer posted three comments in a row and  included  comments  like  “She has PMS 24/7” and “She should be fired.” The long ranty stuff was removed either by the reviewer or by RMP, but the “She should be fired” remains. Or so students tell me. The reviews I receive through the department are far more helpful.

Indeed, on RMP, the inherently discriminatory character of rating systems in general is routinely   expressed   through   the   particular prejudices of contemporary U.S. society. Above all, RMP reviews are sexist. In addition to numerous cases of baldly misogynistic reviews (one instructor received a review, quickly taken down, calling her “a real bitch,” although comments calling her “uptight” remain on the site), a 2015 study indicates that reviewers tend to rate male professors on a so-called “intelligence scale” and women professors on a so-called “nurturing scale” notwithstanding the fact that male and female instructors perform the same job.[7]  One analyst has generated an interactive word-counting chart demonstrating the consistently gendered language RMP reviewers use to describe their professors, with women being far more likely to be described as “unreasonable,” “shrill,” “hateful,” “lovely,” and “beautiful,” while men are far more likely to be described as “pompous,” “egotistical,” and “brilliant.”[8] While one might object that reviewers’ biases undoubtedly preexist RMP and affect department student evaluations as well, RMP ratings’ public and student audience, as well as the site’s explicit encouragement to rate professors’ “hot- ness,” clearly promotes the objectification of teachers, a phenomenon that traditionally dis- proportionately harms women.[9]

. . . [R]eviewers tend to rate male professors on a so-called “intelligence scale” and women professors on a so-called “nurturing scale” . . .

There is also evidence that RMP reviews discriminate against instructors, specifically Asian ones, who have accents. A recent study argues that given the rigorous English proficiency requirements placed on college and university instructors, such bias likely reflects a parochial inclination among U.S. students to make little effort at understanding those who are racially categorized as “Others,” a tendency that, notably, does not apply to instructors with British accents.[10]

Education as a Consumer Good

More  broadly,  the  consumer  basis  of  RMP threatens to deskill all precarious instructors, who are pressured to subordinate rigor to laxity and pedagogical autonomy to indulgence. One adjunct writes,

They have added to the drive to make higher education an infotainment industry. What students think is fun in the classroom is not necessarily what or how they should be learning. By way of example, one of my colleagues is repeatedly reviewed positively on RMP because his students note that “he can be bribed on grades with cigarettes.”

Others observe, “Holding students to any sort of standard is seen as a bad thing to do; doing so leads you to being labeled as ‘mean.’ Administrators frown on this.” And “Yes. I was guilty of expecting students in the final required course of an English composition to be able to read and write. This did not please my administrators.”

. . . [T]he site’s explicit encouragement to rate professors’ “hotness” clearly promotes the objectification of teachers, a phenomenon that traditionally disproportionately harms women.

Numerous instructors  report  feeling  that they are increasingly caught between fulfilling their professional desires to teach and the need to get positive reviews, reflecting the mutually antagonistic aims of student satisfaction and student learning that crystallize in RMP reviews. One respondent wrote that:

Overall, negative student evaluations have had a bearing on my confidence in seeking full-time employment—it is not only information that is publicly available—it’s publicized as a selling point by my employers in trying to attract students to enroll. Future employers can easily access it.

Another instructor echoes this concern, writing,

Reviews can be harmful, especially RMP, because they are so unreliable, but can have a big impact. Any individual review may not have a big effect, but the overall cumulative effect of having these reviews floating around definitely creates a “consumer” mentality, which can cause teachers to second-guess their pedagogical decisions based on what reviews might say. I’ve seen it happen and felt that pressure myself, though I try to resist it. In the end it’s something everyone feels, but that’s difficult to quantify.

Between a Rock and a Hard Place

Although many respondents acknowledged that RMP reviews could affect their job prospects, just as many respondents seemed disillusioned about those very prospects, commenting: “. . . what job prospects? Other crappy adjunct jobs? It’s not like we’re in line for any soon-to-be- obsolete full-time positions”; “Not really, since the probability of being hired full time was very low”; and “Maybe but I don’t think so b/c as an adjunct I have no job prospects.”

Ironically, disillusionment over poor professional prospects might have emboldened numerous respondents to adhere to professional grading guidelines in spite of departmental pressure to pass failing students, an apparent point of pride among instructors who are poorly remunerated and often treated disrespectfully by administrators and students. At the same time, the intense competition for dwindling tenure-track jobs has invested student reviews— both department and RMP—with a lose–lose quality, a phenomenon keenly recognized by several respondents: “Bad ratings kill job prospects. Good ratings don’t have much effect.” At the same time, other respondents noted that ratings that seem inordinately high could also draw scrutiny from department heads concerned about “pandering.” And another adjunct suggested that negative student reviews might only be “scrutinized when an excuse is sought” by the department to terminate or otherwise punish the instructor. It is within this context that many adjuncts are fighting a rearguard battle to maintain grading professionalism while not scaring off students and reducing their class sizes to the point where they lose the limited work that  they  do  have.  In spite of many adjuncts’ devotion to their teaching, there is enormous institutional pressure on adjuncts to make things easier on their students and them- selves by passing undeserving students—though not being too easy, an arbitrary standard that little reflects actual assessments of learning.

. . . [T]here is enormous institutional pressure on adjuncts to make things easier on their students and themselves by passing undeserving students . . .

It is arguably because adjuncts are in the crossfire of the ongoing conflict between the competing conceptions of the modern student that adjuncts answered the question of whether students   have   become   empowered   through reviews   by   referring   explicitly   to   students becoming consumers.  Respondents’ answers include the following: “Yes, as ‘consumers’”; “They believe reviews give them consumer clout. The End of Days is nigh, for higher ed, anyway”; and “Absolutely.  Students are now customers seeking service rather than teaching.” As noted, adjuncts reasonably assume that, irrespective of whether their chairs consult RMP, they will not be rehired if their courses are regularly under-enrolled. Due to its ability (clearly recognized by adjuncts) to affect course sizes through generating and publicizing instructor reputations, RMP has accordingly affected the way some instructors teach.  Indeed, the large majority of instructors I interviewed and surveyed internalize their own surveillance and management by reading RMP reviews, an act they presumably perform during their unpaid free time. RMP has thus harnessed students’ collective power in changing the ways that some courses are taught. As universities and colleges increasingly rely on adjunct employment, the power of online student reviewers will arguably only increase.

The Reality of Student Empowerment

But what kind of collective power is this? On the one hand, student reviewers produce wealth for RMP through writing and posting the unpaid reviews that are the basis of that site’s online traffic and advertising revenue. This unpaid work can be viewed as the fare reviewers must pay to amplify their voices on RMP’s mass platform. On the other hand, reviewers derive a sense of power, enjoyment, and satisfaction from participating on the site. Many students take their reviews seriously and adhere to for- mal and informal guidelines centered around their notions of fairness, honesty, and duty to fellow students. Indeed, RMP provides users with a community of like-minded users devoted to common goals, harnessing users’ senses of cooperation and altruism. However, such val- ues are produced through the concretization of consumerist identities and goals that ultimately challenge not students’ actual economic antagonist in the form of the university—let alone the broader economic system, in which would- be workers’ dependence on unaffordable higher education functions as a form of indentured servitude—but the university’s far more vulnerable agent—the  instructor—and  specifically the adjunct instructor.

Consumer activity as directed through RMP is accordingly not fundamentally designed to advance students’ economic interests or, arguably, the broader interests of society. RMP reviews are neither designed nor equipped to challenge the costs of tuition and are thereby incapable, for instance, of ameliorating crippling student debt.  Furthermore, the greater availability and concomitant commercialization of higher education have predictably come at the price of both inflationary grades—arguably accelerated by RMP—and inflationary degrees that have not only lost their status as elite signifiers but also fail to provide most students with the future income required to tackle that debt. RMP has helped accelerate the conversion of  students  into consumers but  not  powerful actors in the modern  university. While students must  pay their debts in the future, adjunct instructors are experiencing the costs in the here and now.

By Joshua Sperber

NOTES

[1] Swapceinski subsequently became a co-founder of RateMyBoss 
[2] Rate My Professors (RMP) changed its ratings criteria in May 2016, replacing “clarity,” “helpful- ness,” and “easiness” with “overall quality” and “difficulty.” RMP told me that they instituted these changes “after receiving feedback from students.” 
[3] James Otto, Douglas A. Sanford, and Douglas N. Ross, “Does Ratemyprofessor.com Really Rate My Professor?” Assessment & Evaluation in Higher  Education  33,  no.  4  (2008):  355- 68; James Felton, Peter T. Koper, John B. Mitchell, and Michael Stinson, “Attractiveness, Easiness, and Other Issues: Student Evaluations of Professors on RateMyProfessors.com,” Assessment and Evaluation in Higher Education 33,  no.  1  (2008):  45-61;  Janice  Fiamengo, “How Well Does ‘Rate My Professors’ Rate?” PJ Media, February 23, 2013, available at https://pjmedia.com/blog/how-well-does-rate- my-professors-rate/?singlepage=true. 
[4] Wolfgang Stroebe, “Why Good Teaching Evaluations May Reward Bad Teaching,” Perspectives on Psychological Science 11, no. 6 (2016): 800-16. 
[5] The Facebook student groups had anywhere from thirty to over ten thousand members, among  whom  forty-one  students  completed one general survey and thirty-one completed a shorter survey (focusing specifically on the RMP “chili pepper”). They represented a broad distribution  of  majors  across  the  humanities,  natural  sciences,  and  social  sciences and a broad geographic range of U.S. liberal arts colleges and medium- and large-public universities. 
[6] Stroebe, “Why Good Teaching  Evaluations May Reward Bad Teaching.” 
[7] William Huntsberry,  “How We Talk  about Our Teachers,” WNYC, February 23, 2015, available at     http://www.wnyc.org/story/ how-we-talk-about-our-teachers/. 
[8] Ben Schmidt, “Gendered Language in Teacher Reviews,” Gendered Language in Teaching Evaluations, February 2015, available at http://bit.ly/2btMfJ5 
[9] David Epstein, “‘Hotness’ and Quality,” Inside Higher Ed, May 8, 2006, available at https:// www.insidehighered.com/news/2006/05/08/ rateprof. 
[10] A Scottish brogue for many may very well be just as challenging to understand, but the aver- age U.S. student is arguably likelier to identify such accents as charming and worth the effort to  pay  attention.  See  Scott  Jaschik,  “Accent on Bias,” Inside Higher Ed, March 2, 2015, available at https://www.insidehighered.com/news/2015/03/02/study-finds-instructors-asian- last-names-receive-lower-scores-rate-my-profes- sors; and Nicolas Subtirelu, “‘She Does Have an Accent but . . .’: Race and Language Ideology in Students’ Evaluations of Mathematics Instructors on RateMyProfessors.com,” Language in Society44, no. 1 (February 2015), available at https:// www.cambridge.org/core/journals/language- in-society/article/she-does-have-an-accent- but-race-and-language-ideology-in-students- evaluations-of-mathematics-instructors-on- ratemyprofessorscom/1D4C9E099B351BAD- E27C9211526C4E01.

2 thoughts on “Making the Grade: Rating Professors

  • Joshua Sperber

    That’s an excellent point, Eva Swidler.

  • An additional important consideration is the effect that such sites have on the content of what is taught and discussed in all classrooms, but especially adjunct-taught ones. What vulnerable professor wants to raise an issue that might anger even only a vocal and aggressive five percent of students? Any adjunct knows that all you need for a lousy showing on a ratings site is a handful of angry trolls sitting in the back row of class, getting resentful about class time spent on a topic they disdain.

Comments are closed.