An Essay On, Well, Essays

By Noah Caldwell

I remember the childish joy of going to my elementary school’s computer lab. In hindsight, I’m sure the miniscule screens we were plopped down in front of were downright archaic—clunky machines still years away from being blessed with Windows 98. But at the time, visiting those rows of interminable beige cubes was as fun as any field trip Ms. Case could dream up, and we giddily made the long walk down the hall and up the stairs when it was that time of the week.

The Paint application was a revelation. Geometric designs far beyond the capabilities of our shaky fingers were just a click away. No faint, uneven crayon lines. As time went by, we realized we could clandestinely play Oregon Trail in a different window, something that provided my first memory of a few key things: procrastination, the curious joy of digital interaction, and, of course, the adrenaline of guiding my pixelated family safely across the Great Plains in a covered wagon, hoping the wheels would hold as we forded the Willamette River.

That, as far as I’m concerned, was how you learned from a computer. But, sadly, in researching the current age of standardized testing for this issue, I came across one of the main modes of digital interaction that kids now have to look forward to: writing essays on computer-generated grading programs.

Take, for example, the Intelligent Essay Assessor (IEA), widely considered the most high-tech artificial intelligence grader in the world. IEA is trained in “domain-representative texts”—essentially, it learns a bunch of books on a given subject—and uses something called latent semantic analysis to evaluate how well students can touch on the concepts in those texts. It compares conceptual and contextual relevance between the writing of a seventh-grader and that of an industry-leading author. It knows mechanics and grammar better than you or I do.

Imagine how much or how little joy—or, merely, interest—is garnered from a student’s interaction with IEA. What goes through his or her mind? The realization that this computer has more power over the grade of the current project than the class teacher? The instinctive understanding that one must write according to the computer’s guidelines? The eventual learning experience of failing and curbing your creativity to please an algorithm?

IEA isn’t a gimmick—it’s not a futuristic toy that defense engineers at DARPA play around with while joking about robots taking over. It’s the essay-grading platform used by Pearson, which is the largest education publisher in the world and the company that was recently chosen to be the grader of Colorado’s Common Core-aligned standardized tests. If you’re reading this on the Front Range and have a child between the ages of six and 14, he or she will likely use IEA in the near future. (It was also, coincidentally, developed by Professor Thomas Landauer at CU-Boulder.)

On the face of things, it looks like the wave of the future. There are definite benefits to automated essay grading. It saves loads of time for teachers, who can fill newfound hours with instruction. It can easily find plagiarism, meaning kids will be held accountable for their own work, and presumably study harder. It provides an objective standard, taking away the wild card of a teacher grading two different essays in two different moods, giving vastly different scores.

In most schools, some of the best-trained graders are capable of scoring about 30 essays an hour. IEA moves a little quicker—the official specs haven’t been released, but a similar grading program can churn out 16,000 essays in 20 seconds. For schools short on people qualified to grade, funding and resources, an automated system could be a viable option.

As a typically self-critical writer, I have always looked to the creativity of literary peers to eliminate pessimistic thoughts about my own composition. I stand agape when I see an approach that breaks molds and slyly prods the reader to carry on in awe. It is what lets our collective opus of writing evolve. But that trait, that skill, is meaningless to an automated essay grader. The famously guttural first line of The Illiad—“Rage!”—wouldn’t even register as a sentence to IEA. What if a wily student wanted to emulate David Foster Wallace’s habit of footnoting his work, hoping to catch his teacher’s eye with little scribbles cramming the bottom edge of a lined piece of paper? IEA wouldn’t recognize the creative ploy, nor would it even be possible if the student were typing his essay into a text box on a computer. This certainly isn’t to say that the public education system is filled with Homers and Wallaces that have flunked out due to restrictive essay grading, but rather that if children never have the chance to write creatively—because they’re only learning to write in a way that will pass a test—those same children might never learn to write creatively at all.

I recently stumbled upon the musings of Doug Hesse, who teaches writing at the University of Denver and is an outspoken critic of automated grading. Hesse’s commentary goes well beyond mine in scope and approach, but, in my opinion, his greatest contribution to the topic is a simple exercise he made up—a “parlour game” in his words.

Hesse tried to write the worst essay that would get the best score on IEA. What resulted was his masterpiece “An Essay on Aphasia.” The content was well composed, but incoherent to a human reader. It expounded at length about the merits of aphasia, without ever specifically describing what it was. (Don’t worry; I didn’t know what that word meant either. I had to look it up: aphasia is when one’s language comprehension is hindered by an episode of brain dysfunction. Now you know.) The essay was, to a human, meaningless.

In the four categories of scoring (Overall, Content, Style and Mechanics), Hesse got 10, 10, 10 and 7, respectively. That’s 37/40, or an A-. When I read about Hesse’s brilliantly simple experiment, I felt both self-righteously vindicated and horrified at the results. On one hand, he had proven that IEA is faulty; on the other, every day automated grading is more present in classrooms.

Another experiment with automated essay graders was even more unnerving. Les Perelman, director of writing at MIT, ran his own experiment on similar grading software. His findings? He says the programs have a limited standard for good writing, can’t identify whether facts are true or not, and can easily be gamed. If you’re interested in getting a better grade, he found that programs assigns higher grades to essays with these attributes: long paragraphs, long sentences, big words, and sentences that start with “however” or “moreover.” Also, sentence fragments or ones that start with conjunctions are a sure way to lose points. Within these parameters, he managed to craft an essay nonsensically arguing that the reason for high college tuition was greedy, overpaid teaching assistants. It even had a line from Allan Ginsberg’s “Howl” thrown in randomly for a laugh—and his composition received a perfect score from the software.

Ten minutes of researching America’s current levels of public school funding can teach the average Googler one thing: all schools are obviously not created equal. And the lines that divide them are exactly what you would expect—they are the same socioeconomic, ethnic and geographic separations that have plagued American schools for generations. Some have Smart Boards. Some have chalkboards. Some plead with school boards for new textbooks, and some use surplus revenue to take kids on language immersion trips. Some will receive more federal funding because they used IEA—and some won’t.

Computers have come a long way—instead of the simple Oregon Trail machines of yesterday, they’re now determining which kids write essays well enough to pass their classes or even get into the college of their. And while it’s almost impossible to know whether automating grading is the next big thing or the beginning of the end for our nation’s schoolchildren, but the most important aspect is that we take an active role in our students’ education and start the discussion. If we’re the ones making big decisions that can dictate the rest of our children’s lives, the least we can do is education ourselves on the topic.