Robot Overlords

Maybe Black Mirror has broken me, but I've come to believe resistance in futile.

Engineers are going to develop tools that evaluate writing as effectively as a human.

We may be a long way from that reality, but even as we approach, what computers can glean from a text is impressive.

This worries a lot of people, and I get that.

Writing assessment is very complex work. It defines so much of what writing teachers do. But more importantly, writing assessment feels like something humans should own. Writing is, after all, a form of person-to-person communication. Right?

A computer-produced report on my writing is not a valid assessment, because it is not measuring a person's response to the text.

Only people can provide a valid assessment of a text written as person-to-person communication.

And you know what?
Teachers are people!

Teachers can do things computers can't do, like... be people.

This is a reasonable argument, up to a point.

See, teachers are people.
And people are the intended audience of most writing.
But, here's the thing, teachers are not the intended audience of most writing.

This is particularly true in college composition. As students begin college, they enter a new stage in their writing development. They are preparing to join various scholarly, civic, and professional communities.

Writing teachers do not belong to all of those communities.

As a writing teacher, if I am the only person a student has learned how to "write for," I have failed.

So, we cannot say 'the writing teacher provides a completely valid assessment.'

The only completely valid assessment of a text takes place when the actual intended reader reacts to the text.

I need my students to learn to write for a variety of readers.

My job is to help students understand this: From now on, as a writer, you will have to use what you know today in order to learn the rules anew each time you decide to write for a new audience or a new purpose.

There are tools I can give students to do that. So the function of college composition courses remains important.

But I need to leave some of the assessment to others - to my colleagues in other departments, to other students in the class, to other writing teachers, to potential employers, and the list goes on.

So, why not computers?

Are we afraid we are going to be replaced by robot writing assessors?

There is some legitimacy to that fear. If we believe that keeping people in "the assessment loop" is too expensive, or that the labor conditions writing teachers work under are unsustainable, then maybe it's time to welcome our robot overlords.

Maybe.

But that assumes we are satisfied with writing assessment as it is performed today. Are we comfortable saying that?

Try saying it: 

We know how to perform effective writing assessment that reliably predicts a person's ability to write across a wide variety of circumstances. 

I don't buy it. What we do is still very messy, and it's always going to be. That's what drew me to this work.

It was Dalí who said, "Have no fear of perfection - you will never reach it."

So, when IBM rolls out a tool that produces "Personality Insights" based on text samples, I don't see that as IBM working to replace human readers. I see IBM tinkering with just how much computers can learn from writing - and maybe just how much they can contribute to a more comprehensive assessment of writing ability.

Here's what they learned about me in a Sunburst Chart:

Reading that analysis, it feels like I visited a psychic, and not in the "Wow, that's uncanny" way.

It feels like someone sized me up as I walked through the door and then made some vague observations using language intended to sound specific.

In other words, it still feels like a trick. It's a cool trick, and it's rooted in some pretty sophisticated understandings of human behavior and communication, but... it still feels like a trick.

Trick or no, it is progress. That computer made observations about me based on my writing.

I hope these tools get better, and as they do, I hope to use them to add more dimensions to the work I do when I assess the writing my students produce.

As it stands today, I recommend my students consider assessment from their peers, from me, from tutors, from roommates, from professors in their major, from grammar checking software, from study groups, and anywhere else they can get some assessment. And then I ask students to consider the feedback critically and use it to improve their work.

I can't think of a good reason to treat assessment software differently.

If someone tried to suggest that such software will replace me and all those other sources of assessment, I'm ready to tell them why that is absurd. But I think they know. If not, the computers will probably tell them...

Or will they?

On Response to Writing

From Incidental Comics

Why Johnny Can't Teach

A lot of my colleagues are super upset about an advice piece published in the Chronicle of Higher Education.

For those of you who live outside of the world of composition studies, I'd like to explain what all the fuss is about.

I kid you not, the advice piece opens with this:
"My students can’t write a clear sentence to save their lives, and I’ve had it."

From barefootattitude

The author has dug himself into a hole right there. This is a teacher who starts the conversation by blaming students.

For professional compositionists, that's like blaming the controller in a video game (or the sound engineers at a presidential debate). Except, in this case, the author is blaming the people who have come to him for an education, blaming them for making his job difficult.

Please understand, this teacher's job is difficult.
Teaching students to write in a college setting is a profoundly difficult task...
But that is not the students' fault.

Students arrive at colleges and universities each semester with some writing skills, and then something frustrating (but important) happens.
The rules change.
Institutions ask students to repurpose those skills and apply them to a set of activities most students have never engaged in before.
Writing is no longer a way to show a teacher that "I did the reading."
Suddenly students are expected to use writing to critique a reading, analyze lab results, generate new understandings of a complex topic, and the list goes on.
Our most fortunate students may have started doing some of that in 11th or 12th grade, but it is an ability that requires years to develop. And while that ability is developing, general writing performance almost always dips a bit.

So, this advice piece starts in a bad place, a place that demonstrates a failure to understand what it is we are asking our students to work towards.

And that's just the opening sentence.

The author goes on to argue there's something else making his job difficult: The discipline of Composition Studies itself.

I'm not going to get into the weeds on this, but here's a breakdown of what happens.
The author lists three core principles associated with quality writing instruction.
The author states he has tried to teach using those principles.
The author claims the principles "rarely work."
The author goes back to blaming students.

From barefootattitude

That's right, back to blaming the students.

The author tells us that... 

• "Students do not revise." 
• Students do not know how to give good feedback to their peers 
• Students don't use good advice when they get it
• Students don't know how to use basic argument structures 

To which, I imagine many of my colleagues responded with (and please feel free to insert the profanity of your choice), "Do your job!"

It's all hard, but that's the job. 

We are teaching people how to write for academic purposes. Revision is necessary. The ability to recognize good writing is necessary. Understanding feedback from peers and mentors is necessary. The ability to make complex arguments is necessary. 

The ability to "write a clear sentence" is not enough, and if we ask students to do all the necessary work, the students' sentences are going to get messy. 

Stop complaining about how hard their work is. 

Get in there and help them do the hard work.

Well, That's Original

When most of my students think about plagiarism, they are afraid.

They are afraid of harsh consequences, marred permanent records, and potential expulsions.

This bothers me.

A lot.

Students are not introduced to the concept of academic integrity; they are introduced to the violation of that integrity. And often the introduction has a "guilty-until-proven-innocent" vibe.

I work hard to show why that fear is unproductive.

There is a reason universities treat intellectual property the way they do. It is a value the community holds, and not every community feels the same way. Students should know that when we enforce rules about academic integrity, we are not teaching a universally held belief. We are teaching the values of the institution.

I ask my students to explore this and write an informative essay for incoming freshmen that explains why higher education places such an emphasis on crediting our sources.

I'm looking forward to adding this set of posts from Imgur to the readings.
In it, one user animated another user's comic about how the community on Imgur.com treats original content.