These graduate students had no idea their teaching assistant was a robot

'Jill Watson' now answers students' questions with 97 percent certainty

On the Internet, "nobody knows you're a dog," as the old meme goes, and today, the same can increasingly be said of robots.

There are already scheduling robots that are virtually indistinguishable from humans, and recently students at the Georgia Institute of Technology learned that "Jill Watson" -- a teaching assistant they had relied upon all semester -- was in fact artificially intelligent.

“The world is full of online classes, and they’re plagued with low retention rates,” said Ashok Goel, a Georgia Tech professor who teaches a class entitled Knowledge-Based Artificial Intelligence. “One of the main reasons many students drop out is because they don’t receive enough teaching support. We created Jill as a way to provide faster answers and feedback.”

The class is a core requirement of Georgia Tech’s online master’s program in computer science, and it tends to draw a lot of questions from students. It's offered every semester, and each time, the 300 or so students enrolled post roughly 10,000 messages in the course's online forums, Goel estimates.

That volume has often overwhelmed Goel and his eight teaching assistants, so this time, he added a ninth: Jill.

Jill is a virtual TA, and -- as her name suggests -- she's implemented on IBM’s Watson platform. Goel and his team of graduate students started to build Jill last year, including training her on the roughly 40,000 questions that had been asked in the KBAI class since it was first offered in fall 2014.

“One of the secrets of online classes is that the number of questions increases if you have more students, but the number of different questions doesn’t really go up,” Goel said. “Students tend to ask the same questions over and over again.”

Jill started work as a TA in January, but she wasn’t very good for the first few weeks, often giving odd and irrelevant answers. Her responses were posted in a forum that wasn’t visible to students.

“Initially her answers weren't good enough because she would get stuck on keywords,” said Lalith Polepeddi, one of the graduate students who co-developed the virtual TA.

For example, on one occasion, a student asked about organizing a meet-up to go over video lessons with others, and Jill gave an answer referencing a textbook that could supplement the video lessons: same keywords, but different context.

Further tweaking ensued, and today Jill can answer questions with 97 percent certainty. Initially, the human TAs would upload her successful responses to the students, but by the end of March, Jill didn’t need any assistance: She wrote to the class directly if she was 97 percent positive her answer was correct.

The big reveal didn't happen until late April, when Goel informed his AI students that they had actually been interacting with a bot all semester. One reportedly said her mind was "blown." Since then some students have organized an alumni forum to learn about new developments with Jill after the class ends; another group has launched an open-source project to replicate her.

Next semester, Jill will return with a new name and a new goal: to answer 40 percent of all class questions by the end of the year.

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags education

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Katherine Noyes

IDG News Service
Show Comments

Father’s Day Gift Guide

Most Popular Reviews

Latest News Articles

Resources

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?