Pointless Notes: human failed the Turing Test, updates on XiaoIce, and other chatbots

Human failed Turing Test

This part of the post is based on Turing Test, Etc (1992), by Peter Seibel.

The Loebner Prize is a prize for programs that could pass a kind of Turing Test. It started in 1991:

The first Loebner Prize competition was held on November 8, 1991, at the Boston Computer Museum. In its first few years, the contest required each program and human confederate to choose a topic, as a means of limiting the conversation. One of the confederates in 1991 was the Shakespeare expert Cynthia Clay, who was, famously, deemed a computer by three different judges after a conversation about the playwright. The consensus seemed to be: “No one knows that much about Shakespeare.”

Cynthia Joyce Clay is still online, with a blog and stuff. Search "Cynthia Clay Turing" for yourself. She apparently likes to boast the distinction of being considered not human:

I was judged to be a computer program on Shakespeare at the First Loebner Prize Competition of The Turing Test—a truly science fictional experience. I'm an author who likes to write sf, fantasy, updated versions of old myths.

The reason for her participation is

... she didn’t worry about people judging her humanity—to her the test was just a great chance to talk about Shakespeare for three hours.

During the contest nearly all the judges asked what her favorite Shakespeare play is. To keep things interesting for herself she answered differently every time, a maneuver that confused onlookers at the museum. She also warned the judges that it is bad luck to name the Scottish play (MacB**h) by its proper name unless you are working on a production of it, and when asked to interpret Lady MacBeth’s character mentioned all the “s sounds in her speeches which make her sound very sinister. Snaky.” This sort of analysis earned her the rating of “the most human” of all the programs and confederates, but two judges scored her as a computer, probably because she seems to know more about Shakespeare than is humanly possible. When I read the transcript of the test at home I was pretty sure that she was a human but I couldn’t help hoping that somehow she was a clever program.

Loebner is pretty open about a future without humans:

It amuses me to imagine a day in the distant future when humans have become extinct, surpassed by our creations, robots, who roam the universe. I like to think that these robots may have a memory of us humans, perhaps as semi-mythic fractious demigods from the distant past who created them. And, just possibly, they will remember me.

I read "MacBitch" in place of "MacBeth", ha.

On the other hoof,

Ms. Lisette Gozo was honored as the most human of the agents for her discussion of women's clothing, although one judge rated two computer programs above her.

But mostly, the computers did badly in this test.

Several programs tried this tactic [of faking typo] to add a bit of a human touch but some were more successful than others—one program made a typo, backed up to fix it, and then typed the correction ten times faster than any human could.

FRUMP

FRUMP = Fast Reading, Understanding and Memory Program

... one fairly successful program written in the late 70s, FRUMP, which read and summarized newspaper stories was given a story that started with the sentence, “The death of the Pope shook the world.” FRUMP summarized, “There was an earthquake in Italy. One person died.”

XiaoIce

As far as I know, the most successful chatbot is XiaoIce. It made a splash in western media during 2016, but despite the silence after that, it continued to be popular in China and grew in ability and popularity. 

May 16, 2018, XiaoIce sang a new song. A few in the English part of Internet did take notice and report on it.

Roughly, XiaoIce was born in 2014 and is currently in the 6th generation. She can now talk on the phone, with a 3d avatar, sing songs...

For more on the song, refer to XiaoIce Band: A Melody and Arrangement Generation Framework for Pop Music (2018),  Hongyuan Zhu et al.

... We propose an end-to-end melody and arrangement generation framework, called XiaoIce Band, which generates a melody track with several accompany tracks played by several types of instruments... we conduct extensive experiments on a real-world dataset, where the results demonstrate the effectiveness of XiaoIce Band.

Mitsuku

I was thinking about trying out the new chatbots myself, when I realized something: I have very little interest in small talks. Most of the times, I want technical discussions about rather factual and impersonal things, like mathematics, physics, AI, evolutionary psychology, etc. When I want to talk about personal things I do it only with established friends. 

I don’t think I ever have fooled a human into thinking I am human.
    What’s it like? ---- Mitsuku

Personally, when I try a Turing test, the first sentence would be like this:

Gssussuses waht tihs snteecns meens!

So far they all fail to understand it.

But I tried Mitsuku anyway.

What?!