Pointless Notes: human failed the Turing Test, updates on XiaoIce, and other chatbots

Human failed Turing Test

This part of the post is based on Turing Test, Etc (1992), by Peter Seibel.

The Loebner Prize is a prize for programs that could pass a kind of Turing Test. It started in 1991:

The first Loebner Prize competition was held on November 8, 1991, at the Boston Computer Museum. In its first few years, the contest required each program and human confederate to choose a topic, as a means of limiting the conversation. One of the confederates in 1991 was the Shakespeare expert Cynthia Clay, who was, famously, deemed a computer by three different judges after a conversation about the playwright. The consensus seemed to be: “No one knows that much about Shakespeare.”

Cynthia Joyce Clay is still online, with a blog and stuff. Search "Cynthia Clay Turing" for yourself. She apparently likes to boast the distinction of being considered not human:

I was judged to be a computer program on Shakespeare at the First Loebner Prize Competition of The Turing Test—a truly science fictional experience. I'm an author who likes to write sf, fantasy, updated versions of old myths.

Let's read: Sutton's RL, week 3, chap 3

For week 3, we will do:

• Rest of Sutton & Barto Chapter 3
• Sutton & Barto Summary of Notation, 
• Sutton & Barto Section 4.1 

Optimal Policy

Define: given a MDP problem, we have a corresponding policy spaces $\Pi$ of all possible policies for the problem. $\Pi$ is preordered by Pareto ordering:

\[\pi_1 \succeq \pi_2 \quad \text{iff} \quad \forall s\in S, v_{\pi_1}(s) \ge v_{\pi_2}(s)\]

That is, a policy is better than another iff using it does not deprove the expected value in all situations, and improves in at least one.

A policy $\pi$ is optimal iff it is a maximal element in $\Pi$ thus preordered.

The optimal state-value function is defined as as the best that can be done by any policy at a certain state, it's a bit subtler than that

 \[v_*(s) = \max_\pi v_\pi(s)\]

and similarly for $q_*$

Let's read: Sutton's RL, week 2, chap 3

For week 2, we will do:

• Read the definition given for artificial intelligence in Wikipedia and in the Nilsson book on p13; 
• google for and read “John McCarthy basic questions”, “the intentional stance (dictionary of philosophy of mind)”
• Sutton & Barto Chapter 3 to Section 3.5

Reading

From Wikipedia: 

intelligence demonstrated by machines, ... "intelligent agents": any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. Colloquially, ... a machine [that] mimics "cognitive" functions that humans associate with other human minds, such as "learning" and "problem solving".

From The Quest for Artificial Intelligence (2009), Nils Nilsson:

intelligence is that quality that enables an entity to function appropriately and with foresight in its environment... Because “functioning appropriately and with foresight” requires so many different capabilities, depending on the environment, we actually have several continua of intelligences with no particularly sharp discontinuities in any of them. For these reasons, I take a rather generous view of what constitutes AI.

Let's read: Sutton's RL, week 1, chap 2

Chapter 2

This chapter is about multiarmed bandits (MAB). 

There are two kinds of feedbacks to a RL agent:

Evaluative feedback: how well did it act (a score).

Instructive feedback: best action it could do (a correct answer)

In MAP, the environment is a Markov chain, and thus the past does not influence the future. This makes it easy. 

This is called nonassociative learning.

In general,

\[Q_t(a) \approx q_*(a) = \mathbb{E}(R_t | A_t = a)\]

$Q_t(a)$ is the estimate of $q_*(a)$, estimated at time $t$. It is updated as time goes on, hopefully approaching $q_*(a)$ better and better.

The greedy strategy is 

\[A_t = \underset{a}{\operatorname{argmax}}Q_t(a)\]

The key is to estimate $Q_t$ accurately. There are several ways.

Let's read: Norvig's AI, chap 24, 25, 26, 27

Continuing from last post.

Chapter 24: Perception

This chapter is about feelings, or how to sense the world through sensors. It's too engineering for me so I skipped it.

Chapter 25: Robotics

This chapter is about robotics, actuators, or how to actually move around in the world. This is about the most traditional and engineering part of the textbook, and quite out of my part, so I'll skim it as much as I skimmed technical stuff in mechanical engineering during my study of Newtonian physics.

My favorite part from the chapter is a game:

Exercise 25.11: pretend to be a robot! A game for people from kindergarten to PhD! 

I can think of a few variations on this game: First, to make communication strictly minimal, players should only communicate through text messages and webcams. Second, instead of one person playing the brain, why not use two? One for each hemisphere! The left brain can only see the right eye and control the right hand, and vice versa. Then there will be six players. 

And we can even imagine a game of epiphenomenon.

Wait a minute, this sounds just like specialization...

Let's read: Norvig's AI, chap 22, 23

Continuing from last post.

Chapter 22-25 are about interactions with the environment:

• 22 is about reading and 
• 23 is about writing. 

Listening and speaking are not touched upon, but it's not a problem, considering that current technologies for speech-to-text and text-to-speech are pretty much perfect.

• 24 is about feeling (perception) and 
• 25 is about moving (robotics).

Let's read: Norvig's AI, chap 3

I'm tired of Norvig's AI now. It's too long. Much better if I can just refer to specific chapters of it when I need to get an introduction to a particular topic, rather than reading it in one go.
I will just post reading notes from what I've already accomplished.

---

Continuing from last post.

Chapter 3-6 are about solving given, well-defined problems by various kinds of searching. It is quite basic algorithmic stuff. There's no learning yet, and the agent can't improve itself.