Pointless Notes: human failed the Turing Test, updates on XiaoIce, and other chatbots

Human failed Turing Test

This part of the post is based on Turing Test, Etc (1992), by Peter Seibel.

The Loebner Prize is a prize for programs that could pass a kind of Turing Test. It started in 1991:

The first Loebner Prize competition was held on November 8, 1991, at the Boston Computer Museum. In its first few years, the contest required each program and human confederate to choose a topic, as a means of limiting the conversation. One of the confederates in 1991 was the Shakespeare expert Cynthia Clay, who was, famously, deemed a computer by three different judges after a conversation about the playwright. The consensus seemed to be: “No one knows that much about Shakespeare.”

Cynthia Joyce Clay is still online, with a blog and stuff. Search "Cynthia Clay Turing" for yourself. She apparently likes to boast the distinction of being considered not human:

I was judged to be a computer program on Shakespeare at the First Loebner Prize Competition of The Turing Test—a truly science fictional experience. I'm an author who likes to write sf, fantasy, updated versions of old myths.

Let's read: Sutton's RL, week 3, chap 3

For week 3, we will do:

• Rest of Sutton & Barto Chapter 3
• Sutton & Barto Summary of Notation, 
• Sutton & Barto Section 4.1 

Optimal Policy

Define: given a MDP problem, we have a corresponding policy spaces $\Pi$ of all possible policies for the problem. $\Pi$ is preordered by Pareto ordering:

\[\pi_1 \succeq \pi_2 \quad \text{iff} \quad \forall s\in S, v_{\pi_1}(s) \ge v_{\pi_2}(s)\]

That is, a policy is better than another iff using it does not deprove the expected value in all situations, and improves in at least one.

A policy $\pi$ is optimal iff it is a maximal element in $\Pi$ thus preordered.

The optimal state-value function is defined as as the best that can be done by any policy at a certain state, it's a bit subtler than that

 \[v_*(s) = \max_\pi v_\pi(s)\]

and similarly for $q_*$

Let's read: Sutton's RL, week 2, chap 3

For week 2, we will do:

• Read the definition given for artificial intelligence in Wikipedia and in the Nilsson book on p13; 
• google for and read “John McCarthy basic questions”, “the intentional stance (dictionary of philosophy of mind)”
• Sutton & Barto Chapter 3 to Section 3.5

Reading

From Wikipedia: 

intelligence demonstrated by machines, ... "intelligent agents": any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals. Colloquially, ... a machine [that] mimics "cognitive" functions that humans associate with other human minds, such as "learning" and "problem solving".

From The Quest for Artificial Intelligence (2009), Nils Nilsson:

intelligence is that quality that enables an entity to function appropriately and with foresight in its environment... Because “functioning appropriately and with foresight” requires so many different capabilities, depending on the environment, we actually have several continua of intelligences with no particularly sharp discontinuities in any of them. For these reasons, I take a rather generous view of what constitutes AI.

Let's read: Sutton's RL, week 1, chap 2

Chapter 2

This chapter is about multiarmed bandits (MAB). 

There are two kinds of feedbacks to a RL agent:

Evaluative feedback: how well did it act (a score).

Instructive feedback: best action it could do (a correct answer)

In MAP, the environment is a Markov chain, and thus the past does not influence the future. This makes it easy. 

This is called nonassociative learning.

In general,

\[Q_t(a) \approx q_*(a) = \mathbb{E}(R_t | A_t = a)\]

$Q_t(a)$ is the estimate of $q_*(a)$, estimated at time $t$. It is updated as time goes on, hopefully approaching $q_*(a)$ better and better.

The greedy strategy is 

\[A_t = \underset{a}{\operatorname{argmax}}Q_t(a)\]

The key is to estimate $Q_t$ accurately. There are several ways.

Let's read: Norvig's AI, chap 24, 25, 26, 27

Continuing from last post.

Chapter 24: Perception

This chapter is about feelings, or how to sense the world through sensors. It's too engineering for me so I skipped it.

Chapter 25: Robotics

This chapter is about robotics, actuators, or how to actually move around in the world. This is about the most traditional and engineering part of the textbook, and quite out of my part, so I'll skim it as much as I skimmed technical stuff in mechanical engineering during my study of Newtonian physics.

My favorite part from the chapter is a game:

Exercise 25.11: pretend to be a robot! A game for people from kindergarten to PhD! 

I can think of a few variations on this game: First, to make communication strictly minimal, players should only communicate through text messages and webcams. Second, instead of one person playing the brain, why not use two? One for each hemisphere! The left brain can only see the right eye and control the right hand, and vice versa. Then there will be six players. 

And we can even imagine a game of epiphenomenon.

Wait a minute, this sounds just like specialization...

Let's read: Norvig's AI, chap 22, 23

Continuing from last post.

Chapter 22-25 are about interactions with the environment:

• 22 is about reading and 
• 23 is about writing. 

Listening and speaking are not touched upon, but it's not a problem, considering that current technologies for speech-to-text and text-to-speech are pretty much perfect.

• 24 is about feeling (perception) and 
• 25 is about moving (robotics).

Let's read: Norvig's AI, chap 3

I'm tired of Norvig's AI now. It's too long. Much better if I can just refer to specific chapters of it when I need to get an introduction to a particular topic, rather than reading it in one go.
I will just post reading notes from what I've already accomplished.

---

Continuing from last post.

Chapter 3-6 are about solving given, well-defined problems by various kinds of searching. It is quite basic algorithmic stuff. There's no learning yet, and the agent can't improve itself.

Let's read: Norvig's AI, chap 2

Continuing from last post.

Chapter 2 overviews the idea of a rational agent: agents, environments, rationality, and some basic designs.

Agent

An agent in an environment receives percepts from sensors, and acts by actuators. Mathematically, we can model it as an agent function:
\[\text{Agent}: \{\text{Percept}\}^* \to \{\text{Action}\} \]

Let's read: Norvig's AI, chap 1

This is the first post in a series of posts as I read through Artificial Intelligence: A Modern Approach (3rd edition, 2009), by Peter Norvig, Stuart J. Russell. It's a behemoth with 1152 pages, eww. The book is unfortunately not free, but its pdf file is pretty easy to find online anyway, and most schools have a copy, too, just because it's so popular, as the official site says:

• The leading textbook in Artificial Intelligence. 
• Used in over 1300 universities in over 110 countries. 

We will study this book, not fully, it'd take about a year to do that! We'll just read as fast as possible to get the overview.

Today we read chapter 1.

Let's read: Sutton's RL, week 1, chap 1

This is the first post in a series of posts as I read through Richard Sutton's Reinforcement Learning: An Introduction (2nd edition, 2018), which is freely available on Sutton's site, thanks to his philosophy of GNU.

We will follow the textbook and do some assignments as seen from the folder, following the course schedule. The course schedule gives it as a 10 week course though we won't go through them all.

And a warning: I wrote this as review, so you'd better read the book yourself, since I will only write the bare minimum needed and not give more explanations. I will also post my exercise solutions.

Week 1 tasks

• Section 1 of the Wikipedia entry for “the technological singularity”; see also Species of Mind (2010), Vernor Vinge, and When will computer hardware match the human brain? (1998), Hans Moravec. 
• Sutton & Barto Chapters 1 and 2.
• Read probabilities-expectations.pdf in the dropbox.

Pointless Notes: Murder and Complex Analysis

Theodore John Kaczynski, BA (Mathematics, Harvard University, 1962), PhD (University of Michigan, 1967), is famous for being a series bomber, but to mathematicians, he's also known as a complex analyst. Not a (mathematically) famous one, but a competent one at least, and some also said he was brilliant. In 1969, he resigned as an assistant professor at Berkeley.

There was a news article about his math, and it contained the usual kind of flailing about how research level math is too hard for layponies. No need to be so dramatic about it... 

I was amused by this:

Writing in the first person plural is common in mathematics, said Kenneth Ribet of UC Berkeley. So in hindsight, the Unabomber's manifesto may provide an obscure hint at its alleged author's former profession. 

The mathematician's habit of writing in the Royal We has been exposed!

Let's read: Machine Super Intelligence, preface and outline

Machine Super Intelligence (2008) is the PhD thesis of Shane Legg, a previous student of Marcus Hutter, and a cofounder of DeepMind.

Let's start at the Preface:

Meaning of UAI

This thesis concerns the optimal behaviour of agents in unknown computable environments, also known as universal artificial intelligence [UAI]. These theoretical agents are able to learn to perform optimally in many types of environments... Moreover, these agents can be proven to upper bound the performance of general purpose computable agents.

Okay, the first paragraph is heavy, and summarizes the whole point of UAI. Let me unpack it for you.

Let's read: Spinoza's Ethics, You Can't Not Believe Everything You Read, and why Elephants don't play chess

Descartes and Spinoza are two 17th century philosophers. Descartes's philosophy is the one that's shaped how people think the world works, but are they true? Today we read two papers that suggest otherwise.

You Can't Not Believe Everything You Read (1993), Daniel T. Gilbert, Romin W. Tafarodi, Patrick S. Malone. It's a paper written by psychologists, but touches on a lot of philosophy and neurosciences in it.

Let's start by the abstract:

Can people comprehend assertions without believing them? Descartes suggested that people can and should, whereas Spinoza suggested that people should but cannot.

Let's play: Fermi estimates

Fermi estimates are about getting rough guesses from as little data as possible. You win if you get it within a factor of 10.
I took the Fermi estimate problems from this site. You can read how it's done over there. I'll just record my own estimates here.

1. How many new passenger cars are sold each year in the USA?

• There are 3e8 people in America. Source: my memory.
• A citizen of a developed country lives 1e2 years. Source: common knowledge.
• About 1/2 Americans are rich enough to own cars. Source: A feely guess. 
• Each American car owner goes through 3 cars in their lives. Source: blind intuition. After the facts, I can justify by saying  a car usually breaks down in 20 years, and a person's adult life lasts about 60 years.

So in total, we get
$$(3e8 \text{ person}) * 1/2 * (3 \text{ cars/person }) * / (1e2 \text{ year }) = 4.5e6 \text{ cars/year}$$
The right answer is, according to Wikipedia:

In the year 2009, about $5.5$ million new passenger cars were sold in the United States according to the U.S. Department of Transportation.

Experimental result of taking notes without passing through the brains

College is a place where a professor’s lecture notes go straight to the students’ lecture notes, without passing through the brains of either.

So today I tried again going to a lecture and managed to take the entire notes, and somehow indeed, the notes did not pass through my brain at all. It's the last lecture of the whole course, which is why I went there hoping for a bit more interesting things than just a mass of technical details presented uglily through use of verbal speech, I was wrong.

Well, too bad. At least I felt kinda happy that my LaTeX skills are enough for note taking.

Let's Read: In Praise of Lectures by T. W. Körner

Alright, let's read! I'm on a roll of venting against lectures.

In Praise of Lectures (2004), Thomas William Körner

The essay is off to a strong start that lifted my mood.

For these students attendance at lectures has a magical rather than a real significance. They attend lectures regularly (religiously, as one might say) taking care to sit as far from the lecturer as possible (it is not good to attract the attention of little understood but powerful forces) and take complete notes. Some lecturers give out the notes at such speed (often aided by the technological equivalent of the Tibetan prayer wheel — an overhead projector) that the congregation is fully occupied but most fail in this task... 

The notes of the lecture are then kept untouched until the holidays or, more usually, the week before the exams when they are carefully highlighted with day-glow yellow pens (a process known as revision). When more than 50% of the notes have been highlighted, revision is said to be complete, the magical power of the notes is exhausted and they are carefully placed in a file never to be consulted again.

The invention of the Xerox machine [or Internet] means that only one student need attend each lecture, the remainder being freed ... Nor would this student need to take very extensive notes since everything done in the lecture is better done in the textbooks.

Stop math lecturing

There are many bad things and few good things about teaching math by lectures.

Personally I have not learned anything from math lectures since the end of high school. As I said before, lectures are bad for higher math. It's either too fast, then it's not understandable, or too slow, then it's boring. If it's written in a book, it's much better presented, than if it's written on a board live with a human hand. Using slides is mildly better, but slides are ugly, and why even use slides when you have a book?

And what's the point of taking notes? If the lecturer has prepared a lot about the lecture, to the point that the lecture notes are already done and so they could just refer to it during the lecture, why not print it out and give to the students so that they don't have to waste their time writing down their own lecture notes? To allow some idiosyncrasy for the notes, large margins can be put into the printed out notes so that the students can put in their own stuff there.

My LaTeX template

Today I tried going to a lecture and as expected still learned nothing. Seriously, how the heck do people learn at lectures? I mean, I can kind of understand how people learn during history or psychology lectures since those are heavy on story-telling, but there are no stories in real mathematics and so the college tradition of learning through oral instruction seems a bizarre oral-fixation, in grave need of therapy.

Anyway, I tried to at least try live-LaTeX-transcription of what's written on the board, and almost managed to do that, until I tried compiling and the thing does not compile since the tikz-cd package for drawing commutative diagrams doesn't work, and debugging spent so much time that I completely lost hope of keeping up, not that I originally had any hope of keeping up even without note-taking.

Because really, either it's too easy or it's too hard, it's never at the right speed. If it's too slow I get bored, and lose concentration. If it's too fast I can't understand and lose hope. Books never frustrate me like lectures.

Anyway, here's what I ended up using as a latex template. Might be useful for you.

Let's read: $C^*$-algebra, quantum mechanics, Pascual Jordan

A modern mathematical proof is not very different from a modern machine, or a modern test setup: the simple fundamental principles are hidden and almost invisible under a mass of technical details. ---- Hermann Weyl

After 3 courses in analysis I have finally reached Banach algebras, I feel exhausted by the climb and yet there's more theory ahead until the true pinnacle: the theory of $C^*$-algebra, for quantum mechanics!

Let's read, what is being promised?

$C^*$-algebras were first considered in quantum mechanics to model algebras of physical observables... began with Heisenberg's matrix mechanics and in a more mathematically developed form with Pascual Jordan around 1933. Subsequently, John von Neumann attempted to establish a general framework for these algebras which culminated in a series of papers on rings of operators. These papers considered a special class of $C^*$-algebras which are now known as von Neumann algebras.

So experiments in quantum mechanics → weird behavior of observables in quantum mechanics → Heisenberg matrix mechanics → Jordan purified it into Jordan algebra → von Neumann generalized to rings of operators → von Neumann algebra generalized to $C^*$-algebras.

Let's Read: Star Swirl the Bearded's Compiled Papers on Amniomorphic Magic, Part 275/500

Let's read: Bringing about the past, and Backwards causation

Backward causality is not something only physicists think about. Philosophers have thought about it a lot. We'll read about a classic paper Bringing about the past (1964), Michael Dummett.

I'm going to just pick out memorable and easily understandable tidbits that collage to a picture of what the theory of backward causality is about, and not try to summarize the whole thing -- if you want the whole thing, read the paper in whole.

Let's read: weird things on arXiv: ponies, particle card games, statistical mechanics of poker

Search "pony" on arXiv, there are two results:

Complementarity of Resonant and Nonresonant Strong WW Scattering at SSC and LHC, Michael Chanowitz (Submitted on 6 Oct 1992), with a comment:

Comments: 8 pages (figures on request by Pony Express), LBL-32938

No-clicking Event in Quantum Key Distribution, W.-Y. Hwang, I.-T. Lim, J.-W. Park (2008)
with the memorable phrase

quantum Trojan-pony attack

I see that arXiv is yet unponified. No matter. Try "card game", magic happens!

Let's Read: Nielsen and Ninomiya (2007)'s backward causation, LHC, and the end of mathematics

I considered writing a blog because it's better than writing things on the wood paper, which is hard to search, even more prone to be forgotten than electronic writing, and almost impossible to be read by anyone but me.

I mean, there's a slight chance that my papers will get buried by volcanic ash and become dug up by future archeologists (like the scrolls in the Pompei library), while there's pretty much no hope of any server surviving such a treatment, but that's too unlikely to be considered a valid argument for using wood paper.

Today we write about LHC, SSC, and Nielsen and Ninomiya's crazy proposal in 2008 about why LHC shut itself down. Then it digresses to a depressing thought about the ends of scientific and mathematical fields.