Let's read: Norvig's AI, chap 2

Continuing from last post.

Chapter 2 overviews the idea of a rational agent: agents, environments, rationality, and some basic designs.

Agent

An agent in an environment receives percepts from sensors, and acts by actuators. Mathematically, we can model it as an agent function:
\[\text{Agent}: \{\text{Percept}\}^* \to \{\text{Action}\} \]

Rationality

Rational: do the right thing. 

The right thing: the rational thing.

Just kidding. The right thing is foundational, it is given as a brute fact. It is whatever. Rationality is about achieving the final goal, not what the final goal is.

Final goal

To make AI do the right thing, set its final goal to be your final goal, not the action you think would help achieving it. Like, a vacuum cleaner who wants to suck dust would dump the dust to suck again. It should instead want a clean floor.

Fundamentally it's philosophical: what is a good life?

Rationality

Maximize the expectation of final goal achievement. Probability, basically. Omniscience would make rationality trivial, then we can do the perfect thing. But we are not omniscient, so we can only do the best guess thing.

Mathematically, we can write the rational agent's utility, value, and agent function as:

\[\text{Utility}: U: \{\text{Percept}\} \to \mathbb{R}\]\[\text{Value}: V = \mathbb{E}(U)\]\[\text{Rationality}: R: \{\text{Percept}\}^* \to \{\text{Action}\}\]\[R(p_1, \cdots p_n) = \underset{a}{\operatorname{argmax}} \mathbb{E}(\text{expected future value} | p_1, \cdots p_n; a)\]

Task Environment

Rational agents are solutions to problem, the problem is a task. To define the problem fully requires "PEAS":

• Performance score (final goal)
• Environment
• Actuators (actions)
• Sensors (percepts)

The hardest tasks are partially observable, multiagent, stochastic, sequential, dynamic, continuous, and unknown. Like driving!

Types of agent programs

The book uses Cartesian dualism: agent = body + mind = architecture + program. The book concentrates on the mind, though will touch on the body in chap 25, 26.

To implement a rational agent function into a program, there are several engineering styles in increasing complexitie:

Simple Reflex

Basically, it's Skinner's behaviorism. The agent is then simplified to a reflex function:

\[\text{Reflex}: R: \{\text{Percept}\} \to \{\text{Action}\} \]

\[\text{Reflex Agent}: A: \{\text{Percept}\}^* \to \{\text{Action}\} \]

\[A(p_1, \cdots, p_n) = R(p_n) \]

It's the simplest agent you can think of, kinda like an ATM, and yet it can already do a lot.

Model-Reflex

Basically, it's reflex with a model of the world. The model is updated by an update rule that describes the laws of the environment.

\[\text{Update Model}: U: \{\text{State}\} \times \{\text{Percept}\} \times \{\text{Action}\} \to \{\text{State}\} \]

\[U(\text{previous state}, \text{current percept}, \text{last action})= \text{current state}\]

and the reflex depends on the state in the modeled world, since the model is strictly more informative than the bare percept, since the model already incorporates the percept:

\[\text{Reflex}: R: \{\text{State}\} \to \{\text{Action}\} \]

Goal Utility

Basically utilitarianism. Maximize expectation of utility function.

Learning

A learning agent should have 4 parts:

• Critic: gives feedback. Cannot be modified, else wireheading can happen.
• Learner: based on the feedback, change the performer so it acts better.
• Performer: the one doing the actions.
• Problem generator: thinks up problems to solve, exploration, so that the learner can learn more things. 

I like the funny story about Galileo: he was a problem maker! He dropped rocks from the tower not because he wanted to change others' brains, but his own!

Representation complexity

• atomic: just a label, a bare name, a set element. Used by game-playing, search, Markov.
• factored: a vector, a tuple, a list of attributes. Used by planning, constraint satisfaction, prop logic, Bayesian networks.
• structured: a gear model with small components interacting. Used by 1st order logic, 1st order probability model, natural language, knowledge-based learning.

Humans use structured representation, and AI should too.