"What is the price of friendship?" and other evil questions from a rational monster

“Nowadays people know the price of everything and the value of nothing.”

― Oscar Wilde

We follow the theory of Philip E. Tetlock, who has studied several related topics: forecasting future events (as detailed in a book Superforecasting (2015)), counterfactual history ("What if...?"), and some "taboo thoughts" that despite being rational, often provokes "moral outrage" (precise definition given later).

These topics are related, as we will see later. But first: What's the price of an average human life?

Answer: In America, between 1 million to 10 million.

Now, this is perfectly reasonable, as a Fermi estimate quickly shows: An average American works 50 years, with yearly salary 30000 dollars. So in full that gives 1.5 million dollars. That's a lower bound, considering that it's just cash salary and doesn't include all the other non-salary productive work, like friendship, volunteering, and writing free blog posts (like this one lol). But the order of magnitude is very much right.

Okay, if that's right, then isn't it wrong to say "A life is priceless?"

Yes. But don't say it out loud, else you get the dreaded moral outrage.

Counterfactual history

Let's start with counterfactual history, since it's quite entertaining. For an example of how counterfactual history is done, search "what if..." and pick your favorite scenario.

Close-Call Counterfactuals and Belief-System Defenses: I Was Not Almost Wrong But I Was Almost Right (1998), Philip E. Tetlock.

This paper talks about two tendencies of experts when doing counterfactual history. Basically, Tetlock used two studies to show that experts, especially those that has a strong theory of how history works ("high scorers... of need for closure and simplicity"), would:

• reject close-call what-if scenarios when the actual history went as they expected: "I was not almost wrong!"
• embrace them when not. "I was almost right!"

Simplicity is a virtue in fundamental physics and mathematics, but not in history (or many other kinds of complex things). Statistical laws can be simple and predictable, but any particular case would be far less so.

Closure is a feeling that a satisfying explanation for an event has been found, and therefore there's no need for further search for explanation.

Unfortunately for the people who value simplicity and need closure, complex things like history are unsuited for such studies. Often, the more one learns about a historical event, the less it seems inevitable.

Second-order counterfactual

Imagine if Franz Ferdinand wasn't killed. Would there still be WWI? Many historians seem to think so, and their most common argument is a "second-order counterfactual", that is, "if this happens, then later that would probably happen".

"Yes, war probably would not have broken out that summer." but... some other match would have ignited the conflagration... Second-order counterfactuals allow for deviations from reality but ... soon bring events in the simulated counterfactual world back toward the observed historical path.

Such arguments are often used in "I was not almost wrong" arguments. As in "Even if this happened, soon enough that would happen, and thus WWI was inevitable, not almost preventable."

The forbidden calculations

The Psychology of the Unthinkable: Taboo Trade-Offs, Forbidden Base Rates, and Heretical Counterfactuals (2000), Philip E. Tetlock et al.

Three models for human behavior

In basic economy theory and game theory, humans are considered perfectly rational players trying to get what they want. This is the "homo economicus" model, or "economist human".

In science and rational thinking in general, humans are implored to be perfectly rational explorers trying to find the best explanation for what they observe. This is the "scientist human".

The economist human model is too simple, as the numerous demonstrations of the errors humans make when in certain economic tradeoffs (a very nice book about this is Thinking Fast and Slow (2015), David Kahneman). A better model is "psychological human", where a human acts according to psychological principles that are sometimes rational and sometimes predictably irrational.

And in this paper, the authors propose that people are also "moralist-theologian humans".

... people engaged in a continual struggle to protect their private selves and public identities from moral contamination by impure thoughts and deeds.

This is the sacred-profane dichotomy. Some subjects are sacred, and others are profane. Mixing them causes great trouble for the moralist-theologian within.

Moral outrage

Moral outrage is the sum of three things:

1. (cognitive) think someone is bad
2. (emotional) feel anger, disgust, or contempt for someone
3. (behavioral) punish someone

It sounds like moral panic. Maybe a moral panic is when lots of people are in a moral outrage together.

Moral Cleansing

Moral cleansing is when people make an effort (or say that they will make an effort) to support something they think is moral.

Not this kind of cleansing.

Funny story. Just as a test, I thought to myself, "What's the price of my friendship with my best friend?" and after thinking about it a while, I felt that about 0.5 million dollars is about a good price. And after that I had a sudden urge said to her how much I love her (I did.). 

Then it dawned on me that I was doing the moral cleansing thing... (And no, I didn't tell her that I felt that the love was worth 0.5 million dollars.)

Taboo tradeoffs

There are sacred goods and profane goods. Sacred things are like life, liberty, and the pursuit of ponies. Profane things are things that can be usually bought with money. Sacred and profane things aren't comparable.

• A profane-profane tradeoff is routine, like choosing what to buy at the supermarket.
• A sacred-sacred tradeoff is tragic, like choosing to be or not to be.
• A sacred-profane tradeoff is taboo, like buying indulgences (according to Martin Luther)

It's not just the "apples-oranges" problem of incomparability. For example, people thinks this question is weird:

Do you prefer a bottle of aspirin or a bag of potato chips?

because these two items are in incomparable categories, "medicine" and "snack food". But still, people have no problem doing shopping in supermarkets where they choose between incomparable goods.

This question, though, would be quite shocking:

How much money would you request for selling one of your kidneys?

Which is probably why economists, medical priority setters, organ traders, utilitarians, or anyone who prices the sacred things, sometimes provoke moral outrage.

In this experiment, people are asked to consider four possible stories:

Director of Health Care Management, Robert, can save the life of Johnny, a five year old who needs a liver transplant, but it would [cost \$1 million that can be used for other hospital expenses]/[deprive another six year old who also needs a liver transplant]. Robert [decided quickly and easily]/[decided after much thinking], that he would [save Johnny]/[save $1 million]/[save the other boy].

It was found that

• The tradeoff between Johnny and \$1 million is taboo. 
• The tradeoff between Johnny and the other boy is tragic.
• Thinking long on a taboo tradeoff makes people morally outraged at Robert, regardless of Robert's decision. This is because Robert has paid too much attention to a mix of sacred-profane, and thus threatens the moral order of the world.
• Thinking long on a tragic tradeoff makes people admire Robert more, regardless of Robert's decision. This is because Robert has paid proper attention to the seriousness of the tragic tradeoff, and thus affirmed the sacredness of the lives being traded.

The paper concludes that

People who treat these rights and responsibilities as open to the monetary trade-offs of market-pricing relationships show at best ignorance and at worst contempt for the spheres of justice that society insulates from the universal solvent of money.

After people feel tainted by contemplating taboo tradeoffs, they often engage in moral cleansing. In the experiment, it was measured by asking how much effort the participants would put in a campaign against orphan-auctioning. It was found that people would put in more effort after being tainted by the impure thoughts of taboo tradeoff.

In this experiment, four types of subjects were used: libertarians, conservatives, lift-liberals, Marxist socialists. It was found that political leanings were important in how forbidden the calculations were felt. In general,

• libertarians (they really liked trading with money) do not react differently about taboo tradeoffs and taboo tradeoffs.
• Marxist socialists have a lot of problem even with routine tradeoffs compared to conservatives, like using money to buy better healthcare, lawyer service, schooling, etc.
• left-liberals are in the middle between Marxist socialists and conservatives.

Mean response = amount of moral outrage

The libertarians do get moral outrages about others trying to disallow taboo tradeoffs. This is similar to how conservatives get moral outrages about Marxist socialists trying to disallow certain routine tradeoffs.

Forbidden base rates

There are sacred beliefs and profane beliefs. Science and math are in the business of dealing with profane beliefs, not sacred beliefs. At least, that's what it usually is framed as. Many beliefs are said to be "what science cannot study". They are either too vague to be studied, or too sacred to be.

The base rates fallacy is a common fallacy. It happens when people ignore base rates, as in this illustration:

A cab was involved in a hit and run accident at night. Two cab companies, the Green and the Blue, operate in the city. 85% of the cabs in the city are Green and 15% are Blue.
A witness identified the cab as Blue. The court tested the reliability of the witness under the same circumstances that existed on the night of the accident and concluded that the witness correctly identified each one of the two colors 80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accident was Blue rather than Green knowing that this witness identified it as Blue?

The right answer is 41%, by Bayes formula, but most people got an answer closer to 80%. This is the base rate fallacy, where people ignore the base rates and get the answer wrong.

The forbidden base rates, however, is when this kind of calculation threatens sacred beliefs. Calculations are profane, not sacred.

Decision-makers who use statistical generalizations about crime, academic achievement, and so forth, to justify disadvantaging already disadvantaged populations are less likely to be lauded for their statistical savvy than they are to be condemned for their moral insensitivity.

While disadvantaging the already disadvantaged is certainly bad, if the disadvantage is bad, some people are offended when statistical information derived from forbidden sources like race, gender, nationality, etc, are merely considered, or used more subtly (like in affirmative action).

In the experiment, moral cleansing measured by asking how much participants would be willing to support racial equality and understanding of black American culture. It was found that tainted participants cleansed more.

In a related issue, people prefers the "inside view" (what this particular case/individual is like) and doesn't consider the "outside view" (what the general case is like) enough. Kahneman in Thinking Fast and Slow recounted:

I once asked my cousin, a distinguished lawyer, a question about a reference class: “What is the probability of the defendant winning in cases like this one?” His sharp answer that “every case is unique” was accompanied by a look that made it clear he found my question inappropriate and superficial. A proud emphasis on the uniqueness of cases is also common in medicine, in spite of recent advances in evidence-based medicine that point the other way. Medical statistics and baseline predictions come up with increasing frequency in conversations between patients and physicians. However, the remaining ambivalence about the outside view in the medical profession is expressed in concerns about the impersonality of procedures that are guided by statistics and checklists.

Heretical counterfactuals

Some whatifs make people uncomfortable, because they require people to imagine possible histories that threaten their sacred beliefs.

Particularly irksome are counterfactuals that apply normal laws of human nature and of physical causality to heroic founders of the movement. Consider... Salmon Rushdie's heretical counterfactual in Satanic Verses that invited readers to imagine that the Prophet Mohammed kept the company of prostitutes. For this transgression, the theocratic regime in Iran sentenced Rushdie to death (the ultimate expression of moral outrage).

I found some whatif scenario quite funny:

If Joseph had left Mary because he did not believe she had conceived a child with the Holy Ghost, Jesus would have grown up in a one-parent household and formed a different personality. 

If Mary had given birth to more children after Jesus, she could not be portrayed as the Holy Virgin central to Christian beliefs.

If Jesus had allowed himself to be saved by his apostles or through divine intervention, Jesus would not have died on the cross and thus would have failed in his divine mission.

Someone could make a "choose your own adventure" game for Jesus's life... but that'd be a taboo mix of the profane (text adventure games, random chance) and the sacred (Jesus, divine mission).

As one fundamentalist commented: "God did not send his only Son to die for our sins in a careless or casual way that left the success of the mission to depend on chance. God foresaw and foreclosed these possibilities."

Typical result of contemplating such counterfactuals.

Fundamentalists were especially likely to engage in moral cleansing after contemplating heretical counterfactuals, but not secular counterfactuals.

Coping with forbidden calculations

Outrages and cleansings aside, sometimes forbidden calculations need to be made. Life might be priceless, but sacrifices must to be made in a hospital that has too many patients and not enough medicine or equipment. A student's development might be priceless, but No Child Left Behind would cost a lot of money that might be better used somewhere else.

Reframing a taboo tradeoff

One way to deal with a taboo tradeoff is to reframe the tradeoff, so that it's no longer taboo. It could be made routine or tragic. For example, for the tradeoff of a planner deciding between saving a life and saving \$1 million, if the planner promises to use the \$1 million dollars to save two specific lives (bonus points if the two lives have names), it suddenly becomes a tragic tradeoff, and the \$1 million becomes irrelevant.

For organ trading, it can be made routine by adding more secular details, such as:

the poor will be assisted in purchasing needed organs and that they will not be compelled to sell their organs in "deals of desperation".

Changing the sacred and the profane

The sacred and profane can change. Before American Civil War, it was okay for people to pay to have someone else to serve the conscription. Now it isn't. Before capitalism, it was not okay to buy and sell land in England. Now it is.

Additional discussions

Connections to Terror Management

Terror management theory posits that people who are reminded of their mortality seek out the existential comfort of a collectively shared worldview that transcends their mortal life spans and endows their lives with moral significance.

Tetlock suggested that, since sacred values are transcendental and morally significant,

people reminded of their mortality should be especially outraged by taboo trade-offs, forbidden base rates, and heretical counterfactuals that destabilize their worldview, and should be especially inclined to moral cleansing.

I could not find experiments that tested this.

The mere contemplation effect

It's strange that merely thinking about a forbidden calculation can cause moral outrage and cleansing, even if the subject is satisfied with their decision in the end. 

One possible explanation is that people, by merely contemplating a belief, momentarily believe in it, before explicitly rejecting it. This is the Spinoza theory of belief. As such, merely contemplating can taint you.

Another explanation would be that people feel a need to express moral outrage and moral cleansing, to maintain the social norm of separating the sacred and the profane.

Sacred entitlement

This is a side note, but this paper reminds me of entitlement. As mentioned in Thinking Fast and Slow, (Chap 28, "Bad Events", "Loss Aversion in the Law"), Fairness as a Constraint on Profit Seeking: Entitlements in the Market (1986) reported on a study of how people perceive fairness in various situations:

A small photocopying shop has one employee who has worked there for six months and earns \$9 per hour. Business continues to be satisfactory, but a factory in the area has closed and unemployment has increased. Other small shops have now hired reliable workers at \$7 an hour to perform jobs similar to those done by the photocopy shop employee. The owner of the shop reduces the employee’s wage to \$7. 

The respondents did not approve: 83% considered the behavior Unfair or Very Unfair. However, a slight variation on the question clarifies the nature of the employer’s obligation. The background scenario of a profitable store in an area of high unemployment is the same, but now

the current employee leaves, and the owner decides to pay a replacement \$7 an hour.

 They did not report on this, but I think the following tradeoff would be considered equally Unfair:

A new potential employee comes along demanding \$7 an hour, and the owner decides to replace the original employee with the new one.