We Are Beautiful Devices

The Nobel prize-winning psychologist Daniel Kahneman lives in an airy penthouse on the 14th floor of an apartment block in downtown Manhattan, not far from the Eighth Street subway station. But never mind that for a moment. Instead, without thinking too hard about it, try answering the following question: roughly what percentage of the member states of the United Nations are in Africa? (I'll wait.)

The correct figure isn't what's important here. What matters is that your answer is likely to be lower than if you had first been informed that Kahneman is 77 years old, or if I had claimed his apartment – where he lives with his wife, the British-born psychologist Ann Triesman – was 60 floors up, and near the 86th Street station. This is the phenomenon known as the "anchoring effect", and it is typical of Kahneman's contributions to psychology in that it suggests something rather disturbing about the human mind: not just that we're susceptible to making skewed judgments, but that we're influenced by factors more subtle and preposterous than we could ever imagine.

Kahneman's new book, Thinking, Fast and Slow, is a meaty memoir of his life's work that describes countless such cognitive quirks – but don't imagine that reading it will cure your irrationality. "It's not a case of: 'Read this book and then you'll think differently,'" he says. "I've written this book, and I don't think differently." Kahneman, whom Steven Pinker calls "the most important psychologist alive", is twinkly and energetic. But beneath the surface, he is a pessimist. And he is allergic to the notion that his book might be mistaken for self-help. It's his first work aimed at a mass audience, and he hated writing it: "I really did not want to disgrace myself in front of my colleagues, and I worried the public wouldn't like it if it read like a textbook. Also, I really don't like old men's books, and I felt I was writing an old man's book." Eventually, in despair, he arranged to pay four younger psychologists $2,000 each to review his manuscript anonymously, and to tell him the brutal truth: should he bother finishing?

They liked it. So did I. It's hard not to: Kahneman's approach to psychology spurns heart-sinking tables and formulae in favour of short, intriguing questions that elegantly illustrate the ways our intuitions mislead us.

Take the famous "Linda question": Linda is a single 31-year-old, who is very bright and deeply concerned with issues of social justice. Which of the following statements is more probable: a) that Linda works in a bank, or b) that Linda works in a bank and is active in the feminist movement? The overwhelming majority of respondents go for b), even though that's logically impossible. (It can't be more likely that both things are true than that just one of them is.) This is the "conjunctive fallacy", whereby our judgment is warped by the persuasive combination of plausible details. We are much better storytellers than we are logicians.

If any of this sounds familiar, it's because Kahneman and his collaborator Amos Tversky, who died in 1999, are the primary inspiration for many of the past decade's pop-psychology books – the publishing phenomenon that brought you tipping points and freakonomics, the wisdom of crowds, black swans, and "predictable irrationality". It is a trend that one unimpressed reviewer of Kahneman's book labelled "the effect effect". In the early days, academics took a similarly sniffy view of Kahneman and Tversky's research: Kahneman recalls one well-known American philosopher turning his back on him at a party with the disdainful words: "I am not really interested in the psychology of stupidity." That soon changed, though, as the pair's influence spread rapidly throughout the social sciences, culminating in 2002, when Kahneman became one of a handful of non-economists to win the Nobel prize in economics.

"The psychology of stupidity" is not, in any case, a very apt summary. Kahneman's point isn't that we're all wildly bizarre or idiotic, but that our mental apparatus, which works so well most of the time, sometimes leads us astray in predictable ways. "We're beautiful devices," he says. "The devices work well; we're all experts in what we do. But when the mechanism fails, those failures can tell you a lot about how the mind works."

In Thinking, Fast and Slow, he presents this as a drama with two "characters": System One, which is the domain of intuitive responses, and System Two, the domain of conscious, effortful thought. System One – the kind of mental ability celebrated in Malcolm Gladwell's book Blink – kicks in without our needing to think about it. The problem is that it always tries to help, even when it shouldn't, and that it works with whatever it's got, which isn't always the most sensible information.

The biggest challenge this posed was to economists, most of whom assumed that people were basically rational and selfish and acted in their own best interests. The work that won Kahneman the Nobel showed otherwise. For example, we hate losing things more than we like gaining them, which is why people refuse to sell their home for less than they paid, even if it makes financial sense to do so. Similar biases make us behave strangely where risk is involved, too: if forced to choose between being given £500 for certain, or a 50% chance of winning £1,000, most of us will opt for the sure thing. But if the choice is between losing £500 for sure, or a 50% chance of losing £1,000, most of us will take the gamble.

Then there's the much-cited thought experiment involving tickets to the theatre. Suppose a woman plans to buy a ticket for a play costing £40, but en route to the theatre she realises she has lost two £20 notes in the street: would she still buy the ticket? Most people, when asked this question, assume that she would. But what if she bought the ticket in advance, then arrived at the theatre to find she'd lost it? In that case, people assume she'd go home without buying another ticket – even though the scenarios are financially identical. As Richard Thaler, another leading light in the revolution that became known as behavioural economics, told an interviewer, Kahneman and Tversky's research meant that "rationality was fucked". Kahneman, on the other hand, likes to say that you'd need to study economics for years before you'd find his research surprising: it didn't surprise his mother at all.

Kahneman was born in 1934, the son of Lithuanian Jews, and grew up in France. Life was generally good until 1940, when German forces swept in. He recalls drawing, around that time, "what was probably the first graph I ever drew", showing his family's fortunes over time – "and around 1940 the curve crossed into the negative domain." His father was captured during a large-scale sweep of Jews in France, but somehow escaped being sent to a concentration camp and was let go instead. ("The story of my father's release, which I never fully understood, also involved a beautiful woman and a German general who loved her," he wrote.) The family kept moving across France. "The feeling was of being hunted," Kahneman recalls. At one point their home was a chicken coop at the back of a pub. In 1944 his father died of insufficiently treated diabetes, six weeks short of D-day. As soon as the war ended, his mother took the family to live in Palestine, in what would soon become Israel.

Kahneman was drafted into the Israeli army in 1955, where he served as an infantryman for a year – "it was a very tense time, but I never fired a shot in anger" – then worked as a military psychologist. One of his roles was to evaluate new recruits by watching them perform the "leaderless group challenge", in which teams of eight men had to transfer themselves, and a large log, over a 6ft-high wall, without anybody, or the log, touching the wall. The task was designed to reveal the participants' true character, and thus demonstrate who had the making of a future leader. As a method of psychological evaluation, it wasn't much good: Kahneman made predictions, but follow-up research revealed them to be little better than guesses. What the experience taught him, in the end, wasn't how to spot a future hero, but rather how hard it was to expunge his own confidence in his predictions. "We knew as a general fact that our predictions were little better than random guesses," he writes. "But we continued to feel and act as if each particular prediction was valid." Confidence is a feeling, not a logical conclusion reached after analysing statistics. Kahneman would later encounter the same phenomenon among investment advisers, who clung to their belief in their abilities even after it was demonstrated that their stock-picking skills left their clients no better off than rolling dice.

The intellectual relationship that defined his career began in the late 1960s at Hebrew University in Jerusalem, when he met Tversky, a young colleague. Kahneman describes their bond as "magical", and it sounds much more like a loving friendship than a scholarly collaboration. For several years, the two spent hours every afternoon in freewheeling conversations, examining their own hunches and intuitions, gradually developing the list of biases and fallacies for which they became famous. "He got up late, and I was a morning person, so we started with lunch, and took it from there," Kahneman remembers. "This kind of collaboration is very unusual in science. We were just extraordinarily lucky, and we knew it." The editor of the journal to which they submitted their first major paper rejected it; their work seemed too frivolous for the academic establishment. "Psychologists really aim to be scientists, white-coat stuff, with elaborate statistics, running experiments," Kahneman says. "The idea that you can ask one question and it makes the point ... well, that wasn't how psychology was done at the time."

With hindsight, however, those single questions seem anything but frivolous. The irrational traits they uncovered are, to pick one notable example, hugely important in understanding the causes of the current economic crisis, which has its roots in (among others) the overconfidence bias and the illusion of skill. If we can't hope to correct such biases in any lasting way, we can perhaps seek to cultivate some humility about the limits of our mental powers. Being the puppet of subtle psychological influences we cannot even recognise is annoying. But at least we can try to remember that that's what's likely to be happening. Well, it's a start.