Humans are not automatically strategic — Anna Salamon

LessWrong reply by Anna Salamon to Lionhearted's post on the "Failure to Evaluate Return-on-Time" fallacy. The piece explains why even smart, numerate, goal-oriented people routinely pursue their stated goals through far less effective routes than are available to them.

The core claim: most courses of action are ineffective by default. There is no strong evolutionary or cultural force that automatically focuses humans on the narrow set of behaviors that would actually work. Like a random eight-year-old taking a calculus test, failure is the default because most answers are wrong and nothing guides you to the right ones.

Weak Goal Representations

People do have limited senses in which they "have goals." They:

Tell themselves and others stories about aiming for things
Search for activities consistent with the role and goal-seeking they claim
Feel glad or disappointed depending on whether outcomes match the story

These are real but weak. They do not automatically produce effective action.

The Missing Heuristics

Salamon lists concrete useful techniques that humans do not carry out automatically:

(a) Ask ourselves what we’re trying to achieve.

(b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress.

(d) Gather that information (e.g., by asking how folks commonly achieve our goal or similar goals, or by tallying which strategies have and haven’t worked for us in the past).

(e) Systematically test many different conjectures for how to achieve the goals, including methods that aren’t habitual for us, while tracking which ones do and don’t work.

(f) Focus most of the energy that isn’t going into systematic exploration on the methods that work best.

(g) Make sure that our "goal" is really our goal — that we coherently want it, are not constrained by fears or uncertainty as to whether it is worth the effort, and have thought through questions and decisions in advance so they won't continually sap our energies.

(h) Use environmental cues and social contexts to bolster our motivation, so we can keep working effectively in the face of intermittent frustrations or temptations based in hyperbolic discounting.

Instead, we mostly act from habit, impulse, convenience, or whatever feels associated with the goal in the moment.

The Reasoning–Motivation Gap

Verbal and conversational systems can understand the above heuristics once pointed out (perhaps ~5% of people have the abstract reasoning). Motivational and reward-gradient systems that actually pull behavior do not update automatically from that understanding. Conscious overrides are rare and costly. Training the automatic systems (visualization, repeated safe exposure) is possible but is itself not automatic.

Examples of Default Unstrategic Behavior

People enter long training programs “to make money” without spending hours on salary comparisons.
Daily typists remain two-finger typists without a tutor program.
People pursue leisure without tracking which activities are actually enjoyable.
Numerate people fear low-probability bogeymen and ignore relative risk statistics.
Most settle into one stereotyped mode of studying, writing, or interacting without trying and measuring alternatives.

Relation to Rationality

Having goals in a much more pervasive, optimization-oriented sense is part of what “rational” should mean. It helps achieve what we care about and had not been taught in detail.

The piece ends by asking how people have trained themselves to be substantially more strategic and what additional heuristics or training methods exist.

The post's core claim and the explicit (a)–(h) list are captured in humans-are-not-automatically-strategic.

See also the endorsement and placement of this piece as the single highest-ROI recommendation in a-reading-list-for-generalists-dylan-bowman. Cross references: match-quality polymath a-reading-list-for-generalists-dylan-bowman.