Week 6: Additional Information: Bayesian Revision, EVSI & Utility

This week’s big question: Week 5 ended with a teaser: a perfect forecast sets the ceiling (EVPI), but real tests and surveys are imperfect. So how should an imperfect signal change your beliefs, and once it does, how much is that signal actually worth paying for?

Before the week

• Read: Taylor, Introduction to Management Science (13e): Chapter 12, “Decision Analysis,” pp. 596–604 (decision analysis with additional information: Bayesian revision / posterior probabilities, EVSI, and utility), building on Chapter 11, “Probability and Statistics,” pp. 550–551, the Bayesian analysis section you skimmed in Week 2.
• Warm-up self-check, conditional probability and what a “prior” means:

Session 1: Mon, Sep 28 · Bayesian revision: prior → posterior (TBD)

🧩 Puzzle

The developer. A developer is deciding whether to invest in a project. If demand turns out High the project is worth +\$200k; if demand is Low it loses −\$100k. Right now their belief is High 0.6, Low 0.4. Before committing, they could buy a market survey, but a survey only ever comes back “favorable” or “unfavorable,” and it isn’t perfect. How should a survey result actually change what they believe about demand?

Warm-up: revising a belief with no money attached

Before payoffs muddy the picture, do one clean revision. An espresso machine is either calibrated (prior 0.5) or miscalibrated (prior 0.5). When it is calibrated only 5% of shots come out bitter; when it is miscalibrated 35% do. A barista pulls one shot and it is bitter. Should they reach for the toolkit?

• Joint(calibrated, bitter) \(= 0.5 \times 0.05 = 0.025\); Joint(miscalibrated, bitter) \(= 0.5 \times 0.35 = 0.175\).
• Marginal \(P(\text{bitter}) = 0.025 + 0.175 = 0.20\).
• Posterior \(P(\text{miscalibrated} \mid \text{bitter}) = 0.175 / 0.20 = \mathbf{0.875}\).

One bitter shot pushed a 50/50 belief to 87.5% miscalibrated. That is the whole move this week: an effect you can see (a bitter shot, a survey result) revises a belief about a cause you cannot (calibration, true demand). Now we add payoffs.

Build the skill 1: likelihoods: how reliable is the signal?

A survey is described by its track record, written as conditional probabilities \(P(\text{result} \mid \text{state})\), the likelihoods. This survey is good but not perfect: when demand is truly High it calls “Favorable” 80% of the time; when demand is truly Low it still says “Favorable” 30% of the time (a false alarm). The four likelihoods:

• \(P(\text{Favorable} \mid \text{High}) = 0.8\), so \(P(\text{Unfavorable} \mid \text{High}) = 0.2\)
• \(P(\text{Favorable} \mid \text{Low}) = 0.3\), so \(P(\text{Unfavorable} \mid \text{Low}) = 0.7\)

These run “state → result.” But on survey day you observe the result and want to reason “result → state.” Flipping that direction is exactly what Bayes’ rule does.

Build the skill 2: Bayes’ rule: joint, then posterior

Bayes’ rule is just three arithmetic steps you can do in a table. Chapter 11 names the ladder you are climbing: marginal (prior) → conditional (likelihood) → joint → posterior. The key identity is that the column sum is the marginal \(P(\text{result})\), and the posteriors in each column must add to 1 (a built-in check you should run every time):

1. Joint = prior × likelihood. For each (state, result) pair, multiply how likely the state was by how likely that result is given the state.
2. Marginal \(P(\text{result})\) = sum the joints down the result’s column.
3. Posterior = joint ÷ marginal. Renormalize the column so the revised state probabilities add to 1: \(P(\text{state} \mid \text{result}) = \dfrac{P(\text{result} \mid \text{state})\,P(\text{state})}{P(\text{result})}\).

Step through the table below. Pick a result and watch the prior bar get reshaped into the posterior:

Now work both columns by hand so the machine is never a black box. Taylor lays this out as a five-column table, state, prior, conditional (likelihood), joint, posterior, and shows both report outcomes side by side. Doing the same here lets the two checks appear twice:

Read the bottom row twice. The joint column sums are the marginals \(P(\text{Favorable}) = 0.60\) and \(P(\text{Unfavorable}) = 0.40\), and they themselves add to 1, because the survey must return some result. The posterior columns each sum to 1, confirming you renormalized correctly. So a Favorable survey lifts your belief in High demand from 0.6 → 0.8, and an Unfavorable survey flips you the other way to High 0.3 / Low 0.7. Those four posteriors are the inputs to everything in Session 2.

Bayesian revision with software

The two-by-two table above is the whole method, and it stays manageable by hand. Real problems chain several signals together, and at that point the bookkeeping is what breaks first. Netica (free limited edition) draws these as decision networks: nodes for the uncertain states and the signals, arrows for what informs what, and the posterior probabilities update automatically when you enter evidence.

It is optional here. Use it to check a revision you have already done by hand, not to skip the hand calculation. The take-home asks you to produce the posterior yourself, and so can a checkpoint.

In practice: the inputs are estimates, and they come from different places

• Priors are often subjective; likelihoods are often a track record. Chapter 11 separates classical, relative-frequency, and subjective probability. A prior like “High 0.6” is usually a subjective read of the market, and “different people arrive at different subjective probabilities,” so two analysts can start from different priors and reach different posteriors from the same survey. A likelihood like \(P(\text{Favorable} \mid \text{High}) = 0.8\) is usually a relative-frequency number: the vendor’s hit rate across past jobs.
• Vendors quote their best case. That 0.8 is the flattering number. Treat it like any other input. Ask how it was measured, and re-run the table with a less generous value to see if your decision flips.
• Base rates matter. Even an 80%-accurate signal leaves real doubt when the prior is lopsided. The famous reason a “positive” rare-disease test can still be probably-wrong. Always carry the prior through the math instead of trusting the headline accuracy.

Back to the puzzle

The developer doesn’t have to treat the survey as gospel or ignore it. Bayes’ rule converts “High 0.6 / Low 0.4” plus a survey result into a revised belief: Favorable → High 0.8 / Low 0.2, Unfavorable → High 0.3 / Low 0.7. Those posteriors are the inputs to Session 2’s real question. Given the revision, is the survey worth its price?

In-class group practice (wrap-up)

To close Session 1, run one Bayesian revision by hand in your group. Write your group’s names on the sheet and hand it to your TA before you leave. It counts toward participation.

• Download: Week 6 group practice (Word)

Session 2: Wed, Sep 30 · EVSI, efficiency & utility (TBD)

🧩 Puzzle

The survey would revise the developer’s beliefs, but it isn’t free. Given those revised beliefs, is buying the survey actually worth it? And separately: the project risks a real \$100k loss. Even if the expected value says “invest,” why might a cautious owner still walk away?

Build the skill 1: the survey-or-not tree, and EVSI

Lay the whole decision out as a sequential decision tree and fold it back the Week 5 way. Average the circles, take the best branch at the squares. There is a clean structural rule for building the “buy the survey” side, and it is worth memorizing because it transfers to every problem of this type:

The report nodes carry the marginals; the state nodes under each report carry the posteriors.

So the survey-result branches get .6 Favorable / .4 Unfavorable (Session 1’s marginals), and the demand branches under each report get the matching posteriors (.8/.2 after Favorable.3/.7 after Unfavorable). The “no survey” side just uses the original priors. All payoffs are in \$000:

Folding back, branch by branch:

• No survey. Invest EV \(= 0.6(200) + 0.4(-100) = \mathbf{80}\); Do not invest \(= 0\). Choose Invest, 80.
• Buy survey → Favorable (prob .6). Invest EV \(= 0.8(200) + 0.2(-100) = \mathbf{140} > 0\), so invest.
• Buy survey → Unfavorable (prob .4). Invest EV \(= 0.3(200) + 0.7(-100) = \mathbf{-10} < 0\), so don’t invest (take 0). This is the whole point of the survey: it tells you when to back out.
• Survey branch \(= 0.6(140) + 0.4(0) = \mathbf{84}\).
• Root: Buy survey (84) beats No survey (80) → buy the survey; value 84.

The expected value of sample information (EVSI) is the gain from having the (imperfect) survey:

\(\text{EVSI} = 84 - 80 = \mathbf{\$4k}\).

That \$4k is the most this survey is worth. Buy it only if it costs less. For context, a perfect forecast would be worth \(\text{EVPI} = (0.6 \cdot 200 + 0.4 \cdot 0) - 80 = 120 - 80 = \mathbf{\$40k}\). The survey’s efficiency measures how close it gets to perfect:

\(\text{efficiency} = \frac{\text{EVSI}}{\text{EVPI}} = \frac{4}{40} = \mathbf{10\%}\).

This survey captures only a tenth of the value of perfect information, useful to know before paying for it. And notice EVSI (\$4k) \(\le\) EVPI (\$40k): an imperfect signal can never be worth more than perfect information (they are equal only in the special case where the “sample” is itself perfect).

A second efficiency, for calibration. Ten percent feels low, and it is, because this survey’s 30% false-alarm rate makes it noisy. To see the other end of the scale, picture a clinic deciding whether to open a new urgent-care site (High traffic +\$300k, Low −\$150k, prior High 0.55) that can buy a paid foot-traffic study with a much sharper track record: \(P(\text{Busy report} \mid \text{High}) = 0.85\) but only \(P(\text{Busy report} \mid \text{Low}) = 0.10\). That tiny false-positive rate makes a “Busy” report almost conclusive, the posterior jumps to \(P(\text{High} \mid \text{Busy}) = 0.4675/0.5125 \approx \mathbf{0.91}\). Folding that tree gives EVSI \(\approx \$36k\) against an EVPI of \$67.5k, an efficiency of about 53%, over half of a perfect forecast. Two reference points now anchor the 0%-to-100% scale: a near-useless study (10%) and a near-perfect one (53%).

Why EVPI is also the lowest expected regret. There is a second, intuitive route to that \$40k ceiling. Taylor notes that the EV criterion and the expected-opportunity-loss criterion always pick the same decision, and that EVPI equals the expected opportunity loss (EOL) of the best decision, the expected regret you cannot avoid even when you choose well. Here the best no-info act (Invest) still leaves \$40k of unavoidable expected regret, which is exactly EVPI. So whether you think of it as “the value of a crystal ball” or “the regret you’re stuck with,” the ceiling that bounds EVSI is the same number.

Build the skill 2: utility & risk attitudes

EV-maximizing says “invest without the survey, EV = \$80k, done.” Yet a real owner staring at a possible \$100k loss might still decline. They’re not bad at arithmetic, they value outcomes nonlinearly. That’s the idea of utility: instead of maximizing expected dollars, you maximize expected utility, where utility \(u(x)\) captures how much each dollar amount is actually worth to you.

• Risk-averse: utility curve is concave (bends down). The pain of losing \$100k outweighs the joy of gaining \$100k, so you’ll skip some positive-EV gambles. Most people and firms are risk-averse for big stakes.
• Risk-neutral: utility is a straight line; maximizing expected utility is identical to maximizing expected dollars. This is the EV decision-maker of Weeks 4–5.
• Risk-seeking: utility curve is convex (bends up); you’ll accept fair or even unfavorable gambles for the chance at the upside (think lottery tickets).

(Taylor’s own text calls these risk averters and risk takers, with the linear case “indifferent to risk”. The same three attitudes under older names.) Toggle the three shapes below and watch the certainty equivalent (CE) and the risk premium open up on the developer’s invest gamble (High +\$200k at 0.6, Low −\$100k at 0.4, EV = \$80k; amounts shown in the widget are in \$000):

Beyond the book. Taylor introduces utility with utiles and the risk-averter/risk-taker idea but stops short of drawing the curve or naming the certainty equivalent and risk premium. The curve, CE, and risk premium used here are a standard decision-theory extension we add to make the idea visual and quantitative. the underlying logic (replace dollars with utility, then re-fold) is exactly Taylor’s.

Make it bite. The food-truck insurance gamble. A food-truck owner is offered \$400 of event-cancellation coverage against a 1.5% chance of a \$20,000 rained-out festival. The expected loss is only \(0.015 \times 20{,}000 = \mathbf{\$300}\), which is less than the \$400 premium, so pure EV says skip the insurance. Yet a cautious owner buys it anyway, because a single \$20k washout could end the business. Same punchline as the textbook’s auto-insurance example: EV says no, a risk-averse human says yes. Buying insurance is the purest everyday proof that people maximize utility, not dollars.

Put a number on it. Certainty equivalent and risk premium. The certainty equivalent (CE) is the guaranteed amount you’d accept in place of a gamble; the risk premium is the gap EV − CE. For the developer’s invest gamble (EV \(= +\$80k\)), suppose this owner’s concave utility makes the gamble feel only as good as a sure \$30k. That \$30k is the CE. Then the risk premium is \(80 - 30 = \mathbf{\$50k}\): the EV they’d willingly give up to escape the uncertainty. For a risk-averse person CE \(<\) EV, so the risk premium is always positive. (Push the curve concave enough and the CE can fall below \$0, at which point a sure \$0 “do not invest” beats the +\$80k gamble, and declining is perfectly rational.)

The mechanic, in one line. You don’t invent a new tree for utility. You take the same tree, replace each dollar payoff with its utility \(u(\$)\) (measured in utiles), and re-fold to maximize expected utility. Pin the scale at the extremes, \(u(+\$200k) = 1.0\) (best) and \(u(-\$100k) = 0\) (worst), and read the middle outcome off the curve. A straight (risk-neutral) line would put the sure \$0 at its linear share, \(1/3 = 0.33\); a concave curve lifts it higher, say \(u(\$0) = 0.65\), because avoiding the loss is worth a lot to this owner. Now re-fold: the Invest branch is worth \(0.6(1.0) + 0.4(0) = \mathbf{0.60}\) utiles, while “do not invest” is worth \(u(\$0) = \mathbf{0.65}\) utiles, so 0.65 beats 0.60 and the tree now recommends walking away, even though in dollars Invest won (\$80k vs \$0). The −\$100k leaf was punished harder than the +\$200k leaf was rewarded, which is exactly what dropped the gamble below the safe branch. Curve shape, the positive risk premium, and the tree’s new recommendation are all the same fact seen three times.

In practice: neither perfect information nor perfect rationality exists

• Imperfect is the normal case. “Perfect information” almost never exists. Surveys, tests, and pilots are all imperfect, which is precisely why EVSI \(<\) EVPI. The efficiency ratio tells you how much of the ideal a real study buys you, so you don’t overpay (10% here; ~53% for the sharper clinic study).
• The revision is only as good as its inputs. The posteriors rest on estimated priors and estimated likelihoods. If either is off, EVSI is off. Re-run it with pessimistic likelihoods before trusting it.
• People are not EV-maximizers. Loss aversion, limited capital, and “bet-the-company” stakes mean a positive-EV decision can still be the wrong human decision. Utility doesn’t excuse the math: it extends it, by letting you re-fold the tree in utiles instead of dollars.

Back to the puzzle

For an EV-maximizer the survey is worth at most \$4k (EVSI), capturing 10% of the \$40k a perfect forecast would be worth, buy it only below that price. That \$40k is also the unavoidable expected regret of investing blind, the same ceiling seen from the opportunity-loss side. And the cautious owner who passes on a +\$80k gamble isn’t being irrational: a concave utility curve drops the certainty equivalent to roughly \$30k (a \$50k risk premium), and re-folding the tree in utiles can tip the recommendation to “do not invest.” Decision analysis gives you both the dollar value of information and the language to respect risk attitude.

After-class check

A few questions spanning the whole week: Bayesian revision, posteriors, EVSI vs. EVPI, efficiency, and risk attitudes. Retry as many times as you like; nothing is submitted.

Weekly take-home (graded: submit on Canvas)

This week’s take-home is a Bayesian revision, EVSI and utility model for a state workforce agency deciding whether to fund a job-training program. The likelihoods are not given to you: the case brief reports an evaluator’s validation study as raw counts, and the prior comes from a different source altogether. The agency’s own portfolio. In the Your model block you say which number is the prior and write both conditional probabilities out in words, because confusing them is the error this week exists to fix. Then you revise, price the assessment against its own ceiling, and finally re-fold the whole decision on the director’s utility scale, where the recommendation changes. Two short responses plus the required AI-use disclosure go on the Free responses sheet. Upload the completed .xlsx on Canvas. That one file is your entire submission.