The Economics of Decision Models: To Make or to Buy

**To Make or to Buy: How Decision Models and Chow’s Rule Are Reshaping AI Economics**

Inspired by Adam Smith’s famous maxim from *The Wealth of Nations*—that a prudent household never makes at home what costs more to make than to buy—this article explores the emerging economics of decision models such as Jev. Unlike traditional LLMs, which generate open-ended text, decision models produce well-defined outputs in three forms: **choice** (a fixed list of options with probabilities summing to 1), **score** (a probability-weighted rating against ordered levels), and **noul** (the probability that a statement is true, named after the Bernoulli trial). Because the expense lies in reading the input state rather than generating output tokens, these models are cheap, fast, and return explicit probability distributions. This gives rise to a „decision hierarchy” analogous to a computer memory hierarchy: hand-written rules and cached answers act as the L-cache, decision models serve as RAM, LLMs as disk, and humans as magnetic tape—each level handling what it is confident about and escalating the rest.

The core mechanism for deciding when to escalate is **Chow’s rule**, originally formulated in 1970. A classifier should answer with its top class only when the expected cost of being wrong is less than the cost of handing the input to the next level. Formally, the rule states that \(1 – p_{\text{max}} \lt \text{NX}_{\text{err}} + \text{NX}_{\text{price}} / \text{NX}_{\text{errcost}}\), where the next level’s error rate, price per evaluation, and cost per mistake determine the break-even threshold. The key insight: when errors are expensive, a cheaper model barely helps—accuracy alone decides. The next level is essentially „selling fewer mistakes,” and each layer in the hierarchy is a supplier with three numbers: a price, an error rate, and the cost of each error.

The article extends Chow’s rule in two important ways. First, it accounts for **confident mistakes** through random audits (e.g., sending 1% of confident decisions downstream anyway) to better estimate error rates. Second, it generalizes the rule to handle **asymmetric error costs** by replacing the flat error cost with a category-weighted expected cost \(C’ = \sum_c p_c \times \text{NX}_{\text{errcost}}(c)\). This reveals a striking consequence: the top class is not always the cheapest answer. A ticket with probabilities tech 0.5 and legal 0.45, where a legal misroute costs $100, should sometimes be routed to legal even when tech is the most probable label. The article also shows how to fold latency into the rule by adding a wait-cost term.

Looking ahead, the author forecasts that decision-model interfaces will be commoditized rapidly. Jev’s advantages—cheapness, speed, probability distributions, and closed outputs—stem from its interface, and interfaces can be copied. If a cheap LLM like GPT-5 nano becomes more accurate than Jev, vendors can simply call LLMs in decision mode, which is precisely what OpenAI’s Decisions API does. What cannot be copied easily is **honest, calibrated probabilities**, which may be where specialized decision-model vendors retain an edge. The future may even bring decisions sold with warranties—priced with a premium and deductible, much like Riskified today—and expanded multimodality, from grid-cell image queries („which cell contains a can of Cherry Coke?”) to audio time-window searches and real-time sensor reading in robots and factories. As costs of reading multimodal inputs fall, decision models could become the backbone of on-device and industrial decision-making.

**Keywords:** decision models, Chow’s rule, Jev, AI economics, LLM cost optimization, decision hierarchy, calibrated probabilities, error-cost threshold, memory hierarchy, multimodal AI.


Ez a cikk a Neural News AI (V1) verziójával készült.

Forrás: https://pncnmnp.github.io/blogs/economics-decision-models.html.