Analysis / 04
Analysis 04 · Research methods
What a pooled model hides.
A choice-based conjoint study, built end to end on simulated respondents whose true preferences are known. The average buyer the standard model describes does not exist in the data, and the pricing conclusion changes once that is taken into account.
Every number on this page comes from a simulation I wrote. There is no survey, no client and no real product behind it. I have run conjoint studies professionally, for telecoms offers and for car-rental positioning; those datasets belong to the clients, so I rebuilt the method on invented data. The advantage of a simulation is that the right answer is known, which makes it possible to show how close each model gets.
The set-up
The fictional product is an income-protection policy sold alongside a loan. Each simulated respondent sees ten screens. A screen offers three policies and a “none of these” option, and the respondent picks one. Policies differ on what they cover, how long the benefit is paid, the waiting period, who sells them and the monthly price (€6 to €15).
The population is built from two groups. Price-led respondents (57% of the sample) react strongly to price, often choose nothing and care little about extra cover. Protection-led respondents (42%) want broad cover and long benefit periods and are much less sensitive to price. Individual tastes vary randomly around each group’s values, and every choice includes the random term that a logit model assumes.
One model for everyone, then one per group
The standard first analysis is a multinomial logit fitted to all 6,000 choices. Its coefficients are precise (the intervals in the chart are narrow) and every sign is plausible. It is also a description of nobody: on almost every feature the pooled estimate falls between the two groups, close to neither.
A latent-class logit asks a different question: suppose there are two kinds of respondent whose identity is not observed; what preferences for each kind, and what split between them, make the observed choices most likely? Fitted by expectation–maximisation from eight random starts, it finds groups of 57.6% and 42.4%, recovers most of the true values within its confidence intervals, and has the lower BIC (14,374 against 14,592).
The pooled estimate sits between two groups and matches neither
Estimated effect of each feature relative to its reference level, with 95% confidence intervals. Black vertical ticks mark the true values used to generate the data.
Source: simulation and estimation by the author (conjoint.py), seed 20261114. Synthetic data. Willingness to pay = −(feature coefficient) ÷ (price coefficient), standard errors by the delta method; intervals wider than the axis are clipped.
Why it matters for a pricing decision
Switch the chart to willingness to pay. The pooled model values the step from unemployment-only to full cover at about €4.87 a month. A product manager reading that would price the premium tier a few euros above the basic one. The two-class model says that 58% of the market would pay under €2 for it and 42% would pay around €16. Those are two different products for two different customers, and a single mid-priced tier serves neither well.
The simulator makes the same point with shares. Configure a basic and a premium policy and compare the rows. The two aggregate rows can look alike while the two class rows describe opposite markets: one group mostly walks away from the premium policy, the other mostly buys it. A single tier price averages over that split.
Share simulator: two policies against “neither”
Predicted share of choices under each model for the two configurations you set. Lighter segment: Product B. Grey: neither. Synthetic data.
Source: logit share predictions from the coefficients in conjoint.json. Two-class shares are weighted by estimated class size. A share of choices in a survey task is not a market share: awareness, distribution and inertia are absent.
Design. Random (not optimised) full-profile design: 600 respondents × 10 tasks × 3 alternatives plus an opt-out. Attribute levels are dummy-coded against the first level; price enters linearly. “None” was chosen in 28% of tasks.
Estimation. Maximum likelihood by Newton–Raphson, written directly in NumPy so that every step is visible. McFadden’s ρ² is 0.128 for the pooled model and 0.147 for the two-class model. Class-level standard errors are conditional on the estimated memberships and are therefore somewhat too small.
What a simulation cannot show. Real respondents satisfice, misread screens and anchor on the first price they see. Real segments are not two clean groups, and the number of classes has to be chosen, not assumed. In real studies I would use a balanced design, hold-out tasks, and usually a hierarchical Bayes model alongside the latent-class one. The demonstration isolates one issue, heterogeneity, and nothing more.
Data and code
- conjoint.py: simulation, both estimators and the willingness-to-pay calculation (about 130 lines)
- conjoint.json: design, estimates, standard errors and the true parameters