Resources · PM Prep · How do you measure success of the Hot Home feature…
Measuring Redfin's Hot Home: Start Here, Not There
Before you pick a metric, you need to understand what the feature actually claims — and why that changes everything.
THE QUESTION
How do you measure success of the Hot Home feature in Redfin?
The way you answer this question reveals how you think about prediction features. Make a choice at each step — the reasoning unlocks after you commit.
DECISION 1 OF 3
Before defining any metrics — what's the first thing you'd want to know about the Hot Home feature?
THE PLAYBOOK
WHAT THIS QUESTION IS REALLY ASKING
This question is really asking whether you know the difference between measuring a feature and validating a claim — Hot Home makes a prediction, and most PMs skip straight to engagement metrics without checking if the prediction is any good.
HOW TO THINK ABOUT IT
When you see any prediction or recommendation feature, resist the pull toward engagement metrics. Ask first: what is the feature claiming, and is that claim correct? Accuracy is the foundation — every other metric is noise until you know the prediction works. Then ask separately for each stakeholder whether accuracy is actually creating value for them.
THE ANALOGY
"Measuring engagement on a prediction feature is like rating a weather app by how often people open it — not by whether the forecasts were right."
EXPLORE FURTHER
Once you can measure a prediction feature well, the harder version is: what do you do when accuracy is good but engagement is still low? That's the trust problem — and it shows up in LinkedIn's connection suggestions, Spotify's Discover Weekly, and anywhere a recommendation engine has to earn belief before it can create behaviour.
See more questions →