Zillow's price estimates are screwing up homebuying
Alvaro Dominguez for BI
When Zillow debuted in 2006, the fledgling site bore little resemblance to the real-estate behemoth it is now. There were no options to find an agent, get a mortgage, or request a tour — the search portal couldn't even tell you which homes were actually for sale. There was, however, the Zestimate: a "free, unbiased valuation" for 40 million houses around the US, based on a proprietary algorithm. Half the single-family homes in America suddenly had a dollar figure attached to them, and anyone could take a peek. Zillow's site crashed within hours as a million people raced to ogle at the results.
The initial rush was a sign of things to come. Nowadays, the Zestimate is arguably the most popular — and polarizing — number in real estate. An entire generation of homeowners doesn't know life without the algorithm; some obsessively track its output as they would a stock portfolio or the price of bitcoin. By the time a seller hires a real-estate agent, there's a good chance they've already consulted the digital oracle. For anyone with even a passing interest in the housing market, the Zestimate is a breezy way to take the temperature. Keep tabs on mortgage rates all you want, but they can't tell you that your house has appreciated 20% over the past year, or that your annoying coworker's property is worth more than yours.
Many industry insiders, however, regard the number as a starting point at best and dangerously misguided at worst. Real-estate agents recount arguments with sellers who reject their pricing advice, choosing instead to take the Zestimate as the word of God. One meme likens its disciples to adults who still believe in Santa. Zillow itself lost hundreds of millions of dollars during the pandemic when it relied on its algorithm to buy homes at what turned out to be inflated prices, part of an ill-fated attempt to flip homes at scale.
The Zestimate is just one of a slew of automated valuation models that are increasingly used by banks, investors, and laypeople to estimate the value of homes. No other model, however, has wormed its way into our culture like the Zestimate. The model, like other consumer-facing AVMs, is prone to errors that render it more of an amusement than a serious pricing tool. But while the algo's price-guessing skills may be suspect, it's undeniably elite at one thing: luring people to Zillow-dot-com.
The Zestimate is both everywhere and an enigma. About 104 million homes, or 71% of the US housing stock, have a little dollar figure hovering above them on Zillow's website. One of them is the house in Austin where I was raised until the age of 10. It's not for sale, but right underneath the address, in bold, is the Zestimate. Next to it is a "Rent Zestimate," or the amount the owner could probably charge a tenant each month. You can click to see a graph of its Zestimate over the past decade — the Zillow-fied value of my childhood home rose a staggering 72% from May 2020 to its peak in May 2022 but has since dropped 24% from that top tick thanks to the chill running through the Austin market. In just the past 30 days, the Zestimate has dropped by $4,455. Ouch.
Just how accurate are those numbers, though? Until the house actually trades hands, it's impossible to say. Zillow's own explanation of the methodology, and its outcomes, can be misleading. The model, the company says, is based on thousands of data points from public sources like county records, tax documents, and multiple listing services — local databases used by real-estate agents where most homes are advertised for sale. Zillow's formula also incorporates user-submitted info: If you get a fancy new kitchen, for example, your Zestimate might see a nice bump if you let the company know. Zillow makes sure to note that the Zestimate can't replace an actual appraisal, but articles on its website also hail the tool as a "powerful starting point in determining a home's value" and "generally quite accurate." The median error rate for on-market homes is just 2.4%, per the company's website, while the median error rate for off-market homes is 7.49%. Not bad, you might think.
When you think of the Zestimate, for many, it gives a false anchor for what the value actually is.
But that's where things get sticky. By definition, half of homes sell within the median error rate, e.g., within 2.4% of the Zestimate in either direction for on-market homes. But the other half don't, and Zillow doesn't offer many details on how bad those misses are. And while the Zestimate is appealing because it attempts to measure what a house is worth even when it's not for sale, it becomes much more accurate when a house actually hits the market. That's because it's leaning on actual humans, not computers, to do a lot of the grunt work. When somebody lists their house for sale, the Zestimate will adjust to include all the new seller-provided info: new photos, details on recent renovations, and, most importantly, the list price. The Zestimate keeps adjusting until the house actually sells. At that point, the difference between the sale price and the latest Zestimate is used to calculate the on-market error rate, which, again, is pretty good: In Austin, for instance, a little more than 94% of on-market homes end up selling for within 10% of the last Zestimate before the deal goes through. But Zillow also keeps a second Zestimate humming in the background, one that never sees the light of day. This version doesn't factor in the list price — it's carrying on as if the house never went up for sale at all. Instead, it's used to calculate the "off-market" error rate. When the house sells, the difference between the final price and this shadow algorithm reveals an error rate that's much less satisfactory: In Austin, only about 66% of these "off-market Zestimates" come within 10% of the actual sale price. In Atlanta, it's 65%; Chicago, 58%; Nashville, 63%; Seattle, 69%. At today's median home price of $420,000, a 10% error would mean a difference of more than $40,000.
Without sellers spoonfeeding Zillow the most crucial piece of information — the list price — the Zestimate is hamstrung. It's a lot easier to estimate what a home will sell for once the sellers broadcast, "Hey, this is the price we're trying to sell for." Because the vast majority of sellers work with an agent, the list price is also usually based on that agent's knowledge of the local market, the finer details of the house, and comparable sales in the area. This September, per Zillow's own data, the typical home sold for 99.8% of the list price — almost exactly spot on. That may not always be the case, but the list price is generally a good indicator of the sale figure down the line. For a computer model of home prices, it's basically the prized data point. In the world of AVMs, models that achieve success by fitting their results to list prices are deemed "springy" or "bouncy" — like a ball tethered to a string, they won't stray too far. Several people I talked to for this story say they've seen this in action with Zillow's model: A seller lists a home and asks for a number significantly different from the Zestimate, and then watches as the Zestimate moves within a respectable distance of that list price anyway. Zillow itself makes no secret of the fact that it leans on the list price to arrive at its own estimate.
Other sites have their versions of the Zestimate, too — there are actually about 25 different AVMs in the market, says Lee Kennedy, the founder and managing director of AVMetrics, a company known for independently testing these models. Realtor.com will show you three estimates, each from a different AVM provider. Redfin, a Zillow competitor, also has its own model. Kennedy has been studying AVMs for more than three decades, but it wasn't until the advent of Zillow that the masses became aware of them. Consumer-facing AVMs, like the Zestimate or the Redfin Estimate (Restimate?) are meant to be used informally, he says, as casual starting points before consulting real experts. They're not supposed to be used for real pricing, which should be left to the big guys — the "business-to-business" AVMs used by banks, investors, and the government-sponsored enterprises Fannie Mae and Freddie Mac. Lauryn Dempsey, a real-estate agent in the Denver area, gives similar advice to her clients.
"They're tools that provide information," Dempsey says, "but they should not be used in a vacuum to make decisions."
Joe Raedle/Getty Images
The business-to-business models are so costly to develop, Kennedy tells me, that they'll probably never be offered to regular people for free. But his testing indicates they're much more reliable. His firm has unveiled blind testing that looks at how models perform before taking into account the list price, a method that penalizes those aforementioned bouncy algorithms. The standard measurement breaks down how often the model can get within 10%, in either direction, of the actual selling price. In a highly urbanized area with lots of housing transactions, some of the models can correctly get close to the final selling price about 80% to 90% of the time — "not bad," Kennedy says. AVMs of all kinds work best in areas with a lot of homes that look and feel roughly the same. Cookie-cutter suburbs are heaven; areas with a wide range of home styles and ages, like Boston, pose a greater challenge. The value of a ranch home in the middle of nowhere is even tougher to peg.
So the Zestimate isn't exactly unique, and it's far from the best. But to the average internet surfer, no AVM carries the weight, or swagger, of the original. To someone like Jonathan Miller, the president and CEO of the appraisal and consulting company Miller Samuel, the enduring appeal of the Zestimate is maddening. "When you think of the Zestimate, for many, it gives a false anchor for what the value actually is," Miller says.
Miller is no unbiased observer. Given that he's an appraiser who estimates the value of homes for a living, it should come as no surprise that he's siding with the humans over the robots. But he raises real issues, highlighting the disconnect between the public's continued use of the Zestimate and its actual track record.
I could say that I virtually stalked my childhood home for "research," but let's be real: By the time I scrolled to the bottom of the page, I had fully surrendered to the voyeuristic urges that draw millions of visitors to the Zillow website each month. It's been almost two decades since I've stepped inside the house, and I can only imagine the changes its new owners have made to my old room (sadly, no pics of the interior). But with the aid of Zillow, my trip down memory lane was lined with data: I walked away with intimate knowledge of the home and its occupants. Prior to 2006, no regular person had this kind of power.
The launch of Zillow spawned a whole genre of internet snooping that, if anything, has only intensified in the years since. When I call up John Wake, a former economist and real-estate agent who now writes the newsletter Real Estate Decoded, he reveals that he, too, looked up his childhood home only a few months ago. "That part is really fun," he tells me. Keeping tabs on your own Zestimate, though, can provide less of a thrill. In December 2022, after interest-rate hikes tamped down home prices, Wake shared with his followers on X that his Zestimate was down 18% from May: "YIKES!" In a 2020 column, the Wall Street Journal editor Kris Frieswick opened up about the difficulty of quitting the algorithm: "My self-worth is defined by my Zestimate. Each day I approach Zillow.com filled with hope, and fear." The column reads mostly as tongue-in-cheek, but plenty of people take their number very seriously. As Frieswick pointed out, at least several disgruntled homeowners have actually sued Zillow over Zestimates they said were inaccurate.
Looking up other people's houses, by comparison, is a mostly harmless pastime. Bosses, neighbors, lovers, and exes — all are fair game in the all-seeing eyes of the tool. During the heat of the 2021 homebuying frenzy, a "Saturday Night Live" sendup of a Zillow ad declared: "The pleasure you once got from sex now comes from looking at other people's houses." The skit, which featured a lot of moaning and sultry mood lighting, was mostly about the fantasies of browsing homes for sale on Zillow — as one YouTube commenter observed, "They didn't even get into the naughty pleasure of looking up all your friends' Zestimate values." This kicked off a thread of others chiming in with "guilty!" and lots of cry-laughing emojis. "OMG I thought this was just my kink," another person replied. I imagine all of these people at a raucous dinner party, bonding over their exploits on zillow.com. And here I am, the buzzkill in the corner talking about median error rates.
Virtual spelunking aside, the hazards of the Zestimate are most obvious when a seller actually decides to list their home. Francine Carstensen, a real-estate agent in Alabama, says those in her line of work have a complicated relationship with the Zestimate: "We love it, and we hate it." A lofty estimate might jolt a homeowner into action — "I could sell my house for what?!" — and drive more business her way. But the number can also make it hard to do her job. A few times, she tells me, she's lost clients over a pricing disagreement involving the Zestimate. It can be difficult enough to pry a seller away from their unrealistic expectations without a number on a screen confirming their hopes for a bigger payday.
"I hate it when they tell me, 'Well, this is what Zillow tells me my house is worth,'" Carstensen says. "Because it's very rarely accurate."
Accuracy may not even be the point. It didn't appear to be in 2006, when the beta version of the Zestimate launched. "The Zestimate started out fairly inaccurate, but it didn't matter," Rich Barton, a Zillow cofounder who was then its CEO, recalled in a 2021 podcast episode. "It was provocative." Spencer Rascoff, another cofounder and former CEO, sold his own home in 2016 for 40% less than its Zestimate. The next year, the company offered $1 million to whoever could improve the Zestimate algorithm the most. The winning team, a group of three data scientists working remotely from the US, Canada, and Morocco, beat the Zillow benchmark by 13%.
I hate it when they tell me, 'Well, this is what Zillow tells me my house is worth.' Because it's very rarely accurate.
No misstep appeared more damning, however, than the implosion of Zillow's homebuying business. In 2018, the company launched Zillow Offers, making all-cash offers to sellers looking to move quickly and seamlessly. In theory, Zillow could then turn around and offload the home in short order for a modest fee, plus however much the home had appreciated. The company used a combination of internal algorithms and human analysts to value the home and predict what it could sell for in a few months — in some cases, homeowners could get an immediate cash offer based on their Zestimate with just a few clicks. But the company's forecasts turned out to be way off base. Zillow Offers squandered $422 million in the third quarter of 2021 alone — a Business Insider investigation found that almost two-thirds of the homes listed by Zillow in Atlanta, Phoenix, Dallas, Houston, and Minneapolis were being marketed at a loss. Amanda Pendleton, a Zillow spokesperson, tells me it was the volatility of the market, not the Zestimate, that really led to the program's downfall. Once the losses came to light, the company swiftly shuttered the division and laid off a quarter of its staff.
I remember wondering whether this would be the death knell for the Zestimate, a kind of algorithm-has-no-clothes moment. I was wrong. Zillow and its best-known creation haven't gone anywhere — the company continues to highlight its progress, providing periodic updates as its data scientists tinker away at the formulas. As search portals like Homes.com and Redfin jockey with Zillow for dominance, the Zestimate is too valuable of an asset to give up. People still flock to Zillow for those little numbers next to each home, for the thrill of feasting their eyes upon something that, like salaries, is considered taboo to talk about in person. For Zillow, that's an unequivocal win.
"It's 100% a marketing tool," says Mike DelPrete, a scholar-in-residence at the University of Colorado Boulder who studies the intersection of tech and real estate. "Like, not even 99%. It's a marketing tool."
James Rodriguez is a senior reporter on Business Insider's Discourse team.