I totally agree. It's not impossible to imagine their model working: why couldn't you serve as a market-maker for homes at a large scale, especially with the unique insights Zillow could have based on their datasets.
However I think where the hubris lay is in how they thought they could leapfrog all the way to an automated solution before building a competency as a house-flipping company.
From what I understand, where they failed was partly in building a rich enough model to properly account for the less easily quantifiable elements which ultimately account for a property's value. I.e. the price per square foot might make a property look like a steal, while something like a sewer main nearby, or problematic neighbor could radically change the value proposition to anyone standing at the site. That's a non-trivial problem to solve for even the best ML and it's not clear how you would automate this.
If you ask me, instead of focusing on building an automated price discovery system, they should have started by trying to build a quality home-flipping organization, and figuring out how to super-charge manual work using their datasets. Over time you might find ways to optimize the process and increase the level of automation to scale output relative to head-count.
If you pledge to purchase at the Zestimate then people who reasonably think they can get more than the Zestimate on the open market don’t have an incentive to sell their house to Zillow (besides convenience). But people who think the Zestimate is an over estimate will of course sell to Zillow. So instead of a normal distribution of actual value:estimated value you end up with a skew towards the end where the estimate is over the actual value.
Trading housing is very different from normal market making because houses are not fungible commodities like most securities are. For most entities trading securities at low frequency it does not really matter whether a market maker skims off a few pennies on their trade; it’s worth it for the liquidity. Houses are less liquid (because they are non fungible) so the liquidity is more valuable, but the price improvement routing around a MM can also be many percentage points of a trade because there are not only so many factors affecting their valuation, but also just chance and random noise (bidding war, a particular buyer falling in love with the property, not-price-conscious buyers).
According to Matt Levine's recent column, while you might think that, it wasn't what sunk them in practice. Bidding low in fact worked; it just was inherently limited in scale, which is why they switched to bidding higher. Unfortunately, being wrong in the other direction is very bad.
"I know, I know, the traders are saying: “No, this is stupid, your algorithms will not be 100% precise, some of your ‘lowball’ bids will in fact be too high, and those will be the ones that sellers accept. You’ll get adverse selection and end up losing money.” But that was not Zillow’s actual experience in the first quarter! The actual experience is presumably that some people accidentally got too-high bids, realized they were good and accepted them, but mostly Zillow sent too-low bids to everyone, and some people, for whatever irrational reason — market ignorance or financial necessity or laziness or whatever — accepted the too-low bids. The general point is that there is no reason at all to think that the people on the other side of these trades from Zillow are generally better informed than Zillow is. Sure they know more about their houses than Zillow does, but Zillow knows more about the market, and has more money"
"If you systematically bid too low, you will not do many trades, but you will make a lot of money on each trade. If you systematically bid too high, you will lose money on each trade, and also you will do a whole ton of trades. This is much worse!"
Indeed, I believe this is what OpenDoor does. From The Economist article [1],
"They [OpenDoor] charge a fee for the services they provide: buying and selling homes immediately, with zero fuss. The quick in-and-out makes them more like marketmakers than property investors, who buy to hold.
...
"A former Zillow employee told Business Insider that management had been hellbent on catching up with Opendoor, the front-runner. In order to compete, the employee alleged, the company pushed to offer generous deals to potential clients. It called this “Project Ketchup”. Now it has its own fake blood on its hands."
[1] https://www.economist.com/finance-and-economics/2021/11/13/a...
These are different things.
Archetypal market making involves simultaneously buying and selling an asset. Flipping involves buying, improving and later selling. One might be able to deal with the heterogeneity of houses by operating at scale. (Zillow attempted this.) One might also deal with the delay between buying and selling by hedging. (Zillow never seems to have thought about this.) But the improvement function makes what Zillow attempted fundamentally separate from market making.
They weren’t paid to provide liquidity. If anything, they paid a premium for scale and immediacy. They were a real estate operation masquerading as a tech outfit. WeWork in different stripes.
The main insights are that market makers hold assets for a short period of time making money on the spread between buyers and sellers offers. Zillow had to hold on to houses for a long time and was speculating that the houses would be worth more in the future which is not market making.
Does it? I worked for a few years for a market maker, and that's not what we did. Simultaneous buying and selling is what the arb guys did. We'd buy and sell with generally short hold times. Which makes sense to me given that the exchange has market makers to provide liquidity. If something can be simultaneously bought and sold, then the market-maker is unnecessary.
It reminds me of one of my favorite Bill Gates quotes:
"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency."
Starting from scratch can be a huge advantage.
Not to mention that generally ML models are not useful for assessing risk. ML nearly always focuses almost exclusively on some point estimate rather than a distribution of what you believe about a value. The former case is all about expectation and the latter about variance. Correctly modeling variance is far more essential to risk modeling than expectation alone.
I recall talking to a startup that was attempting to model credit risk by building a binary classier for defaulting, and trying to figure out a way to use this to score people for credit (obviously they chose to ignore the fact that there is a huge industry with decades of experience in assessing consumer credit risk).
They focused exclusively on finding more advanced models to get better AUC without even realizing that that's not important. I mentioned that the most simplistic credit score model should at least model P(default|info) and then set the interest rate to - P(default|X)/(P(default|X)-1) to break even and they couldn't comprehend this basic reasoning. It was doubly hilarious since their population's base default rate was such that the solution to this equation was higher than the legal limit they could charge for interest.
In the early part of the current startup/tech boom there was a focus on "disruption", the idea that new ideas could easily dominate old ways of doing things. But for many industries, such as credit/lending and real estate, you should at least understand the basic principles of how these "old ways" work before trying to disrupt them.
It is actually quite a common practice to design neural networks that output probability distributions.
This is the real problem.
Even if they have the historical data for that exact house/unit, it won't help them in cases such as:
* That nice view of the woods out the window is now blocked by a massive radio antenna that was just built there
* The river running through the back yard is now heavily polluted by something up-stream
* The new neighbor across the street is a huge nuisance and says they will never move
* The house just had a mass-murder event in it
Just because something is now cheaper than "comps" at price/sq ft and other metrics doesn't mean it's comparable.
Minor? Usually.
Random? Not at all. A minor annoyance like a cracked driveway ($1,500 to fix) is also likely to be associated with older kitchen appliances, faulty water pressure, deteriorating deck; poorly seated windows, etc. And then, buying that house for what the algo tells you -- or even algo minus 3% -- isn't likely to be a happy choice. Its fair market price may be algo minus 10% or worse.
Also worth bearing in mind, the Realtor community is not going to make life easy for Zillow. Once it's known that Zillow is loading up on clunkers, buyers' agents are likely to tell their customers: There's a Zillow house on the market, too. It's probably got problems. I'd demand a full inspection and some indemnities if I were you.
Common flaw of market disruptors. They assume that the existing players will remain neutral and indifferent to their arrival. The real world tends to be much tougher.
Originally the estimate on Zillow said my house was 20% over the value I actually sold my house for just last month. I listed with a traditional realtor for a 5% commission, because when I looked up the service and other fees for Zillow sales, I found they included around 20% of cost for buying homes and closing within generally 10 days.
As I listed my house, and as I reduced price on it for it to gain attention, I noticed the zillow estimate also went down to always stay below my listed price. I believe the estimate that both Zillow and Redfin display prominently were purely based on what my list price was changed to last, not on any meaningful algorithm, which can be very harmful to sellers and buyers, because it makes the process a bit deceptive by nature. Luckily Zillow also displays the price history on homes, which apparently cannot be "gamed" as much as the "zestimate" can be. Another thing I noticed was that the view stats on my listing that zillow regularly provided changed, even after days passed, that was very concerning because stats of that kind aren't supposed to change... They indicate real interest in a property, that guide decisions for sellers to reduce price, and they also indicate what is truly a "hot home".
No matter what, there is always the "human factor" that can corrupt or even destroy any company, where realtors can game the process to maximize their own sales profit or positions, or where appraisers can inflate an estimate as a favor for a personal friend, even despite laws against doing so. In a bad economy, the lengths people will go to to suit their advantage are wild. This type of issue can never be properly addressed by any algorithm, and that's why trusting technology too much can so easily lead to failure in any setting.
Ultimately I am glad I did not sell to Zillow, because of all of the potential for hidden costs and because they manipulate the process even when you don't use their service, but I am not feeling sorry for them as a company... I felt the impact of their presence in the market whether I involved them or not, and that's a big problem when it comes to preserving the value of traditional investment and stable investment in a house that should be properly addressed by regulation.
The fact that a house is for sale at a given price, but has not sold after some time, is a strong signal that it's overpriced. The longer it's been sitting, the stronger that signal is. They'd be crazy not to include that data in the Zestimate.
Now, if it's extremely fast, eg they adjust the price down within a day or so, then it seems a little ridiculous. OTOH the Zestimate has always been a rough indicator at best.
I wish I had the data that I assume they have internally, because watching their actions I’m not convinced they understand what questions would actually be interesting to explore with ml.
Why would Zillow have unique insights? With the exception of Texas, I thought real estate sales information is public information in the US.
How many people search on bedrooms but not bathrooms? When people search on both, what’s the pattern they use? If we highlight prices and BRs on the map does that give more clicks than just prices? How important are photos (times 50 different questions there)? How strong a signal is repeat views spaced over time? Saving a house to favorites? Sending a link to a friend? Clicking on comps in the neighborhood? Which comps do people zero in on (as evidenced by spending more time on the page)? How strong a signal is sending a message to the real estate agent on the listing? What areas of the country are seeing an uptick in search traffic? How long between claiming a house as an owner on the site, updating the information, and listing it for sale?
They are sitting on a (well-earned) treasure trove of data and it’s not unreasonable to think they could use that to be better informed than another buyer without that information.
Where they seem to have failed is in not augmenting that advantageous data with regular old boots-on-the-ground observations.
In my mind this is the problem with consultants who try to automate processes. It’s really difficult (maybe even impossible?) to successfully write a program to make a computer do $thing if you don’t understand the intricacies of how to do $thing manually.
Houses trade slowly, so would sit on Zillows books for a long time (days/months). Market makers on the stock market can have assets sit on the books for under a second. Houses are not fungible, which extenuates the slow trade problem.
Not in my book. All I see is the price of real estate being driven up by corporate greed and the individual home-buyer being shut out of the market.
Is it wrong of me to hate "flippers" (be they corporate or private)? Pure capitalists will tell me that every property sold went to the highest bidder — in the case of a flipper winning they were willing (able) to risk the capital to hopefully turn a profit on the flip.
I suspect if you dig deeper you might find sales going to flippers because they had 100% cash offers, because they are better at "the game". I see no reason to punish prospective first-time home owners in this sort of market.
But I don't know what the answer is either.
If houses were a (much) smaller bet for the buyer, there would be more flexibility to build houses where demand exists and a faster, lower-drama exit for people who don't like the changing nature of their in-demand neighborhood.
The inertia created when people have their life savings tied up in their house perpetuates the problem of affordability, by making the areas that have the most mismatched supply vs demand the least likely to deal with the problem.
The process of building new structures is filled with so much regulatory friction that it is impossible for the average person to even consider building their own home.
That housing should also be up to a similar standard in terms of its externalities like pollution and energy efficiency etc.
We have regulations for air travel, for car emissions and efficiency, why should housing be any different?
All successful work probably displaces someone else in some way. If you're good at your job, you're "denying" that job to someone less skilled. If you work in software, you're automating things that would require more labor if done manually. Fortunately, humans can pivot.
Either hate everyone, or hate no-one. You can't just hate flippers.
House flipping isn't a social negative. They're doing a productive activity and producing value. They aren't long-term speculators removing housing stock from the market. It's essentially home renovation, done by a 3rd party owner.
Who is doing the selling? "Wall St Fat cat Co" or the average Joe who saw his house value go up by a LOT?
When average homeowner Joe sells their house they still have to live somewhere. They must immediately use that money for another house, which is also inflated. The higher sale price doesn’t matter.
Average non-homeowner Joe trying to buy a first house is SOL.
Flippers take the risk of the market falling while they're flipping - that's the price they pay for their profits.
It pains me to watch people apply simplistic theoretical laws of supply and demand to something as complicated as housing. The map is not the territory. There are massive costs to increasing supply, as well as psychological/community costs to moving homes, which are not cleanly captured in any Economics 101 textbook.
At a guess, in our county, 20%+ of the housing is idle, owned by out-of-state companies, some of whom pay property taxes and some dont. The county isn't auctioning off because of tax default anymore, no one was buying these places at $100. Many of these places are complete teardowns now; some actually no longer exist, having burned or apparently been scrapped. The tax assessments on those have not been adjusted, for the few i checked.
I think the housing market is so fucked no one really grasps the scale of the problem.
I don't think I agree with this assessment. I live in a very rural area two hours northwest of Austin, literally in the middle of nowhere. I've studied the local economy and understand how things work here.
I think the characteristics you've identified in the rural housing supply are not unusual and also not as serious in a practical sense as you seem to be indicating. For example, in San Saba, Texas, 20-30% of the households are under the federal poverty threshold. The median household income in the town of San Saba is about $32K/yr. People just don't have any excess cash so the maintenance on dwellings is neglected. That means folks become extremely thrifty and resourceful patching what needs to be patched, very cheaply, if not for free. Some dwellings simply aren't maintained and one day won't be there anymore.
Families live on small budgets, don't require much and generally just "get by". The municipal and county governments have very small budgets but extremely resourceful staff who accomplish a lot with very little. Everyone comes together as a community when needed (see: February 2021 freeze event) and it all works very efficiently, actually.
To someone who is not from here and who doesn't understand that dynamic, they might see those properties as you described and believe a tragedy was unfolding. But that doesn't reflect reality on the ground vis-a-vis my neighbors.
I've seen this personally, too. A house I rented until a couple of years ago was owned by a Chinese company, which also owned half of the other houses on the block. We all paid rent to the same LLC that forwarded the cash overseas, and did almost zero maintenance.
I think the housing market is so fucked no one really grasps the scale of the problem.
One thing I don't see discusses very often is the affect that large "master-planned communities" have on a city's housing prices. I've seen at least three cities where mega developers like Howard Hughes Corp own massive tracts of land, but instead of building houses, sit on that land waiting for the price of housing to go up. Sometimes the developers are very open about it. Sometimes not. But instead of allowing a free market to develop 5,000 new homes, they develop one lot here and one lot there.
Or worse — I've seen them build hundreds of homes and then sit on them, empty and vacant, waiting for prices to climb high enough to put the houses on the market. Again, a drip at a time, to keep the housing supply artificially small so they can boost their profits. Meanwhile, people have nowhere to live.
What's the issue with out of state companies owning rural properties nobody wants? If the market is heating up, maybe it's time to run tax auctions again.
In WA state, if there's no bidders, the county retains the land and will auction it again when someone expresses interest (or it some cases, can sell it to a neighboring land holder without auction, like for the 1930s era tax foreclosure I bought last year)
Jobs generate demand for homes. Homes cost a lot in areas where there's been more jobs added than homes. In the last decade, the bay area has added seven jobs per every unit of housing constructed.
The amount of social media content revolving around "how I became a milionaire/how I reached my first million" and the common factor is "I bought a house in 201*", then I'd say something is a bit off...
Either there's massive speculation, or 1 million isn't what it used to be, or worst: both.
The problem is that their blogging about it attracts the people that want to get rich quick and they are the ones likely to lose their shirts.
If this kind of algorithmic speculation took off, not only are we likely to see the algorithms themselves form feedback loops to push prices up far higher than otherwise, even if they don't sell directly to each other, I believe it will drastically increase the size and rate of the boom bust cycle.. and we're doing that to _peoples homes_.
The scope of human suffering possible here is huge, the societal damage massive.
This needs to be so very very illegal.
If buyers want more now for their house, than it can be sold for in a few months time (which is necessary for renovations and other prep for sale), then there is no ML (and no non-ML) method to make money. Either you overpay and lose money, or you don't overpay and you don't buy any houses.
In that situation, the only smart play, is to get out of the market. Zillow is, no doubt, not perfect. But they have a lot of knowledge of the housing market, and they thought it was time to get out entirely. I think the author of the article either isn't able, or doesn't want, to consider that Zillow might have been exactly correct in doing so.
1) Why layoff your data science division if they are predicting with accuracy?
2a) If you have enough conviction to call the top of the market, why sell off so much housing at a huge loss? Zillow are the only participant in the residential real estate market losing money right now.
2b) If you see signals of a forthcoming housing crash, why not short the housing market?
The simplest explanation is that Zillow was poorly run.
- they could not buy houses without overpaying (relative to what they could sell them for a few months down the line)
- the housing market would not recover for several years (so no need to keep that extra 25% of your labor force, especially if you anticipate a decline in revenue from real estate agents coming soon)
The issue is that the housing market is just unsuitable for this strategy. Houses aren’t fungible, and they are very slow to trade. So Zillow ended up in a position where rather than clipping the ticket on spread, they were actually quite exposed to house price movements.
This kind of thing is difficult to confirm from the outside, of course. But that they adjusted the model to pay more towards the end is pretty widely known.
Vast swaths of rural Midwest and northeast with little industry and declining population definitely did not make out, especially factoring in property taxes and the opportunity cost of not investing in VOO as a near risk free alternative.
Zillow realized the only time their ask was hit is when it was at a premium to the actual market price. If they used competitive offers, they’d never have the winning bid. In a hot market where you’re offering a premium, you’re going to have owners of lower quality properties accepting your offer, while owners of higher quality properties have more offers to select from.
Zillow got left holding a bag of lemons and decided to get out before buying the whole lemon grove.
Why do you assume that, seems like a cash buyout would be a great deal for many sellers if it was at the appropriate price. Issue is I think that Zillow's information was less granular than what the buyers/sellers had. Let's say Zillow priced two houses near each other at 1million each. However one was close to a busy road so would only sell for $900k while the other could sell for $1.1. Zillow made the right average offer of $1million to both but the buyers/sellers actually had more information. So the 1.1m seller didn't take Zillow's offer while the 900k seller did. Now Zillow was out $100k essentially not counting fees.
The problem seems more that they were not getting “enough” houses doing it this way, especially competing against Opendoor, and so they had to bid higher and on more properties in order to hit “scale”. And that lack of selectivity is what led to the bad basket of houses they now own.
Planning to lose money takes nerve. Zillow tried to avoid avoid the pain, and ended up abandoning what might be a profitable enterprise (for someone else) in the future.
The most valuable data is not social data, ... but your own data because every dataset that you’re looking at internally describes your own process, including your bugs, ... building models from your own data is the only way to build a really successful system.
Facebook and Google's data are not their own. That data is comprised of private lives, stripped bare pixel by pixel, bit by bit, and it's offensive to frame it as if they're doing something alchemical and special with it. Google's search dominance came from something special, creating the right algorithm and seizing the first mover advantage, but the relentless and ruthless invasion of privacy is a rent seeking race to the bottom.
All of the ills of the internet and political turmoil in the west from algorithmic amplification are the brainchilren of Facebook and Google. It turns out that "tailoring search results" and "targeted advertisement" are excuses for something that can cost far more than a society might want to pay.
Most data Facebook collects is of the form (user saw this post, user clicked/did not click this post). That data's value is tightly coupled to the process Facebook used to decide whether or not to cause the user to see that post. The data only has value in the context of iterating on that process.
You’re also absolutely right that the social media content: the photos, the sentiments, the likes, the connections, should not in any way “belong” to FB/G.
The data that does belong to them, and that is useless to anyone else, are the outputs from their sentiment analyzer service, the weights and trigger conditions for their content ranking algorithms, the intermediate outputs of their ML evaluations, etc.
GP, and the article, are saying: look there first. Try to start by truly understanding “what you already know, but aren’t paying enough attention to,” and don’t just treat the problem as “needs more data.”
> rent seeking
> first mover advantage
This reads like an HN buzzword bingo card.
But seriously, a lot of your claims are flimsy or misinformed, which goes to credibility. Cambridge Analytica was a huge nothingburger that had no actual effect on US elections or Brexit. Google did not have first mover advantage, they were so late to the search engine game that it caused them trouble in their early financing. Show us a real, known harm from Clearview. Google and Facebook are not breaking any laws, so how can they be "invading privacy"? The bottom line is, people love FAANG tech, and are happy to trade their data to use it. And one of the reasons is because they are not experiencing real harm, in spite of what HN's white knights would have us believe.
Some domains are intricately mapped in available data (e.g. equity pricing), but most, and especially most physical, are not (e.g. freight transportation).
This is similar to 'adverse selection' in real life & in Zillow's model. The article makes a nod to this, but seems to imply that if you train your model on that adverse selection, you can come out ahead after paying to learn about it.
To me that kind of misses the point. Adverse Selection isn't a static feature of the landscape you can identify and avoid, it is people understanding what you understand, adapting, and responding. Train your model with adversaries trying to beat it, then you'll maybe counter the specific first round strategies they use, and they'll learn new ones and beat your new model with their 2nd round strategies. It's a continuous game. Your requirement to gather a corpus of training data will keep you in the 2nd turn of a game where the wins are biased to whoever has the 1st move.
Sure they failed. But the only data we have is that they failed because of something very specific which doesn’t relate to much else.
This is what most profit seeking strategies can miss. Their designers (consciously or not) can't help but to stop thinking through their plan at the profit step and just assume "rinse and repeat" forever after.
Many seasoned wall street algorithms have suffered many times over 5 decades, and when they fail we call them black swan events.
The author is arguing that they should have pivoted from “we already have models” to “we’re intentionally gambling hundreds of millions of dollars so we can build good models over the next few years”. That might be a good strategy for a startup with loads of VC money and no other products, but it makes less sense for a more established company to risk going under on that bet
He uses Zillow to explain how datasets – especially the ones with money tied-in – can’t be trusted blindly. Building a high-quality dataset is an expensive endeavour.
Good tweetstorms with technical explanations on how that happened:
https://twitter.com/macrocephalopod/status/14558873523715973...
Ultimately they were really bad as flippers. More often than not paying more than market price for the homes they bought.
I think the root problem is that this was a panic move. They saw Open Door's success and thought they had no choice but to try and replicate it. But its a questionable business move for Zillow and ultimately they couldn't make it work
Opendoor's primary benefit is to enable people to move when they otherwise could not easily do so, creating more liquidity and matching supply and demand (often number of bedrooms in house to number of bedrooms now needed).
The challenge with moving is that most people need to sell their current house before they can afford (or even know what they can afford) to buy their next home. Opendoor lets a family buy that next home with its cash, then list their current home on the market or sell it to the company so they avoid the double mortgage or double move (home->rental->home)
In contrast, “Zillow Seeks to Sell 7,000 Homes for $2.8 Billion” so Zillow lost more than a few percentage points.
One example: The MLS in Austin, TX recently banned publicly sharing a home's sold price. https://www.zillow.com/austin-tx-78701/sold/
It's a hybrid model trading in an adversarial, real-dollar environment. The leverage comes from having a small human team trade big volume, much more than they could possibly trade directly, by augmenting their human abilities with automation and a model. Or seen from the other side, it's a model with human oversight.
Any system like that is high risk, high reward. All the successful ones started out by losing a lot of money. Paypal lost an incredible amount to fraud before they started breaking even. OpenDoor lost an incredible amount to mispricing, and took on a ton of balance sheet risk, before their business really started working.
"To live, you must be willing to die"
- poker legend Amir Vahedi
A big part of Opendoor is creating the right apps and processes to collect this information to feed their models. The machine learning part is important, but can give the false impression it's just about data scientists crunching numbers at head office, when in reality there's a huge real-world operational machine that's driving it.
I see this as a victory for us calling out companies for immoral behavior.
It's telling that the article opens with a clip of Alec Baldwin talking about needing "brass balls" from Glengarry Glen Ross, seemingly oblivious to the fact that it's a dark comedy mocking cutthroat sales culture.
This article does not mention that. Instead, the rest of the article deals with Linkedin-wisdom and hard platitudes, such that it is not possible to build a good model on someone else's data (as if Zillow even was).
Data scientists remarking on the Zillow fold, are like psychiatrists or engineers remarking on non-clients and bridges build by others. They know nothing about the business, about the constraints, about how the estimates are consumed. They end up silly, but without good information coming from Zillow, we assign value to their analysis, purely on Twitter-soundbite-ability and internet-authority.
During the time of the pandemic, the house prices rose and so did the volume. If anything, they made out like a bandit.
No pricing model will ever get this right.
What does this mean? 50% of the money is held as debt? Or 50% of the money is lost to fraud?
i think zillow also did not understand what fungibility means. real estate is not fungible. in fact, it's anti-fungible- that's why there are huge diligence processes that exist around most real estate transactions. maybe floors in an office building may be fungible, but residences are definitely not- with all their quirks, customizations and problems.
this whole argument that they failed because the ceo of zillow didn't have big balls is pretty putrid. pair this with the word salad of misused words and twisting of quotes and i'd say this is probably one of the worst pieces i've ever seen posted here. i feel worse for having given it any time at all.
the simple fact is that the ceo of zillow didn't know what they were doing, had a team that (supposedly) applied facebook's infrastructure scaling prediction library to house prices and then attempted to apply a market making mindset to the real estate market at scale. not only is this probably something nobody should try to do, considering we contribute so much in tax dollars to first time homebuying incentives as it's recognized that the housing market is where j q public can start to build wealth, it's also probably something that nobody could do (well, at scale, with machines) given that housing is not fungible.
sure, financial instruments are fungible, maybe even late model cars, but definitely not houses. doesn't take a scientist to spot that.
In particular when you’re auto-underwriting credit it’s not typically an origination-for-sale model. So the value of the loan is the present value of the future payments, less the future value of defaults, less the cost of acquiring the customer.
Historically those things can be modeled pretty accurately and the aspects that can’t be modeled accurately can often be hedged or eliminated by the law of large numbers. The innovation of the new ML underwriting with respect to accuracy is at the margins. The real disruption is the speed and cost. (Disclosure: I worked at a SMB fin tech and we reran multiple credit models for a million customers and past customers every night.)
If Zillow were getting into the rental business, in some ways it might have been easier for them. But they needed to model where they could sell an illiquid asset which is a much harder and much less well understood problem. And yes with enough capital to plow through and the appropriate risk attitude they could likely have gotten the handle on what their pipeline was really going to look like. But it’s hardly the same problem as credit underwriting.
False. You can definitely bootstrap and adjust the model as you either gather more data yourself or get more outside data. You can also build confidence intervals around the model predictions and decide how you want to proceed based on that. There is lots you can do with that initial model.
> A machine learning organization thinks of risk entirely differently than an automated risk underwriting organization.
It's possible and maybe even advisable to use machine learning in the automated risk underwriting business, but it is a different setup / set of objectives.
As the author notes, IMO the adversarial and antifraud aspect of risk underwriting turns it less into a straight-up estimation problem and much more into a game theory type of problem. ML models can assist in evaluating risk, but you do indeed have to be preocuppied by your risk as a party to the transaction in the first place, and not just trying to predict prices as a third party observer (which by itself is pretty riskless).
And if you have billions of dollars in cheap capital, everything looks like an investment problem.
Which is ultimately the suggestion of this article: "Why aren't you more like Wall Street?"
The implications are exactly the opposite of Zillow being an innovative company. If they require billions of dollars in deep pockets (nbd) and a restructuring of their org to be more like old-school operators, all signs point to existing players as more fundamentally correct about the strategy required to succeed in the space.
If Zillow thought they had all the data they needed, there would have been little harm starting with $100 million in properties -- if the loss there ended up being $5 million, they would have known immediately something was up and that they had work to do.
This is ridiculous, we need much better regulation on this stuff.
I wonder if higher property taxes would help a bit? If you own a 'home' then you're going to be paying for the water, school, electricity infrastructure whether you use electricity, water, or not.
Of course, that would be gamed hard and would have to be strongly regulated as well.
But that, and vacant property taxes, limits on some other things, and some other adjustments might help.
You're not going to be able to reliably model asset prices at the resolution and accuracy needed to front run the market for a long period of time. This case was worse because the "Zestimate" directly created a feedback loop that moved the underlying asset prices higher.
Most AIs today are for augmentation, not replacement. Vehicle autopilots are a perfect example. The ones that are commercially available aren't capable of replacing the human, they just augment the human's abilities.
Always has been that way. Always will be that way. AI is great for when you need to tame a firehose and make millisecond decisions. But there's a 90 year old in Omaha who is better than the best AI.
How ?
(A) Both Wall Street and Machine Learning Modelers struggle with tail risk. Hedge funds measure performance against
https://en.wikipedia.org/wiki/Sharpe_ratio
which assumes risk is (i) normally distributed and (ii) a source of reward. For most people, however, risk looks like Theranos or the Fukushima accident or the Challenger distaster.
It's unbelievable that a machine learning model trained to predict house prices based on experience would be accurate in the face of events like the COVID-19 pandemic or what will happen when the Fed raises interest rates. You can model risks like that, but to the extent that you're working from experience you are working from a database from the 1929 Crash, South Sea Bubble, etc.
(B) Mark Levine wrote a good article about how you'd exploit such a predictive model. If you consistently gave people low offers, a few people would accept them. You would get a high rate of return but could invest little capital.
To invest more capital you have to make more offers that get accepted, that is, give better prices. Your rate of return goes down and if there is shrinkage from errors, accidents, etc. you could get a negative return.
It's that "tendency towards a declining rate of profit" that Marx warned about.
(C) The analogy with stock market market makers doesn't sound good when you consider the differing timescales.
Market makers are isolated from some risk because of the length of their holdings. Yet, they make profits by exploiting the stochastics of a stationary market (e.g. if you don't like the price at time t1, you will usually get a better price at t2) but they lose money when markets move definitively in one direction or another.
That kind of trader heads for the bathroom when things go South and in the interest of being orderly markets impose sanctions on market makers who do the natural thing and press the "STOP & UNWIND ALL POSITIONS" button when it gets tough.
In the case of Zillow I see holding times that go on for weeks or months and all kinds of real world risk like planning to do certain renovations but having to delay the work because out of 20 things you need from Home Depot they only have 16 of them.