Where do you get that idea from? or maybe, how did you come to that conclusion? It's not evident that would be the case. Traffic mirrors demand. Demand goes both ways, unevenly. Prior to the end-of-the-work-day, you would see an increase in empty vehicles consolidating for the imminent exodus of a city, but that would hardly be the common case.
> that supposes that most people will give up the idea of personal car ownership entirely instead of just buying their own AVs
There are multiple ways to ensure this occurs. The same reason horse carriages aren't allowed on Freeways (legal/safety barriers), or almost nobody uses gliders or natural gas cars to commute (impractical), etc. US Society is subject to change/more malleable than you might imagine. Maybe not in our lifetime, but ubiquitous phone booths disappeared in mine, so I hold out hope.