As somebody who is a landlord, albeit far from the bay area, I've never seen any tenant that expected the security deposit back. Usually, they break the lease before the lease terms are up, and want to have the security deposit applied as last month's rent. I've had nondestructive tenants and am a generally over-nice guy, so I usually go with it...
I rented a house for a week in Michigan this summer --- the whole rent was paid up front --- and the landlord illegally retained a substantial portion of the security deposit. They have the money. Why would they ever give it back if they didn't have to?
I'm sure there are scummy ones that do that. The other shoe is that landlords have to deal with the shifty end of people a lot of the time. When I bought my current rental, the previous owner went to great lengths to educate me on what could be done with non-paying tenants and eviction procedures; it seemed that knowledge was hard earned.
