Because the wording of such restrictions are often just window-dressing on the true motive, which is racial segregation.
The first zoning law in the country was in Los Angeles in 1908. Part of the law was crafted to force certain businesses to relocate retroactively. It was crafted specifically to target the Chinese businesses, which were the laundromats. White people just did not want to live alongside Chinese people.
Redlining was a further practice of both bankers and zoning having restrictions which were explicitly racially motivated. Levittown, the first planned community in the United States, was very explicitly "not mixed" - it had racial covenants that restricted blacks or jews from moving in. The intent was to sell more homes, as they presumed white people would not want to live in "mixed" areas with neighbors of an undesireable race. That practice continued throughout planned communities and zoning laws all over the country. Many of those racial covenants are still in place today.
As an aside: you can let your children go outside in most cities. The moral panic over children getting kidnapped or abused if they step foot outside the house has been overblown for decades, and at this point is just a bizarre psychological trope of America.
