That is true! The biggest downside with channel factories is that noncompliance from one party forces the whole factory to close and reopen without them, which gets much more likely with every added participant. But if there is only one participant which is "untrustworthy" (on some level, in the sense that they aren't trusted to stay online as much as well known nodes are), then the system could work fine.
That's really interesting, and opens up a lot of ideas. Still, I think we should avoid trying to bring up channel factories for now. They are still a few steps away as they add more complexity to an already complex system. Focus on getting LN up and working, then add in nice-to-haves like channel factories slowly once vendors are on-board.