AR simply has no market. Apple could sell Vision Pro for $10,000 apiece,
if there was natural demand for high-quality AR hardware. But the product isn't competing for Hololens' commercial contracts, and it forfeit the consumer VR segment on release. The remainder of consumer-forward AR experiences are even less lucrative than Zuckerberg's commoditized Quests. Apple wants to build a nonexistent market with an inaccessible product.
Until that demand actually materializes, Apple's ability to miniaturize the tech is inconsequential. Vision Pro's "iPad for your face" philosophy is not enough to carve out a niche, and definitely not enough to displace the iPhone or the Mac.