Like, yes, we allow calculators for highschoolers. But we'd never give a calculator to a 3rd grader learning their times tables. Because then they'd never learn that 9 * 9 is 49, and now they're permanently handicapped in math. And now, when they are in high school, the calculator doesn't even matter, because they can't do math at a basic level.
Or, consider reading. Certainly high schoolers can use audio books. But imagine if we didn't teach reading at all, and just used audio books, from kindergarten. Would anyone know how to read? I doubt it.
Learning is unique in that it builds on lower-level stuff. If you just skip that stuff, sure, it might look efficient - but it's not. You actually learn much less if you do that. It's not like me at my job, where if I use AI I finish a project in half the time and I'm better off. No, if I learn in half the time, I've learned less!