Lots of teams have different expectations about how granular a user story should be, and some of this has to do with the project. How fast can you design, develop, test and deploy a meaningful amount of new content?
One explanation for the longer time is that we had a process where for each user story we'd write a short, informal design document and send it to the team and stakeholders for review. For non-trivial stories we'd then meet to discuss them and come to a consensus about how to implement it. This probably added about a day or two to a story's duration, but it meant that we had a solid system at all time. It also often served a similar purpose to a retrospective, or produce topics for a retrospective, because these meetings would surface any technical debt and other impediments to progress. It also served as knowledge transfer and helped the team converge on design principles and expectations.
So these meetings had a cost, but we felt it was essential for practicing agile design. We didn't find this made us slow to react. On the contrary, because this kept our technical debt low, and our software well designed and well understood by the whole team, it meant we could pivot on a dime.
I think there's a real risk in going too fast. Maximum speed should not be the goal of a software development process, and I don't think any business really wants that. The two primary goals should be: 1) to make predictable, steady progress over long periods of time, and 2) the ability to change priorities as quickly as possible as new information emerges. If you have to sacrifice some speed to get there, it may be worthwhile.