From what I've gathered, getting access to a single user-picked file via SAF isn't too painful, but for browsing a whole directory tree it's more annoying having to use a completely new API for that, doubly so when you're using a library that expects regular files, plus performance is somewhat lacking, too (judging from complaints and bug reports I've seen, both in that it is easy to run into performance pitfalls, but also that even best-case performance is still worse than classic file access).
[1] I think something like that was actually implemented for Android 11 onwards, judging from https://source.android.com/docs/core/storage/scoped#using-sc...? Although possibly still at some noticeable performance cost.