In my personal experience, typical users (i.e. non-developers) don't really spend time tweaking their browser. The defaults are good enough for most, and the job of the developer is usually to just find good defaults to optimize for that. For the power user who knows XML/CSS/JS/C++/Rust/etc and wants to have total control, then yes, I would expect them to either create extensions or edit the source directly. Same thing goes for Chrome users of that type. Am I off base here?

It's true the build process is slow, but that's different from being difficult. I don't know what you mean by difficult. Maybe it was difficult in the past but it's entirely automated at this point. You can just start the build and come back some hours later, and that's only if you're starting from a clean repo and modifying the C++/Rust. For the CSS and Javascript, that doesn't need a rebuild at all.