A lot of exploit mitigations, especially spectre/meltdown mitigations, allocator hardening, auditd, toolchain hardening flags that introduce runtime checks, syscall filtering, etc. have introduced major slowdowns. I'm sure this isn't the only reason, but it's the only reason I am familiar with. Ask someone familiar with a different sector of osdev and they'd probably rattle off a few more.

I have noticed that lots of tasks that would be fast a decade or two ago are slow today on a rbpi: tasks like switching workspaces, switching browser tabs, etc. have noticeable delays. Sway makes switching workspaces and some window management functionality nearly instant in comparison. I've compared it with i3 and openbox; others have compared it with dwm. I'll see if I can find a link.

A lot of tasks become faster with GPU acceleration, which is why OpenGL and Vulkan accel are helpful.