RWKV and Mamba families LLM-s have recurrent blocks instead of attention, and they get really close to performance of Transformer on small scale. What you've probably tried was some outdated tech. The biggest advantage of Transformer is that they are easy to scale, and the scale itself brings some quality to the table.
For example, some time ago Mamba-3B matched performance of Pythia-7B:
https://www.reddit.com/r/singularity/comments/18asto2/announ...
The main drawback of the legacy models was that they were hard to scale, both due to slow training (sequential processing) and poor stability with depth increase (i.e. poor gradient flow). Modern implementation no longer have these problems and able to scale decently.
