It is not surprising to me that there is no text support yet, given the issues of integrating 3D (batch x sequence x features) RNN support (for CUDNN). This issue has prevailed so long that I came to believe it is impossible to integrate with the current Flux.jl stack.
Transformers.jl and TextAnalysis.jl already provide quite a bit of functionality for NLP, though to my knowledge neither makes use of RNNs. You may be interested in commenting on https://github.com/FluxML/Flux.jl/issues/1678.
