Once those basics are setup it's a matter of attracting users. First and foremost your software needs to solve a problem that people are encountering. Once there's a reason to use the project, then there need to be incentives to assist in the development. Tell users clearly how they can help and provide some issues in the style of first-timers-only issues. There's going to be a good number of drive by contributions, but if they're handled correctly some of them can turn into repeat contributions. In that vein, issues/PRs/etc need to have prompt responses to keep people interested, they need to point out resources, and otherwise convince the community members they're being heard. Overall growing and cultivating a community is a ton of work.
For context I've been maintaining a fairly popular open source synthesizer project for over a decade. Feel free to ask more specific questions and I should be able to point out some ideas.
I do not and will not use Discord, and do not like Discourse. For fast communication, I would use IRC (although do not currently have one set up); if someone else wants to provide a bridge of the IRC to Discord I would accept that though. For slow communication, I would use NNTP (which I already have set up), but I would accept contributions to provide a web interface and mail interface to the same messages (the messages are stored in a SQLite database, and I can provide a link to the database schema and documentation for those who are interested).
In the case of my projects, I do not allow others write access to my repositories. However, if someone writes a bug report or other message with a patch included, then I can review it and then possibly apply it.
I agree with your second paragraph ("Once those basics are setup it's a matter of attracting users ... Overall growing and cultivating a community is a ton of work") and unfortunately, I am not really sure what to do. Hopefully, if someone on here looks and is interested in these projects (whether mine or someone else), then they can go on there. You can also post here, if wanted.
I also agree that documentation is important (comment 26106008). (I often see projects without good documentation.)
I will allow others to propose a diff, and others can also fork the repository if they want to; nothing is stopping that. (Although, those who wish to fork it must store their own copy (or use chiselapp); I will not host it for them.) If other people want to use GitHub, that is also OK.
Threaded discussions are possible with NNTP. Making a threaded conversation about each line of code (or a group of contiguous lines of code) is also possible; if the message body is formatted in a certain way to find the code in there, then a header could be added to reference the individual line of code (or range) being commented on. (The relevant line(s) of code should also be quoted in the follow-up message.)
However, code patches are not the only kinds of contributions that can be made; there are also documentation, bug reports, feature requests, FAQ, artwork, discussion about how something should be implemented, etc.
It's 100% fine to develop things which are openly available, but not try to build a community. That seems to match your description with your current setup.
Documentation is always important, and what I am working on for my own project exclusively right now.
