Having Open Source Project on GitHub does not mean you need to become less agile. First there will not be instantly 1000 of people sending you pull requests, second even if they do - you have no obligation to take them in.

However if you say, we can write crap code... and no one would know because they only see binaries, this is not really a way gain a good confidence

In my opinion those days just having a "binary you can use" is the worst way to go for something like database/data store. If you are not sure about your Open Source plans having something like SaaS solution with free tier so users can experiment easily could be way to go