The goal here is not tax evasion but reducing some of the bureaucracy of founding and running the company.
Assuming the company isn't operating at a loss it is producing company income. Companies have to pay taxes on their profits too! AFAIK that income (excluding loopholes) needs to be taxed where it is generated. So if that company made 100k Euro using software you developed in Berlin, you'll pay the company taxes for 100k revenue in Germany. Not 100% sure, but that is my understanding of the situation.
Also note: You need to be quite careful, tax law is complicated in Germany. E.g. if you work for the company but you are not paying yourself a salary, but just profits, this can in itself be considered tax evasion! (since in Germany the corporate tax can be lower than your personal income tax)
But again, ask a tax lawyer or call your tax authority if you want dependable information on this.
P.S.: I don't believe bureaucracy is an issue here, you quite likely increase it by setting up a multinational.