No, it's not. Well, if you are doing software related stuff. (Almost any other occupation is highly regulated in Germany (be it electrician, painting service or tax consultant, you need specific degrees to be allowed to open a specific business)).
To do (software) business those are relevant options (there are more, but those are more exotic or for larger businesses or startups):
a) "Freiberuf" (often translated as freelance, but it is actually something different). This is originally intended for architects and similar professions. It is hard to get, and even harder if you don't have some kind of engineering degree.
b) "Gewerbe". You just need to register, pay a few minor fees, and you can do business. However, you are personally liable for the business you do (so get proper insurances).
c) GmbH or UG. An LLC. This _used to be_ expensive (GmbH requires at least 12.5k of base capital to be put into the company). This establishes an own legal entity and the liability is restricted to the money you put into it. "UG" is a "new" (>10y old now!) version of a GmbH which only requires 1 Euro to be put into the company (should be more like 2k) ...
Costs
a) and b) are essentially "free". You don't necessarily need a tax consultant and all the filings are not that hard anymore (all online AFAIK, though I do let my tax consultant do that). a) is a little better because the taxes are setup a little different.
c) Setting up a UG is not _expensive_, but not exactly free either :-) Plan ~1k for setup, and then ~1.5-2k yearly for letting the tax consultant do the necessary paperwork. It's very easy too, you just setup the company at a notary, and they will essentially do everything for you.
Notably there are also lots of funding and startup programs in Germany, many of which are actually quite good.
Probably the most important thing is to find a reasonable tax consultant. It's not rocket science, but it's worth the money.
Disclaimer: I have a) for ~15 years, and I've setup two c)'s and an AG (a stock based company).
Yes, corporate tax rate in Bulgaria is attractive. However, how does this help if op wants to stay in Germany. Immediately cross-border tax issues are introduced you have to deal with. Only the tax accountants and lawyer are winning in this case with their billable hour. Also do not discount the language barrier too much.
The problem is that if you have a single client, the government may claim that you are trying to avoid tax and other obligations --- that you have a fake company.
Couldn't I just pay all of that stuff by my own to show them I have no intention to do so? I want everything to be legal. I do not want to save any money in taxes, health insurance or social insurance. I am actually happy to pay all this. Nevertheless I heard horror stories even by doing so you are at risk they will f..k you anyway after a Prüfung (audit) within the first five years or so.
Wouldn't you actually be paying more taxes as a one person company? Why does it matter if the company has only one client and one employee if everything is paid properly?
Because as managing director you don’t have to pay pension insurance (and others). So if you are employed, the social system will get more contributions.