Maybe government should invest in companies instead of giving grants. So if the company fails, it is money lost like a grant, but if there is success, then the government can get its money back.

Then you would have state capitalism, and the government might be biased towards state owned enterprises, ruining the free market.

Good news, there's no free market to be ruined. Free markets are a fiction. Governments already influence the market in far, far more extreme ways than this.

So long as any profits from these investments are not ploughed back into general revenues, your concerns are moot. For example, you could establish an independent body—let's call it INVEST1—to oversee these investments. INVEST1 would be required to divest ownership of successful enterprises at a threshold that ensures money spent roughly matches money earned (and thus self-funding). Once it reaches a stable equilibrium, you spin off INVEST1 as a fully independent not-for-profit. Government then establishes INVEST2 and the process starts again from the beginning. Rinse and repeat.

Loans that only come due upon breakout success, more precisely.