Very interesting. In the US, you may need to register for a EIN (Employer Identification Number) for most of our equivalent (in cases where there is an equivalent) to those things. You can do so online quick and easy, and as a sole proprietor. https://www.irs.gov/businesses/small-businesses-self-employe...
Depending on what you are doing, you may also need a similiar tax id from your state (and possibly any other states you are doing business in, if you have a physical presence there). Most states have on online process similar to what the federal government has.
Then, you may need to register with your city and get a city business license- some cities don't require this for all types of businesses, but this can be one of the most expensive and difficult parts of the process, as you usually have to go to city hall and pay money for it.
But all of the above can be done as a sole proprietor. It is also fairly easy in many states to get an LLC setup online for not too much money if you want to go that route... though this depends wildly by state, I have heard California is a (comparative) nightmare to do this in!
The big diff apparently is that everyone in the US is sole proprietor by default, while in France you need to ask an agreement from the government.