Wikipedia gives 22 deaths per million flights for 2002 to 2011, while england and wales have about 5 traffic deaths per day and about 15 million people commuting daily by car.
The car numbers are very rough, but flying will stay at least one order of magnitude more dangerous than driving by this measure, which makes sense: large heights are dangerous.
Also, "if you flew everywhere you would go by car", you wouldn't feasibly be flying in airliners for 5-mile trips, but in helicopters or GA craft, which are statistically way more dangerous.
The car numbers are very rough, but flying will stay at least one order of magnitude more dangerous than driving by this measure, which makes sense: large heights are dangerous.