Pollution has a cost and production has a benefit. If $BENEFIT - $COST > 0, we want more pollution.
The problem is that since pollution is an external cost, the producer/consumer has no incentive to accurately make this calculation. As long as $BENEFIT > 0, the producer/consumer will pollute, and society will pay $COST.
The point of the Pigouvian tax is to give the producer/consumer an incentive to make this calculation correctly, and to compensate the rest of society for the costs they impose.
Actually, the end goal would be to reduce the total amount of pollution.