Violence has no religious morality baggage - religions were extremely violent. Sex, however, has all the baggage from all the angles. Even if we aren't religious, it doesn't matter, religion still dictates huge chunks of our lives and our mindsets. It has thousands of years of inertia.
Buddhism literally teaches celibacy. Mostly for monks but if you get serious about your practice the advice is always to become celibate in order to make progress. It makes sense as you're trying to remove your desires.
