It depends, but overall it might be risky if you don't pay the necessary attention.

In my case - I've kind of outsourced it, but every user submission is being reviewed before it's published.

Yes, and yes.

Care to elaborate? How much risk is there to the site owner which accidentally hosts the content?

You'll want to make sure all models are of legal age, and that the person submitting it has the rights to upload the material. Yes there's DMCA clauses to live behind that help make such tasks easier but in adult DMCA is so widely abused that it's best to make sure your stuff is 100% legal and legit to stay out of trouble. Not worth the returns in my opinion.

Source: I have just spent the last 12 years in adult and recently just severed all ties with the imploding industry.