Can LLMs be reliably restricted to produce any specific subset of content? AFAICT they're still consistently jailbroken.

It’s easy to do at the syntactic level by controlling the sampling. For example, it’s easy and common to restrict output to be valid JSON but just not allowing any tokens that would make it not valid JSON.

But reliably restricting output at the semantic level is very much an open problem.

How are you going to ensure that it is impossible for the student to work around whatever measures you take?