You don't need to escape all your ampersands and quotes. An ampersand that doesn't form an html entity will render as is, quotes are perfectly fine unless used in an HTML attribute. And in an html attribute, less thans are fine. The body of a script tag has different parsing rules.
You don’t need to escape them, but it certainly doesn’t hurt. And script tags are generally not a problem when building something from a template, but they are a reason why XML is more “pure” than HTML.