I generally agree with your line of thought (CS is too accustomed to toying around with imaginary models/rules and not real-world physical ones), but I don't get the connection between mathematically sound models and manufacturing problems. Are even Genetic Algorithms (a bunch of metaheuristics) really mathematically that robust? I thought GAs were mainly discovered and developed purely empirically, and there's not that much theoretical guarantees behind it even as of today. The reason it's being used in the context of engineering is that it seems to solve non-convex or combinatorial problems really well compared to previous approaches, and that's it. And if GANs or Deep RL can explore solution spaces more effectively than GA algorithms, then why not?
Many engineering problems can be reduced to optimization problems, and if a certain toolset can be used to effectively solve these optimization problems, then it should be applied pragmatically. You could still question if ML brings anything new to the table ("isn't it just curve fitting in higher dimensions?"), but I think there's some value in using data-driven ML toolsets judiciously. For example, a lot of engineering tasks are repetitive but vary in very slight deviations (like screwing things to holes), and for these cases statistical models might help in bringing efficiency and safety. Of course the system still cannot handle real catastrophic deviations from the norm (ex. small earthquake happens and the robot is shaking) - but at that point human intervention should definitely happen. A truly intelligent AI would handle even the exceptional cases with some degree of pragmatic decision-making, but that's not the goal we're chasing for here.
I still agree with the sentiment that there needs more work to make ML methods safe though. This is in two grounds: one is to make it easier for the programmers to add domain-specific hard constraints to the model they're learning (enforce the output of the model always adhere to certain rules), and the other is to make the ML system report to the human operators for manual intervention when any abnormalities to the input and the output are detected. It's a bit disappointing that much of "safe" ML has focused on ethical issues mainly arisen from Silicon Valley (like gender/race issues in models: duh, your model will probably have the same bias as your data! The conception of using ML for solving social problems is fucked up in the first place...). The real focus should be the ability to steer, control, and constrain ML models as human operators would like, and I think the neglecting of this is the reason why ML has largely failed to be incorporated in industrial domains outside of SV as of today.
It would be dumb to say that it's not worth it to pursue ML research and apply it to engineering problems to improve well-established solutions.
What bothers me the most however and is the reason I was tongue-in-chick in my first comment was the insane overrepresentation machine learning receives in virtually every interaction related to tech, considering its success to non-physical problems. I get that hypes come and go and will continue so, but it's another thing having a trend spreading out in almost every scientific discipline advertising it as a discovery by big brain computer people that have come to salvage their poor inferiors (that its ardent proponents have zero experience in engineering design is another story). You may think this is hyperbole but it's really a common sentiment among ML skeptics. And it's especially frustrating when it's presented as a subtitute, and not as a supplement, to well established mathematical ways, e.g. machine learning vs control theory in AVs as you've said.
On the other hand, although SV startup culture is definitely to blame for all this to an extent, I can definitely understand some subfields being way too conservative to trendier topics just for the sake of not blending in with the hype. There have been some decent attempts lately to bridge the numerous chasms each discipline has and get the best out of each world. Hopefully something useful comes out of it.
Many engineering problems can be reduced to optimization problems, and if a certain toolset can be used to effectively solve these optimization problems, then it should be applied pragmatically. You could still question if ML brings anything new to the table ("isn't it just curve fitting in higher dimensions?"), but I think there's some value in using data-driven ML toolsets judiciously. For example, a lot of engineering tasks are repetitive but vary in very slight deviations (like screwing things to holes), and for these cases statistical models might help in bringing efficiency and safety. Of course the system still cannot handle real catastrophic deviations from the norm (ex. small earthquake happens and the robot is shaking) - but at that point human intervention should definitely happen. A truly intelligent AI would handle even the exceptional cases with some degree of pragmatic decision-making, but that's not the goal we're chasing for here.
I still agree with the sentiment that there needs more work to make ML methods safe though. This is in two grounds: one is to make it easier for the programmers to add domain-specific hard constraints to the model they're learning (enforce the output of the model always adhere to certain rules), and the other is to make the ML system report to the human operators for manual intervention when any abnormalities to the input and the output are detected. It's a bit disappointing that much of "safe" ML has focused on ethical issues mainly arisen from Silicon Valley (like gender/race issues in models: duh, your model will probably have the same bias as your data! The conception of using ML for solving social problems is fucked up in the first place...). The real focus should be the ability to steer, control, and constrain ML models as human operators would like, and I think the neglecting of this is the reason why ML has largely failed to be incorporated in industrial domains outside of SV as of today.