You can achieve this with LMStudio's UI to test it today. You can switch between different local models in the same chat context. You can also edit previous chat results to remove context-poisoning information.

LMStudio has an API, so it should be possible to hook into that with relatively little code.