What you're describing is basically a modern FPGA[1]. You can wire it up as you want at runtime, and they can contain specialized hardware like hardware multipliers and fast local memory to accelerate certain workloads.

[1]: https://en.wikipedia.org/wiki/Field-programmable_gate_array