Would there be some way to “launder” the model to make it plausibly viable for commercial use? Train a new model with the weights of this model with some kind of noise added to make it hard to tell what it is based on?
Distillation would be the ideal way (especially because it also has efficiency gains), but as far as I know distillation for LLMs is kinda unproven.
Honestly though, even if you just finetune it, which you will want anyway for any serious commercial application, it's essentially impossible to determine the origin.
Randomly perturbing the weights and then finetuning would probably make it impossible. If someone had access to the finetune dataset and you didn’t add noise, they could see if the finetuning curves intersect.
I guess in practice, it’ll look suspicious if you have an identical model architecture and have similar performance.
