When I hear 'poisoning' I think of an internet vandal trying to destroy things for fun.
But a more likely motivation is someone who doesn't like what an AI says about them (think: powerful individual, company, or government) manipulating training data to their advantage.
There are companies who will try to scrub unfavourable web search results for you (for a price). Perhaps this is the next iteration of that.
I'm gonna write (generate I guess using GPT..) a thousand articles about how matsemann is the greatest developer ever. And in 5 years when a recruiter uses recruitGPT to ask "should we hire matsemann" it will give a stellar recommendation.
I also invite you to contemplate the potential for brand marketing campaigns if your "poison" is intended to boost the prominence of a well-known product, so that ChatGPT relentlessly pushes it in the face of users. (Brand marketing isn't just about advertising, it's about familiarity.)
There are also darker uses for political propaganda, such as asserting as a positive some contested territorial/ethnic/religious claim, eg. Nation A's claim over the territory of Nation B, or reinforcing Group C's belittling ethnic stereotype of Group D.
I imagine, some sort of fractal pocket dimension, capable to hypnotize a Neural Network.
Like a sort of sphinx of all answers, a TRIZ https://en.wikipedia.org/wiki/TRIZ like system claiming to have content for all the problems, occuring again and again, because its optimized to cuddle to the reward function..
This is fascinating concept.
I'm fairly convinced this is the next iteration of that. In the last 6 years I feel like there has been a massive focus on 'content moderation' which was just an attempt to direct and bias information that was being used to power a lot of the models we see today.
But a more likely motivation is someone who doesn't like what an AI says about them (think: powerful individual, company, or government) manipulating training data to their advantage.
There are companies who will try to scrub unfavourable web search results for you (for a price). Perhaps this is the next iteration of that.