> Detect if someone is using a covert breakthrough AI system in the world.
I bet the real purpose of this is not
"It seems important to detect this", at least against superhuman AGI.
As that's asking you to win a game against a more intelligent game player, who is going to be able to out-think you and hide.
It seems more likely that:
The real purpose of this problem is to impose a 'secrecy tax' on entities who would like to recoup investment by covertly using such a system, thus reducing the incentive to build one in secret. I.e. if you know someone has invested in research that's going to limit your ability to profit from covertly building an AGI, this means you can only use it up to the limit of game theoretic detectability, reducing the profit motive of acting covertly in the first place.
Its a bit like once you break Enigma, you have to be careful how you use it. Having alternative explanations for how you got your edge then becomes valuable.
So, if you see a corporation building something that could provide an alternative explanations for huge returns, other than AGI - ideally for something that's very noisy (high variance) - and where it'll be hard to tell whether their success was by chance or on purpose, that's probably (very weak) evidence (in the bayesian sense) they are working on a secret AGI.
I suggest YCombinator is an ideal candidate... huge returns, very hard to tell if its luck or skill.
I bet the real purpose of this is not "It seems important to detect this", at least against superhuman AGI.
As that's asking you to win a game against a more intelligent game player, who is going to be able to out-think you and hide.
It seems more likely that:
The real purpose of this problem is to impose a 'secrecy tax' on entities who would like to recoup investment by covertly using such a system, thus reducing the incentive to build one in secret. I.e. if you know someone has invested in research that's going to limit your ability to profit from covertly building an AGI, this means you can only use it up to the limit of game theoretic detectability, reducing the profit motive of acting covertly in the first place.
Its a bit like once you break Enigma, you have to be careful how you use it. Having alternative explanations for how you got your edge then becomes valuable.
So, if you see a corporation building something that could provide an alternative explanations for huge returns, other than AGI - ideally for something that's very noisy (high variance) - and where it'll be hard to tell whether their success was by chance or on purpose, that's probably (very weak) evidence (in the bayesian sense) they are working on a secret AGI.
I suggest YCombinator is an ideal candidate... huge returns, very hard to tell if its luck or skill.