You have to have at least two independent compiler stack development processes occurring in separate light-cones. If one happens far enough within the cone of the other you cannot trust it.
(Not actually speed-of-light-cone of course. You have the lead-time required to develop general "nanites" and then their travel time to reach the opposite side of the Earth (assuming no one is working on this off-planet. The first thing a paranoid nanotech-haver would does is detect and suborn all other nanotech labs. I call this the "Matter-Lock".)
That goes a bit beyond the "trusting trust" scenario, though, and into subsuming "hardware" rather than "compiler software", which DDC admits it doesn't control for...
I don't have enough backing physics to be confident on the possibility of a "matter lock". How do you detect without being detected a nanomachine that was designed vs one of the nanomachines that already exist in living organisms? Can you also expand on the goal of subsuming all the nanotech of other labs? If things obviously start breaking, that's especially going away from the "trusted trust" scenario which implies things like trojans used to passively sniff secrets and gain advantage through information when an opportunity to export the information arises. I would expect physics might allow some workarounds for that, on top of methods of outright detection like conservation laws, spectrometry...
I've finished the first of three in a sci-fi book series that has introduced the problem of what to do with an adversary that subsumes basic physics research to halt general advances, perhaps there's an entertaining hard sci-fi book you'd recommend for the "matter lock" idea? Or arxiv papers if you have any.
The problem with nanotech is that unlike software it can transmit itself. That's what make the "trusting trust" problem so severe in this context.
On the scale of hypothetical nanites the world is really really huge, so the first hurdle is figuring out how to integrate the incoming information and control the machines.
> How do you detect without being detected a nanomachine that was designed vs one of the nanomachines that already exist in living organisms?
If there are already other machines to detect then you're too late and the scenario is "toner war" as per "Diamond Age" (probably the best nanotech sci-fi novel; or maybe "Blood Music" by Greg Bear.)
If you do get there first (and you've correctly identified this as a huge existential challenge: how can you know that you're not being fed false information by the person who got there before you? You can't. If "matter lock" is possible there's no way to know if you're really first, except to try some shit and see if anyone notices and can stop you) then you have the relatively easy task of locating the other nanotech labs in the world and infecting them with your malware.
> Can you also expand on the goal of subsuming all the nanotech of other labs?
Well, if you're reading "The Three Body Problem" then that's one way. Eventually some people would start to get wise. But nanotech: you detect them and alter their brains to forget. There's always another way to contain the information if you get there early enough.
It would be easy to infect the other labs because you would be infecting every lab everywhere already.
And of course, you can always just declare yourself. Wear a purple silk cape and call yourself the Robot King. Who's going to stop you?
Anyhow, if you wanted to keep your "matter lock" a secret you would have to minimize your interventions, restrict yourself to subtle sabotage, and program every instrument to ignore the fact that every computer and robot in the entire world had a massive Trojan in it. More than that, to actively lie about it and alert you if anyone starts doing weird experiments.
Even then I suspect things would come to a head somehow and... and then I don't know what would happen.
> perhaps there's an entertaining hard sci-fi book you'd recommend for the "matter lock" idea?
Nah. There is one novel about a megalomaniacal mad scientist who achieves "matter lock" and immediately begins editing the world as he pleases. It's grotesque. FWIW it's called "The Goliath Stone" by Matthew Joseph Harrington with some sort of involvement of Larry Niven (who is otherwise one of my favorite authors, but this book is a stinker.) Just one example: the mad scientist is violently opposed to rape (okay) but he makes womens' breasts larger without asking them.
I do recommend these if you haven't read them already:
I've read the Diamond Age, and the first book in the Three Body Problem series. I don't think the tech will play out like in Diamond Age, and the book I mentioned was indeed 3BP but since I haven't finished the other two books my final thoughts have to wait. (Only thing I didn't like so far was the sudden FTL comms at the end...)
I think your best metaphor is either cracking root access to the Matrix or simply becoming God. Very far removed from the "trusting trust" scenario. But also removed from physical systems. Using that sort of metaphor instead of "matter lock" will insulate any criticisms from hard science. It also reduces the existential concerns to the same level as the question of "what if we're living in a simulation?"
People do publish technical details on both MNT and non-MNT... To use an older reference I would bet that if you ran your idea by someone who has read Drexler's Nanosystems they could point something out at some layer that forbids your idea in principle at least insofar as current understanding of physics, chemistry, and biology go. If we (or some other species) can create machines that can move along a spatial dimension outside our normal 4D space-time but project itself back inside at will, sure, that's one way we're screwed, but that AFAIK has no real basis yet, it's the same concern as if we (or some other species) can root the Matrix...
> I don't think the tech will play out like in Diamond Age
Well (SPOILER ALERT!!!) the whole point of Diamond Age was that nanotech could play out in one of two ways: metered by a central authority to extract rents vs. imitating natural self-replicating systems. The deeper issue being control vs. wilderness.
That's really a psychological issue, and one we are already facing today: witness how the idea of building self-replicating 3D printers ("RepRap") to alter economic conditions became subsumed by companies trying to sell 3D printers to consumers. Most printers cost between $300 to $3000, when I should be able to go down to Noisebridge and print my own for $10.50. People have to make a living; Noisebridge is soliciting donations because their lease is up and they have to move. Can I really fault the folks trying to make a living selling printers?
Bucky Fuller pointed out that we would have the technology to take care of ourselves by sometime in the 1970's, no nanotech required, if we would just apply our resources and existing technology to our problems in an efficient manner.
> the book I mentioned was indeed 3BP but since I haven't finished the other two books my final thoughts have to wait. (Only thing I didn't like so far was the sudden FTL comms at the end...)
I've only read the first two, has the third been released in paperback yet? As for the FTL comms, I think it's really hard to make a hard-sci-fi story that's realistic and emotionally engaging over lots of light-years.
> I think your best metaphor is either cracking root access to the Matrix or simply becoming God. Very far removed from the "trusting trust" scenario. But also removed from physical systems. Using that sort of metaphor instead of "matter lock" will insulate any criticisms from hard science. It also reduces the existential concerns to the same level as the question of "what if we're living in a simulation?"
I don't think there's any hard science consideration preventing the development the machinery for "matter lock" (I'm getting tired of my own jargon at this point, lol.) At the most general level of analysis you have a decay rate and a regeneration rate and as long as the latter is sufficiently greater than the former you're golden. Keep in mind, you would control all atomic energy on the planet in this scenario.
I think it's physically, mechanically possible to suffuse the planetary envelope (the bubble-shaped space between the hard vacuum and the magma) with a communicating network of machines that could sense and affect conditions globally. (After all, life did it.)
The problem I foresee is command and control: could you coordinate it? How does one person (or group) receive, process, and transmit information to and from this system? Here we are pressed up against the so-called Hard Problem of Consciousness, which of course is directly related existential question you mention! That's the weird thing about self-reflexive consciousness: it's still a problem whether your system is "hard science" or "metaphorical" or "I'm dreaming" or whatever.
> People do publish technical details on both MNT and non-MNT...
I didn't mean that they don't, I meant that the (theoretical) people researching how to use nanotech to become Robot King don't publish.
Attaining the "ML" would be akin to becoming a local god, but how would you have to transform yourself to manage it? I believe that is the barrier, if any.
In any event, after reading "A Planet of Viruses" by Carl Zimmer [1] I'm pretty sure that they already have things locked down. It's a non-fiction pop-sci covering recent discoveries in biology of viruses, only 109 pages and nearly every one mind-blowing.
Read that, then "Blood Music", then Gregory Bateson's "Mind and Nature: A Necessary Unity (Advances in Systems Theory, Complexity, and the Human Sciences)" I think the Matrix is rooted... ;-)
You have to have at least two independent compiler stack development processes occurring in separate light-cones. If one happens far enough within the cone of the other you cannot trust it.
(Not actually speed-of-light-cone of course. You have the lead-time required to develop general "nanites" and then their travel time to reach the opposite side of the Earth (assuming no one is working on this off-planet. The first thing a paranoid nanotech-haver would does is detect and suborn all other nanotech labs. I call this the "Matter-Lock".)