"No one outside Apple can audit their products" is obviously an overstatement. Obviously people can take Apple's products and test them, subject them to fuzzing and reverse-engineering. But no one outside of Apple has access to their design documents and source code, and that limits the extent and effectiveness of those efforts. I probably should have hedged with, "No one outside of Apple can FULLY audit their products" or something like that.
Also, the focus on iOS is a distraction. Apple makes their own silicon. Vulnerabilities are probably more likely to be hiding there than in the software. Modern silicon is very hard to reverse-engineer.
None of your arguments in this thread cohere. You wonder upthread why I'm not volunteering more of my own perspective on this. It's because I don't want to arm you with more weird tangents to pursue. Let's try to break this down:
1. We're talking about SS7 security.
2. Whether or not SS7 is secure, if your phone is compromised and hostile, it's game-over. So it's hard to see how this debate is even relevant.
3. You introduced the argument that Apple and Google phones were untrustworthy because nobody outside Apple (or Google) can audit them. You now say that's an overstatement.
4. When informed that there's in fact a whole cottage industry of people outside Apple who do audit iOS from binaries --- effectively enough, I'll add now, that they routinely find vulnerabilities that Apple missed despite Apple's privileged access to "design documents and source code" --- you express incredulity.
5. You denominate your incredulity in a "bytes per second" rate of reverse engineering based on compiled image sizes.
6. Later, you derive a rate from the number of characters you can fit on a VT100 screen.
7. You later clarify: we have to audit everything, you see: not just the kernel and privileged services but also every pixel in the Apple logo that appears when you boot the phone, and, of course, the security of the weather app.
8. Now we can't trust silicon either. Vulnerabilities are (???) more likely to be hiding there than in software.
This is a kaleidoscope of weird, wrong arguments about security, and, once again, has nothing to do with the thread. If you simultaneously believe all these things, it still doesn't make sense to try to protect phone calls by securing the SS7 network.
What I think you should do is take a simple binary from your desktop OS, download a copy of radare, learn how to use it, and blow your own mind about how even the free, open source reversing tooling works. I'm not being snarky. I think you'll be surprised by how this stuff works.
First of all, thanks for taking the time to write this detailed comment. I really do appreciate it.
Second, let's try to achieve clarity on what it is we're actually disagreeing about here, because I'm not sure we even agree about that.
> no mass-market computing platform has been safer than the flagship hardware/software platforms from Apple and Google
I actually agree with that (though I wonder if leaving Microsoft off this list is actually justified, but that is neither here nor there). What I don't agree with is the implication that Apple and Google flagships are plenty good enough, and that buying phones from Apple or Google is the right answer to all our security concerns.
An analogy: before Fukushima it could be said of boiling water reactors that "no reactor design has been safer". Indeed, even after Fukushima the safety record of BWRs compares very favorably overall (in terms of casualty rates) with almost all other forms of energy. That doesn't mean that we can't do substantially better, or that we shouldn't try. But humans have always had trouble with low-probability high-impact events.
> 1. We're talking about SS7 security.
Well, sort of. The original topic was SS7 security (or the lack thereof). This is a somewhat unfortunate distraction because it led down a rabbit hole: yes, SS7 is insecure. But that doesn't matter much because most sensitive data that is transmitted by a cell phone nowadays is locally encrypted. But it matters some because most != all.
> 2. Whether or not SS7 is secure, if your phone is compromised and hostile, it's game-over. So it's hard to see how this debate is even relevant.
Well, SS7 is a data point. It is an existence proof that modern cell phone contain security flaws because they are the product of a long design process that has a lot of legacy from a time when security was less of a concern. If one such flaw exists, others might as well.
> 3. You introduced the argument that Apple and Google phones were untrustworthy because nobody outside Apple (or Google) can audit them. You now say that's an overstatement.
Yes, I chose my words poorly. A apologize for that.
> 4. When informed that there's in fact a whole cottage industry of people outside Apple who do audit iOS from binaries --- you express incredulity.
No, that's not fair. I am well aware that this industry exists. But I am skeptical that the existence of this industry is sufficient reason to accept the proposition that anyone who owns an Apple or Google flagship need not be further concerned about security, and I think my skepticism can be justified. Even if I'm wrong, I think my position is defensible. And I may well be wrong. That would actually be a good outcome.
> 5. You denominate your incredulity in a "bytes per second" rate of reverse engineering based on compiled image sizes.
I have a lot of reasons to justify my skepticism. I could write a paper (and maybe I should). But HN comments do not lend themselves well to long-form communications so I advanced what I thought would be a compact argument: a quick back-of-the-envelope calculation of the amount of effort required to audit iOS (which is just one component of an iPhone) would reveal that to be infeasible. Now, that calculation may have been way off. That entire argument may have been wrong. But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem (specifically, you would have to solve the lambda-equivalence problem, which reduces to the halting problem). So I'm pretty sure I'm on solid ground with at least some level of skepticism about the effectiveness of reverse engineering.
> 6... 7... 8...
Yes, because the position that I'm arguing for is that there are a lot of potential problems that the reverse-engineering industry is not well equipped to address no matter how effective its tools are. And furthermore, even if I'm wrong about that, that is still not enough to justify the conclusion that owners of Apple and Google flagships need not be further concerned, because it is not just the abilities of the reverse-engineering industry that matters, but also their motives. And I see a lot of grounds for skepticism about that.
> it still doesn't make sense to try to protect phone calls by securing the SS7 network.
Yes, we agree about this too.
> radare
I was not aware of radare, so thanks for that pointer. It does appear to be a very impressive and comprehensive collection of tools. But one thing that it doesn't have (AFAICT) is advanced AI that automates the process of extracting algorithms and intent from object code. So at the end of the day, you still have humans searching for needles in a 1.6GB haystack.
No, I don't think you have a handle on what reverse engineers are actually looking at when they look at binaries (semi-spoiler: they're not necessarily even looking at assembly instructions), and I don't think explaining it in a comment is going to be nearly as useful as urging you to try it for yourself on a toy problem would be.
Another thing to acquaint yourself with --- orthogonal to the Radare pointer --- is the concept of a "lifter". Or, in another direction, with symbolic execution. Or, still another, with modern decompilation tools.
Instead of addressing the substance of my argument you are instead attacking my familiarity with the particulars of the the tools and techniques used to do contemporary reverse engineering. This is a logical fallacy known as the Courtier's Reply:
Hang on, the substance of your argument is a sort of mysticism - first you claim that an iOS image is a black obelisk fathomless to man and when challenged and essentially forced to concede this is completely inaccurate you fall back on saying your position is generally right and what's more, both right and unfalsifiable because of Archangels Turing & Church.
Neither of these are serious, engaged-with-the-actual-topic sorts of arguments. It's a little rich to be coming back with a grumpy note on the taxonomy of logical fallacies.
"A straw man is a common form of argument and is an informal fallacy based on giving the impression of refuting an opponent's argument, while actually refuting an argument that was not presented by that opponent."
Yes. Really. (Note that there is no mention of obelisks.)
To be perfectly clear (since misunderstandings seem to be running rampant in this thread) I did not literally pull that number out of my hat. That was a figure of speech, meaning that I picked a number that seems not-entirely-unreasonable but which I didn't really give a whole lot of thought to because the exact value is not essential to my argument. No matter what wizardry your tools provide, it seems exceedingly unlikely that you're going to burn through the code at, say, 100 bytes per second without running the risk of missing something. So the question of what you consider a reasonable estimate was deadly serious. If you think my estimate is wrong, then tell me what you think the right answer is and why. The fact that I didn't put a lot of effort into my analysis is not in and of itself a valid criticism. Sometimes people get the right answer for the wrong reasons. (In fact, I am in the middle of preparing a series of lectures on the history of science, and it turns out that it is usually the case that the first time someone gets the right answer that they get it for the wrong reasons!)
No matter what wizardry your tools provide, it seems exceedingly unlikely that you're going to burn through the code at, say, 100 bytes per second without running the risk of missing something.
The fact that you are focused on insisting every byte of the code indicates that you are not yet familiar with the process of how this works.
By way of background, I have done security source code audits of systems on the order of 750,000 lines of code. This was done in a 12 week effort. The approach taken with source code review is possibly different than you think. One part of the approach is to look for patterns of code that are known vulnerability patterns, such as sql injection, or opening a socket. You then trace back to code paths that lead to that to determine how external input (that is user-controlled or attacker-controlled) can be use to trigger those vulnerable pieces of code. Another part of the approach is to look at each of the inputs (or interfaces) to the code to determine how those inputs can influence behavior of the program. One is likely to switch back and forth between these instances.
One key approach in looking at the code is to ask "can user input cause the program to choose one path of a branch or another." Another key approach is to ask "can the user's actions cause a change in one word of the programs memory." From that, an exploit can be crafted.
So you might well now ask "ok so that is source code. Object code is orders of magnitude more difficult." This is not really the case. The tools that 'tptacek mentioned take apparently impenetrable object code and transform it to assembly language (as well as to an intermediate language ESIL), and answer many questions about the static and dynamic nature of the code under inspection. Also, you can get differences of the call graphs from one version to the next. This trick was used to detect a vulnerability resolved by a Windows patch in a common library. It was noted through this tool that there was another use of this library elsewhere in the system that left the vulnerability in. This is without having source.
In fact, for the most serious level of analysis, one should go directly for the binary, as who knows how the source code actually corresponds with what binary actually gets shipped.
And it turns out one can effectively audit code for a language that one is not an expert in. The key elements are "where are the branches" and "what are the call graphs" and "what are the inputs and outputs".
In another thread, you note that you are an expert and that you are involved in the production of a security product. I am as well, having been in the software development business for 52 years, the last 10 in the security field, focusing on software security. And I can testify that these are two different fields of expertise. An expert in software development, even of security products, does not automatically mean that one is an expert in finding security flaws in code.
I've trained many software engineers in software security, and a key part of that training is to note that software engineering builds up programs and solutions by using previously developed abstractions, and making new abstractions that use existing ones. A penetration tester will develop skills in penetrating abstractions. It is a different way of thinking, a different kind of expertise. It is clear from your work that you are excellent at building up abstractions.
There are a couple of ideas that you are missing, I think. One is that evaluating the "rate of burn" through the bytes of a binary blob is a useful way to determine the difficulty of assessing the security. (Nor is it a useful way to evaluate software productivity) It is not necessary to look at every byte. Think of looking at every basic block. I suspect you will get a number that is different by two orders of magnitude than what you are currently thinking.
> you are not yet familiar with the process of how this works
That is true, I'm not, certainly not with the details. I hire people to do audits and pen-testing for me. But in order to be able to distinguish people who are competent from snake-oil salesman, I have to have a pretty good grip on some fundamentals, and I believe I do. So even though I have not used a modern decompiler, I certainly know that they exist, I know the fundamentals of how they work, and I know their limitations. I know, for example, that information is lost in the process of going from source code to object code to decompiled code, so the process of auditing decompiled code is necessarily strictly more difficult than auditing source code. I also know that the relative sizes of the source, object, and decompiled code is more or less linear. (A few things deviate from this, like C++ templates and unrolled loops, but it's true to first order.) So using a linear approximation to get an estimate of the amount of work required to audit a binary blob is not entirely unreasonable, particularly since the only question I'm trying to answer is whether or not it is even plausible that iOS could be effectively audited by an unauthorized third party given its size. (Note that even if the answer to that turns out to be "yes", that still doesn't demonstrate that iOS actually is being effectively audited.)
So:
> It is not necessary to look at every byte.
I never said it was. Nonetheless, unless some auditing technique is effective enough to change the (first-order) linear relationship between source code, object code, and decompiled code, or effective enough to introduce a radical linear multiplier (i.e. eliminate 90% of the code from consideration) the fact that not every byte needs to be examined is irrelevant. My first-order estimate will still be good enough to provide a plausibly correct answer to the question I'm asking.
> In fact, for the most serious level of analysis, one should go directly for the binary, as who knows how the source code actually corresponds with what binary actually gets shipped.
Yes, exactly right. Nonetheless, this process is easier if you have the (purported) corresponding source. More information is always better.
There is one more important point that you seem to have missed (along with everyone else): there is a big difference between trying to find a vulnerability that is the result of an inadvertent bug, and one that has been introduced deliberately by a competent adversary. Finding the latter is vastly more difficult than the former. Decompilers operate on heuristics. Heuristics can be fooled. One of the things that a competent adversary would do if they were trying to hide a back door would be to put it in a form that causes it to be hidden by decompiler heuristics. So this entire discussion about reverse engineering techniques is actually totally moot. Writing a decompiler that did not have this problem would require solving the halting problem, so even though I don't know the details, I know they have this shortcoming. Not only that, but I know how I could identify the specifics of this shortcoming, and I know how I could write code to take advantage of this shortcoming to conceal a back door. And if I know these things, then there are certainly people at Apple who know these things.
This is the reason that I am quite confident in my position despite my ignorance of the details of modern reverse engineering techniques. The halting problem gives the advantage to the attacker, and no technological advance is going to change that. It's like a second law of thermodynamics for security. I don't need to know the details of contemporary technology to know that, all else being equal, the person trying to hide a back door in a 1.6GB binary is very likely to beat the person trying to find it. And the problem is not just technological. As I've pointed out elsewhere in this thread, even if someone does find it, that knowledge will be extremely valuable. It is far from clear that anyone who succeeds in finding a back door in iOS would use that knowledge for the public good, particularly when you consider the personality types and economic circumstances that lead people to pursue a career in reverse-engineering in the first place.
This situation is ridiculously complex, and that complexity extends far beyond the size of the iOS binary. Anyone who is quibbling over my 1-byte-per-second estimate, or my personal proficiency with radare, is missing the point rather badly.
Your first-order estimate is not in fact good enough to provide a plausibly correct answer; you're not even in the ballpark of correct. The challenges facing reversers are in fact degrees different than those facing Apple itself, with its access to original source code; they are not entirely different kinds of challenges. And in neither case can the challenge be measured in "compiled image bytes reviewed per second".
Your arguments here pile non sequitur atop non sequitur. I'm left with the impression that this is a topic in which you've decided you're unwilling to concede anything. No doubt, if I challenged you about how difficult it is to reverse hardware, you'd pull some other weird rabbit out of your hat, like quantum states or interactions between circuits and cosmic rays.
Of course, if you'd led off your argument with "cosmic rays make all of information security in some ways unknowable", we'd all have simply said "sure, but that's besides the point". But that's not your argument. Your argument is that telcos should provide cryptographic security for telephone users, because Apple has insurmountable advantages against its users due to its access to its own source code. No, that doesn't make any sense.
I'm not taking it personally or anything. You've just gone on tilt. It happens to all of us. For what it's worth: I think you still share a Slack with several of us? You could ask this question there, and I'd be more comfortable responding in detail there.
I've already conceded that I am ignorant of many of the details of modern reverse-engineering techniques, so that is manifestly untrue.
> you're not even in the ballpark of correct
You keep saying that, and yet you don't back this up with any details or supporting arguments. In what way am I incorrect? Is my estimate too high? Too low? By how much? Am I wrong when I claim that a lower bound on the computational complexity of auditing is O(n)? If so, what is the correct result? Is it O(sqrt(n))? O(log(n))? O(1)?
BTW, I would actually love to be convinced that I'm wrong about this. That would be a huge two-fold win for me. It would mean that 1) I can stop worrying about security (as long as I use an iPhone) and 2) I would learn something new and almost certainly very interesting. But you (or someone) have to tell me how and why I am wrong, not just that I am wrong.
It would also help if you would stop stop advancing logical fallacies like this straw man:
> cosmic rays make all of information security in some ways unknowable
I don't understand why you're more comfortable discussing this on Slack, but OK, I've fired up my Slack client.
I don't want to pretend that there's a meaningful relationship between image size and reversing challenge, but to the extent there is, it's something more like O(log n).
I did not literally pull that number out of my hat. That was a figure of speech
Fair enough.
No matter what wizardry your tools provide, it seems exceedingly unlikely
Wait no, this is wrong. Experts are telling you this is wrong. Non-experts (say, me) looking at the same stuff can easily see that it's wrong.
If you think my estimate is wrong, then tell me what you think the right answer is and why
I think I saw some tptacek comment in another branch of this thread about 'how many bushels does it take to get to space'. I think he's right that your arguments are in that exact realm of underinformed inarguability. I don't have to prove the halting problem to reasonably say that an airliner, with all its software, is a more reliable and safer form of transportation than a snowboard. Yet you bring this up as some sort of meaningful retort. The onus is not on me to show reasonableness.
So, first of all, I am an expert. I am not on the level of tptacek, but I did not just fall off the turnip truck either. I have a Ph.D. in computer science plus ~30 years of industrial and research experience. I've been studying security and cryptography for ~20 years. I run a company that sells a security product (https://sc4.us/hsm) for which I wrote much of the code myself. One of the reasons I'm skeptical about the security of iOS is because I know how easy it would be for me to conceal a back door in my own product, and my product has orders of magnitude less code and correspondingly fewer places for vulnerabilities to hide (which is supposed to be one of its main selling points, BTW).
Second, it's simply not true that experts are telling me that the "this" you are referring to is wrong. The comment to which you are responding is the first time in this thread that I have raised the amended claim that the rate of decompilation is <100 BPS rather than ~1 BPS. You are the only one who has responded to that so far.
Third, while it's true that tptacek has been telling me that I am wrong, he has been very vague about what exactly it is that I am wrong about. He has not directly challenged any of my claims, not even (if you read carefully) my conclusion. Neither he nor anyone else has actually made a made a statement of the form, "You're wrong, a reasonable estimate of decompilation rate is X", or, "You're wrong, a complete audit of iOS can be done in X work-days." All of the responses have been of the form, "You're wrong because you're an idiot," or, "You're wrong because you don't know the ins and outs of contemporary reverse-engineering techniques." But that, as I have noted before, is the courtier's-reply fallacy. Even if I am ignorant it does not follow that I am wrong. People get things right for the wrong reasons surprisingly often.
So, first of all, I am an expert. [...] You're wrong because you're an idiot
You're right, this line of argument about expertise was generally unproductive and I shouldn't have gone down it. I apologize for starting it. It's dumb and it's dumb no matter how much of an expert you are or are not.
I stand by 'unserious', though. 'image size' and 'halting problem' are two deeply unserious retorts to the points others were making in this thread.
I would accept your apology, except that you immediately go on to commit the exact same sin using different terminology. "You are deeply unserious" is no different than "you are an idiot." Neither one tells me anything other than that you disagree with me, and that you think that it is so self-evident that I am wrong, or that I am so mentally deficient, that it is not worth your time to actually advance an argument. Seriously, how exactly did you expect me to respond to that?
BTW, you can only "stand by" something that you have previously said, and AFAICT you just now introduced the word "unserious" into this exchange. That, to me, is an indication that you are not taking this very seriously.
"first you claim that an iOS image is a black obelisk fathomless to man and when challenged and essentially forced to concede this is completely inaccurate you fall back on saying your position is generally right and what's more, both right and unfalsifiable because of Archangels Turing & Church. Neither of these are serious..."
You're right, neither of those are serious. But neither of those are my argument.
Ok, let's wrap this up since I don't believe you're actually interested in making an argument but in writhing out of whatever corner you've decided to plant your flag in. This is a thing you said:
"But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem"
This is not a serious argument. You can say it about anything and declare up-front that no matter what, you're going to be 'right'. It would be mildly amusing in a dorm room chat around a bong. It's an embarrassment to say it in full seriousness and then indignantly point at your CS PhD. Now you won't even admit you even made it. The (now-distant) sibling comment is right - even in the unlikely event we moved past this, your next Atlantic Wall would be Penrose quantum consciousness.
Well, yes, the halting problem does have some rather far-reaching consequences. (There's a reason Turing is famous.) But there is an important sense in which the halting problem is relevant here: security is adversarial, and so the limits on computability matter more than they do in a non-adversarial context. In a non-adversarial context, the odds that my system will behave according to some simplified model are higher than in an adversarial context, and that often makes reliable analysis feasible, the halting problem notwithstanding.
BTW, I am not waving my Ph.D. around to convince you that I'm right. I'm waving it around to try to persuade you that if you disagree with me, it might be because I have a point that you are failing to grasp, and it might be worth your while trying to understand what that point is even -- perhaps especially -- if it turns out to be wrong. But simply proclaiming my argument to be "non-serious" is not a productive first step.
Of course I believe it's true. I also believe it's true that there is a non-zero probability that if my phone were stolen, the air molecules around the fingerprint sensor could spontaneously come to an electromechanical arrangement that unlocks it, giving the thief full access to my bank account and terrible taste in music.
I (and I imagine you) don't claim I was late to an appointment because of heretofore unforeseen difficulties in hidden-variable theory.
Now you are the one raising non-serious arguments. The probability of quantum fluctuations unlocking your phone is indistinguishable from zero.
On the other hand, the probability of a skilled adversary introducing a back door that evades known methods of detection is very much distinguishable from zero. Heck, the probability of a bug introducing a back door that evades known methods of detection (at least for a while) is distinguishable from zero! That's why jailbreaks are possible.
No. You explicitly said (I quoted you verbatim about it more than once) that any argument or assessment of the security of a phone can be dismissed by virtue of the halting problem. If you'd like to retract that, fair enough. If not, you are staking out what I believe is a silly and unserious position. Is this a position you hold or not?
> You explicitly said (I quoted you verbatim about it more than once) that any argument or assessment of the security of a phone can be dismissed by virtue of the halting problem.
No, I did not say that. You keep putting words in my mouth, starting with "an iOS image is a black obelisk fathomless to man". And yes, you did quote me, but you took the quote out of context and then twisted its meaning.
Here is what I said:
> I have a lot of reasons to justify my skepticism. I could write a paper (and maybe I should). But HN comments do not lend themselves well to long-form communications so I advanced what I thought would be a compact argument: a quick back-of-the-envelope calculation of the amount of effort required to audit iOS (which is just one component of an iPhone) would reveal that to be infeasible. Now, that calculation may have been way off. That entire argument may have been wrong. But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem (specifically, you would have to solve the lambda-equivalence problem, which reduces to the halting problem). So I'm pretty sure I'm on solid ground with at least some level of skepticism about the effectiveness of reverse engineering.
That does not mean the same thing as "any argument or assessment of the security of a phone can be dismissed by virtue of the halting problem." If you don't understand that, I would be happy to try to explain the difference to you. But you have reached the limits of my patience with your straw-man arguments. Do it one more time and we're done. Life it too short.
But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem
I mean, it's in your own damn text. What on earth does this mean that is different from what I think it means. I'm perfectly happy to hear your explanation.
Also, the focus on iOS is a distraction. Apple makes their own silicon. Vulnerabilities are probably more likely to be hiding there than in the software. Modern silicon is very hard to reverse-engineer.