Neither of these are serious, engaged-with-the-actual-topic sorts of arguments. It's a little rich to be coming back with a grumpy note on the taxonomy of logical fallacies.

https://en.wikipedia.org/wiki/Straw_man

"A straw man is a common form of argument and is an informal fallacy based on giving the impression of refuting an opponent's argument, while actually refuting an argument that was not presented by that opponent."

I mean, really?

To be perfectly clear (since misunderstandings seem to be running rampant in this thread) I did not literally pull that number out of my hat. That was a figure of speech, meaning that I picked a number that seems not-entirely-unreasonable but which I didn't really give a whole lot of thought to because the exact value is not essential to my argument. No matter what wizardry your tools provide, it seems exceedingly unlikely that you're going to burn through the code at, say, 100 bytes per second without running the risk of missing something. So the question of what you consider a reasonable estimate was deadly serious. If you think my estimate is wrong, then tell me what you think the right answer is and why. The fact that I didn't put a lot of effort into my analysis is not in and of itself a valid criticism. Sometimes people get the right answer for the wrong reasons. (In fact, I am in the middle of preparing a series of lectures on the history of science, and it turns out that it is usually the case that the first time someone gets the right answer that they get it for the wrong reasons!)

The fact that you are focused on insisting every byte of the code indicates that you are not yet familiar with the process of how this works.

By way of background, I have done security source code audits of systems on the order of 750,000 lines of code. This was done in a 12 week effort. The approach taken with source code review is possibly different than you think. One part of the approach is to look for patterns of code that are known vulnerability patterns, such as sql injection, or opening a socket. You then trace back to code paths that lead to that to determine how external input (that is user-controlled or attacker-controlled) can be use to trigger those vulnerable pieces of code. Another part of the approach is to look at each of the inputs (or interfaces) to the code to determine how those inputs can influence behavior of the program. One is likely to switch back and forth between these instances.

One key approach in looking at the code is to ask "can user input cause the program to choose one path of a branch or another." Another key approach is to ask "can the user's actions cause a change in one word of the programs memory." From that, an exploit can be crafted.

So you might well now ask "ok so that is source code. Object code is orders of magnitude more difficult." This is not really the case. The tools that 'tptacek mentioned take apparently impenetrable object code and transform it to assembly language (as well as to an intermediate language ESIL), and answer many questions about the static and dynamic nature of the code under inspection. Also, you can get differences of the call graphs from one version to the next. This trick was used to detect a vulnerability resolved by a Windows patch in a common library. It was noted through this tool that there was another use of this library elsewhere in the system that left the vulnerability in. This is without having source.

In fact, for the most serious level of analysis, one should go directly for the binary, as who knows how the source code actually corresponds with what binary actually gets shipped.

And it turns out one can effectively audit code for a language that one is not an expert in. The key elements are "where are the branches" and "what are the call graphs" and "what are the inputs and outputs".

In another thread, you note that you are an expert and that you are involved in the production of a security product. I am as well, having been in the software development business for 52 years, the last 10 in the security field, focusing on software security. And I can testify that these are two different fields of expertise. An expert in software development, even of security products, does not automatically mean that one is an expert in finding security flaws in code.

I've trained many software engineers in software security, and a key part of that training is to note that software engineering builds up programs and solutions by using previously developed abstractions, and making new abstractions that use existing ones. A penetration tester will develop skills in penetrating abstractions. It is a different way of thinking, a different kind of expertise. It is clear from your work that you are excellent at building up abstractions.

There are a couple of ideas that you are missing, I think. One is that evaluating the "rate of burn" through the bytes of a binary blob is a useful way to determine the difficulty of assessing the security. (Nor is it a useful way to evaluate software productivity) It is not necessary to look at every byte. Think of looking at every basic block. I suspect you will get a number that is different by two orders of magnitude than what you are currently thinking.

That is true, I'm not, certainly not with the details. I hire people to do audits and pen-testing for me. But in order to be able to distinguish people who are competent from snake-oil salesman, I have to have a pretty good grip on some fundamentals, and I believe I do. So even though I have not used a modern decompiler, I certainly know that they exist, I know the fundamentals of how they work, and I know their limitations. I know, for example, that information is lost in the process of going from source code to object code to decompiled code, so the process of auditing decompiled code is necessarily strictly more difficult than auditing source code. I also know that the relative sizes of the source, object, and decompiled code is more or less linear. (A few things deviate from this, like C++ templates and unrolled loops, but it's true to first order.) So using a linear approximation to get an estimate of the amount of work required to audit a binary blob is not entirely unreasonable, particularly since the only question I'm trying to answer is whether or not it is even plausible that iOS could be effectively audited by an unauthorized third party given its size. (Note that even if the answer to that turns out to be "yes", that still doesn't demonstrate that iOS actually is being effectively audited.)

So:

> It is not necessary to look at every byte.

I never said it was. Nonetheless, unless some auditing technique is effective enough to change the (first-order) linear relationship between source code, object code, and decompiled code, or effective enough to introduce a radical linear multiplier (i.e. eliminate 90% of the code from consideration) the fact that not every byte needs to be examined is irrelevant. My first-order estimate will still be good enough to provide a plausibly correct answer to the question I'm asking.

> In fact, for the most serious level of analysis, one should go directly for the binary, as who knows how the source code actually corresponds with what binary actually gets shipped.

Yes, exactly right. Nonetheless, this process is easier if you have the (purported) corresponding source. More information is always better.

There is one more important point that you seem to have missed (along with everyone else): there is a big difference between trying to find a vulnerability that is the result of an inadvertent bug, and one that has been introduced deliberately by a competent adversary. Finding the latter is vastly more difficult than the former. Decompilers operate on heuristics. Heuristics can be fooled. One of the things that a competent adversary would do if they were trying to hide a back door would be to put it in a form that causes it to be hidden by decompiler heuristics. So this entire discussion about reverse engineering techniques is actually totally moot. Writing a decompiler that did not have this problem would require solving the halting problem, so even though I don't know the details, I know they have this shortcoming. Not only that, but I know how I could identify the specifics of this shortcoming, and I know how I could write code to take advantage of this shortcoming to conceal a back door. And if I know these things, then there are certainly people at Apple who know these things.

This is the reason that I am quite confident in my position despite my ignorance of the details of modern reverse engineering techniques. The halting problem gives the advantage to the attacker, and no technological advance is going to change that. It's like a second law of thermodynamics for security. I don't need to know the details of contemporary technology to know that, all else being equal, the person trying to hide a back door in a 1.6GB binary is very likely to beat the person trying to find it. And the problem is not just technological. As I've pointed out elsewhere in this thread, even if someone does find it, that knowledge will be extremely valuable. It is far from clear that anyone who succeeds in finding a back door in iOS would use that knowledge for the public good, particularly when you consider the personality types and economic circumstances that lead people to pursue a career in reverse-engineering in the first place.

This situation is ridiculously complex, and that complexity extends far beyond the size of the iOS binary. Anyone who is quibbling over my 1-byte-per-second estimate, or my personal proficiency with radare, is missing the point rather badly.

Your arguments here pile non sequitur atop non sequitur. I'm left with the impression that this is a topic in which you've decided you're unwilling to concede anything. No doubt, if I challenged you about how difficult it is to reverse hardware, you'd pull some other weird rabbit out of your hat, like quantum states or interactions between circuits and cosmic rays.

Of course, if you'd led off your argument with "cosmic rays make all of information security in some ways unknowable", we'd all have simply said "sure, but that's besides the point". But that's not your argument. Your argument is that telcos should provide cryptographic security for telephone users, because Apple has insurmountable advantages against its users due to its access to its own source code. No, that doesn't make any sense.

I'm not taking it personally or anything. You've just gone on tilt. It happens to all of us. For what it's worth: I think you still share a Slack with several of us? You could ask this question there, and I'd be more comfortable responding in detail there.

I've already conceded that I am ignorant of many of the details of modern reverse-engineering techniques, so that is manifestly untrue.

> you're not even in the ballpark of correct

You keep saying that, and yet you don't back this up with any details or supporting arguments. In what way am I incorrect? Is my estimate too high? Too low? By how much? Am I wrong when I claim that a lower bound on the computational complexity of auditing is O(n)? If so, what is the correct result? Is it O(sqrt(n))? O(log(n))? O(1)?

BTW, I would actually love to be convinced that I'm wrong about this. That would be a huge two-fold win for me. It would mean that 1) I can stop worrying about security (as long as I use an iPhone) and 2) I would learn something new and almost certainly very interesting. But you (or someone) have to tell me how and why I am wrong, not just that I am wrong.

It would also help if you would stop stop advancing logical fallacies like this straw man:

> cosmic rays make all of information security in some ways unknowable

I don't understand why you're more comfortable discussing this on Slack, but OK, I've fired up my Slack client.

Fair enough.

No matter what wizardry your tools provide, it seems exceedingly unlikely

Wait no, this is wrong. Experts are telling you this is wrong. Non-experts (say, me) looking at the same stuff can easily see that it's wrong.

If you think my estimate is wrong, then tell me what you think the right answer is and why

I think I saw some tptacek comment in another branch of this thread about 'how many bushels does it take to get to space'. I think he's right that your arguments are in that exact realm of underinformed inarguability. I don't have to prove the halting problem to reasonably say that an airliner, with all its software, is a more reliable and safer form of transportation than a snowboard. Yet you bring this up as some sort of meaningful retort. The onus is not on me to show reasonableness.

So, first of all, I am an expert. I am not on the level of tptacek, but I did not just fall off the turnip truck either. I have a Ph.D. in computer science plus ~30 years of industrial and research experience. I've been studying security and cryptography for ~20 years. I run a company that sells a security product (https://sc4.us/hsm) for which I wrote much of the code myself. One of the reasons I'm skeptical about the security of iOS is because I know how easy it would be for me to conceal a back door in my own product, and my product has orders of magnitude less code and correspondingly fewer places for vulnerabilities to hide (which is supposed to be one of its main selling points, BTW).

Second, it's simply not true that experts are telling me that the "this" you are referring to is wrong. The comment to which you are responding is the first time in this thread that I have raised the amended claim that the rate of decompilation is <100 BPS rather than ~1 BPS. You are the only one who has responded to that so far.

Third, while it's true that tptacek has been telling me that I am wrong, he has been very vague about what exactly it is that I am wrong about. He has not directly challenged any of my claims, not even (if you read carefully) my conclusion. Neither he nor anyone else has actually made a made a statement of the form, "You're wrong, a reasonable estimate of decompilation rate is X", or, "You're wrong, a complete audit of iOS can be done in X work-days." All of the responses have been of the form, "You're wrong because you're an idiot," or, "You're wrong because you don't know the ins and outs of contemporary reverse-engineering techniques." But that, as I have noted before, is the courtier's-reply fallacy. Even if I am ignorant it does not follow that I am wrong. People get things right for the wrong reasons surprisingly often.

You're right, this line of argument about expertise was generally unproductive and I shouldn't have gone down it. I apologize for starting it. It's dumb and it's dumb no matter how much of an expert you are or are not.

I stand by 'unserious', though. 'image size' and 'halting problem' are two deeply unserious retorts to the points others were making in this thread.

BTW, you can only "stand by" something that you have previously said, and AFAICT you just now introduced the word "unserious" into this exchange. That, to me, is an indication that you are not taking this very seriously.

"Neither of these are serious, engaged-with-the-actual-topic sorts of arguments. "

That's the bit I stand by.

"first you claim that an iOS image is a black obelisk fathomless to man and when challenged and essentially forced to concede this is completely inaccurate you fall back on saying your position is generally right and what's more, both right and unfalsifiable because of Archangels Turing & Church. Neither of these are serious..."

You're right, neither of those are serious. But neither of those are my argument.

So I guess we actually agree about that.

"But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem"

This is not a serious argument. You can say it about anything and declare up-front that no matter what, you're going to be 'right'. It would be mildly amusing in a dorm room chat around a bong. It's an embarrassment to say it in full seriousness and then indignantly point at your CS PhD. Now you won't even admit you even made it. The (now-distant) sibling comment is right - even in the unlikely event we moved past this, your next Atlantic Wall would be Penrose quantum consciousness.

Why not? Do you not believe that it is true?

> You can say it about anything

Well, yes, the halting problem does have some rather far-reaching consequences. (There's a reason Turing is famous.) But there is an important sense in which the halting problem is relevant here: security is adversarial, and so the limits on computability matter more than they do in a non-adversarial context. In a non-adversarial context, the odds that my system will behave according to some simplified model are higher than in an adversarial context, and that often makes reliable analysis feasible, the halting problem notwithstanding.

BTW, I am not waving my Ph.D. around to convince you that I'm right. I'm waving it around to try to persuade you that if you disagree with me, it might be because I have a point that you are failing to grasp, and it might be worth your while trying to understand what that point is even -- perhaps especially -- if it turns out to be wrong. But simply proclaiming my argument to be "non-serious" is not a productive first step.

Of course I believe it's true. I also believe it's true that there is a non-zero probability that if my phone were stolen, the air molecules around the fingerprint sensor could spontaneously come to an electromechanical arrangement that unlocks it, giving the thief full access to my bank account and terrible taste in music.

I (and I imagine you) don't claim I was late to an appointment because of heretofore unforeseen difficulties in hidden-variable theory.

On the other hand, the probability of a skilled adversary introducing a back door that evades known methods of detection is very much distinguishable from zero. Heck, the probability of a bug introducing a back door that evades known methods of detection (at least for a while) is distinguishable from zero! That's why jailbreaks are possible.

No, I did not say that. You keep putting words in my mouth, starting with "an iOS image is a black obelisk fathomless to man". And yes, you did quote me, but you took the quote out of context and then twisted its meaning.

Here is what I said:

> I have a lot of reasons to justify my skepticism. I could write a paper (and maybe I should). But HN comments do not lend themselves well to long-form communications so I advanced what I thought would be a compact argument: a quick back-of-the-envelope calculation of the amount of effort required to audit iOS (which is just one component of an iPhone) would reveal that to be infeasible. Now, that calculation may have been way off. That entire argument may have been wrong. But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem (specifically, you would have to solve the lambda-equivalence problem, which reduces to the halting problem). So I'm pretty sure I'm on solid ground with at least some level of skepticism about the effectiveness of reverse engineering.

That does not mean the same thing as "any argument or assessment of the security of a phone can be dismissed by virtue of the halting problem." If you don't understand that, I would be happy to try to explain the difference to you. But you have reached the limits of my patience with your straw-man arguments. Do it one more time and we're done. Life it too short.

I mean, it's in your own damn text. What on earth does this mean that is different from what I think it means. I'm perfectly happy to hear your explanation.

The quote is accurate. Your paraphrase of it is not.

> I'm perfectly happy to hear your explanation.

OK.

My invocation of the halting problem did not apply to "any argument", it applied to one specific argument, namely, the argument that because I was ignorant about the details of modern reverse-engineering techniques, that therefore my conclusion about the likely security of iPhone was wrong. There is still the possibility that someone else could raise a different argument for why iPhones should be trusted to which the halting-problem objection would not apply.

Also, I did not "dismiss" the reverse-engineering argument. It's possible that the reverse-engineering community has figured out some devilishly clever way to work around the halting problem for all practical purposes, and that I'm just unaware of this. If they have, I'm perfectly open to being convinced. (But I'll give you long odds against.)

did not apply to "any argument" [...]

What you said is plainly preposterous and ridiculous. I'm happy to discuss anything you said and anything I said. But, sputter this came out of your own effin mouth. I am not going to talk about how it's something other than its plain meaning.

"But I can always fall back on the fact that to prove that there are no security holes in iOS you would have to solve the halting problem"