Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You are right, but I have not seen any published data about the properties of some concrete 128-bit or 256-bit CRC's that could be chosen.

For 32-bit or smaller CRC's there are a large number of publications analyzing various useful polynomials and there is also some published data about a few 64-bit polynomials.

So if you want to use a CRC larger than 64-bit, you must first spend some time to learn and understand very well the theory of CRCs, then search for some suitable polynomials and test them.

It is unlikely that this would be worthwhile in most cases.

In practice, CRCs of up to 64-bit are very useful to detect errors in data packets of limited length, like in the data packets sent and received through some communication protocol, or in the headers or other components of limited size of some structured kind of data file.

CRCs of the available lengths are usually not suitable to check the integrity of entire data files, because in modern computers the files have become too large and too many.

Therefore a file utility like the traditional UNIX cksum has become obsolete when used to compute CRC values for files.

(The latest GNU cksum has become a generic utility, where you can specify other hash algorithms instead of the default CRC-32)

While this is expected due to the short length of CRC-32, I have actually verified that when computing the CRC-32 values for all the files on my 2 TB SSD I have obtained about seven hundreds of CRC collisions, i.e. pairs of completely different files that had identical CRC-32 values.

Even with the obsolete 128-bit MD5 hash there were no collisions, but as another poster mentioned, on modern CPUs with hardware support MD5 is slower than SHA-1 or SHA-256, so there is no reason to ever use it.

While SHA-1 is insecure for digital signatures and other applications for which an opponent must not be able to find collisions, because SHA-1 remains faster than SHA-256, it remains useful for error detection in many cases, e.g. for detecting whole file corruption.



> You are right, but I have not seen any published data about the properties of some concrete 128-bit or 256-bit CRC's that could be chosen.

I mean, sure. But what published data do you have about SHA1's error detection properties?

SHA1 doesn't have a hamming distance analysis associated with it, not at all... nor is there any kind of "burst error" analysis (or the like) associated with it. The only analysis you got on SHA1 is differential cryptography, and other such maths that have little or no relevance with regards to the kinds of errors we expect to see in data.

Its not too hard to pick a 128-bit burst error CRC-128. In fact, all CRC-128 would detect 128-bit burst errors. The "high quality CRC" is about the difficult question of hamming distances and other such properties.

-----

IMO: picking SHA1 is basically saying you don't care about those details. Which is a fine choice to make. But it seems odd that you'd criticize CRC for "not having enough analysis" when SHA1 barely has any at all.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: