Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For me it is more clear to say that not the absolute probability, but the conditional probability of a word (in its typical contexts) determines word length. Now thinking about it: it is trivial that this is optimal for efficiency. Not a very deep result.


One thing that isn't trivial is the nature of the constraints that produce the various designs of human language. Are they constrained by some sort of innate "Universal Grammar" coded for in DNA? Or by ease of learning, given general learning mechanisms (not specific to language)? Or by efficiency of use? Or by a balance of information theoretic efficiency (high bandwidth) and redundancy (high accuracy despite noise)? Or something else more significant? Maybe some form of flexibility (greater efficiency in low-noise and redundancy in high)? Or all of the above, in differing proportions for different aspects of language?

We need to measure a lot of things in a lot of different ways in a lot of different languages before the contributions of the various constraints reflected in "the design" become clear. It's not trivial that we will discover (nor has this study come close to proving) that "optimal efficiency" is the bottom line.


It may not seem deep if you're knowledgeable about information theory, but information theory is still primarily known about only by mathematicians, cutting-edge physicists, and computer scientists with a lot of academic study (i.e., your average four-year graduate probably has never even heard of it). It really hasn't penetrated anywhere near as far as it needs to. In the meantime there's a lot of rather "obvious" papers in numerous disciplines to be written that amount to "Hey, information theory is cool and helpful to us" yet.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: