Actually, the PECL compiler he mentions generates C, and then compiles it using GCC with -O3.

So in a way, yes it is measuring the speed of a C program. Not a hand-optimized one, but it's compiled the same.