I didn't know about BVH! Right now I'm using COLLADA (.dae). I'd like to decide on a single file format and write my own parser for it, but I feel like there must be a slightly less complex format than COLLADA that still does everything I need. I'll probably spend some time comparing it to e.g. FBX.