Pretty much. Usually you do the rough animation by manipulating the objects in the 3D viewport (with the manipulator using euler angles even in the cases where the angles are then converted to and stored as quaternions) and marking keyframes. The graph view is then useful for tweaking the interpolation between keyframes, minor adjustments, copying and pasting keyframes and such.

The graphs are time on X against rotation/translation/scale amount on Y, yes, with separate graphs for each axis.