Maybe you just had a bad approach? Linear algebra is loads of fun, methinks. And you need a good teacher or appreciation of the material for that.

And it's something you slowly forget if you don't put it to use.