> How does the memory usage scale as the sequence length grows?

That's a good question. I was under the assumption it's linearly proportional, but I can test it out I guess.