Right. I do share your concerns about performance, but I'm sure it will improve considerably. A few months ago on reddit, one of the Matrix core devs (ara4n) answered a question about how much they can improve performance:
"enormously. much of synapse's algorithms and DB schema are still unoptimised, plus python is not exactly renowned for being super-fast. we're effectively going through rewriting chunks of synapse - e.g. the new state storage representation in 0.18, and meanwhile Ruma is going through writing a cleanroom impl in Rust that should be a bajillion times faster :)"
"enormously. much of synapse's algorithms and DB schema are still unoptimised, plus python is not exactly renowned for being super-fast. we're effectively going through rewriting chunks of synapse - e.g. the new state storage representation in 0.18, and meanwhile Ruma is going through writing a cleanroom impl in Rust that should be a bajillion times faster :)"
(https://www.reddit.com/r/linux/comments/56xzyo/matrix_is_a_n...)
Ruma: https://github.com/ruma/ruma