We hear you and agree, GitLab.com is too slow. We're working on multiple things to improve it https://gitlab.com/gitlab-com/infrastructure/issues/947 GitLab 9.1 will have lots of performance improvements for database queries. The GItaly team is working hard to move git commands to the fileservers to reduce latency, that will happen in about a month from now.
As a thought for the performance diagnosis part of things, if your network switches have the option to enable NetFlow (also known as IPFIX), it can be extremely useful.
It does need to be paired with something that can capture and display the NetFlow data well. But with that, you can literally see what data is going where in your network, accurate to the byte. And with a good visualisation solution, you have the data historically too, so you can (easily) match things up time wise.
eg a developer writes a bad query which pulls 1/2 the database over the network to the front end server for filtering results there, instead of filtering them in the database. You can see that on the network fairly easily, and let the developer self-educate (if given access to the tools). ;)
As an aside, the gold standard used to be a commercial product called "NetFlow Tracker". It was amazing (fond memories), but the company behind it (Fluke Networks) didn't seem to know anything about software sales. So, it's now discontinued. :(
Hopefully there's a modern version of that somewhere. :)
We're working very hard on making GitLab.com much faster. Right now, it's measurably (and more importantly: measured [0]) faster than it was a month ago.
If you would like to follow progress you can dive into it all here [1]. One of the spearpoints is Gitaly, which should significantly improve git access.
