This is shoehorning something into a model that doesn't fit it :/
For example, on Jenkins right now, we have a job which can trigger a bunch of load tests against our website. We can run this job indefinitely, give it arguments, it'll even run a local web server. By default, the job only runs the load test for ~10 mins.
The job is run periodically. When it fails, we get emails. After it completes, regardless of failure, it triggers a "cleanup" job which deletes the loadtest-related data from the website.
This is what I mean with "arbitrary jobs". I think it's absolutely fair to consider them out of scope, but if you want to replace Jenkins, this is what you're going up against.
Thanks for sharing your use case. I think that code of the cleanup job can be in the same repo as the load test itself. Would that be a fit for you? Of course the load test code can have the testing code of the application as a dependency.
Maybe you prefer the load testing code in the same repo as the application. In that case it should be two jobs that are triggered periodically I think.
no, it does not fit the bill. It needs truly arbitrary job running.
I want to be able to just drop in some arbitrary command (e.g. df -h /some/build/agent/dir/ | grep 99%) that would not be worth it to me to write a script, commit it to git, get that deployed and running.
In fact, I AM a fan of this being in git, but the example I'd use is an improvement on TeamCity, where you can configure your job and project configurations to "commit to git" so that you can easily restore when you need to rebuild a server or reload a previous version.
Sooooo many use cases for arbitrary jobs. Many of which may be short lived, but want them automated in some way.
Can Gitlab CI do manual triggers yet? (Drone can't, last time I checked, and that was a huge turnoff)
The "code" in this case really lives in the Jenkins database, as it's a shell snippet. Admittedly I'm not a fan of this. However, since you're a FOSS company as well, you may have hit the case where you want to separate the code for your infrastructure from the code you're releasing in the general repository, but still build off master.
This is where Jenkins' model is quite good. I strongly dislike that it's so hard for Jenkins' metadata to be versioned in proper git repositories, but the separation of the actual code, and the CI-related code is great.
In my taste the shell snippet that is infrastructure specific should like in a small repository. An example of that for us is https://gitlab.com/gitlab-com/runbooks (although that is more process since we try to keep the code generic).
