> Showing 122 changed files with 4,156 additions and 2,915 deletions.
Wow, that's a lot of code for a tool which is doing little more than creating, starting, stopping, deleting, and zipping-up VMs. Looking at this diff, all I can say is: Holy Java-ish Ruby, Batman!
Most of these classes contain two non-helper methods: init and call. That's not a class; that's a function! I'm reminded of this talk, titled "Stop Writing Classes": http://news.ycombinator.com/item?id=3717715
I've used Vagrant before, but it always felt like overkill. When I had an issue with it, I cracked open the code and was quickly deterred from digging deeper. I hacked my VM back to working by hand and decided to avoid mucking with Vagrant.
The next time I need automated development VM provisioning, I plan to write a very small shell script. Here's an example of getting a box up and running with 7ish commands: http://www.linux-mag.com/id/7673/ For a complete development environment, you only need to additionally mount a shared filesystem and write an entry into your ssh config.
All that said, I do think that there is value in having a command line tool which abstracts various machine provisioning APIs. I know that such a thing exists for cloud providers, but I suspect local VMs have some unique concerns. Regardless, it seems like a task that a shell script is best suited for.
Vagrant author here. Allow me to address both of your points.
First, the "Java-ish" Ruby: I am using the middleware pattern. The classes are used to build up complex sequences of actions[1]. They are reusable and very easily testable. Why classes? Why not modules with functions? It actually started out as modules with functions, but middleware was so _painfully_ the RIGHT way to do things, I can't imagine doing it any other way. Instead of explaining it all, I guide you to this talk I gave on the pattern: http://confreaks.com/videos/961-mwrc2012-rack-middleware-as-...
Next, on Vagrant being "overkill." Write a shell script that does the following: Import a virtual machine, setup host only networking, setup shared folders, boots the machine, runs a provisioner within the machine. Wow that is simple. Oh, wait, I need that script to work on Mac, Linux, and Windows. Oh, VirtualBox has some bugs so can you work around that?[2] Also, host only networking requires some guest OS specific setup (Ubuntu different from Fedora different from FreeBSD etc.), so you have to handle that. Right, and it'd be nice if the script didn't setup networking that collided with other machines on the network, so please write in checks that ports and IPs don't collide. Maybe the script should also verify that Guest Additions are properly installed. AND shared folder performance in VirtualBox sucks, so can you modify that script to use NFS? But only on non-Windows, because Windows doesn't support NFS. Did I mention that NFS requires specialized configuration depending on the host OS AND guest OS? So if you're running Mac + Ubuntu its totally different from Linux + FreeBSD. Have fun!
Now, make all of that REUSABLE and EXTENSIBLE, while adding in some other features (other types of networking, host name setting, and so on).
Yeah, I'd say Vagrant's 3000 line of code count is pretty spot on, and pretty good.
It is worth searching through all these comments and reading what Mitchell (mitchellh) has to say. It's been very informative getting the author's perspective on every important issue brought up in this thread.
Mitchell: First allow me to apologize for the tone of my comment. Re-reading it now, I realize that I glossed over a key point: I used Vagrant every day for a team of 5 people. In general, it worked very well, rarely broke in any way, saved us a whole bunch of time getting up and running. So thank you for your small, but real, contribution to my startup.
> middleware was so _painfully_ the RIGHT way to do things
I did see how you were composing the operations and I'm familiar with the middleware pattern; thanks for sharing your talk. However, I think this is one of those cases where OOP has a name for something that FP just takes for granted: function composition. Regardless, Clojure's Rack equivilant does provide middleware as functions of request -> response maps. It's much nicer to work with, but that might be a result of Clojure's design vs Ruby's and unrelated to Vagrant: https://github.com/ring-clojure/ring/wiki/Concepts
> Write a shell script that does the following
I don't have any of those requirements. I only need to support OSX as the host OS. The team's MacBook Airs don't have enough resources to run more than one VM, so I don't ever worry about port collisions. I need to configure my network adapters anyway for my production deployment, so I just need an if statement or two in that config file template. Hell, I don't even need the ability to turn off a VM gracefully, since kill -9 seemed like a decent way to test crash recovery.
Even thought I don't have those requirements: I used Vagrant. It saved me some time up front.
But that complexity has a cost. Running `vagrant ssh`, for example, took 3ish seconds on my machine. Again, this is probably due to RubyGems, which is painfully slow. I had to explain to the team to run `vagrant ssh-config >> ~/.ssh/config`, then to go tweak that a bit, so that they could write `ssh dev` and have it be instantaneous.
Worth noting however, is that I already needed a way to generate local ssh config for each production box to be able to talk to specific other production boxes, and for client machines to talk to our cluster. Vagrant's ssh-config command was just one more extra moving part to concern myself with, so I manually put the correct config into a cat script along with the production needs, and added it to the build process.
There was one or two more of these little things that escape my memory now.
> make all of that REUSABLE and EXTENSIBLE
I think my main complaint about Vagrant is one which you seem to be fixing, starting with the very commit that this HN thread is attached to: You're pulling out the REUSABLE bit of abstracting away various virtualization platforms across a variety of host and guest operating systems.
However, I'm a big believer that EXTENSIBLE does not belong in library code. Extensibility belongs at user layers. Furthermore, extensibility is really fucking hard. 99% of the time, I'd rather the core be more reusable and allow me to bypass extensibility.
I think that Git is a perfect example of this design. They call it plumbing and porcelain. The plumbing commands offer zero extensibility. They don't have any creature comforts, as their output is tuned for parsing. The goal of these commands is to manipulate the abstractions of the underlying system. The porcelain, by contrast, does offer extensibility via git-foo.sh scripts on your path, aliases and configuration, commit hooks, etc. I'd see a lot of value in something like this for virtual machine management.
Another example would be Chef & Puppet's facts systems. I forgot what they are called. Ohai? Something like that. They provide OS-independent interfaces for gathering facts about machines. Those components should be hoisted to first class project status!
Unfortunately, Vagrant does not, yet, provide a plumbing vs porcelain split. In many ways, I view Vagrant like SVN: It's a damn solid piece of software. Lot's of people get a ton of milage out of it. However, it's underlying concepts are slightly muddled together and there are huge gains to be reaped from deep thought and careful modularization. When those big gains do come, they will only work on a single platform, maybe one or two guest operating systems, people will bitch about it's low ease of use, despite it's inherent simplicity, but the people who deeply understand machine provisioning and management will adopt it in droves, until eventually Windows support doesn't suck too bad, all the various use cases are more or less covered, and people are wondering "How can you possibly still use that old thing?"
I think you've got a good start towards that goal in Vagrant, but my complexity sense is tingling.
> I'd say Vagrant's 3000 line of code count is pretty spot on
I just ran cloc on the master branch locally; I see 12355 lines of Ruby ;-)
Thanks for your response. Lots of points in here so I'll try my best to respond to each in turn. I apologize if I miss anything (just point it out, I'm not trying to avoid anything).
1. Function composition middleware
Actually, I simply use classes as a way to do function composition. The reason I chose classes over functions I think doesn't really matter in the grand scheme of things, but in my talk I mention that what I am doing is function composition, and I spent a solid 5 to 10 minutes explaining how function composition solves the problem. :) We're on the same page here.
2. The fact you don't have the requirements for the situation I posted.
I understand this, and that is why Vagrant works for you. But it also works for people who do have requirements other than yours. There is a _huge_ amount of Windows users out there that love Vagrant because it now makes Ruby on Rails and so on development work well! If I catered Vagrant to only _my_ requirements (Mac OS X host, Ubuntu guest), then Vagrant would indeed be a lot simpler, but Vagrant as a tool gets a lot of its power and usefulness out of its ability to work in heterogenous environments.
3. `vagrant ssh` is slow. This must be caused by complexity.
This has nothing to do with complexity. This is caused by poorly optimized code paths and how things work up to that point. I just want to note that nowadays `vagrant` commands across the board are a LOT faster. But there is still a lot of work to be done. Specifically the `vagrant ssh` case will get a massive speedup due to cached SSH info.
I specifically have issue with "complexity causes slowness." Sure, it happens, but it doesn't HAVE to, and I'll show that to be true.
4. Plumping and porcelain Git vs Vagrant
Perhaps there is an option here to use unix-type piping to provide an API via plumbing commands. I've decided to take a different approach. Will this work well? Time will tell, but I don't think its fair to criticize Vagrant at this point before this is proven/disproven yet.
5. 12000 lines of Ruby.
~3500 is test code.
~5500 is "plugins" but there is a lot of boilerplate here. The boilerplate is on purpose, I didn't want any "magic," hence it being somewhat Java-like, I suppose, but this is a lot more friendly to non-Ruby developers, and a vast majority of Vagrant users are not Ruby developers. Note that ~3000 of this is ONLY the VirtualBox plugin, which is a pretty complex piece, so that makes sense. The other ~2500 is _every other_ feature of Vagrant.
~3300 is core code. This is mostly a glorified plugin runner that handles plumbing for you, such as hooking up the right host/guest combination, choosing the right communication (SSH, WinRM, etc.) for the machine, error handling, etc.
Hi Mitchell, it's really great to have you explain all of these things here. Thank you both for this and for writing Vagrant, there is no doubt that it is a valuable piece of software. I have a few more questions if you don't mind:
1. When you first started with Vagrant, did you consider using Python at all? Did you consider using libvirt? Because VirtualBox offers a Python API out of the box (or it does now) and libvirt comes with direct support for Python (http://libvirt.org/bindings.html). I like Ruby and generally I prefer it to Python but I am interested in your view of whether it was the right tool for the job in this case or whether the language choice didn't matter much.
2. In your view what is the overlap in functionality between libvirt and Vagrant and what are the differences? As far as I can see libvirt does let you spin up boxes, talk to them and tear them down. The notable difference is that there is no Puppet of Chef provisioning, is there anything else beyond that?
3. veewee in version 0.3.0 (currently in beta) seems to offer the same functionality as Vagrant plus the ability to create boxes using templates, minus the ability to run provisioners. If they add that say in 0.4.0 then it will be 1:1 in terms of features with Vagrant. Any thoughts on this?
Thank you again for your time and for answering in such detail :-)
No problem, I'm glad you're getting value out of the responses. In response to your questions:
1. When I first started Vagrant, I was a full time Ruby developer, so Ruby was really the obvious choice for me. I don't really see language as a barrier for using a tool, my opinion is generally use the best tool for the job. And as a Ruby developer that at the time had 4 years of Ruby experience, that was the best tool for the job since I'd get the most productivity out of it. To date, I don't regret this decision. There are some things Ruby is bad at, some things it is good at, but I think I can overcome the bad with time.
2. I think the difference is that Vagrant is a tool focused on user experience and workflow. i.e. it'd be possibly to build Vagrant on top of libvirt. I think libvirt is technically stronger than Vagrant, but Vagrant provides a better overall experience. The overlap is small, we both have a lot to gain from working with each other.
3. A handful of people keep saying that Vagrant is just a "VM setup + provisioning" tool. Vagrant does quite a lot more. The main example I always use is networking because it is usually the most complex. Vagrant makes networks work across Linux, Mac OS X, and Windows, and sets up the hosts and guests properly. VeeWee doesn't do this and won't do this. VeeWee at its core was built to be a VM image creation tool. VM lifecycle control was bolted on later. VeeWee filled a major gap in the Vagrant ecosystem for a long time.
I'm good friends with the creator of VeeWee and we talked all the way back in Oct, 2011 of merging the functionality. He was all for it. Its almost been a year but this work is finally going to get started.
I don't want to talk too much about it until I really start coding, but I can say that no one in his thread has really seen the true scope of what I'm trying to build here. They will. :)
In particular, I've found the discussion of autoconf to be fascinating. I think what fundamentally bothers me about Vagrant is that it feels like autoconf. By that, I mean it's a system which 1) hides a ton of hard work that users don't need to do to target multiple backends 2) accomplishes that hiding through complexity 3) further hides that complexity through an "easy" interface 4) generally just fucking works most of the time 5) is completely maddening to deal with when it does break 6) ideally shouldn't need to exist.
Anyway, I realize that ideals != reality. I also realize that I've got a learned behavior of reinventing the minimal viable tool from fundamentals to avoid dealing with the complexity of problems that I don't have. And furthermore, I realize that my learned behavior is of limited applicability outside of narrow environments, such as my own startup where I own the full stack and make all the rules.
"However, I think this is one of those cases where OOP has a name for something that FP just takes for granted: function composition. Regardless, Clojure's Rack equivilant does provide middleware as functions of request -> response maps."
I'd like to address some of the finer points in the comparison of middleware vs function composition that have been overlooked.
First class functions - While Ruby does indeed have lambda's and blocks it does not have first class functions, or partial application. This makes state and argument management much more difficult. That is, Vagrant would have to cram _everything_ into `env`...
State - The middleware(s) operate on a common state container `env` and this is where the initial composition comparison arises. The common argument is consistently the `env`. The problem with cramming everything into `env` is that you discard data encapsulation. All the things are known to all the people, which is unnecessary and can cause implicit dependencies between middleware(s). By instantiating a class with values specific to its operation we avoid this issue all together. To achieve a similar result with pure functions you would need something like partial application which Ruby simply doesn't support. It's also worth noting that with pure functions you could adopt a monadic pattern for composing and managing the state which is, IMO, a more accurate analog [1].
Method dependencies - With regular functions you have two choices, you can simply use a global reference to a helper function to maintain discrete testable operations or inline all the different bits of functionality into the function itself thereby making it difficult to test. Generally speaking I'm a fan of keeping those helpers namespaced and easily sharing the small bits a state specific to the instance's operation between them via context. Keep in mind that small classes with sparse state access through the implicit self can actually be more readable, and the ability of the reader to reason about execution isn't hindered too horribly by the implicit state.
Warden - Implementing a recovery pattern is hard using just pure functions. In Vagrant a watcher middleware (Warden) is inserted between each middleware to catch exceptions. When an exception is thrown, because the parent is the Warden it can then recognize its place in the stack and work backwards calling a `recover` method on all the preceding middleware in the stack. This means that each middleware is effectively two methods. One for the initial operation and a second for recovering in the case of failure. You can implement something _similar_ using Either, but in reality you need a custom record data type that carries both functions in which case you're not composing a simple function any more and the difference is really just that Vagrant is written in Ruby. I've written a blog post on the inspiration for Vagrant's Warden implementation if you're interested [2].
Vagrant's Middleware != Rack - PEP333 makes no assertions about state cleanup because in an http request there really isn't much state (the prescription is to return a helpful error message[3]). Vagrant on the other hand is writing hundreds of MB to the disk and needs to clean up when something goes wrong, so I'm not sure that drawing a comparison with a Rack implementation makes sense.
I've been using Vagrant since the early days. While I have to admit I've spent zero time in the Vagrant source and I'm sure your assessment is accurate, I'm not sure how much I really care that it might be java-ish or bloated or not well written. I like well written code as much as the next guy. But I'm not deploying Vagrant to my production machines to be accessed by millions of users where I need to squeeze a few milliseconds out and I'm not building on top of vagrant as a framework. I use it behind the scenes as a tool to save me time and bootstraps my development servers. And it does that well. You don't need to maintain the internals, which means you can focus on your actual business, which I don't think is writing provisioning tools :)
Rather than just an echo chamber of praise? It's good to see multiple points of view and criticism. There's always a place for improvement, and he's right, the codebase doesn't seem particularly ruby-ish.
I was at an academic conference recently where an researcher I respect was talking about paper reviews at a top journal much the same way. His point, if I can attempt to do it justice--was that we sometimes get so caught up in proposing ways to "polish the diamond" that sometimes we forget that it's a diamond we're polishing. Some things are good in their own right, and while yes, they are likely improvable, we as "reviewers" don't usually offer enough recognition of the submission's value, or do it only in a token way in order to excuse the criticism that follows.
The researcher used as an example a recent submission with a novel research method. The other reviewers were caught up in the minutia of the paper, and missed the forrest for the trees: the method itself was more than a worthy enough contribution to the field to be accepted, and they were arguing about significance of the results. Over the course of the next couple of review rounds, this researcher's focus shifted from reviewing the paper to defending it to the other researchers. In the end, after multiple rounds (and likely a couple of years), the vote was 2-1 to reject. At this point the Senior Editor stepped in and published the paper without further revision.
Do we do the same with submissions here?
That conversation was hit me pretty hard, and I hope it will change the way I review.
Honestly, does it matter? Do I have to always code in "ruby-ish" while coding in ruby. What should take more preference? Solving a problem or maintaining "ruby-ish", "python-ish" idioms. Besides if the author feels that he can maintain his codebase this way then I don't see any problem in it. I also do not like tone of trashing vagrant just because it's code base is 'shitty'. Vagrant has been godsend for our office synchronizing configurations across multiple machines. It may be easy for some geeks to create a development system using seven command. But there are people who aren't bothered or have less time digging very deep into every documentation on earth. They want to quickly find a base solution for their problem which they can tweak to their need. Vagrant solves problems for those people perfectly.
I don't see any mention of 'shitty', just over-engineering. For the record, I would use Vagrant if I had the need, and probably will in the future (this kind of caveat seems sadly required otherwise I would be painted as somehow opposed to a software project), that said..
Fitting in with the community style/idioms is rather important for a project, especially as it grows in size. When it's a huge codebase, used by many, in many different use-cases, people being able to look into the code to help sort out their own issues is hugely valuable. If someone who knows your programming language and knows your problem domain gets scared off by how your software is architectured/functions, that's not a particularly good sign.
But maybe the problem is huge, and all that complexity is needed, and will pay-off now they're looking to have multiple VM providers. But criticism of complexity shouldn't just be tossed aside as 'trashing'.
That doesn't follow at all. You can still fix bugs in open source programs, even if you disapprove of the coding conventions. It's only equivalent to closed source if you're irrationally attached to a particular style.
That you don't like the design and coding style strikes me as a poor reason not to use a system. It's utterly orthogonal to whether that system solves your problem.
I am prepared to bet $250 -- your choice of USD or AUD -- that within 1 year your shell script will have grown beyond seven commands.
I've been using my own bash scripts for a few years to manage local VMs with KVM for dev and I have exactly 6 commands in 249 lines (excluding blank and comments) and never needed more: image, create, start, stop, delete, list. Where can I collect my $250 USD?
> I've used Vagrant before, but it always felt like overkill.
I'm finding Vagrant is an indispensable tool for learning Chef and testing my recipes. Also, having a complete multi-VM setup as a development environment that replicates my production environment has its advantages.
So this is good news for me. I very much prefer VMware Fusion to VirtualBox. Can't wait till the next Vagrant release.
Bit of a non-sequitor, but why are you using Chef instead of Puppet? I don't usually deal with VM stuff, but I've recently started looking into it and it seemed like Puppet had a clear lead in functionality, or perhaps my mind is just playing tricks on me at this time in the morning.
I have used both and find that wile Puppet is a good tool, Chef has a clear lead in functionality and in community strength. Also, the Puppet DSL is a pain as u have to learn both it and ruby.
The simple interface of Vagrant is intentional. It is built on the Rack spec, which uses middleware that builds on top of each other. There is a talk called "Rack Middleware as General Purpose Abstraction" where John Bender, the lead developer on Vagrant project, explains why he chose to use that interface. You might want to watch it.
Yes, the project seems to have around 380 files. I'd be interested to hear from the author if he's reading this thread why the codebase is at this size. Perhaps VirtualBox quirks necessitate a large part of it?
Still a lot of files but not 380, just wanted to point that out. Each individual file is actually rather small. There are only a few larger files in there (Vagrant::Environment actually perhaps being the only one).
"Why the codebase is at this size." Well, Vagrant started very small, but in the 3 years of its existence, it has gained a lot of what I've previously heard called as "code wisdom:"
* Generic host support since some tasks are host-specific. For example, setting up NFS on Mac OS X differs from Ubuntu which differs from FreeBSD. Vagrant knows this.
* Generic guest support. Setting up a variety of things is guest OS specific, so again, Vagrant knows how to configure networks, mount folders, etc. on a variety of different guest operating systems.
* VirtualBox bugs. Vagrant has quite a bit of code to work around VirtualBox bugs.
* Networking is hard. Vagrant does quite a lot with networking to ensure your networking works as expected. It also does things like port collision detection to verify that any ports you choose won't collide with any other VMs as well as making a best effort to check if that port is open, and erroring in that case as well.
* Plugin interface. Almost every part of Vagrant at this point in git master is customizable. This isn't overkill, this was built out of seeing actual human need for this. This lowers the number of features I need in core and actually fights being overkill, because more things can be built with plugins.
* Communication abstraction. Linux and POSIX-like guests with SSH is great, but what happens when you want to run Windows guests? SSH CAN work but isn't ideal. For this reason, Vagrant is started to abstract out communication so that Windows can use things like WinRM, completely transparently. Also, if you use something like VMWare, it uses the VMWare guest communication API, which is blazingly faster than SSH. Again, this is all transparent to plugin developers, provisioners, etc. It "just works."
It does much more, but I think you get the idea.
Building something that "just works" comes at a cost. And that cost is usually complexity. Can you build a very basic tool to control VirtualBox virtual machines in a small shell script? You betcha! Vagrant started this way. Can you make that reusable? Yeah, but prepare to add some complexity. Then make it run on every major OS and support some nifty features and you got yourself a big project.
Vagrant does have support for the general case, where something in 7 lines will be a bit more specific. For example, Vagrant will handle cross distro/OS NFS shared folders.
The integration with Chef is enough for any engineer using Chef to configure and deploy their software to choose Vagrant over any other solution.
The Chef community has had major success in automated testing of their Cookbooks (units of configuration in Chef) and it is all made possible by Vagrant.
I know one fellow who built a large private cloud for his organisation (about two racks of topline gear) so that they could simulate an architecture of several dozen servers and thousands of complex, concurrent transations.
What's it built on? VMWare. He was not totally amused when I sent him a link to Vagrant.
(I understand VMWare have similar fleet-orchestration tools, but not development-oriented as vagrant is -- awesome for production though).
Putting aside my friend, I can also see performance gains from in-kernel virtualisation schemes such as KVM^w lxc or Solaris Zones.
Bravo to the Vagrant team. This tool has become one of the centrepieces of how I personally do development.
Thanks! Vagrant will soon be able to control development cycles around private clouds just like this. This is exactly the use case I am building for. Let your friend know!
LXC is great but does have its limitations. We are currently building a vagrant like solution and GUI for LXC/Vagrant/vSphere/ESXi/Xen/AWS etc which will launch in beta in the next couple of months - sign up at http://10xlabs.net if you are interested in following progress
There seems to be no other way to communicate with you or 10xlabs, so excuse me for cluttering this thread.
You have an error on the sign up page - 'lounch' should be 'launch'.
I'm not sure what are the benefits of supporting many VM providers for a tool that is meant for development (as far as I understand at least). Isn't VirtualBox sufficient? It is open source and runs on Linux, OS X and MS Windows. Perhaps I am missing some core idea.
Then there is veewee[1] which started life as a Vagrant plugin but as of version 0.3.0 it can be used standalone and one can ssh to the boxes using it, just like with Vagrant. If they were to add provisioning it would probably be an alternative to Vagrant feature for feature so maybe we're seeing a bit of competition between these two tools there.
On Ubuntu there is VMBuilder[2] written in Python but it doesn't offer provisioning as far as I know.
Last, as others have mentioned there is libvirt[3] which offers tools for communicating with different VMs but I don't think it offers a tool that matches what Vagrant does 1:1 (most notably the provisioning part).
The power in Vagrant is not the technology, it is actually the workflow. Vagrant is now in use by thousands of companies worldwide (some of the bigger ones you can see on vagrantup.com), and they consistently tell me that the power in Vagrant is that every developer, designer, etc now knows they can get some code and just type `vagrant up` and be working within a few minutes. When they're done? `vagrant destroy`. Small example, but Vagrant provides a consistent workflow.
There is a benefit to having this workflow on other systems.
First, VirtualBox has major problems. VirtualBox routinely kernel panics on a variety of operating systems, even leading to the point that there is a commit to the linux kernel that ignores crash reports from VirtualBox.
Second, the whole goal of Vagrant is to have development mimic production as closely as possible, and to make development fun. Some companies that use Vagrant can't mimic production very well on a single laptop, so they have a private cloud. They still want to use Vagrant -- because of the workflow! -- and this caters directly to them.
Also, for people on Linux, I know its not full virtualization, but some problems can be solved with simple LXC. Those people write scripts around LXC because LXC is pretty simple, but it would be pretty nice to get the benefits of Vagrant there, too. That is coming! Vagrant will support LXC.
Libvirt doesn't compare to Vagrant at all. If anything, they compliment each other.
And veewee-like functionality will make its way into Vagrant core very soon.
I had so bad experiences with Virtualbox (Dropping interfaces, crashing, ...) and switched to VMWare (Mac OS X). So I applaud the support of different providers.
This is really great news. Vagrant has really made my life easier as a developer, thanks Mitchell! I prefer VMware so this is awesome!
I personally think vagrant needs a UI manager of some sorts (I 'm thinking about doing this when I have more free time). I have about 10-15 vagrant boxes now and it can be a pain to manage. Having to go into each project directory and manually up'ing each vagrant box can be tedious.
Good to hear this. I had several issues with Virtual Box in the past on OS X(Lion, Mountain Lion) and it even produced 3-4 times a kernel crash on my MacBook. I switched to VMWare Fusion then and all the complains were gone.
This is incorrect. I am not trying to reimplement libvirt at all. If anything, I'm providing the infrastructure to use libvirt WITH Vagrant. Actually, surprise! I'm working with RedHat to do JUST THIS.
I applaud the libvirt project for everything they've done. We've had our difference, but I have no intention of competing with that project. The power in Vagrant is the workflow of Vagrant, not Vagrant's ability to abstract away the hypervisor (and I certainly don't want to play that game).
The power in Vagrant is the workflow, and that is the game that I'm playing.
Everytime I try to like libvirt, I start reading the documentation and want to claw my eyes out when I remember just how complicated it makes things.
libvirt might be ok if you really need to try to be independent of virtualization technology (though from the looks of it, doing so still requires you to limit yourself to a small subset of functionality) and have really complicated needs, but for a lot of use cases it is ludicrous amounts of overkill.
libvirt is (internally) complicated because it really does a huge amount of work for you [the vagrant developers will no doubt discover this over time].
The API isn't so complex, since there are only perhaps 10 calls that you need to know, plus some XML, and it's all documented in extreme detail. If anything I'd say the problem with libvirt is the documentation is too detailed, and there still aren't any good starter API tutorials.
It's the usage, not the internals, I find overcomplicated.
Every time I've looked at it, I've just ended up writing a small shell script to do what I needed instead rather than having to deal with the convoluted XML config and read through massive amounts of documentation.
If you're writing or editing XML, then you're probably doing it wrong (which to be fair could be a problem with the documentation). You should probably be using commands like:
virsh start MyGuest
To create guests: virt-install, virt-manager, Boxes, Heat, OpenStack or one of the many other libvirt users. None require XML editing.
In any case, that's not an issue for the topic here. The Vagrant developers will certainly need to read up some libvirt API documentation, but once they have made the sensible decision to use libvirt, then that will be completely hidden inside Vagrant, and wouldn't affect end users of Vagrant in any way.
I added a libvirt backend to libguestfs (previously we were managing KVM processes by hand). The change is completely transparent to libguestfs users.
> To create guests: virt-install, virt-manager, Boxes, Heat, OpenStack or one of the many other libvirt users. None require XML editing.
All of which adds more complexity to do something I can do trivially without libvirt.
I might just not be the right target for it, given that I have the luxury of being able to use only LXC and OpenVz, and in both cases creating and managing VM's with the tools provided is extremely easy.
> In any case, that's not an issue for the topic here. The Vagrant developers will certainly need to read up some libvirt API documentation, but once they have made the sensible decision to use libvirt, then that will be completely hidden inside Vagrant, and wouldn't affect end users of Vagrant in any way.
Of course as someone using mostly LXC and OpenVz, to me Vagrant also seems totally over-engineered for what it does. It's at least a factor of 10 larger than what we use to build and deploy our entire production clusters. I'm very happy I don't have to deal with that kind of complexity just to get some dev environments set up.
They are right now in the stages of removing proprietary code towards a modular architecture. I think it's more likely they could implement libvert as a plugin. The focus isn't to make a pluggable architecture for vm emulators(libvert). But to make vagrant itself more robust, by removing it's ties to virtualbox with a plugin system.
I've always liked what vagrant did for the other systems (re: non-linux). However, with this change, it seems they are indeed going in this direction and, well, that is a shame. I might be missing something but it seems they would do much better to build on other tech.
The best I've seen is Ubuntu and how it does it (using libvirt among others, KVM etc - http://www.ubuntu.com/business/server/virtualisation). Would be awesome if they were all playing well together. And, perhaps, vagrant is going to actually do that in the future.
Ubuntu's business server is hardly something as turn-key as vagrant.
We use Ubuntu on EC2 and Canonical has made so many atrocious decisions since the release of 10.04, I am not sure if I ever want to run anything from them again. =( We are however heavily invested with custom PPAs and all that.
Back to Vagrant - as others said before: `vagrant up` and productivity starts. At work I manage local and remote people and they all work with Vagrant. It works the same for developers and frontend people. And most importantly: it works.
Yes, there are ins and outs, but they are mostly Virtualbox related and I cannot wait until other virtualization is supported.
Everywhere I look with Ubuntu they are making the right decision, so I'm not sure which decisions you mean. Can you elaborate?
My thoughts are:
Server: Juju is just plain awesome. If you manage hundreds or thousands of servers, it is Fan-Freakin'-tastic. Seriously, that alone is worth the price of admission.
They backed openstack very early, which looks like the winner. They are THE option for EC2. Heck, even Azure is picking up Ubuntu. More here: http://www.ubuntu.com/cloud If I was deploying in the Cloud, there is no other OS I would choose (and I am). IMO, it would be crazy to not use Ubuntu.
Desktop: Unity is an interesting choice, and if you asked me 3 years ago I would probably have said it was a bad choice, but seeing the state of GNOME these days, it is easily the best choice Canonical has made in the past few years.
And then there are the demo products like Ubuntu for Android, which is just plain awesome. The TV, though I'm not sure how viable it is as a commercial entity, is slick. And there is talk of a phone/tablet. Is there room for a third or fourth person in this market? With the patent problems, probably, I don't know. Though, I have to admit, I'd like to see what an Ubuntu phone and tablet would look like!
Wow, that's a lot of code for a tool which is doing little more than creating, starting, stopping, deleting, and zipping-up VMs. Looking at this diff, all I can say is: Holy Java-ish Ruby, Batman!
Most of these classes contain two non-helper methods: init and call. That's not a class; that's a function! I'm reminded of this talk, titled "Stop Writing Classes": http://news.ycombinator.com/item?id=3717715
I've used Vagrant before, but it always felt like overkill. When I had an issue with it, I cracked open the code and was quickly deterred from digging deeper. I hacked my VM back to working by hand and decided to avoid mucking with Vagrant.
The next time I need automated development VM provisioning, I plan to write a very small shell script. Here's an example of getting a box up and running with 7ish commands: http://www.linux-mag.com/id/7673/ For a complete development environment, you only need to additionally mount a shared filesystem and write an entry into your ssh config.
All that said, I do think that there is value in having a command line tool which abstracts various machine provisioning APIs. I know that such a thing exists for cloud providers, but I suspect local VMs have some unique concerns. Regardless, it seems like a task that a shell script is best suited for.