As a Rails developer doing a fair amount of TDD and some BDD, I have a first-hand view of the state of the art of view testing for the web. Tools like webrat, selenium and cucumber are compelling because they hold the promise of end to end testing.

I'm really glad that people are working on solving these hard problems, but, I'm also a designer and usability guy, and I can tell you that these tools are still very very crude. You know the sayings "when all you have is a hammer everything looks like a nail", and "use the right tool for the right job"? Well, to developers, code is the hammer. Developers (especially consultants) would like to believe that if they write a cuke test and it passes, then their job is done. But the fact of the matter is that test code is not free to write or even to run, and the amount of things that a QA person is actually testing when they click through an application is several orders of magnitude more coverage than an state-of-the-art acceptance test gives you (this may change eventually, but it's an AI-level problem).

I see a lot of value for the initial acceptance tests that verify basic flows, but they are just one tool in your belt. Usability testing and QA provide orthogonal human perspectives that provide more bang for your buck then increasingly fine-grained automated tests. Think of it like this, automated tests take your product from complete failure to mediocre, but design, usability testing and QA can take it from mediocre to great.

I've seen brilliant programmers get obsessed with "all green" to the point that they do all their work without firing up a browser, and miss dozens of terrible usability flaws that would be immediately obvious the first time you click through.

None of this is in any way disparaging of GUI testing per se, but just that it has limitations which I've seen ignored for the sake of code fetishism.

Not sure if it's any different or better, but Project Sikuli (http://groups.csail.mit.edu/uid/sikuli/), an api for scripting visual actions in guis could be useful for testing layout and interaction.

Here's an example of using it for unit testing a gui: http://sikuli.org/documentation.shtml#examples/TestJEdit.sik...

The way I see it is that the code to test if a GUI element is properly formed is basically the same code that would just properly form the element in the first place. So by automatically testing the GUI you are basically just repeating yourself over and over.

Not to mention that when you want to improve the usability by rearranging things, it will be twice as hard to do if you need to change both the code and the tests.

Unless you have to render something pixel-perfect, it is probably not worth it to write automated tests. A human can look at your screens and verify that they aren't (or are) completely crap fairly quickly.

Ultimately humans will be the ones finding bugs in your software, and when they find UI layout bugs it should be immediately obvious by looking at the code what is wrong. If it isn't then you might need to separate UI from the rest of the gunk better.

You are right of course, but nobody said that GUIs weren't testable. Just that some bugs cannot be expressed as tests.

To do be basic GUI testing, you don't actually need to test aesthetics, just some basic morphological/topological perperties like "is this button touching that button", "is this window visible?" etc.

But this is not likely to happen either.

Part of the problem with making a windowing system query-able is that your talking about a number of distinct components that each separately paint themselves onto the screen. The screen's properties vary by system's configuration, the graphics driver, and the size of the overall window being painted. Sure, all of those components could be rewritten so at least relationships like "above" and "beside" and "touching" could be queried. But this stack involves hardware vendors, OS creators and software tool vendors. Moreover, hardware vendors and OS creators wouldn't want add anything that got in the way of performance since that's their main claim to fame.

> So it is true that GUIs are untestable, but this doesn't have to be true.

I respectfully disagree. You can't codify aesthetics or usability, and even if you could, the return on investment would be abysmal.

"Aesthetics" or "usability" as a whole? Sure, of course not. No more than you can codify "works correctly" as a whole.

Codify huge swathes of it, though? Absolutely. Assert that correct fonts and font sizes are used. Assert that color schemes are correctly followed, or that certain widgets always have attached desirable behaviors, assert that all widgets have keyboard shortcuts, assert that those keyboard shortcuts are coherent (CTRL-F is not assigned to three actions), assert that using the keyboard shortcut has the same behavior as whatever it is the shortcut to, assert that automated validation of appropriate accessibility guidelines hold (ie, for web interfaces assert that a Bobby run over your interface holds no surprises), and so on.

Sure, you can't "automate usability" but if you just give up at that point you'll miss out on a lot of things you can automate... or could automate if GUIs let you, getting back to my original point.

All of the examples you give are functionality, not how the thing actually looks when it renders on the screen.

Is there some way to write a test that determines whether my page renders correctly on Firefox, Chrome, Safari, IE 6, IE 7, IE 8, and Opera? If so, please show me the way.

In web development, these sorts of things end up being a large percentage of bugs filed in FogBugz, and there's no way to test for them but to have someone look at the page and make sure.

We had manual test case for "page looks correct" test. It consists of more than 10 checks, i.e. "no JavaScript errors", "no miss-spelled words", "consistent look across pages", "no broken links", etc.

More than half of that list is already automated.