An interesting approach to this is the one used in Go, where the "defer" keyword schedules code to run on function exit. It's simple and doesn't require adding more nesting to your code.
Ruby has "ensure" if you want something to be guaranteed to run on exit - it can be used at the end of methods, or by explicitly wrapping code in begin .. end.
(Assuming it works the same way as Smalltalk's ensure...) a really nice aspect of this is that the resource is held open while you're in the debugger, but cleaned up when you finish debugging.
