I still love most aspects of the Amiga user experience, but a lot of Amiga applications would need libraries installed to Libs: and deleting the application's "drawer" would leave those libraries behind. (Having said that, by default libs: is assigned to sys:libs but you could assign extra targets, so that libraries would be sought from application-specific directories.)

Also, it suffers from the same problem as Windows here, in that you can't delete a file or directory which is currently open. The executable itself wouldn't be open after launch is complete (with the possible exception of overlay executables, but they were pretty rare) but the directory would be locked for as long as it was the program's current directory. If a subdirectory with app-specific libraries was assigned to Libs: that would also prevent deletion.