I'm thinking of a similar approach with text files and collections of other file types like pdfs, images, etc as you note.

For tags, there are some packages for tagging I've come across like TMSU [1], taggo [2], dantalian [3]

I'm thinking of tagging by just having a tags folder with a txt file for each tag. You could just stick the path to files you want to "tag" in there. Then if desired, have a script to create a bunch of symlinks for a browsable tag file structure. Maybe even have some custom grep code to find specific keywords in my notes and "tag" them automatically

[1] https://github.com/oniony/TMSU [2] https://github.com/xeor/taggo [3] http://darkfeline.github.io/dantalian/