I'm a victim of "Information Paralysis" as well. One thing that has helped me was to join a meetup or group or even discussion forums related to a particular field that i want to spend time on for the next few months and frequently catchup with them so that you're always motivated and there's a win-win kind of situation for the group as well.
I think you should start working on your Math (Khan academy courses) and ML foundations (Andrew Ng's coursera course). Then Geoffrey Hinton's coursera course on Neural networks could be a gentle introduction to Neural networks, deep learning and their applications. Last but not least, do a small project on Deep learning or try out few kaggle competitions to deepen your understanding.
I think the kind/amount of data and type of analysis that is to be performed on the data should probably lead the decision. Scenarios where you have hundred of GBs of data having to perform text analysis on them, probably would make me think in terms of flat files. But for simple web apps, probably RDBMS or a NoSQL store is the way to go.
