You can crowd source this by allowing people to rate the material. Sounds like a good idea for a startup. A "rottentomatoes.com" for self-published books? User rating and "pro" rating?
A rating can tell potential readers that a book is worth reading or not. It can't grow a talented but unskilled author into a great author. Most successful authors will tell you that they had to write hundreds of thousands of words worth of crap before they were able to produce anything worth reading, and then only because they were getting quality coaching while writing all of that crap.
As far as I can tell, the closest things currently in existence are:
1: Writing workshops
2: Short-fiction magaziness
#1 is open to anyone who has the cash to get in the door, but isn't well set up for establishing long-term mentorship relationships.
#2 is good for long-term mentorship relationships, but you have to already be a pretty good writer to get your foot in the door and the price of entry is selling the rights to your work. Also, a "slow-starter" with the potential to become a world-class novelist might never get off the ground under this system.
I'd love to see someone establish independent coaching services for developing writers, but I think there are some significant difficulties. The biggest problem is: where do you get the coaches? Good traditional publishers and editors are good at mentoring authors because they have years of experience acquired by slowly working their ways up through the traditional publishing industry.
I'd like to believe that there are other ways to make great editors, but there's no substitute for experience. You would need a seed of experienced editors from the traditional industry and they would have to do double-duty mentoring both aspiring authors and aspiring editors. Eventually you might be able to get to the point where you have a self-sustaining community where the authors have reputations based on their work and editors have reputations based on the quality of the authors and editors they have mentored.
