Some time last year I learned of an example of such a project (peerreview on GitHub):

The goal of this project was to create an open access “Peer Review” platform:

Peer Review is an open access, reputation based scientific publishing system that has the potential to replace the journal system with a single, community run website. It is free to publish, free to access, and the plan is to support it with donations and (eventually, hopefully) institutional support.

It allows academic authors to submit a draft of a paper for review by peers in their field, and then to publish it for public consumption once they are ready. It allows their peers to exercise post-publish quality control of papers by voting them up or down and posting public responses.

---

I just looked it up now to see how it is going… And I am a bit saddened to find out that the developer decided to stop. The author has a blog in which he wrote about the project and about why he is not so optimistic about the prospects of crowd sourced peer review anymore: theroadgoeson.com/crowdsourcing-peer-review-proba…

It is only one opinion, but at least it is the opinion of someone who has thought about this some time and made a real effort towards the goal, so maybe you find some value from his perspective.

Personally, I am still optimistic about this being possible. But that’s easy for me to say as I have not invested the effort!