You display the problem with that system in your own comment through your wording of "should be". Everyone knows what downvotes and upvotes should be, but due to their intent being unenforceable they just become "that person liked or didn't like this for reasons beyond my knowledge and understanding"