When you hover of the "+1" icon, the tooltip says "Rate this post as valuable". It does not say "I agree".
Similarly, hovering over "-1" says "This post is not helpful". Again, it does not say "I don't agree".
Also, most people also use the system as it was intended.
Look for instance at this post, which got upvoted several times, and rightly so!