This still poses a bit of a problem for me so far and I have experimented a bit (apologies to those I downvoted in those experiments).
On Stack Overflow it's pretty clear what warrants an up- or downvote. If the answer is correct then it gets an upvote, if it's incorrect it gets a downvote.
Now I have observed, at least in me, that I hesitate giving any votes to answers on this site. Speaking with another user he told me similarly. He basically avoids voting on other answers unless it doesn't impact his position in the »vote ranking« for that question (so to speak). I tend to not upvote Golfscript solutions unless they are really very clever (personal thing) and other answers also rarely and if so, then mostly based on how nice it is.
Downvotes are another matter as well. I may have made some enemies by downvoting answers on Shortest floor function that I perceived as incorrect (for not handling negative numbers correctly) – I made a mistake in one instance.
Generally, how should votes be given on this site? Are downvotes acceptable for answers that are wrong according to the task specification (such as this one)? Is everyone encouraged to upvote the current shortest answer?
I guess it's a bit of a problem in how these sites are designed, since essentially votes can only ever be a popularity contest but what good is in using humans to impose an algorithm here (namely, bubbling the shortest answer to the top)? Should upvotes be only a general measure of »I like this solution because it looks nice or uses tricks I never would have thought of« and objective criteria are tallied separately?