27
\$\begingroup\$

The default sorting for answers (by votes) seems somewhat inappropriate here on PPCG. In any case it tends to put early answers over late answers, because once a few are there, few people scroll to the bottom to see newer answers and even fewer people sort by activity.

If you consider that better answers will usually take longer to come up with and implement, that means the answers that deserve the votes are actually at a disadvantage.

This is problematic for different reasons on different challenge types:

Popularity Contest

Sorting by votes is still useful here, because that actually shows you the ranking of the answers by the winning criterion. However, for that very reason (that votes determine the winner) a bias towards early answers can't be in the interest of the community.

Other Challenges

Here, sorting by votes doesn't even make really make sense in the first place. The reason this is the default sorting on other SE sites is that it's a decent heuristic for which answer is the best. But we don't need that, because the quality of an answer can be determined objectively by the winning criterion. Of course, sorting by winning criterion is not an option, but sorting by votes doesn't seem useful either. Hence, some other sorting could at least help later and better answers get the votes they deserve.

So, I'd say we should think about different, more appropriate, sorting modes and also consider whether the default sorting should be changed. If we can come to a community consensus here, we might be able to bring this to the attention of SE staff.

The questions to discuss are: are the existing sorting modes sufficient or do we need new ones (which ones)? Should a different (new or existing) sorting mode be the default?

\$\endgroup\$
  • \$\begingroup\$ See also: meta.codegolf.stackexchange.com/questions/900/… \$\endgroup\$ – Ilmari Karonen Aug 21 '14 at 14:09
  • \$\begingroup\$ @IlmariKaronen While useful this doesn't really solve the problem, because it's especially about new/one-time users who get here via HNQ or otherwise and are just here to vote on popular questions. \$\endgroup\$ – Martin Ender Aug 21 '14 at 14:12
  • \$\begingroup\$ True. I was just pointing out that a similar issue has been discussed here before, and that, based on the feedback I've got for the script (and Nathan Osman's earlier similar script), a significant fraction of CGSE users do seem to feel that the existing sorting methods are inadequate. That said, the issue you raise seems to be somewhat different, as it mainly concerns popularity-contest questions, whereas my script is mainly useful for code-golf. Still, I felt it was potentially relevant enough to at least link to. \$\endgroup\$ – Ilmari Karonen Aug 21 '14 at 14:19

12 Answers 12

20
\$\begingroup\$

Measuring the Strength of the FGITW Effect

Although this answer isn't a suggestion for a solution, it should provide a better understanding of the problem. A proper understanding is necessary to create a solution.

The FGITW effect ("fastest gun in the west" effect) is the phenomenon in which the earliest answers receive the most votes and always stay at the top of the answer list. The feedback loop is this:

early answer -> some votes -> top of the answer list -> more visibility -> even more votes

Although I think everyone recognizes the existence of FGITW, people might not have an idea as to its power. How can we measure the FGITW effect in an objective way? With the help of user Eric Tressler, some data was collected and measured.

We decided to look at the RPSLS tournament as an example of FGITW. This challenge was posted about 3.5 weeks ago. Here is the data (collected by Eric):

// Score, date posted rank, #votes for Rock Paper Scissors Lizard Spock
n = 65
64  28 18
62  55 1
62  65 2
61  64 1
60  29 5
58  27 3
57  32 2
53  61 1
51  42 1
51  16 20
49  31 2
49  49 1
47  34 3
46  56 1
45  24 4
44  48 1
43  40 4
42  20 4
42  53 1
41  12 2
41  13 3
40  23 3
40  46 2
39  62 1
38  38 4
38  33 2
38  43 2
37  50 1
35  15 8
35  58 1
35  54 1
34  3  4
33  10 3
32  4  8
32  44 1
32  14 5
32  17 2
31  19 8
31  45 2
31  51 1
30  1  3
30  60 1
29  22 4
28  25 3
27  57 1
25  30 2
24  2  7
24  39 3

The three variables are:

  • Score (s), showing how well the post performed in the contest
  • Date rank (d), with lower numbers being the earliest answers
  • Votes (v), which is just the net number of votes (up - down)

We wanted to find the relationships between these variables. We used Spearman's Rank Correlation Coefficient (Rho) to measure the correlation between the variables.

Dates-Scores:

datesscores

The above dates-scores graph shows how performance was related to submission time. You can see how the very best entries (high 50s and 60s) came later in the competition, while the earliest submissions generally performed below-average (the average score was 34.6615).

We calculated rho = 0.243443, so there was a moderately-weak correlation between newer posts and higher scores. This is as expected, since the later bots could be tailored to the competition. Also, the best posts would probably take longer to write.

Given a sample size of 65, the standard error for rho is approximately 0.0791, so these results are significant.

Dates-Votes:

This is the interesting part - the FGITW. Here is the graph, which speaks for itself.

datesvotes

This is the FGITW in full force, you can clearly see how the most up-voted answers are typically early answers. The most-voted answers were all early in the competition. After a certain point (the latest 18), no answer has more than 2 upvotes. Although we would expect older posts to have some more votes, simply by virtue of being older, it is clear that the FGITW Effect is strong.

We calculated rho = -0.766193, which is a moderately strong correlation between earlier posts and more votes. This is objective evidence as to the strength of the FGITW, and I believe that the age of the post is one of the most important factors in determining the number of votes an answer will receive.

Scores-Votes:

While the relationship between dates and votes should show the strength of the effect, the relationship between scores and votes should show why this is a problem.

scoresvotes

The above graph shows the relationship between performance and the number of votes. With the exception of a few outliers, this third graph looks rather similar to the second graph. The submissions with lower scores actually had more votes than most of the submissions with higher scores.

We calculated rho = -0.250174, which is a moderately weak correlation between "higher score" and "fewer votes."

A negative correlation is the opposite of what we should want. Submissions with better performance should receive more votes. I suspect that this is due to the FGITW as well. As we saw above, earlier posts receive more votes, while later posts perform better. It turns out that this actually causes a negative correlation between score (performance) and votes.

\$\endgroup\$
  • 3
    \$\begingroup\$ As a side note (since this is not the main issue here), if the winning answer (or the successive winning answers in the case of a koth that runs during a few weeks) has been accepted, it would have given it more visibility (and thus theoretically more votes), hence the importance of accepting answers even if that means unaccepting later when a better answer comes along. \$\endgroup\$ – plannapus Aug 20 '14 at 6:21
  • 1
    \$\begingroup\$ @plannapus But that still only fixes the problem for one answer at a time. PhiNotPi, it should also be noted that the two outliers on the final plot are also outliers on the previous plot, so they aren't outliers because they go a lot of votes for being good, but they are outliers because they were good despite being early posts. \$\endgroup\$ – Martin Ender Aug 20 '14 at 13:24
16
\$\begingroup\$

Random Sorting

I propose that a purely random default sorting would solve the above problems. It would simply remove the bias towards any kind of answer and would distribute the votes more fairly. Of course, the existing sorting by activity will also remove the bias towards early answers, but the fact that it's deterministic is still problematic.

Especially in cases where there are so many answers that few people read all of them, some answers are just almost never seen because they are too far down the list (depending on which sorting is active). If the sorting was random by default, people still wouldn't scroll through all of them, but at least each answer would get the same number of views on average.

I would even go so far to suggest that this default sorting is not remembered between visiting different questions (currently, if you switch to sorting by activity, all questions will be sorted by activity until you switch back).

Note: random sorting has been suggested before a few years ago back on SO but without much response. I don't even think it's that useful on other SE sites, but PPCG could really use it.

Pros

  • Trivial to implement.
  • All answers are equally likely to be seen. So there is no bias towards any answers, hence it's as fair as it gets.

Cons

  • All answers are equally likely to be seen. Even the bad ones.

(However, this would lead to really bad answers getting more downvotes and hence being more likely to be deleted, resulting in less overall clutter. So maybe this is another pro after all.)

\$\endgroup\$
  • 3
    \$\begingroup\$ I think we need a compromise that gives new posts an equal footing without being random. Ideally we want good posts at the top, with posts with few votes getting some time at the top but only until they have been judged, then not seeing the top again if they were not well received in the first few days. \$\endgroup\$ – trichoplax Aug 12 '14 at 14:24
  • \$\begingroup\$ @githubphagocyte that sound's great, but might be somewhat hard to implement. Good idea though. \$\endgroup\$ – tomsmeding Aug 12 '14 at 20:37
  • 1
    \$\begingroup\$ @tomsmeding I think it might be easier than you'd think. Even something as simple as votes/time has that basic behaviour. The algorithm described by Randall Munroe in my answer also has that behaviour, although on this site we'd need to include time too as we don't use downvotes in quite the same way. \$\endgroup\$ – trichoplax Aug 12 '14 at 21:51
8
\$\begingroup\$

Randall Munroe - a statistical estimation ordering

When reddit had a similar problem with new comments getting lost at the bottom, Randall Munroe brought in a new sorting algorithm. Obviously don't just vote for this because you like Randall Munroe - it's an interesting algorithm well worth reading about before you decide.

It only requires the information that stack exchange already uses to sort by votes anyway (upvotes and downvotes), but deals elegantly with precisely the problem we're discussing. The example at the end demonstrates this nicely.

The algorithm has the best features of sorting by votes, sorting randomly, and sorting by time, without actually using any randomness or time, just upvotes and downvotes.

In short: The Randall Munroe algorithm works well on reddit because it uses both upvotes and downvotes to measure how statistically likely a post is to get to the top, and orders by the expected future position rather than just its current votes. So it doesn't need time because the downvotes give a percentage measure rather than just a plain count of upvotes The problem with applying that to stack exchange is that downvotes are a rare thing here - most answers have zero downvotes, so we might need to take a different approach that bases the statistical measure on upvotes, downvotes, and time. These are all already available to the sorting code of stack exchange so it is feasible to implement.

\$\endgroup\$
  • \$\begingroup\$ Ah - I see Firefly beat me to it by 59s... :) \$\endgroup\$ – trichoplax Aug 12 '14 at 13:42
  • \$\begingroup\$ Actually this isn't the same algorithm Firefly is describing - reddit uses "hot" and "best". This one is "best" and isn't about time but statistical confidence. \$\endgroup\$ – trichoplax Aug 12 '14 at 13:48
  • 1
    \$\begingroup\$ This is a very interesting approach though as mention in several places, our downvotes don't work like the ones on reddit, so we would either need to use time, (unique) question views, total votes or a combination thereof. I'd also suggest using a more optimistic approach to the statistics, using the best score with 5% confidence for instance, thus favoring new answers during their first few votes, even if things don't look superb. \$\endgroup\$ – overactor Aug 12 '14 at 14:53
  • 1
    \$\begingroup\$ It could also be interesting to use higher statistical confidence for lower rep and users and people that are not logged in, users with higfh rep would use a low statistical significance. This way trusted users get presented with more content that still needs a concensus and new users see the entries that can safely be assumed to be at least decent to good. \$\endgroup\$ – overactor Aug 12 '14 at 15:01
  • \$\begingroup\$ I don't think that this method would have much impact, since downvotes are really quite rare. It is entirely feasible that the top N posts to a question all have 0 downvotes, in which case the ordering would be the same and we would still encounter the problem of the first posts getting all the visibility and votes. \$\endgroup\$ – PhiNotPi Aug 14 '14 at 15:00
  • \$\begingroup\$ @PhiNotPi yes I agree. That's why I think it would need to include time too. \$\endgroup\$ – trichoplax Aug 14 '14 at 15:04
3
\$\begingroup\$

Reddit-style sorting

I don't remember exactly how it works, but Reddit's sorting algorithm (or HN's, or ...) takes both time and votes into account to present "hot" content, i.e. content climbing faster than typical or with higher than typical votecount. In practice high-voted content tends to be the norm, with new or rising content spliced in occasionally.

Our situation is a bit different: we don't want sufficiently old answers to suddenly drop quickly the same way as Reddit does, and we have an additional "active" parameter separate from the time since initial answer. Still, Reddit's (or a similar service's) algorithm could serve as a base for a good sorting algorithm. Reddit-style sorting might also be of interest in other SE subsites as well (moreso than random sorting, at least), so there's more reason for SE to provide it.


I think it'd be unfair to high-voted entries with completely random sorting, and even though we don't need the vote-based sorting for questions with something other than popularity-contest as the winning criterion it still makes sense to promote high-voted answers since they presumably follow the winning criterion well (or otherwise entertain the voters).

\$\endgroup\$
  • \$\begingroup\$ Well SE has this feature for questions already (the HNQ sidebar). Nevertheless, this would probably be a nice solution for most challenges, but for pop-cons it would still bias some answers which may or may not be fair (in this case, the bias might be reasonable, because it favours new answers which get lots of votes, which means they are likely actually better). \$\endgroup\$ – Martin Ender Aug 12 '14 at 13:45
  • \$\begingroup\$ Since you got in just before me, feel free to edit in the link from my answer and combine it into one. The "hot" sorting algorithm is actually unrelated to the one I mention, and I agree wouldn't suit our purposes, but Randall Munroe's "best" sorting algorithm maintains an order based on statistical confidence in the comment which favours new answers but only until they have enough votes to order them better - so old answers still stay high if voted highly, and never move off the top spot for long. \$\endgroup\$ – trichoplax Aug 12 '14 at 13:45
  • 1
    \$\begingroup\$ @githubphagocyte I think we shoud keep them separate as the posts are about different algorithms. Actually, my suggestion isn't so much about using an exact algorithm as coming up with a new one based on it... which is a bit vague I guess. The biggest issue with applying Reddit's Hot as is would be how old posts fall off the frontpage. \$\endgroup\$ – FireFly Aug 12 '14 at 13:48
2
\$\begingroup\$

Not enough information given

I can't agree to change the default sort at all unless more evidence is given that the basic premise is true.

PhiNotPi gives some data showing that the FGITW effect happens, and I don't doubt that. However, the only correlation it really shows is between early answers and votes. For one question. With tons of answers.

To be clear, 80 answers is just not typical. The site as a whole averages a bit under 10 answers per question. Certain challenges might garner 30+, but there aren't very many, and they just drag the average up. I don't think the default sort for all questions should be changed for the sake of those few.

...few people scroll to the bottom to see newer answers and even fewer people sort by activity.

This could be true. It also might not be. A simpler explanation to explain FGITW could be that early answers get more page views, so have a better chance of getting an upvote. Even if every user scrolls to see every answer on every question, the early ones still get seen more.

Changing the sort method won't change that, except (most likely) on questions that go 30+ answers and paginate. I think it likely that on most questions, people read at least the majority of answers, because there aren't very many anyway. In addition, since the vast majority of challenges are , there are typically <10 short answers.

\$\endgroup\$
  • 2
    \$\begingroup\$ Note: This doesn't apply to the mobile app. There are no sort options there (votes only), and you can't even get to page 2 of answers from what I can tell. In addition, you can't jump to the newest answer/edit from the main page like you can on the actual site. It's a damn mess. \$\endgroup\$ – Geobits Aug 20 '14 at 13:25
0
\$\begingroup\$

Vote Order + Semi-Random Permutation

A combination of votes and a randomizer might be best.

One method of accomplishing this is to take the placement of each answer (1, 2, 3, etc.) and add a random real number 0-5 to that number. Then, sorting by these new numbers gives the new order. The goal is to create a semi-random permutation of the order such that no answer is too far from its original location.

Pros:

  • The most upvoted answers are still near the top, with just enough randomization that the not-quite-as-upvoted answers also get the change to be near the top.
  • Answers with no votes are still lower down, but aren't always dead last.
  • The "catchy" answers are still visible to random visitors, which helps create interest in this site.
  • Simpler than other methods.
  • Changing the 0-5 range to something bigger (like 0-10) is an easy way to increase randomness. As the range becomes larger, it is closer to the random sorting. If is is smaller, it is closer to vote order.

Cons:

  • Newer answers still have an advantage if the choice of range is small, as highly-voted answers will remain visible.
  • If there are a ton of 0-score answers, most of those answers will end up being swapped around with each other.
\$\endgroup\$
  • \$\begingroup\$ The con of newer answers/highly voted answers still having the advantage outweighs the pros in this one, and thus won't solve the problem. \$\endgroup\$ – Isiah Meadows Aug 14 '14 at 23:30
-1
\$\begingroup\$

Why sorting by votes is a good idea

PPCG benefits a lot from people stumbeling upon popular questions, when they open thsi question, we definitely want to showcase the best and most interesting answers. In many categories, there is a very clear difference between good and interesting answers. This answer explains this better than I ever could..

Why sorting by votes is a terrible idea

Well, basically, for all the reasons you mentioned, late answers don't get the attention they deserve and the gap between early answers and late answers only gets bigger over time (especially when answers exceed one page).

Problems with random sorting

While it is more fair and we could still let new users (or people who aren't logged in) keep their ranking by votes. It's still valuable to have answers which are obviously good near the top and terrible answers (or very unremarkable ones) buried. Random sorting won't do this and whenever I visit a question with a lot of answers (the sort of question where this ranking would be desirable) I'll be presented with mostly 'meh' answers.

An alternative Ranking system

We could opt for a ranking systems that still largely bases on votes but favors newer answers, similar to how hot network questions work A possible formula for this could look something like: votes on this anwer/votes on all answers + a*votes on this anwer/votes on all answer since this answer was posted where a is a carefully selected weight. This would give late answers a fighting chance when they are first posted. If they gather some upvotes then, they'll stay up and even float up, if being at the top doesn't gather them upvotes though, they'll drop down again.

A random addend could also be added to this score to bring some life into things.

\$\endgroup\$
  • \$\begingroup\$ Two things about the specifics of your ranking: you don't need to divide the first term by all votes because it's just a constant that's the same for all answers (and hence can be absorbed into a). Secondly, it would need an additional tweak to place completely new answer without any votes at the top (maybe just count the post itself as one upvote for the purposes of this ranking). Regardless, the same comment applies as posted on FireFly's proposal. \$\endgroup\$ – Martin Ender Aug 12 '14 at 13:48
  • \$\begingroup\$ @MartinBüttner Good point, the formula was more to illustrate the idea rather than a serious suggestion. The formula in githubphagocyte's answer is better by a wide margin. Though it's not time based, it still operates under teh same assumption, that new answers that gathered relatrively many upvotes are good enough to be at the top without having the same absolute vote count of older asnwers. \$\endgroup\$ – overactor Aug 12 '14 at 13:56
  • \$\begingroup\$ My suggestion will probably also need to include time since we don't have the same level of downvotes that reddit does (as ours mean something different). \$\endgroup\$ – trichoplax Aug 12 '14 at 14:27
  • \$\begingroup\$ The only problem with this approach is that although it gives new answers the advantage they need early on, it gives a permanent advantage over all older answers, even after the passage of time when they should be being treated near to equally. This difference will be most pronounced when there are no more votes coming in, but the existing vote distribution can never sort the answers fairly even after a long period of time. \$\endgroup\$ – trichoplax Aug 12 '14 at 14:31
  • \$\begingroup\$ @githubphagocyte you're right, when time approaches infinite (or a few weeks) answers should be treated almost equally. \$\endgroup\$ – overactor Aug 12 '14 at 14:35
  • \$\begingroup\$ Example: 1 answer posted, 50% of votes ever come in. 2nd answer comes in, the remaining 50% of votes ever are shared equally by the 2 equally good answers. 1st answer has 0.75 + 0.75a. 2nd answer has 0.25 + 0.5a. This may be the other way around from my intuition... It doesn't settle down to a fair ratio though. \$\endgroup\$ – trichoplax Aug 12 '14 at 14:35
  • \$\begingroup\$ I think something as simple as Martin's votes/time gives plenty of advantage early on but still evens out with increasing time. \$\endgroup\$ – trichoplax Aug 12 '14 at 14:37
-1
\$\begingroup\$

Rate of votes compared to placement on page?

This is sort of a long-shot thinking-out-loud idea, but I feel that a "hotness" ranking should be determined by two things:

  • The rate of votes that the answer is experiencing
  • The answer's history of placement on the page

My logic is that the rate of votes is determined by two primary factors:

  • The quality of that answer (which may be very roughly approximated by votes/viewer)
  • The visibility of that answer (which determines the rate of new viewers)

Basically, we want to calculate votes/viewer = rate of votes / rate of new viewers.

I think the ranking system should try to determine which answers have the highest votes/viewer ratio. The ranking algorithm would look at the rate of votes and try to determine how much of that rate is due to answer quality and how much is due to visibility.

What we would need to do is take an empirical measurement of how an answer's placement on the page affects the rate of votes it is experiencing. I expect the visibility to decrease with lower placemement, with large decreases once the answers are on second/third pages.

Perhaps here is one way of calculating it. This is by no means the best way, just a possible realization of the main idea of this post. The idea of taking answer placement into account can be incorporated into several of the other suggestions.

A = answer, N = placement position number, votes(A) = all votes, score(A) = up - down

expectedrate(N) = average rate of votes experienced by all answers in Nth placement position
timespent(A,N) = the total time spent by answer A in placement position N
expectedvotes(A) = summation( timespent(A,N) * expectedrate(N) )
result(A) = score(A) * votes(A) / expectedvotes(A)

Basically, the answer's ranking is affected by how that answer's number of votes compares to the expected number of votes. The expected number of votes takes into account the how long the answer spent in each position and the rate of votes experienced by each position.

Pros:

  • High-quality answers with very low visibility will have that taken into account and rise to the top very quickly
  • Highly-visible answers with the most upvotes will be penalized if the only reason for their upvotes is the visibility and not because of above average quality.

Cons:

  • Might not work when the pool of voters is very small and starts to diminish.
  • The popularity of the question probably determines how influential answer placement is.
  • Hard to calculate or measure.
\$\endgroup\$
  • \$\begingroup\$ How do you work out the feedback loop of "placement on page" -> "new ranking" -> "placement on page" -> "new ranking" -> ...? :) \$\endgroup\$ – Martin Ender Aug 12 '14 at 14:49
  • \$\begingroup\$ I tried to resolve that loop by taking into account the amount of time the answer spent at all answer positions in the past to determine the expected votes (I've changed it from current placement to placement history). \$\endgroup\$ – PhiNotPi Aug 12 '14 at 14:51
  • \$\begingroup\$ I would use the views on the question while teh answer wa sin a certain position rather than the time spent there. \$\endgroup\$ – overactor Aug 12 '14 at 15:08
-1
\$\begingroup\$

Sort by votes minus amount of time since post

Here's a fancy userscript for your testing pleasures (doesn't work for multiple pages):

function rank(x) {
  var score = parseInt($('.vote-count-post', x).text());
  var milliseconds = new Date($('.post-signature:last-child .user-action-time span.relativetime', x).attr('title')) * 1,
    hours = milliseconds / 1000 / 60 / 60;
  return score - hours; // or some other factor
}

var ranked = $('.answer').sort(function(a, b) { return rank(a) < rank(b) ? 1 : -1; });
$('#answers').empty().append(ranked);

This will rank a +20 answer posted 10 hours ago the same as a +15 answer posted 5 hours ago. I.e., it will subtract the amount of hours since post from the score. (Technically, hours since the epoch, but they work the same in practice.)

Of course, the amount that time has to do with the sorting could be changed, but 1 hour == 1 upvote seems like a good factor, from brief testing.

Advantages:

  • Very simple.
  • Can be tweaked easily.
  • You see new answers when you come back in a day or two.
  • Answers with a ridiculous amount of upvotes still stay at the top.
  • Answers posted, say, a few months later will have a very large advantage.

Disadvantages:

  • You see new answers when you come back in a day or two.
  • Answers with a ridiculous amount of upvotes still stay at the top.
  • Answers posted, say, a few months later will have a very large advantage.
  • Not very fair for users who post on times like the weekend, whose posts sink down and don't have much of a chance at rising back up.
  • Can still be affected by bias.
\$\endgroup\$
  • 2
    \$\begingroup\$ You'd need an insane amount of upvotes to stay on top over an answer (even if downvoted) posted a month later. I don't think any linear relationship is going do here. But the userscript is really useful! :) \$\endgroup\$ – Martin Ender Aug 12 '14 at 15:41
  • \$\begingroup\$ @MartinBüttner ... I probably should have thought of that. -_- On the other hand, people don't usually come back to challenges posted a month later, and maybe the sorting could be changed to pure votes after, say, no new answers in 3 days, or something along those lines. Also, isn't the "staying on top" thing exactly what you're trying to avoid? In fact, this could be a good thing, as it would mean that you can still get a few votes by answering a 3 month old question with a great answer. \$\endgroup\$ – Doorknob Aug 12 '14 at 15:45
  • \$\begingroup\$ People coming back to old answers happens surprisingly often. I think I see it about once a week that some old challenge gets an answer and then another handful due to being bumped to the front page. What I'm saying is that a new answer on such an old question should go to the top, but should bubble down if it turns out to be worse than the previous top voted answer. With the linear relationship it's impossible to be voted below any top-voted answer after a month. \$\endgroup\$ – Martin Ender Aug 12 '14 at 15:50
-1
\$\begingroup\$

highest = vote count for highest voted answer on the same question

hours = number of hours since answering

votes = number of votes on your answer

sorting: max(0, highest - votes - hours) + votes

By taking into account the highest voted answer, it ensures that all answers appear at the top, at least when first posted. "max" makes sure that the time since posting only benefits the post for a suitable period of time, during which users have the chance to vote on it. And in the end it still comes back to votes, but at least all answers would have had a chance to be upvoted.

note, when I say votes above, of course I mean upvotes - downvotes. Also, hours could be replaced with "number of hours since answering * n", to tweak it.

\$\endgroup\$
-1
\$\begingroup\$

Slightly weighted random sorting

This has in some form or another been suggested, but the weight was a little too inflexibly strong in all of them IMO. How about this algorithm (in Python pseudocode)?

# an arbitrary constant signifying the weight given to either the votes
# (greater than 1), the time (0 < n < 1), or neither (1).
ratio_weight = 1

# an arbitrary constant signifying the weight given to the statistics of
# each entry (greater than 1), the random element (0 < n < 1), or
# neither (1).
stats_weight = 1

def order_list(entry_list):
  total_votes = reduce(entry_list, lambda x, y: x + y)

  # this is the main weighting of the sorting. `entry` is a tuple in the
  # form of (answer_entry, random_index_to_use).
  def get_sorting_index(entry):
    e = entry[0]
    return entry[1] + stats_weight * (1 -
      (ratio_weight * (e.votes/e.time)) / total_votes)

  # get a list of random indices to work with
  random_indices = list(range(len(entry_list)))
  random.shuffle(random_indices)

  # get a zipped list with the random indices
  entry_list = zip(entry_list, random_indices)

  # sort using the weighted, randomized entry indices
  entry_list.sort(key=lambda entry: get_sorting_index(entry))
  return [x[0] for x in entry_list] # strip random sorting index

Please comment if you think something doesn't look quite right (either bug or proposal).

\$\endgroup\$
-1
\$\begingroup\$

Merge Active + Votes + Random Rankings using a Condorcet method

I think each of these has its benefits, which can be merged.

Active sorting means that newer answers are more visible and given a chance to be viewed. This helps to counteract the bias towards older answers. It allows a sort of kick-start to new answers.

Votes sorting means that new users will see primarily see quality posts as opposed to downvoted posts. We don't want to place too much faith in here, but we can't simply ignore it.

Random sorting helps to scramble up the order. It is blind to quality, which can be both a pro and a con.

This algorithm would take the three rankings and use them as the votes in a Condercet election. There would be X copies of the active ranking, Y copies of the votes ranking, and Z copies of the random ranking (each copy is the same ordering). The X, Y, and Z are the weights. There are several Condercet-compliant methods to merge these, but something like Ranked Pairs could work decently.

Pros:

  • The weightings are easy flexible, offering a continuum of options all the way to complete randomness.
  • With the correct weights, the new posts with few votes are on the same footing as the old posts with many votes.
  • This system favors the newer posts with many votes and is against the older posts with no votes.

Cons:

  • Finding the correct weightings would be an empirical thing.
  • The active ranking can be kinda chaotic sometimes when older posts are edited. It can give bad posts a second chance at greatness after being fixed, but could motivate people to create pointless edits. We could replace Active with Newest if this is a concern with little impact on the algorithm.
\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .