14
\$\begingroup\$

In a recent challenge, almost all answers use an algorithm that yields the wrong results due to floating point errors.

The challenge is/was not unclear, but there was no test case that led to the wrong results when most answers were posted.

Now, most of the answers are invalid, and should therefore by community consensus be deleted until they are fixed. However, asking a moderator to delete almost all answers on a challenge doesn't seem like the right course of action.

What to do?

Note: Most (not all) answers got comments saying they were invalid 15-25 hours ago. They are not yet updated.

\$\endgroup\$

3 Answers 3

Reset to default
15
\$\begingroup\$

The challenge was not unclear on that test case

log10(1000) = 3 isn't some weird constraint that the challenge imposed, or unexpected behavior that the OP ruled on where it could have gone one of two ways. It's simply the correct answer to the challenge with its existing specification. Adding it to the test cases doesn't cause answers to become valid or stop being valid, it's just a useful edge-case that happens to show that many existing answers were never truly valid. This isn't a case where the specification was changed due to the OP not liking some answers or the like.

Any invalid answer needs an individual notice that they are invalid

A user won't know that an answer was discovered as invalid because of comments on the challenge or any meta post. The method I know of is to comment on any post which is discovered to not meet the spec, or even propose an edit which resolves the issue. Just using age, or age since the issue was first discovered on one answer, is not useful because the user doesn't know that it happened. I don't necessarily have some good resolution for determining who is responsible for that check, but we shouldn't delete answers made in good faith without giving the user a chance to fix it (though, if something does get deleted, the user still has an opportunity to fix it afterwards).

Deleting isn't the only fix

Sometimes an answer on other challenges is constrained to some seemingly-arbitrary upper bound of input, often having to do with the language's number representations. I am not opposed to answers that are "only guaranteed to work up to inputs of 1000 due to floating point precision errors" in a roughly similar manner. It would be prudent to establish a firm upper bound for it, i.e. you should know with certainty what the lowest number pair that produces incorrect output is. In reality most languages represent numbers in a format with a strict upper bound, or variable precision, and so almost all answers in almost all challenges involving numerical input without upper bounds there is some input that is technically within spec but will cause every answer to break. (1000,10) isn't a particularly high input to be causing such an error, but as long as answers identify such bounds I don't think they should count as invalid.

\$\endgroup\$
8
  • \$\begingroup\$ "a strict upper bound, or variable precision" -- something is wrong here? Maybe "fixed precision"? \$\endgroup\$
    – user202729
    Commented Apr 20, 2018 at 12:21
  • \$\begingroup\$ @user202729 Variable precision means that the precision in some ranges is different from the precision in other ranges. For example, the "floating point" data type in most languages can hold both very small and very large numbers, but large numbers lose precision. The point of that sentence is that problems can be caused by either a number which is too large to be stored at all, or a number which is too large to be stored with enough precision. \$\endgroup\$
    – Kamil Drakari
    Commented Apr 20, 2018 at 13:21
  • 3
    \$\begingroup\$ Large numbers don't lose precision, they lose accuracy. \$\endgroup\$
    – user202729
    Commented Apr 20, 2018 at 13:42
  • 1
    \$\begingroup\$ @user202729 I don't have the actual numbers memorized, but in my understanding of the terms a data type which can express exactly 0.5 but cannot express exactly 1000000000.5 means it loses precision at higher values. \$\endgroup\$
    – Kamil Drakari
    Commented Apr 20, 2018 at 13:51
  • \$\begingroup\$ @KamilDrakari Most floating-point numbers get more coarse the further you move away from zero, which means that there are larger and larger gaps of values which cannot be represented. However, most implementations are statistically stable, meaning that their set of representable values will not change from call to call. Therefore they are more accurate when dealing with values around zero, though their precision is at any value perfectly spot-on. \$\endgroup\$
    – Jonathan Frech
    Commented Apr 20, 2018 at 19:58
  • 1
    \$\begingroup\$ @JonathanFrech I agree with you about how floating point numbers work at a basic level, but I disagree with your conclusion. To my knowledge, there isn't a floating point implementation that, when presented with a number that doesn't perfectly line up with the discrete points that are storable just says "No, I can't represent that". Instead, they do represent that number, it's just that there's a small difference between the representation and the original value. I consider that an imprecise representation. \$\endgroup\$
    – Kamil Drakari
    Commented Apr 20, 2018 at 20:12
  • \$\begingroup\$ @KamilDrakari I would call that behaviour inaccurate. \$\endgroup\$
    – Jonathan Frech
    Commented Apr 20, 2018 at 20:35
  • \$\begingroup\$ Given that the title is "Integer logarithms" answers that fail due to using float point numbers should be deleted or fixed unless that language only operates with floats and doesn't have an integer type. \$\endgroup\$
    – ბიმო
    Commented Apr 21, 2018 at 16:43
-6
\$\begingroup\$

I think the challenge is unclear. If you want to avoid that people use floating point arithmetic that might result in rounding errors this should explicitly be written as a requirement, and if it is ok to find results up to floating point inaccuracies it should also be mentioned. This challenge didn't do either of that. The fact that there are so many "wrong" answers means that the challenge just was not good enough. And this would most certainly have been prevented if the challenge was submitted to the sandbox in the first place.

What to do? In my opinion it doesn't make any sense to change the specs after so many people did answer it with the same interpretation. I suggest closing it as unclear.

\$\endgroup\$
4
  • 2
    \$\begingroup\$ I agree with you, but to be fair: The specs wouldn't really be changed, only clarified. The first sentence makes it pretty clear that 1000, 10 should give 3. \$\endgroup\$
    – Stewie Griffin
    Commented Apr 19, 2018 at 18:24
  • \$\begingroup\$ Submissions in languages with a primitive integer type that can overflow are often seen and fail to pass some test cases. Why should one penalize a submission based on its language-based floating-point limits? \$\endgroup\$
    – Jonathan Frech
    Commented Apr 19, 2018 at 18:26
  • \$\begingroup\$ @JonathanFrech floating point numbers are well able to store the numbers 3, 10 and 1000, so the problem is the algorithm (i.e. using an imprecise logarithm implementation), not floating point numbers. I fail to see what "limit" is getting in the way except doing things to a bag logarithm implementation it wasn't meant for. \$\endgroup\$
    – dzaima
    Commented Apr 20, 2018 at 10:14
  • \$\begingroup\$ @dzaima I partially see your point, though one would expect for log_b(a) === ln(a)/ln(b) to hold. That it results in some inaccuracies when attempting to floor it, I claim, could be seen as a language limitation which does not invalidate the answer. \$\endgroup\$
    – Jonathan Frech
    Commented Apr 20, 2018 at 13:08
-7
\$\begingroup\$

The challenge is/was not unclear

Yes, it was. It was necessary to reverse engineer from the test cases the fact that ^ was an exponentiation operator and not an exclusive or.

With such a trivial question I don't think that anything of value would be lost if the entire question were deleted and all the answers with it, but since I don't expect that to be a popular solution I will instead observe that the consensus which I think you're referring to says

If a sufficient amount of time has passed (say 48 hours) or the answer was deliberately invalid, flag the answer as in need of moderator intervention, providing an explanation of why it is invalid.

Give people a bit longer to fix their answers before flagging, or downvoting and voting to delete for those which haven't acquired upvotes.

\$\endgroup\$
5
  • \$\begingroup\$ Wow. If giving people time to fix the problem goes to +0/-3 then maybe I should post an answer which advocates deleting the question. \$\endgroup\$
    – Peter Taylor
    Commented Apr 19, 2018 at 20:03
  • 4
    \$\begingroup\$ I agree with the part about giving time so I'm upvoting this. Guessing that ^ means exponentiation and not an exclusive or wasn't exactly difficult, so the reverse engineering part is not really that big of a deal in my book. \$\endgroup\$
    – Stewie Griffin
    Commented Apr 19, 2018 at 20:21
  • \$\begingroup\$ For the record, I guess the downvotes are for: With such a trivial question I don't think that anything of value would be lost if the entire question were deleted and all the answers with it, not for suggesting to give people time. \$\endgroup\$
    – Stewie Griffin
    Commented Apr 20, 2018 at 11:20
  • \$\begingroup\$ @Stewie, so you think that there are a lot of people on meta who don't understand the meaning of the word "instead"? Could well be... \$\endgroup\$
    – Peter Taylor
    Commented Apr 20, 2018 at 14:33
  • \$\begingroup\$ You're essentially saying that the effort of every single one who answered that challenge is worthless. You might be right, but I think that's the part people don't like about this answer. \$\endgroup\$
    – Stewie Griffin
    Commented Apr 20, 2018 at 14:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .