I wrote a challenge about sampling a probability distribution: Sample the Pareto Distribution
This probability distribution is defined over the real numbers. However, submissions do not have access to true real numbers, but merely imprecise approximations of real numbers. I wanted to allow submissions using floating point numbers. However, I couldn't find a satisfying way to specify what submissions were allowed.
In particular, many submissions would, very rarely, output
NaN, due to the fact that their randomly sampled floating point number would ocassionally be 0, and
1/0 would be output.
Likewise, in my challenge specification, I said that the output should be greater than 1 with probability 1, which is part of a natural definition of the distribution over the reals. However, many submissions would output exactly 1 with non-zero probability, again due to the fact that the floats are different from the reals.
What's the best way to say in the challenge that it's OK if submissions don't perfectly sample the true distribution due to floating point imprecision?
In general, how should challenges say that floating point imprecision is allowed?
Say that a submission must be written so that if the floating point type was replaced by a true real-number type, and all appropriate built-ins were likewise adjusted, then the program's output would exactly match the specified distribution.
- This is unsatisfying because it asks about a hypothetical analysis of the program, which isn't testable.
Specify exactly how far from the idealized output the submission is allowed to be. For instance, it could say that the difference between the probability of the submission outputting a value in an interval of size
1e-6must not differ from the true probability mass in that interval by more than
- This is unsatisfying because it's very technical and puts a heavy burden on the answerer. In addition, it might be very hard to come up with a perfect specification. Finally, this doesn't add much to the quality of the question.
Just handwave it with something like "The submission shouldn't be very far the ideal distribution."
- This is unsatisfying because it's not objective, and may lead to disagreements that a specification should make clear.
How should challenges specify that floating-point imprecision is allowed?