How do we handle answers that have later become invalid due to external circumstances?

Question

From time to time, answers can be posted to the site which, at the time are valid but later become invalid without any fault of the original answerer. For example, a number of answers to this challenge use domains such as http://ppcg.lol and http://ppcg.ga, which, at the time, worked, but have since expired and so the answers are technically invalid.

Typically, I would advocate for always deleting invalid answers. However, in this specific case, I'm not entirely sure that deletion is the correct approach:

• For the linked challenge specifically, a total of 4 out of the 50 answers do not use a broken URL, meaning that 46 answers would either need to be updated (unlikely, given the question is almost 7 years old) or deleted. Such mass deletion of answers would be incredibly disruptive to the challenge.

• In general, the reason that answers like this are invalid are not any fault of the original answerers. Rather, it's because of external factors out of their control that invalidate their answers, such as domains expiring or even languages being updated.

  In this case, we would effectively be requiring answers to keep up to date over multiple years, or else be marked invalid by something out of the answerers' control.

These are simply my personal opinions on the matter though, and feel free to share your suggestions as well. What does the community believe is the appropriate course of action here?

---

In fact, this issue appears to be not uncommon, especially for internet challenges. For example,

• All answers here are invalid, as https://strawpoll.me/ is closing down
  • Similarly, if https://i.imgur.com, https://goo.gl, or various other sites ever go down, entire challenges would be invalidated.
• Answers here are susceptible at any time to being invalidated by another answer being posted with the same search text/by the same user

Answer 1

Answers invalidated through external circumstances should not be deleted as invalid, but should be clearly marked as no longer working

That said, they should be encouraged to be updated to be fully valid.

I think this is the most fair way to approach such answers. As indicated in the question, I don't fully believe that answers invalidated through changes entirely out of their control should be deleted. That said, such answers don't complete the challenge as written, and we have a rule that invalid answers should be removed from the site.

Therefore, I believe the best compromise here is to leave these answers undeleted, but clearly indicate (through an edit, ideally, but at least with a comment) that they have since been invalidated due to something changing, as well as leave a comment informing that answerer that their answer should be updated.

For the specific challenge linked, especially given the sheer number of "invalidated" answers, I think that editing in a note into the challenge body and encouraging each answer in a comment to update their answer is the appropriate course of action.

Answer 2

Depends on the external circumstance

There are two different types of external circumstances here: reasonably predictable and seriously problematic.

A reasonably predictable external circumstance would be a custom domain registered by a single code-golf user expiring/one day falling out of registration. In this example, an individual can't be expected to host a single domain forever, because people move on, forget things exist or simply decide that it's no longer worth the investment. If such an external circumstance were to arise, then that's on the answerers for utilising something that kinda sorta resembles the loophole disallowing creating a language just for the challenge. These posts should be given a chance to fix the answer based on new information and then handled as any invalid answer would be.

A seriously problematic external circumstance is like the case of a major service like google or imgur going down, or an otherwise reliable service being shutdown. In these cases, either there'll be dedicated teams of software engineers working on restoring these services (or providing an alternative that can be used, in which case, answers should be handled like normal invalid answers and given a chance to update their answer), or the question will need to be looked at to see why a lot of answers refer to a now invalid source. In the example of the strawpoll question, it'll need to be closed (and possibly locked), as it can no longer be objectively answered.

Basically, reasonably predictable external circumstances should be treated as regular invalid answers, while seriously problematic external circumstances need further attention.

Answer 3

Language updates breaking answers is no problem

If I try to run an answer from ten years ago using a modern compiler/interpreter and it doesn't work, a reasonable assumption is that I should use a compiler/interpreter from ten years ago. It is unreasonable to expect anyone to update answers that are broken in this fashion, as that can mean hundreds or thousands of updates all at once. Additionally, any updates would be likely to alter the byte count of the answer, which is a can of worms best left unopened (unless of course the original answerer wishes to take advantage of new features to reduce the byte count).

Answer 4

The answers are invalid ex tunc.

Suppose an implementation of a task uses the system clock and fulfills all task’s requirements exactly on 1673141000 seconds UNIX epoch time. Technically speaking the answer is correct, however, since the implementation depends on it, the system clock is an input.

By analogy the utilization of external URL shorteners constitute hidden data, too. If the answer does not produce this data, e. g. an adequate /etc/hosts or BIND9 configuration entry (and thus contribute to the scoring), it is invalid.