The tag wiki has a few pointers, as does the meta post Geobits linked. I don't know if this is the best way to do it, but here is the general practice we've been using so far (some of these are assuming 1-vs-1-style cops and robbers challenges like all the ones we've had so far; in principle it should be possible to have n-vs-1, 1-vs-n or n-vs-n challenges as well):
- Post separate challenges for the cops and the robbers, linking them to each other. a) this allows the robbers to include explanations with their submissions (or to format longer pieces of code properly) and b) the robbers can gain reputation for their answers and an answer can be accepted.
- Robbers should link to the cop they crack and post a comment on that answer linking back to their own. The cop should then edit their answer to indicate that it has been cracked.
- Robbers generally don't have to find the particular solution the cop intended, but just any one solution that satisfies the specification (otherwise, this would amount to guessing in most cases).
- To ensure that all cop answers are valid (i.e. crackable), set a time limit like 7 days. If this time limit passes after a cop has been posted, the cop may reveal their solution at which point it is safe and can no longer be cracked. I usually allow robbers to crack cops after the end of the time limit until the cop actually reveals the solution (in order to incentivise cops to reveal their solutions as soon as possible).
- It usually makes sense to require cops to be deterministic.
- For the cops, each answer is usually scored individually (as opposed to determine an aggregate score for each user), and increasing that score should make it more likely that the answer can be cracked. A good choice is usually the normal code-golf score, because for most cops and robbers challenges, shorter submissions give fewer options to guard a valid solution.
- For the robbers, people usually form an aggregate score per user (although this is by no means necessary). A common choice is simply to use the largest number of cracked cops, using a tie breaker based on the cop scores. For the robbers it's tempting to use the inverse score of the cops (e.g. a longer cop submission is usually harder to crack, so it should give fewer points for the cops and more for the robbers), but be careful with this: the incentive for the robbers is then to focus on those cop answers which aren't likely to win anyway. One example with such a scoring is this challenge.
- Unlike our other challenges, it makes sense to close the challenge at some point (e.g. once you haven't received a new cop for a week or two). Once the challenge loses attention (from HNQ and being discussed in chat), it's very easy for someone to slip in an easy cop that is simply not cracked because no one noticed it.
Finally, there's one big unsolved problem for cops and robbers challenges: almost every challenge is susceptible to cops that are simply safe for cryptographic reasons. E.g. it's very easy to encode a massive prime factorisation in a cop, so that even though it's clear how the solution looks in principle, the exact solution simply can't be found within the time limit because it would be computationally too expensive. Similarly, one could make use of a PRNG with a fixed (but hidden) seed, which would still make for a deterministic answer. At this point I'm not aware of a good solution and really hope that one day someone will come up with one, because this is what is stopping me from posting the cops and robbers challenges I have in the sandbox myself. The only fix we currently know is to disallow PRNGs, hash functions, encoding functions (like base 64) and cryptographic functions, but it's always possible to implement these things yourself for a few bytes, especially in golfing languages. Ideally cops and robbers would be about figuring out the cop's approach or algorithm, not the parameters (whose search space is way too large), but it's difficult to put that in an objective rule.
