Test a question

Question

The sandbox is a helpful feature^{[citation needed]} but often I have a question that I feel needs to be tested before it can be asked. This is usually the case with a cops-and-robbers question or questions with unique scoring systems. Its hard to predict how you specifications will stack up in the wild making feedback not sufficient. As an example my latest sandboxed challenge is an answer-chaining question that probably needs some fine tuning to make sure it will be neither extremely trivial or impossibly difficult to create new answers.

Since I can't really fine tune the specifications of a challenge after it is release, I often feel like it would be nice to test out the challenge with a soft-release.

Is there any way I might do this? Perhaps I should try the challenge in a chat room? Any suggestions are welcome.

Answer 1

Invite "Beta Testers" To Try It

In cases where a challenge might seem like a good idea on paper but really requires it to be executed and tested to be sure, one could, publicly or privately, invite one or more people to do so prior to posting it to main with the condition that those "beta testers" not be permitted to compete when the challenge is posted to main as they would have had the advantage of having had more time to work on their solutions (although, that may be dependent on the challenge type).

With regards to how that should be implemented:

• When posting to Sandbox, you could include a notice that you're looking for people to test drive your challenge and, possibly create a private chat for you and your testers.
• A chatroom could be set up that would either be invite-only or have a higher rep requirement than normal rooms (assuming either is possible) to serve this purpose. This would have the added benefit of challenges that end up being "bad" never reaching the Sandbox thereby reducing a little bit of the clutter.
• Create a new Meta question, of a similar format to the Sandbox, but specifically for those challenges that need to be taken for a test drive. (Yeah, I'm not a fan of that one, either!)

Footnote: Yes, I'm well aware that people do use the Sandbox to get a headstart on challenges in order to be the FGITW (or whatever their reasons) - I admit to being guilty of it myself when I can't find a challenge on Main that sparks my interest - but, if we adopt my proposal, it should help benefit the community rather than the individual.