While to-be-posted challenges get feedback in the Sandbox (ideally), we don't usually reflect on challenges after they have run. And while issues and ambiguities in challenges do get pointed out, we rarely talk about what worked well in a challenge, or draw deeper lessons. I think doing so would help us become better challenge writers.
This thread is for post-mortems on your own challenges after they seem to be mostly done getting answers. Or, others' challenges, if they ask for it.
I'm not happy with this one. I feel like I could have made something better with the idea.
As a choose-your-own-method type challenge, the appeal is for solvers to look for the map of directions that's golfiest, then implement it. The risk in writing these is that you don't really know what solvers are going to do. Maybe someone finds something that trivializes the challenges and everyone just copies it.
At least, nothing that bad happened here. Credit for this goes to S.S. Anne who took the time to look for solutions to an earlier version in the Sandbox post (deleted). This earlier version had the walk be in steps of up/down/left/right, rather than turning with a facing direction. S.S. Anne came up with solutions that were shorter than I expected, and inspired me to come up with yet more minimal versions that seemed easy to port to most languages. Though these took cleverness to come up with, they would trivialize the challenge once someone posted them.
This was a big saving throw that stopped me from posting a pretty bad challenge, even after it got a good a number of Sandbox upvotes. Really I should have taken time to think up optimized solutions from the get-go. I made the challenge more complicated by including the concept of facing direction and having movement be relative to it. Still, I didn't foresee most of the methods used in the new challenge, many of which involved turning only one direction.
The challenge turned out fine, and I think golfers had some interesting optimization. I think solvers enjoyed it. Still, I feel like the concept had more potential and the challenge could have been a lot better.
Not much to say about it, but I figured I'd post it here anyway as an example of how a reasonably nice challenge can be ruined by cumbersome rules as only allowing full programs and having input validation.
This was my very first challenge I posted when I came to this Stackexchange back in April 2016. The challenge is to post an animation of 9 different states of a stick figure, based on a given input of digits. Sounds quite straight-forward, but here are all the things I did completely wrong with this first challenge of mine:
0); there were no repeating digits adjacent to each other (with wraparound).. All things that distract from an otherwise good challenge.Of course, this was just the very first challenge of mine, so that it contains flaws like this isn't too surprising. Because of this challenge I learned that you can best assume all input given is valid (unless input validation is a core part of the challenge); to use the default of allowing both full programs AND functions; and to have flexible I/O. Which is also the reason I post it here, so other (relatively new users) that might read this will know what type of things to avoid when posting a challenge. (And always use the Sandbox before posting a challenge to main, no matter whether you're an experienced user or a new one, since there can always be something you might have overlooked.)
I was really worried about this one. I think it worked out well, but only because I got lucky and solvers were inventive.
A challenge stapling together unrelated challenges is a bad idea. Even worse is if those sub-challenges are intentionally generic and overdone. The spec needs to be long and complicated to include both the "frame challenge" and the sub-challenges with their technicalities.
Gosh, I didn't even Sandbox it. What a disaster waiting to happen. Sure, the concept is cute, but would solvers enjoy it? Would anyone even answer?
I had thought of a question like this for a while. My original motivation was to parody a common strain of overdone challenges that used to plague the site. I considered making such a question as a mostly-joking dupe target for them. Perhaps by the time I posted it, many users who weren't around then didn't get the "reference", which suggests I should have posted this challenge years ago when I first thought of the idea.
I tried to get across the satirical humor in the challenge but I was worried that my initial attempts seemed too mean, and that someone could interpret it as making fun of their challenge in particular, so I backed off. Still, I feel like I missed an opportunity to lean into the absurdity. Like, if the main appeal of a challenge is that the idea is witty, that humor better actually come across. It's unusual for me to write a thematic challenge like this, so I don't have a good sense on how much to lean on theme for them.
Despite my worries, the challenge got a good number of answers with lots of interesting golfing and optimizations.
I found it impressive how answerers managed to reuse code for computing Fibonacci number/triangular number/multiple of 3 (B). I didn't see that coming for the most part. and in retrospect I got really lucky with the three sequences I chose kind-of arbitrarily. Kudos to solvers for finding some pretty parallel ways to generate the sequences and digit checks, and optimizing those that a bunch. Because of that, answers didn't feel like solutions to separate sub-challenges pasted together.
I'm glad that I resisted the urge to include primes for being so classic. I also originally had more and harder sub-challenge parts, including variable bases, and thankfully I cut down on those.
I tried really hard to write a clear spec given how weird and long it was, and I think I succeeded there. I thought a lot on how to present the different options before coming up with the fill-in-the-blank presentation, which then inspired the Mad Libs title. Still, I wonder if Sandboxing it would have led it to suggestions to make it clearer.
One helpful last-minute change was to assign letters A/B/C to each blank, which made it easier to refer to them in the details and let solvers refer to them in answers or comments. I guess a lesson is to think about clarity not just within spec itself, but for supporting answerers in writing clear explanations. Along these lines, I've found it useful when challenges name variables or concepts so that different answers can refer to them in a consistent way, which in turn makes it easier for readers to understand explanations.
Despite the challenge working out in hindsight, I think it was still probably a bad idea. Maybe it's the type of challenge that can be done once then never again. At least I definitely should have Sandboxed it.
I initially posted this challenge in the Sandbox back in January 2017 when I had a rough idea about involving merge errors into a challenge. Over the course of two years, I modified it every now and then when I looked at the Browse your pending proposals list, but mostly deemed it unfit for an actual challenge. I don't know how many times I've thought about just deleting it, but mid-2019 I decided to finally polish it so it was at least suitable enough for a challenge to be posted, which I did in June 2019, 2.5 year after I had put it in the Sandbox.
Given the just 12 votes and three answers, it wasn't too big of a succes, as I was expecting to be completely honest. But I'm still happy I've posted the challenge instead of deleting it completely in the Sandbox, since the current answers all three use a different technique to solve it. I personally am mostly happy when a challenge I post sees a variety of different approaches in answers. (Which could be seen as ironic, since I also tend to port answers pretty often, when they end up being shorter than my initial own approach in answers I post.. ;) )
I still do have the feeling I could have used the concept of merge errors and the different effects it has, for a completely different type of challenge. This was also the main reason I kept it in the Sandbox for all these years, in the hope I would have had some inspiration of what the challenge could convert to (which I didn't in the end).
I really like this concept, though Arnauld deserves credit for doing it first. I'm charmed at a code golf challenge having a fixed output while the input varies.
The try-everything-and-check-hash method of the initial answers took me totally by surprise. I really should have seen it coming. While I appreciate its brazen effectiveness, I was worried it was too golfy while being too generic, so I was glad when later answers came up with inventive custom schemes.
In retrospect, it might have been better to require or allow answers to output the position and character of the fix, which Arnauld's older challenge had as an option. That way solutions that locate the error aren't saddled with an extra step of producing the corrected text over solutions that enumerate potential texts. I found that this step consumed many bytes where I wrote my Python answer. On the other hand, this change would dampen the thematic purity of a fixed text as output.
I posted a bounty to encourage looking for new techniques, but unfortunately no new solutions did so. Maybe there's not really anything further, and I at least haven't thought of anything better. I wonder if having a longer text would have encouraged more sophisticated methods. Perhaps restricting errors to preserve letter case would have helped somehow.
I like how the challenge is very open to designing your own method to solve it. This use of error-correction makes for a weird and mind-bending twist on kolmogorov-complexity where your code already has basically the whole output given to it. The challenge being code-golf means trade-offs between the checksum/hash size versus the code size, which I think is interesting for an algorithmic challenge. I think there's room for further challenges exploring this kind of thing.
This was the very first challenge I posted here on CGCC (PPCG back then). As far as I remember, I did read most of the important meta posts about writing challenges, I posted the challenge on the sandbox first, waited around a week (trying to fix a lot of problems identified in the way), and then posted to main.
Even with all of this process, it had one problem: it had a custom scoring metric that didn't balance very well, or more precisely, was impossible to balance well. I wanted the solutions to do some juggling between (shorter code + somewhat unbalanced encoding scheme) and (longer code + more balanced encoding scheme). However, because we don't mind solutions taking too long by default, the challenge attracted (quite short) brute-force solutions doing perfect encoding, which got the best scores. Then the challenge became boring.
I believe a lesson from this challenge is do not mix code length and algorithm in the custom score. Just having the code length measured in different ways is fine (as in Modified Boggle Checker, with Modified Boggle-able Code, another code-challenge of mine).
This was my first challenge, and I think it taught me a few things about how certain, very important parts of making a challenge are more important than the base idea. Primarily, I think this comes across in two main ways: I could have made more approaches usable when solving the challenge, and I could have reduced the IO complexity of challenge. Both of these changes would only have had a minor impact on the challenge, but I was too attached the cute idea to realise that it was hurting the challenge.
The simpler change to see, particularly from our more modern point of view, was that the IO format required was needlessly complicated. My motivation for the complexity was clear: I wanted to include the characters from the show in the test cases. What I should have done was allow any distinct objects to represent names. This, along with the requirement that new bodies have new names quite probably cost me an answer from feersum, and maybe other users.
Slightly more subtle, was the effect of requiring minimal additional bodies to the solution. This precluded using an intelligent algorithm, such as from the proof, without needing to prove something that most people without a doctorate in maths would struggle to even approach. I was warned against this in the sandbox, partially, but with the suggestion being to make the challenge fastest-code. I think I was correct in not changing to that (for the same aforementioned complexity) but I think the challenge becomes much more interesting if brute-forcing isn't the only reasonably approachable strategy.
In summary, I think this challenge does a good job of highlighting how trying to force solutions to a challenge to be what you want often causes them to be mindless, while also probably discouraging solutions overall. However, unlike "answer in javascript only" cases which usually get downvoted and closed, most of the community seemed to want this challenge to do well. Hopefully this makes it a useful example of a couple things to avoid.
