17
\$\begingroup\$

All stats given here are correct at the time of writing. I can't promise that they'll stay exactly correct for a significant amount of time (and they most likely won't), but I'd be surprised if the pattern challenges significantly

Of the last 50 questions that were posted to main (and that are still undeleted):

  • 33 were posted by an inexperienced challenge author (\$\le3\$ open, positively scored questions)
  • Of those, 18 are negatively scored
  • And, 20 are closed (7 as duplicates)

Looking at the deletion stats (10k+ only) for the same period, we can change those numbers to

  • 41 were posted by inexperienced challenge authors
  • 25 have a negative score
  • 28 are closed (10 as duplicate)
  • 8 are deleted (6 deleted by Roomba)

How can we improve these stats?

We have the Sandbox specifically to avoid this - somewhere that newer users, less experienced with challenge writing, can go in order to learn how to post high quality posts. Clearly, however, it isn't working, and in fact, some users aren't even aware of it before posting their first question.

In order to see exactly what it's like to post a question as a new user, I logged into one of my sockpuppets and went through the motions of posting a question. My observations:

  • Our info box is obviously visible, but doesn't stand out in a page full of links and more. In fact, visiting my latest question, more space was given to ads than the info box

  • When I click the "Ask Question" button, I was greeted by our custom modal, which, as I've said before, isn't bad, it's just not great.

  • I then drafted a low quality question, and hit "Review your question". Something that immediately caught my eye:

    Our automated system checked for ways to improve your question and found none.

    Granted, we shouldn't expect an automated system to be our absolute filter for low quality questions, but I can think of a few automated checks that could be helpful, such as checking for winning criteria tags, or looking for patterns that may indicate the post is off-topic or low quality.

    Additionally, as noted by Unrelated String, for users who are less familiar with the SE system, this could be seen as an "endorsement" that their question is good to go, rather than the system telling them there are no "system" errors (e.g. missing title, no tags etc.)

  • During this process, there was only ever a single time where I was directed to our Tour page and our how to ask pages, both of which are supposed to be there to help newer users ask high quality questions, and that was in the Step 1 while asking my question, along with a lot of information in a fairly small box. Even if I had visited the Tour/asking pages, though, I'm not sure it would've helped:

    You should only ask practical, answerable questions based on actual problems that you face. Chatty, open-ended questions diminish the usefulness of our site and push other questions off the front page.

    What types of questions should I avoid asking?

    Have you thoroughly searched for an answer before asking your question? Sharing your research helps everyone. Tell us what you found and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and above all, it helps you get a more specific and relevant answer!

    How do I ask a good question?

    Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.

    The Tour page

    We've asked hundreds of times about our misleading Tour/help pages, so consider this one the latest in a long series of similar questions. That doesn't change the fact that the pages are misleading at best, and downright wrong at worst.

    Furthermore, despite it being the go-to, number one piece of advice offered by countless users across the site, our how to ask page only mentions the Sandbox once, and just doesn't mention other useful resources such as our or pages such as Things to avoid when writing challenges

  • "This site is for programming contests and challenges. General programming questions are off-topic here. You may be able to get help on Stack Overflow." is one of our primary close reasons (19.05 % of closed questions in the last 90 days), behind "Needs details or clarity" at 44.44 % and "Duplicate" at 23.81 %. This suggests that a lot of users post here without fully understanding what the site's actually about.

  • Finally, the Sandbox isn't even fully accessible for new users, as due to a bug that has gone unfixed since September 2020 (going on 7 months now), users require 5 reputation to post on Meta.


In short, the systems we have in place in order to help newer users post high quality questions are not working. New users do not get directed to helpful pages until after they post here, they don't get adequately informed about the purpose of the site, and they don't have access to, or the knowledge of, valuable and helpful resources.

So what can we do about it?

What suggestions do you have to make it easier for new users to post good quality questions? What can/should we change (or try to change) in order to decrease the number of low quality posts?

\$\endgroup\$
3
  • 3
    \$\begingroup\$ The overarching problem here isn't the influx of new contributors. That influx is a good thing. The white elephant in the room is simply that Stack Exchange either doesn't have the resources to help us, or doesn't care enough (we hardly bring in any profits). \$\endgroup\$ – Adám Mar 1 at 21:34
  • 1
    \$\begingroup\$ @Adám That influx is only a good thing as long as those new contributors actually improve the quality of the site. When 40% of recently posted questions are closed, and are overwhelmingly posted by inexperienced users, I can't confidently say that they are improving the quality of the site, and I want to change that \$\endgroup\$ – caird coinheringaahing Mar 1 at 21:36
  • \$\begingroup\$ Perhaps, when a new user signs up, he... or she... is directed to the tour page and / or a special page which reiterates the info box's contents. \$\endgroup\$ – ophact Mar 5 at 8:46
8
\$\begingroup\$

As far as I can see, there are two big issues here that both need addressing:

  • Visibility of useful resources
  • How helpful those resources are

Visibility

No one likes having a whole bunch of links to a whole bunch of posts thrust into their face as soon as they do anything. That said, some links are close to essential for new users in order to properly understand the site.

To avoid having a massive standard comment, or heavily cluttered info box or similar, I suggest we collate Meta posts which we believe are all but necessary for new users to read, and tag them with something like to allow newer users to be pointed to the tag and able to browse through the posts.

This guide can then be constantly linked to every step of the way for new users. Link to it from the Tour page, in the sidebar when writing a question, in the Info Box at the side of the site, in the modal that pops up when you ask a question. We can't force people to read something, but by encouraging and linking it enough, we can make sure that the vast majority of people are aware of what the site is about before ever posting anything.

As I said in the question, there's nothing that stands out about places to improve your challenge before posting. When new users answer a question, they get a big yellow info box. Having one of these for challenges could be very helpful, directing new users to resources such as the Sandbox, the Tour/help pages or a .

Helpfulness

As noted in a lot of meta discussions, the Tour and Help pages aren't fully relevant and/or helpful to newer users.

There have also been recent examples of users not understanding how long to leave a question in the Sandbox, suggesting that the purpose of the Sandbox ("A place to leave a draft so that others can suggest feedback until it's ready to be posted") isn't fully understood by all.

And, of course, new users can't post in the Sandbox.

Improving our "automated systems" would be a significant first step. Escalating the reputation issue to SE and emphasising it's importance, customising our asking modals and tag warnings, and correcting our Tour and help pages would hopefully lead to fewer off-topic and low quality questions. And, even if nothing changed, fixing these would be an improvement to the site

Finally, updating our pages, promoting some of the pages, and linking new users to the FAQ would help those users better integrate into the site.


TL;DR: If I had to sum this answer up in one sentence, it'd be this:

Collect all our helpful information, links, pages and more into a single location and promote links to the location at every step of posting a question

\$\endgroup\$
3
  • \$\begingroup\$ Another option is, either instead of or in addition to the tag, we could create a single meta post which has links and summaries for important resources, a brief explanation of how and why to use the sandbox, etc. \$\endgroup\$ – Redwolf Programs Mar 5 at 4:02
  • \$\begingroup\$ @RedwolfPrograms I considered that, but my issue would be that we have a lot of “new user” meta posts, and it could very easily turn into a wall of text unless written well. Having a tag would make it easier for newer users to find exactly what they’re looking for if they have a specific question (e.g. “How do I use the Sandbox?”) \$\endgroup\$ – caird coinheringaahing Mar 5 at 14:04
  • \$\begingroup\$ @RedwolfPrograms Related Meta discussion \$\endgroup\$ – caird coinheringaahing Mar 5 at 15:34
5
\$\begingroup\$

Warnings

From what I can tell, the review screen is somewhat customizable. It doesn't seem like it can test for not having a tag (i.e., not having an objective winning criterion), but we can add regex based warnings for the body or tags.

I suggest the following:

Language tags

Some of the more common of these, like , , , etc., with warnings against language restrictions

Restricted source

Add a warning for Do X without Y, with an appropriate link

Other possible options

Later today, I'm going to download every undeleted challenge on CGCC, and look for the tags and substrings that are most common in low quality questions. Strategically adding warnings prompted by these could catch a fair number of challenges, if it works.

\$\endgroup\$
2
  • 2
    \$\begingroup\$ Actually if you can do a regex test on the list of tags, you can check for not having an objective winning criterion (except that it needs to be changed whenever our list of winning criterion tags changes). \$\endgroup\$ – Bubbler Mar 5 at 1:44
  • \$\begingroup\$ @Bubbler Ooh, I guess that is true. I was thinking it was just per-tag, but if it uses the same format as SEDE (<tag-one><tag-two><tag-three>) or the format you enter them in (tag-one tag-two tag-three) it would definitely be possible. I don't think we'd even need to cover all of them or update it...things like code challenge and fastest code should probably still have warnings anyway since there are additional requirements or warnings needed, and it's fine if a warning is showed for the lesser used OWC tag because they can just be ignored. \$\endgroup\$ – Redwolf Programs Mar 5 at 2:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .