One chat room cannot be what everybody expects from it
The Nineteenth Byte's purpose is labeled as general discussion; the jestful addendum appears to indicate that pretty much everything goes.
A good thing about this is that The Nineteenth Byte is usually crowded. If you need input on something, you can almost be sure that you'll find somebody you can talk to. It also means that there is a place where we can get to know each other better, which I think is useful for community building. None of this would be the case with a strict policy about what is on topic and what not.1
Moreover, I feel The Nineteenth Byte should also be the go-to place for serious discussions about PPCG.2 Idle chatter is not a problem, as long as it occurs during idle times. When a serious, important discussion is taking place, chatter should cease until that discussion is over.
Is it possible to have a calm, extended, one-to-one conversation with somebody about, say, golfing a particular answer? Probably not. I also don't think that falls under general discussion.
Ideally, we'd have topic rooms for these things. That has worked well for the development of esoteric programming languages, Data, the testing room for bots, etc. For whatever reason, the Code Golfer's Corner – which would seem like a popular idea on paper – didn't share that success.
Finally, there's always the option to create a temporary chat room for an extended conversation between a small number of users. That's a useful feature, and it should get used more often.
1 That does not mean I endorse caret chains, conversations that consist solely of animated GIFs, and similarly disruptive interactions that do not even qualify as off-topic chatter.
2 By that, I mean discussions that affect the site, like the recent discussion about the purpose of The Nineteenth Byte.