This site gets a relatively small number of posts. Could we contact IDEOne or somebody, and get a script going to test people's code automatically? Maybe put up a "powered by" logo somewhere?
IDEONE expects some boilerplate for scala for example, naming conventions and a main-method, and a specific scala version. Enforcing compatibility would need to take this into account. I'm pretty sure, for other languages the situation is similar, isn't it? C, C++, Java, C#?
Maybe it isn't that restricted for scripting languages like ruby, python, perl, ... ?