#Pseudoku Cops and Robbers King of the Hill

(I know that another user, @NathanMerrill, is proposing a similar contest. I started playing with the idea for this type of contest independently yesterday, but have since chatted with in The Nineteenth Byte. He is currently undecided on the type of puzzle to use and has some different ideas on how to evaluate participants' performance, so I feel comfortable proposing my idea as a separate challenge.)

Sudoku is a well-known logic puzzle. It is a puzzle of four nines: nine rows of cells, nine columns of cells, nine 3x3 adjacent and distinct blocks of cells, and nine values that any cell can have. A valid Sudoku arrangement or solution is one in which every row, cell, and block has all nine values exactly one time. For example, consider the following valid solution:

+-----+-----+-----+
|4 6 1|5 7 3|2 8 9|
|5 7 8|2 1 9|4 6 3|
|3 2 9|6 8 4|1 7 5|
+-----+-----+-----+
|9 8 4|7 6 2|3 5 1|
|7 5 6|3 4 1|9 2 8|
|2 1 3|9 5 8|7 4 6|
+-----+-----+-----+
|8 3 5|1 2 7|6 9 4|
|6 9 7|4 3 5|8 1 2|
|1 4 2|8 9 6|5 3 7|
+-----+-----+-----+

These are turned into puzzles by removing many of the values in the arrangement in such a way that all blanks are mirrored horizontally and vertically across the puzzle and so there is only one valid way to fill in the blanks to get a valid Sudoku solution. For the above puzzle, this might look like this:

+-----+-----+-----+
|4    |     |2 8  |
|  7  |  1  |    3|
|    9|    4|1    |
+-----+-----+-----+
|     |7 6  |3 5 1|
|     |     |     |
|2 1 3|  5 8|     |
+-----+-----+-----+
|    5|1    |6    |
|6    |  3  |  1  |
|  4 2|     |    7|
+-----+-----+-----+

Someone who wished to play this Sudoku puzzle would then use the information provided to find the original solution.

Sudoku has some interesting properties that allow it to be generalized to similar puzzles with different rules that are sometimes called "Pseudoku" (which is pronounced the same way as the actual puzzle, SOO-DOE-KOO, so please stop saying SOO-DOO-KOO). For our purposes, we will make two differences. First, it may be possible to generate harder puzzles by removing the restriction for symmetric removals. The following is a valid puzzle by Sudoku rules, so why not allow it?

+-----+-----+-----+
|4   1|  7  |2    |
|5    |2   9|     |
|3    |  8  |     |
+-----+-----+-----+
|  8 4|7   2|    1|
|     |3   1|  2  |
|     |     |     |
+-----+-----+-----+
|     |     |6    |
|  9 7|     |     |
|1    |     |5 3  |
+-----+-----+-----+

Second, Sudoku's properties allow us to define games with different sizes. You can define a Pseudoku game with a parameter N where the resulting board has N^2 rows, N^2 columns, N^2 blocks of size NxN, and N^2 values for each cell. Standard Sudoku would be a Pseudoku variant with N=3. So the following would be a valid Pseudoku(2) game:

+---+---+
|1  |   |
|   |  3|
+---+---+
|  1|4  |
|4  |   |
+---+---+

and an example Pseudoku(4) game:

+-----------+-----------+-----------+-----------+
|11         |      10  4| 1     9   | 2       16|
| 6     5   |   15  1   |    3     2|12  8      |
|10    13 14|         12| 5       15| 4     7   |
| 2  3      |    6    13|       8 11|    5    10|
+-----------+-----------+-----------+-----------+
| 7 11 12  9|14        2|16  1  4   |           |
|           |   10  4   |          3|13    16 11|
| 4       10|   16    15|   12      |          6|
| 1 16  2   |11  3      |   10     8|           |
+-----------+-----------+-----------+-----------+
| 3        2|       6   |13         | 5 14     1|
|    7      |          5|           |           |
|   13 14  4|12    16   |           | 8  9      |
|    5 16   |13     9   | 4     2  1|           |
+-----------+-----------+-----------+-----------+
|14 12     7|           |   15     4|11     6   |
|    9      | 6         |11 16      |       3   |
|      11   | 8 13     1| 3 14      |    7      |
|13     8   | 7    11   | 2         |      15   |
+-----------+-----------+-----------+-----------+

Since Sudoku is NP-complete, so is Pseudoku. That means that it gets more difficult to solve a Pseudoku puzzle the larger N gets. However, it can take more time to generate Pseudoku puzzles than it does to solve them, since the naive algorithm for generating a puzzle requires solving the puzzle each time a value is removed! Solving Pseudoku puzzles is fun, but if it takes longer to generate them than it does to solve them, it becomes more work than play.

So help me out! I propose a Cops and Robbers style King of the Hill. The Cops will compete by writing programs to generate lots of Pseudoku puzzles to consume as much time as possible for solving, while the Robbers will compete by writing programs to solve Pseudoku puzzles to consume as little as time possible solving these puzzles.

I need some help ironing out the format, but here is what I have so far:

1. I will provide a Java framework for running the contest. This framework will connect to clients by TCP/IP so contestants can choose whatever language they want to write their Cops and Robbers (so long as I can run them on my system). I will also provide a basic Cop and Robber for these users to try out to see what sorts of times they take. I will publish the times they generate on my system so contestants can estimate how their entries will run on my system.

2. I will give each Cop ten minutes to generate as many Pseudoku(N>= 3) puzzles as they can, but they should be able to generate at least Pseudoku(N=4) puzzles. They can choose what sizes they want the puzzles to be, but they have to be valid with exactly one solution. My server will naively check each one to guarantee their validity; any Cop that generates an invalid puzzle is disqualified. I recommend configuring the Cop programs to be parameterized externally so that Robbers can test their code against basic Cop configurations, but then the Cops can send me secret, optimized configurations before the contest completes for their actual execution. I will provide a couple days after the deadline ends for conferring with the Cop programmers if their settings do not work as expected on my system. Cops should generate different puzzles every time with reasonable expectations; that means no spamming with the same puzzle repeatedly or reading pregenerated puzzles from a file system, Internet source, or internal cache. In addition, I don't want to see a Cop that uses the same removal pattern for every puzzle (that may not guarantee valid puzzles, anyway).

3. Each Robber will be tested against each puzzle generated by the Cops. The Robber will have to generate the correct solution for each puzzle as quickly as possible. I will probably need to see some timings before I make a final decision, but each Robber will be capped with some amount of time to solve a puzzle (maybe an hour?) before the time-to-completion defaults to twice that cap. These Robbers will be permitted to use any technique for solving the puzzles that my system supports except for packet sniffing. I am on the fence as to whether the Robbers will be on an honor code to not study Cop code since I plan to have secret parameterizations anyway.

4. All the times for all the puzzles will be sorted from least to greatest and then assigned an index as one would in a Mann-Whitney U test. Each Cop and Robber will be scored using the sum of the indices of their contributions: Cops for the times the Robbers spent solving their puzzzles, and Robbers for the times they spent solving puzzles. The winning Cop will have the highest sum and the winning Robber the lowest. Cop ties will be broken first by the average time required to solve one of its problems (more is better), then by the number of puzzles generated (more is better), then by the name I deem cooler (here's hoping that doesn't happen). Robber ties will be broken first by the average time spent solving puzzles, then by the sum of the time, then by the standard deviation, then by the cooler name.

5. This scoring scheme poses an interesting challenge to Cops: balancing the size of the problems (and the likely amount of time needed to solve them) against the number of problems generated. A Cop that generates only one puzzle that no Robber can solve in the time limit is likely to lose to another Cop that generates many moderate problems. Similarly, a Cop that spams many small problems is likely to be beaten by another Cop that generates fewer problems of larger sizes. Since the official contest configurations should be kept secret until the contest starts, other Cops can study the other programs to try to determine what their opponents are likely to do and plan accordingly.

I am interested in any and all feedback that the community might have about this challenge.