Go Space

a project by T.J. Gaffney.

Go Space is a program / model that attempts to embed similar Tseumego close to each other. The ultimate goal is to use this as a study tool: As the user solves problems, we can explore / exploit to build a map for that user to identify which parts of the go space the user has difficulty. We can then serve problems from difficult regions until the user gets better.

Today, the UI has not yet been built, but a V1 model has been built, and there is some evidence that it works well.

Code Location

All code is available here: https://github.com/gaffney2010/go-space

Approach

The idea, a small amount of code, and a lot of ideas come from Deep Learning and the Game of Go. The approach is fairly standard with a couple of key changes:

We only care about the corner of the board, where a majority of Tseumego occur.
We want embeddings instead of move predictions.

We started with some freely available games in SGF format. We stuck with 9x9 games as these are mostly Tseumego problems anyway. (Note that we train on games even though the intention is to use this to fit Tseumego problems. There is more data from games, and the decisions should be the same.)

In order to only use the corner, we first rotated each move so that it'd appear in the first quadrant. Normal each move is a data point to use in training a model, but we threw away data points from moves that didn't happen in the first 4x4 spots. Then we masked to those spots, plus a few more that we think may influence the decision to play in one of these locations. Specifically, we masked to the following locations, represented by the #:

# xxxxxxxx.

# xxxxxx...

# xxxxx....

# xxxx.....

# .........

With that data, we built a small CNN. We made the target the next move (among the 16 corner spots), taken with a softmax. We used TF2 with Keras with the following layers, chosen after some tuning:

ZeroPadding2D(padding=2, data_format="channels_last"),

Conv2D(24, (5, 5), data_format="channels_last"),

Activation("relu"),

ZeroPadding2D(padding=1, data_format="channels_last"),

Conv2D(12, (3, 3), data_format="channels_last"),

Activation("relu"),

Flatten(),

Dense(64),

Activation("relu"),

Dense(16, activation="softmax"),

To obtain embeddings from this model, we use the first few layers from this model.

Results

The CNN improved predictions on a holdout while training, which indicates that the training is producing meaningful results.

As a second indicator, we hand-coded 13 Tseumego problems split into "classes". A class represents problems that a human with familiarity of the game would consider similar. We then embedded these classes with our model and with random same-dimension benchmark embeddings, and measured Buhlmann credibility (a within class similarity measure) with class as the only parameter. The embedding model showed a significant improvement over random benchmarks. Finally, we used TSNE to visualize our embedding on these 13 points as embedded by our model. It seems like the embedding is meaningful, but not extremely precise.

Next Steps

We think the existing model is good enough. In the worst case scenario, the model is random, and the resulting UI would just be a random Tseumego tester, which is acceptable program in itself. Our plan is to create the UI, and use it to collect data indicating when boards are/aren't similar. We can use this data to train future iterations, perhaps using a w2v-style fine-tuning of our existing embedding. We may need to figure out ways to weigh validity of human labels, for example doing something like this.

So as a next step, we want to build a UI. For each board, we want to have a "Similar Boards" section on the side. Next to each similar board, we want to have a Thumbs up/Thumbs down button to allow voting if these are actually similar. If the user clicks a similar board it should pull it up; so this will server as a useful UI feature. We want to save this data from our first UI version.

A next step would be to use this ourselves, and try to convince others to use it as well.