This week, our team began work on training our first models! The simulation we’ve chosen to use is called Catanatron. It is an open-source Catan simulation for testing bot strategies.

In Catanatron, we can build player classes using different strategies. Each player will have access to the game state and a list of possible actions to take when it is their turn. We can feed this information into a reinforcement learning model, and then have it predict which action to take.

Compared to previous simulations, this simulation is very stable and easy to work with. It provides a Gymnasium environment to help us train our models and lets us easily compare our players with other bots. It also provides visualizations of specific games of Catan in a web server, allowing us to see what kinds of actions our models are taking.

Here, we’ve created a player that uses a reinforcement learning model with a simple reward function and had it play against 3 other random players in 1,000 games. Random players simply pick a random action in their list of available actions. Our player won the most games, but only by a small margin.

In addition to training models, we also need to work on modifying the simulation. The simulation did not come with a complete trading interface. That is, players can perform maritime trades, but not trades with other players. We plan to enable more features of the game, so our AI players can have the proper Catan experience.

Next week, we’ll be working more on setting up our environment to test our models more rapidly. See you then!