This week, our team visited CAE at their Orlando, Florida office to give our presentation for our Preliminary Design Review (PDR)! Our PDR covers our initial research, requirements, specifications, design, and project plan. Its purpose is to explain to our sponsor what we will do and how we plan to do it. With their approval, we can start focusing on implementing the AI model and simulation for our project.

We had originally planned to visit their facility in Tampa, but due to unforeseen circumstances, we had to see them in Orlando instead. We’ll hopefully get another chance later this year.

In other news, our team has managed to run our simulation on HiPerGator! Using a few basic resources, a single game of Catan takes about 5 seconds to run. That’s about 5,000 games in 8 hours! We’re hoping that by further increasing our resource usage and allowing games to run in parallel, we can increase this amount even more.

Our next plan is to analyze and modify the current simulation of Catan to better fit the design requirements for our project. We also plan to research different algorithms and reward strategies for our AI model. That’s all for now!

This week, our team met in the IPPD lab to play Catan! Given that our project involves training an AI model to Catan, we wanted to take a moment to play the game ourselves and understand its rules better.

Catan is so complex for an AI model because there is a lot of freedom in what moves a player can make. On a person’s turn, one can trade his or her resources with another player, perform a maritime trade (a trade with the bank), build roads, build settlements, upgrade settlements, or buy development cards. Additionally, it’s not obvious which moves are the best. For example, a trade might benefit one’s opponent more than oneself. One has to set goals and take risks to reach victory.

Part of our challenge in this project is figuring out how to tell our AI model if it is performing well or poorly. Winning the game might be an obvious indicator, but without other indicators of success, our model will have a harder time learning. After playing a few turns of Catan, our team went through some brainstorming techniques to try and determine what constitutes a “good move.”

Additionally, our team has been working on our presentation for our sponsor, CAE. Next week, we will be visiting them in Tampa, Florida. That’s all for now!

Our team is excited to use UF’s HiPerGator supercomputer for AI model training. For some of us, it’s our first experience with a supercomputer, so we’re proactively learning how to use it. The simulation we chose as our foundation previously trained an agent for a month on a 32-core machine with a GTX 3090 GPU. While that setup is better than a typical office computer, HiPerGator has us eager to get to training our model.

Why HiPerGator?

HiPerGator, one of the world’s top academic supercomputers, supports UF’s innovative AI research with 70,320 cores, 608 high-end GPUs, and 4 Petabytes of Blue storage. Since Catan is centered on strategy, resource management, and negotiation, having access to a powerful system will let us refine game-winning tactics much more efficiently. We hope to create an AI model that performs exceptionally well and adapts strategically to unique game scenarios with HiPerGator’s AI-optimized architecture.

Using HiPerGator isn’t the only thing we’re excited about. On October 17th, we’re planning a site visit to CAE’s Tampa office to explore their simulators, present our project’s progress, and finally meet our liaison in person. We’re preparing for a productive day trip and are currently getting everything approved and finalized.

In addition to the trip, we’ve recently purchased Catan and an expansion pack to play together as a team. It’s our hands-on approach to familiarize ourselves with the game’s rules, strategies, and dynamics. We think it’s a great way to combine research with team building. Game night and bonding? Say less.

This week, our team has been “settling” on Catan as our game of choice. A lot of thought went behind our selection, including factors such as how popular and complex the game is. We decided to use a decision matrix to rate our game based on different factors and give each factor a weight to indicate how important that factor is to us.

Upon choosing Catan, our next goal was to get a simulation running of the game. We need a simulation that provides an API for our AI model. Open-source simulations exist, though we may also have to consider the possibility of creating a simulation manually.

Additionally, we are preparing to present our design to our Sponsor. This involves making a detailed list of requirements and specifications. One example of a specification is our model’s win rate. Given that Catan is a 3-to-4-player game and requires significant strategic planning, it may be difficult to achieve even a 50% win rate. We’ll have to decide on a goal for our project, and then work to meet those goals.

This week, our team has been researching various strategy games so we can figure out what game we will use for our AI model to learn. One game we considered was Stratego, a 2-player game where players can only see the ranks of their own pieces and must employ guessing and bluffing to come out on top.

Stratego is great because, unlike chess, the outcome rarely depends on a single move, and gathering information on one’s opponent is essential. However, it lacks some aspects of strategy games that we might want to have, such as cooperation. Additionally, it would be difficult to improve upon existing models, which already achieve very high win rates [1].

Another game we considered is Catan, a game where 3 to 4 players compete to gather resources and build settlements.

The addition of a third and (possibly) fourth player introduces another layer of complexity. Additionally, with the game’s trading system, our AI model must consider how trading may benefit other players. Training the model would be more difficult, but it would make for an interesting project!

We also considered other games like Risk, which we mentioned in a previous post, Civilization, a series of strategy video games, and The Battle of Polytopia.

Also, we’ve updated our team logo based on feedback we’ve received from our peers:

We’ve emphasized the “rook/wrench” image while also adding a set of nodes and edges to represent a network, alluding to neural networks in AI.

References

1. J. Perolat, B. De Vylder, D. Hennes, E. Tarassov, F. Strub, and K. Tuyls, “Mastering Stratego, the Classic Game of Imperfect Information,” Google DeepMind, Dec. 01, 2022. https://deepmind.google/discover/blog/mastering-stratego-the-classic-game-of-imperfect-information/ (accessed Sep. 20, 2024).

This week, our team is working on identifying strategy games to have our machine learning model play. A few of our candidates are turn-based strategy board games like Risk, Stratego, or even 4-player chess (pictured). There are several factors to consider, such as how we might be able to simulate the game, how many moves are possible in the game, and whether players must use collaboration to win.

Additionally, we are narrowing down the features that our product will need. We need to make sure we can evaluate the model at each stage and determine if the model is improving its strategies. We also need it to be “human interactive,” i.e., it should be possible for a human to play against the model and recognize the strategies that it employs.

Our team has a new name! Tactica! Our current logo, still a work in progress:

Greetings!

Welcome to Team 3’s space. We are excited to be a part of CAE’s IPPD group focused on developing a model capable of using a faster-than-real-time simulation and machine learning model to automatically generate winning strategies for complex tactical decision-making.

We have set up recurring meetings Wednesdays @ 5pm in MALA, with secondary times Fridays @ 5pm and afternoons on Sunday for when extra group collaboration time is needed. We just met with our coach and plan on returning together for our liaison meeting on Friday after doing individual work researching core concepts and potential simulation game platforms for our model.

Next week, we will share our official team name and logo. So far, we have brainstormed the name “Faster Than Real Time” alongside this mock logo.

Meet our team:

Han Mach

Jason Li

Max Banach

Cathy Quan (not pictured)

Andres Espinosa

Cody Flynn

Brian Magnuson

Mock Logo: