This week, our team gave our System Level Design Review (SLDR) presentation! The event was an exciting experience for all of us as we gathered in the Reitz Union Rion Ballroom to network with the liaison engineers from other teams and learn more about life in the workplace.

The SLDR presentation is the final presentation of the semester, so we wanted to make sure we finished strong. We talked about how the game of Catan is played, previous research done on Catan, our requirements and specifications, our project architecture, and our testing plan. There were a few clarifying questions and mainly clarifying questions. Overall, the presentation was a success!

The event also gave us a chance to relax and celebrate. We learned a lot over the semester: how to create a list of specifications, concept generation, public speaking, and team management. After a long journey, we now have a plan for the spring semester starting in January.

IPPD can be struggling at times, but it’s a great experience as it emulates the engineering design process that professional engineers go through. Additionally, we get to work with real companies and expand our network.

Among the things we’ll be working on next are creating our ensemble model, fixing any remaining issues with the simulation, and creating a script to test our model according to our testing plan. We also have plans to explore other simulations of Catan and see if we can learn from them.

Some of our team members also plan on helping out with the project during the winter break.

Andres plans on reviewing reinforcement learning concepts and playing against the current simulation’s AI to see how they can be improved.

Brian plans on studying other simulations and building a small reinforcement learning model to try and learn more about building AI models.

This will be the last blog post for a while. We hope you’ve found our project interesting! Things are only just getting started! See you next time!

This week, our team has been working hard on our System Level Design Review (SLDR) report. As we mentioned in a previous post, the SLDR is like our PDR but aims to be as specific as possible in describing our project design. For our project, this involves coming up with a design for our AI model that will improve upon existing models, as well as a plan to thoroughly test it to make sure we hit our specifications.

We mentioned that we have access to bigger hardware with HiPerGator, a supercomputer at the University of Florida. To take advantage of this hardware, we plan on utilizing a form of ensemble learning to help our AI make decisions. Ensemble learning is like combining multiple AI models into one. Each base AI model can focus on a particular strategy (such as building roads or buying development cards), and we can add a decision algorithm to decide which strategy works best for the next move. This has the benefit of ensuring that our overall AI model doesn’t get “locked in” to a sub-optimal strategy.

Earlier this week, we gave our SLDR presentations to our peers. Although things went well overall, our peers have helped us learn better ways to present our tables. We’ve also taken some time to reflect on the organization of the presentation, and we hope to improve upon it by December 3rd when we give our real SLDR presentation to our sponsor. Our presentation will be given here at the University of Florida, so we won’t be traveling again for the rest of the semester, unfortunately. However, there will be other opportunities in the next semester!

There will not be a blog post next week as we are taking a break for Thanksgiving. After next week, there will be one final blog post for the Fall semester where we will share details about our SLDR presentation to our sponsor, discuss our plans for the break, and discuss our plans for the next semester. See you then!

This week, the IPPD teams met on Tuesday for Prototype Inspection Day (PID)! On PID, each team gets to present their current prototype live in front of a series of judges. These judges then give us feedback on how to improve our product and development process.

For our prototype presentation, we showed off our simulation and how we plan on training our AI model to employ different strategies for victory.

To train an AI model to focus on a particular strategy (such as building roads or buying development cards), we use a reward function. In reinforcement learning, a reward function gives our AI a “positive reward” for good actions and a “negative reward” for bad actions. For example, winning the game might give +500 points while discarding resources might give -50 points. The model, during training, will attempt to maximize positive rewards while minimizing negative rewards. By adjusting the reward function, we can encourage certain kinds of behavior such as finishing the game quickly or building the longest road. For the game Catan, where it might not be clear which strategy is best, we intend on incorporating multiple strategies in our model to achieve victory.

In addition to showcasing our different reward functions, we also presented a testing plan for our project. When testing the performance of our AI, the most basic measure is the win rate, i.e., the number of games our model wins divided by the number of total games it plays. But there are different ways of measuring win rate. One way is to have our model play against three other models that provide random inputs. While these opponents might be easy to beat, it can be a good indicator if our model is learning a specific strategy. Another way is to play against the model ourselves, which we can do with our simulation. While this won’t provide very objective data, it can be a good source of qualitative data on how our model “thinks”.

Next week, we will be giving presentations to our peers for our System Level Design Review (SLDR). See you next week!

This week, our team has been busy preparing for Prototype Inspection Day (PID). Next Tuesday, we and the other IPPD teams will present our prototypes to several judges to receive feedback on our project. In our case, we will present the simulation we have, hopefully with added visuals to show how the AI model is performing.

We’ve mentioned that we have a simulation and AI model, but we can now show more details about the inner workings of our project.

To keep things clean, we try to keep the AI model and simulation as separate as possible. The simulation’s job is to keep track of the game state and relay state information to the AI agents. State information is all the information about the game’s current “state”, i.e., the cities, settlements, where they’re built, who owns them, what cards each player has, who has the most victory points, etc. The main issues with the current simulation are that it is not well encapsulated from the AI model and its functions are difficult to understand. The programming team’s goal is to address these issues.

The AI agents process this state information into data and feed it into a reinforcement learning algorithm. This should hopefully result in an action that will be sent back to the simulation, allowing the player to “make their move.” The main issue with the current AI model is that the reward function needs to be improved. The reward function determines which moves are good and which moves are bad (something that isn’t very obvious in Catan). The AI team’s goal is to address this issue by coming up with different reward functions and evaluating each of them. Hopefully, this gives a better understanding of our project!

As we work toward our prototype, we also have to be ready for our System-Level Design Review (SLDR). The SLDR is like our PDR, but it aims to be as specific as possible in describing our project design. It also must demonstrate that the project design will meet the specifications set in our PDR.

That’s all for now! See you next week!

This week, our team met several times to develop plans for our upcoming prototype. One thing we needed to decide on was how to develop the simulation. This is important since our team already has a simulation for training reinforcement learning models. However, this existing simulation has a few flaws and will need to be improved somehow. One idea was more or less an overhaul of the entire system; we needed to decide if the benefits would outweigh the downsides.

Without going into detail on our implementation, we can explain our concept generation using an example. This example won’t show what we actually discussed, but it illustrates how considering alternatives requires a lot of thought.

If we wanted to write a program, we’d first have to decide what programming language to use. Our team happens to know Python and C++. Python is often praised for its simplicity and readability. Additionally, it has many libraries with plenty of documentation online. However, when the program increases in size it needs to be broken down and more modular. Python’s dynamic type system would make things more difficult and less readable since it becomes less clear what each part of the program needs. C++ is more type-safe since all variables need to be declared with a type before they are used, and the type of a variable cannot change. Additionally, C++ is often implemented as a compiled language and can be optimized to run faster. However, it may require writing significantly more code to achieve the same results. Furthermore, managing libraries for C++ is more difficult.

From this example, just choosing the language for a program involves weighing various pros and cons. This procedure becomes more difficult when one starts considering the entire architecture of the program.

The AI team has also been working on how to restructure the code to better fit their iterative improvements. If we want to try out different algorithms for machine learning, the code should be designed to easily switch algorithms in and out.

Next week, our team will be focusing on making our project presentable for our prototype inspection day. That’s all for now!

This week, things have started to slow down a little since we finished our Preliminary Design Review (PDR). However, things will pick up again soon as we start working on our prototype. The purpose of our prototype will be to get as much feedback as possible. This way, we can determine if anything needs to be changed or if we need to take a new approach. It also shows our sponsor that our design works and can be iterated upon to create our final product.

Right now, we have split our group into two sub-teams: the AI team and the Programming team. The AI team’s job is to design the AI model process for our project. Right now, they are reviewing the documentation for machine learning libraries to understand how to build the algorithm in Python. They are also reviewing strategies to reward the AI model when it makes good moves and bad moves.

The Programming team’s job is to modify the simulation and understand how to run it on HiPerGator. Right now, they are reviewing code from an existing simulation and taking notes on each function. Once they get a full grasp of the simulation, they can determine if the code needs modification, or if it would be better to create a new simulation based on the existing one.

Throughout November, we hope to have a fully working simulation where we can fully utilize our AI model and have it running on HiPerGator. Then, by December, we will be wrapping things up for the semester with our System Level Design Review (SLDR). Then, by the spring semester, we’ll be working on training the AI model at full speed!

That’s all for now. See you next week!