This week, we collected datasets for individual modalities to create our unimodal model architectures. Our team gained access to our shared Github repository, where we’ve begun uploading our code and data processing methods to integrate our various components. We advanced our audio analysis capabilities, developing code for audio marker detection and spectrogram analysis. Additionally, we implemented a system using Large Language Models to determine emotions from transcripts.

We have focused our development efforts on creating prototypes that we aim to present for Prototype Inspection Day on November 12. For example, we set up facial action unit detection using the OpenFace library:

We also completed our prototype of real-time emotion detection using CLIP:

For the upcoming week, we’ll be finalizing our prototype plans for Prototype Inspection Day while continuing the development of our unimodal models. We plan to set up API key access for the team and enhance our transcript analysis through improved prompt engineering.