This week, we presented our progress at QRB1 and received valuable feedback from various coaches. We shared detailed model performance metrics with our liaison, and we’ve achieved our target accuracy goals for audio, transcript, and action unit recognition as specified in our Technical Performance Measures. We have also implemented a joint-representation multimodal transformer architecture for the fusion model.

For the upcoming week, we’ll focus on enhancing our CLIP model through image-text pair training and hyperparameter tuning. We’re moving forward with end-to-end testing of our fusion model using our highest-performing components. Additional tasks include training CNN and ConvLSTM models for audio emotion recognition, testing FG-Net accuracy for emotion-specific markers, and acquiring supplementary training data. Our project continues to progress on schedule as we move toward our integration goals.