This week, our team presented our QRB2 update to the review committee. During the review, we received valuable feedback that will guide our ongoing development efforts. The committee identified several key areas for improvement in our emotional analysis system. Based on this feedback, we’ll be implementing several changes. We’re going to evaluate using a weighted prediction approach that gives higher priority to better-performing modalities, and we’ll be evaluating an intermediate fusion model that can dynamically learn which inputs deserve more weight.

Data imbalance between emotion classes emerged as another challenge, resulting in rare emotions being predicted less frequently and with lower accuracy. Our solution includes implementing balanced sampling during training, utilizing weighted loss functions to account for these imbalances, and expanding our training datasets.

Additional improvement areas include finding more representative datasets that better match our use case, integrating attention layers to help weaker models focus on informative features, standardizing preprocessing across modalities, and incorporating prediction confidence into our late fusion weighting function.