Following our successful PDR presentation in Jacksonville last week, our team has begun the data collection phase. We’ve started gathering data across all modalities needed to train our individual models and developed preliminary processing methods to extract frames, audio, and transcripts from video inputs. We’ve also reached out to dataset publishers to acquire additional datasets for our training needs. Our team has now split into focused groups for the model architecture development sprint, allowing us to work more efficiently on different aspects of the project.

Looking ahead to next week, we plan to expand our dataset collection with a focus on modality-specific data. We’ll be completing all unimodal model architectures and integrating real-time processing capabilities into our pipeline. We’re also working to ensure all team members can access Hipergator for model training purposes.

This week, our team focused on finalizing our Preliminary Design Review (PDR) materials. We added new sections to our PDR report to meet additional requirements and completed the final version for our sponsor. Based on peer feedback, we also revised our PDR presentation. We’re on track to present our PDR to our sponsor, AGIS AI, in Jacksonville on October 18th. This presentation marks a significant milestone in our project timeline. We’re looking forward to receiving feedback from our liaison engineer after the PDR presentation.

Looking ahead to next week, we plan to divide into teams for product development, start our model architecture development sprint, set up a centralized GitHub repository for our code, and continue developing our data processing pipeline to test with our current dataset.

This week, we focused our efforts on preparing for our PDR presentation peer review session. After developing our PowerPoint slides based on our PDR report, we practiced giving the presentation as a group before we presented it for peer review on Tuesday. Our peer reviewers gave us lots of useful and actionable insights that we will be using to improve our presentation.

Unfortunately, Hurricane Milton forced us to cancel our coach and liaison meeting this Thursday. Despite this, our project remains on schedule. Next week, we plan to fine-tune the presentation and travel to Jacksonville to officially deliver it to our sponsor AGIS AI in Jacksonville.

This week, we concluded our literature review and concept generation phases and finalized the overall system architecture. We decided to implement an intermediate fusion multimodal model, combining multiple unimodal models and synthesizing the outputs of these models with a master neural network model which will produce a final output. Our choice was based on a thorough investigation of state-of-the-art multimodal models from the literature and a consideration of important factors such as speed and interaction between modalities.

In addition to selecting our system architecture, we finalized our PDR schedule with our liaison. We have begun working on our PDR presentation in preparation for our peer review session on Tuesday. We’re looking forward to getting feedback on our presentation and improving it before we present it to our sponsor.

Next week, we plan to begin gathering data for the model and developing our model architecture.

With Hurricane Helene canceling classes and shutting down campus operations, our team’s efforts were primarily virtual. Despite this setback, our team made good progress this week, acquiring facial analysis model training datasets, identifying latency optimization libraries, and finalizing our semester roadmap and risk assessment. We’ve also been finishing up our Preliminary Design Review report draft, keeping us on schedule.

Looking ahead, we’re focusing on audio analysis datasets, audio-to-transcript libraries, and Large Language Model research for transcript analysis. We’ll also begin developing our facial analysis model architecture. Currently, we’re coordinating with liaisons to arrange our sponsor site visit.

Our team has been progressing well in our research and planning for the project. We’ve been compiling relevant papers and examining various approaches to inform our implementation. Additionally, we met with a researcher at UF to discuss his work in real-time facial analysis, which we hope to expand on as we develop our model prototype.

We’ve started working on our Project Roadmap for the year, which will help us stay on track with our goals and deadlines. We’re also beginning to finalize our model architecture as we wrap up the brainstorming process and literature review.

Next week, we plan to complete our general literature review and research specific technologies for our concept. Additionally, we’ll begin the process of exploring relevant datasets and setting up data pipelines. We also plan to start compiling information for the preliminary design review.

This week, our team has been moving forward with the project rapidly. We finalized our team name and logo, coordinated with project advisors, and began to explore relevant literature.

Our team name “Noesys” is derived from the words “noesis” and “system.” Noesis is a philosophical term that refers to the activity of the intellect, which we combined with “system” to highlight the technical aspects of the project. Our logo reflects both of these concepts, containing related imagery such as an eye and a circuit. The logo is encircled by a gradient-colored circle designed to represent a mood ring, highlighting how our project aims to detect emotions.

The week began with an introductory meeting with our coach, where we discussed resources and methodologies for the project. We also held a productive meeting with our liaison and coach, during which we established clear expectations for the product prototype and development timeline. On the technical front, we began to create a schema for the necessary model components, which will serve as the basis for our future model script.

Looking ahead to next week, we’re scheduled to meet with a Ph.D. graduate from UF whose research closely aligns with our project. We plan to review his work and discuss the availability of data and code for our project. Our team will also be conducting an in-depth review of the literature, focusing specifically on temporal context prediction models. Additionally, we plan to test the current Hume.ai emotional analysis prototype to gain insights into its functionality. Finally, we’ll begin work on our own model prototype.

Hello everyone, welcome to our team’s blog page! We will be working with AGIS AI to assist mental health professionals by using AI to analyze emotions in audio and video. We will be documenting our progress on the project here on this page throughout the year.

This week, we met each other in class on Tuesday and spent some time understanding and reviewing the scope of work for our project. We also had our first team meeting on Thursday, where we assigned team roles, brainstormed our team name and logo, and scheduled recurring weekly meetings.

Next week, we will be meeting with our coach and our liaison to gain a more comprehensive understanding of the project and discuss our next steps.

Our team members:

-Albert Li (Meeting Timekeeper)

-Arjun Singh (Team Leader)

-Estefania Rodriguez Da Silva (Template & Design Manager)

-Jason Liang (Meeting Facilitator)

-Peter Salmon (Web / Blog Editor)

-Rafid Chowdhury (Finance and Travel Coordinator)

-Tuan Truong (Research Librarian)