Blog Posts

WEEK #9: Enhancing Prototype 1

Published:November 6th, 2023
Category:Fall Semester

During week 9, the team focused on refining the chatbot prototype and laying the foundation for robust internal testing and feedback integration.

Prototype Progress and Enhanced Transparency

The team refined the architectural model for delivering prototype 1 to our company liaison for internal testing. This is crucial as it allows us to gather essential feedback and insights with the dataset we used for training and the accuracy of our model’s answers. 

To further achieve this, our team engaged in an in-depth brainstorming session implementing feedback loops within the architecture. This includes developing a reranking system based on feedback from responses, which will be pivotal in enhancing the quality and accuracy of the chatbot.

In addition to preparing for the prototype delivery, we introduced the feature of returning the sources from which the chatbot retrieves its information. This enhances transparency in the retrieval process, allowing us to understand how information is sourced and, therefore, carefully finetune the hyperparameters and strengthen our corpus where appropriate. 

Testing and Overall Insights

To ensure the chatbot’s effectiveness, we compiled about 150+ testing queries. These queries are instrumental in evaluating how our bot responds to different parametric values, allowing the team to refine the LLMs’ hyperparameters and reliability further.

Moreover, an eyeball comparison between our chosen base LLM and the implemented retriever was conducted. This further enabled the user to identify areas of improvement and whether or not the models have been successful in meeting the expectations.

Tagged as: , , , , ,

WEEK #8: Advancing Prototypes and Building Professional Skills

Published:October 30th, 2023
Category:Fall Semester

Greetings!

In our weekly recap, we’re excited to share our progress and experiences over the past week as we continue our journey to develop an iterative prototype CoPilot for JAWS.

Prototype Enhancement and Vector Embedding Evaluation

We dedicated our focus to enhancing Prototype 1 by evaluating various vector embedding options. After careful consideration, we made the decision to finalize one of the options, paving the way for a more efficient and accurate chatbot. To accommodate these changes, we meticulously restructured our codebase in Github.

To ensure the robustness of our chatbot, we initiated automated testing for the prototype. This involved creating a comprehensive testing suite, complete with a list of queries that users may pose to JAWS. These tests are pivotal in helping us determine the optimal model parameters for the final product.

In our pursuit of enhancing the chatbot prototype, we also delved into experimenting with different embedding models. This exploration allowed us to gauge how these models impact the accuracy and performance of our Prototype 1. Through extensive discussions, we weighed the pros and cons of continuing to use an embedding service versus transitioning to an on-device operation. This decision will be instrumental in shaping the final prototype.

Professional Exposure

In addition to our technical work, we invested in our personal and professional growth. We attended two enlightening workshops during the week:

1. A Thorough Introduction to the Scrum Framework. Dr. Manuel Bermudez, from UF CISE, guided us through the fundamentals of the Scrum framework. The team recalled the importance of Scrum’s adaptability for development teams facing evolving requirements and dynamic priorities. The iterative and incremental approach, with its short sprints, resonated with the team as a way to improve project outcomes.

2. From College to Career – Success Tips for the New Engineer. We gained valuable insights from Mark Sanders of RTX, who shared tips on transitioning from college to a full-time engineering role. The workshop provided guidance on navigating real-world scenarios and encouraged us to adopt a proactive and adaptable mindset.

As the week drew to a close, we proudly delivered our PDR presentation to our Freedom Scientific liaisons. Their feedback will be invaluable as we refine our presentation for the next stage of our project.

We’re making great strides in both technical development and personal growth, setting the stage for an exciting journey ahead. Keep up the fantastic work, team!

Tagged as: , , , , ,

WEEK #7: Prototype 1 Complete

Published:October 24th, 2023
Category:Fall Semester

In this week’s update we are happy to announce that prototype 1 of our JAWS CoPilot is complete. We have successfully deployed and connected both Pinecone and ChatGPT with our codebase which means that it is accessible to both you and me! We go into more detail down below.

Prototype 1

As a team we worked on finally being able to deploy our first version of JAWS CoPilot. Our website is reachable with the link provided: JAWS CoPilot. You will need an OpenAI key in order to access the LLM. In terms of prototype 1’s capabilities, a user may can ask any question they would like, and the CoPilot will do its best to search inside our corpus compiled of a variety of documents related to JAWS. If the CoPilot cannot find what the user is looking for then it will let the user know that their question is not currently included in Copilot’s capabilities as of yet. If the answer is indeed found by CoPilot, it is then returned to the user in a readable text format. This is something we will definitely have to change the in future as our product is meant to be made for visually-impaired users however this is a good start.

Hope to see you back next week on this journey. Chao!

WEEK #6: Breaking Barriers – Prototyping and Reporting

Published:October 23rd, 2023
Category:Fall Semester

In this week’s update, we’ve hit some significant milestones that will propel us forward on our project journey. As we continue to push the boundaries of our project, here’s a glimpse of what we achieved:

Architectural Diagrams: Unveiling the Blueprint

We’ve taken a significant step forward by crafting architecture diagrams for each of our three iterative prototypes. These visual roadmaps are invaluable tools, helping us conceptualize and streamline our development process. With these diagrams in hand, we’re better equipped to navigate the intricate pathways of our project.

Corpus Processing: Resource Refinement

Our data handling capabilities received a substantial boost. We’ve begun processing and chunking our corpus resources, successfully managing various file formats, including .pdf, .html, .txt, and .srt. This achievement streamlines the team’s data preparation, bringing us closer to our embedding and storage objectives.

Preliminary Design Review (PDR): Written Report Milestone

The PDR Report, a cornerstone of our project, has reached its final form, featuring the addition of architecture diagrams, an engaging narrative, and an executive summary. The PDR included integrating the architecture diagrams, as mentioned earlier, an associated narrative, and an executive summary. The report encapsulates our progress and plans for the project, setting the stage for what’s to come. Simultaneously, we prepared for our PDR presentation, scheduled for next week. This presentation is an exciting opportunity to showcase our achievements and goals, as well as receive constructive feedback from our peers.

HiPerGator: Unleashing Computing Power

Our week concluded with the exhilarating news of gaining access to HiPerGator, a supercomputing resource that promises to be a game-changer. HiPerGator’s computational prowess opens doors to tackling the most complex and demanding tasks our project demands. We’ve secured a powerful ally to propel us further into uncharted territories of knowledge and innovation.

While we are making remarkable strides in our project, it’s important to remember that specific details must remain confidential. We are excited about the opportunities and challenges ahead as we move forward. Our commitment to excellence and continuous improvement drives us to push boundaries and reach new heights in our field.

Stay tuned for more updates on our journey!

Tagged as: , , , , ,

WEEK #5: Navigating the Future with EchoPilot

Published:October 7th, 2023
Category:Fall Semester

Are you ready to take a peek behind the scenes of our ongoing project? We’ve been making significant progress, and we’re excited to share our accomplishments, future plans, and open issues with you.

Corpus Collection

In order to implement a RAG system into ChatGPT, we have been focused on collecting an introductory corpus that will allow us to start making prototypes of the EchoPilot. We’ve expanded our knowledge base by collecting diverse documents, including video transcripts and JAWS Training manuals, enriching the EchoPilot dataset. We have also looked into gathering metadata collected by Freedom Scientific that might be able to provide the LLM with context, so it can make more accurate predictions. Now with the basic corpus collected, we will be able to start working on the LLM next week.

White Belt Training Completed

Last weekend we were gifted with the opportunity to work with Florida Power and Light to complete their White Belt Training workshop. This was a workshop geared towards helping us understand how to optimize efficiency and workflow in a work environment. We would like to give a shoutout to FPL once again for hosting this event and it was a pleasure.

PDR Report Draft

Our team is on track to deliver the PDR Report Draft, marking a significant milestone in our project’s development. This report lists out all of our team’s preparations and contains all of our ideas that we have planned for this exciting project.

That concludes our blog post this week. I would like to thank you for taking the time to read our blog and hopefully, we’ll see you next week. Chao!

WEEK #4: Exploring the Depths of JAWS: Insights from Meetings

Published:September 28th, 2023
Category:Fall Semester

We dove headfirst into our mission to better understand and improve the user experience of JAWS, the renowned screen reader software. Our meetings were packed with valuable insights, both from our own team members and internal experts. In this blog post, we’ll provide an overview of our activities and key takeaways from these meetings.

Understanding JAWS

One of the highlights of our week was our meeting with Liz from the Freedom Scientific team who we met with September 25, 2023. Liz, who uses JAWS daily, gave us a hands-on demonstration of how the software works. She showed us how to navigate Word documents and webpages, shedding light on the challenges users face when dealing with webpages that don’t interact seamlessly with JAWS.

We also had the opportunity to meet with Glenn, a software engineer at Freedom Scientific and one of the founding engineers behind JAWS, who joined us on September 27, 2023. This meeting delved into the technical side of JAWS, including its coding language and infrastructure. Glenn’s insights were instrumental in shaping our project’s direction. We discussed the possibility of launching EchoPilot up in the cloud to solve the issues between different coding languages.

Tasks and Brainstorming

Our primary tasks revolved around understanding JAWS comprehensively, exploring its capabilities, and identifying its limitations. This knowledge will serve as the foundation for brainstorming ways to address these limitations through our project. We also sought to gain insights into the inner workings of JAWS, including the coding language used. In addition to our focus on JAWS, we also delved into research on CoPilot, a related technology that could play a pivotal role in our project.

With a week brimming with meetings and enlightening discussions behind us, our resolve to enhance the user experience of JAWS has grown stronger than ever. The invaluable insights shared by Liz and Glenn from Freedom Scientific have enriched our perspective, and our ongoing research and brainstorming sessions are driving us toward our goal. Keep an eye out for further updates as we navigate the intricacies of JAWS and work tirelessly to improve it for our visually impaired users.

Tagged as: , ,

WEEK #3: EchoPilot Soars

Published:September 25th, 2023
Category:Fall Semester

This past week has been a whirlwind of research, discussions, and team branding refinement as our team, now the EchoPilot team, geared up and laid the groundwork for the preliminary design portion of our project with Freedom Scientific.

Research Mode

The team kicked off the week by diving headfirst into exploring Retrieval Augmented Generation (RAG) and LLM implementations. This groundwork was essential for our initial preliminary design planning and our more in-depth project briefing with our Freedom Scientific liaison engineer.

Product Design Specification (PDS) Planning

During class meetings, our team hashed our initial understanding of the project stakeholders, customer needs, and design requirements. We evaluated vital components and mapped out their relationship, which will serve as a solid foundation for our forthcoming PDS documentation and revision.

EchoPilot Takes Flight

Exciting news! We unveiled our team name, EchoPilot, and our logo. The response was a mix of criticism, praise, and valuable suggestions for improving the logo. We took these comments to heart and incorporated them into our final design, which we proudly present to everyone.

Liaison Engineer Meet and Greet

The highlight of our week was our meeting with Sriram Ramanathan, the liaison engineer from Freedom Scientific. We discussed our evolving understanding of the project, tackled concerns and questions, and requested access to resources crucial for our initial project implementation steps.

As we wrap up another week, Team EchoPilot feels more prepared and motivated than ever. The journey ahead is filled with challenges and opportunities, and we can’t wait to see where it takes us. Stay tuned for more updates as we continue to navigate the exciting world of computer science and bring our project to life. Until next week!

Tagged as: , , , ,

WEEK #2: Promising Beginnings

Published:September 17th, 2023
Category:Fall Semester

Hey there! We are thrilled to share our exciting journey as our team of four works with our coach, Dr. Catia Silva, and our sponsor, Freedom Scientific, to develop a multilingual conversational assistant for screen reader operations using Large Language Model (LLM) Technologies.

Meeting the Coach: Setting the Tone

Despite the delay from the hurricane, the team kicked off with a fantastic start this week. We held our first meeting with our coach, where we discussed the project’s scope of work, expectations, and further preparations to meet with our liaison engineers by the following week.

Delegation and Team Charter: Our Roadmap to Success

One of the highlights of the week was creating our team charter. This document serves as the team’s compass, outlining the team’s values, expectations, and responsibilities. With our coach’s suggestions, the group unanimously settled to delegate roles and tasks with a bi-weekly rotational system in place. This dynamic approach ensures that every team member understands the project comprehensively, enriching their overall experience in a holistic approach.

Kanban: Keeping Tasks on Track

To keep our workflow organized, the team adopted a Kanban board. This visual tool has already proven invaluable in helping us manage our tasks effectively. With its simple and user-friendly interface, members are able to get a glance at where each task stands, making it easier to collaborate and ensure nothing falls through.

Reflecting on our first week, we are optimistic about the journey ahead. Stay tuned for more updates on our adventures as a student team. We can’t wait to share our successes, challenges, and lessons learned as we work together towards our common goals.

See you next week!

Tagged as: , , , , ,

<< Newer