Blog Posts

Week 19: Demos and Renos

The latest updates on SyncAssist’s mobile UI

Over the past week, mobile development has been going swimmingly. We managed to get the mobile development environment set up properly on all our team members’ devices and started making basic screens allowing users to make new conference rooms and join the conference. With the refactoring that we accomplished last week, along with some additional refactoring of conference functions, we’re feeling confident that we can implement the major features requested by our liaisons for mobile in a fairly short amount of time.

Unfortunately, in our quest to streamline our mobile development, we had to somewhat abandon our desktop and web app for a bit, and the repercussions of this became clear at our regular liaison meeting this week. During the meeting, we attempted to do a live demo of our app to show off our improvements in remote control latency and screen-shared video quality, but instead ended up showing off the plethora of bugs in our state management and our rather unremarkable video/audio quality.

But as our IPPD Lab Manager Larry Warren likes to say, “Bad news is good news. Good news is no news. And no news is bad news”. With these bugs exposed, we now have renewed priorities: make our baseline video, audio, and remote control experience as seamless and performant as possible.

With that in mind, we’ve refreshed our isuses list to add priorities to any bugs related to video/audio performance and state management. Here’s a quick sneak peek:

SyncAssist’s issue list

Next week we’ll hopefully have updates on the improved performance of our app, so stay tuned for that!

Week 18: Refactor extractor

SyncAssist’s target supported platforms (mobile, desktop and browser)

This week, SyncAssist has been doing a pretty heavy overhaul of our code! To start our mobile development work, we realized that several of the functions and states that we would be using for our conference handling would have to essentially be copied into our mobile app. So, to avoid this, we began by refactoring our states into global contexts, which allows us to reuse the states on the mobile side as well as avoid having to pass the state down as props.

Along with refactoring, we’ve fixed a number of existing bugs. Here’s the list, pulled straight from our commit history: fixed the home page, fixed user joining getting a message that the host joined, disabled remote control requests for non-Electron users, patched react-native-performance, reset the screen-sharing states, fix low screen-share FPS, update logic to stop screen-sharing, fix share screen, fix div error with RemoteControl.tsx β€” and these are only a fraction of the commits. It’s almost like every time we create something, we introduce new bugs πŸ˜ƒ.

Although it’s been tough trying to implement new features while fixing old bugs, we’re making slow but steady progress and are planning on having most of the mobile app features built out over the next two weeks. Stay tuned for updates on our mobile app, and until next time!

Week 17: Mobile Development, start!

SyncAssist’s issue board

With most of our major presentations out of the way for now, SyncAssist is now ready to start our major project for this semester: development of our mobile app! As mentioned in Week 15, we’d decided to use the Expo Go framework with React Native, which allows us develop iOS apps without building on a Mac. Unfortunately, we’ve run into some minor roadblocks in this area.

It turns out that the Expo Go app that you can download from the App Store to run your app has no built-in support for WebRTC. This means that we’ll have to use Expo Dev Clients instead and build the app locally, which means we need Xcode 😭. Fortunately, one of our team members has a Macbook, so we’ll be offloading all of the iOS app building to him for now.

We also ran into some potential issues with integrating lib-jitsi-meet into our mobile app. Since it was a primarily web-focused library, it uses browser constructs like Document.querySelector and DOM traversal that aren’t a part of React Native. Fortunately, it looks like the developers of the Jitsi Meet mobile app solved this issue with polyfills for all of those browser functionalities, so we shouldn’t have much of an issue copying what they did.

Week 16: Project Pitch Presentation!

Our long-awaited trip has arrived! This week, we traveled to Clearwater, FL to pitch our project to the directors and employees at Vispero.

To prepare for the presentation, we drove down the day before and arrived at our hotel in the evening, where we met our industry liaison, who treated us to dinner at a Brazilian steakhouse and boba for dessert (big thank you to Sriram!) After dinner, we rehearsed our presentation and were given several helpful tips for adjusting our presentation structure by Sriram.

After a long night of presentation revisions and Switch games, we woke up bright and early the next morning to rehearse our pitch one final time before making our way to Vispero’s Clearwater location. At the office, we had the opportunity to network with Vispero employees (several of whom were IPPD alum as well!) over pizza before delivering our final pitch. Here’s a few snapshots from our day!

Our presentation was very well received by our audience (somewhat to our surprise), and they asked us valuable questions about certain topics that we hadn’t considered in depth before, such as the potential security issues of remote control as well as the possibility of remoting into a computer that does not have JAWS open.

Overall, this was an incredibly valuable experience and we’re more ready than ever to tackle our project and bring it to life! And of course, a big thank you to our liaisons Sriram and Jake for helping us along the way and making this day happen!

Week 15: Back from break!

Freedom Scientific’s corporate office in Clearwater, FL

SyncAssist is back from break and hitting the ground running! This past week, we’ve been fleshing out our plans for this semester, preparing for our visit to Freedom Scientific’s office in Clearwater, and cleaning up our code. Finally, the long standing “Remote-Control” branch that’s 62 commits ahead and 12 commits behind main will be merged β€” huzzah!

After we deliver our pitch at Clearwater, our main priorities for the project will be abstracting away the Jitsi internals and having a cleaner interface for sending arbitrary data, as well as beginning development for the mobile app. To avoid having to buy Macs for everyone, we’ve decided to use Expo along with React Native, which manages the underlying iOS and Android systems so that we can focus on using purely React. More detailed mobile development plans are in the works, but our main goals are to reuse as much code as possible and to decouple the logic from the UI.

Stay tuned for updates on how our project pitch goes, and until next time!

Week 14: SLDR + End of Semester!

Snapshots of SyncAssist’s SLDR day

This week, SyncAssist gave our SLDR presentation! After some last minute presentation fine-tuning with the help of our liaison Sriram, we were able to give our best presentation of this semester and really convey the work that we’ve put into our project! During the SLDR day, we also had the opportunity to network with industry professionals and get advice on how to navigate the workplace environment, and it was overall an incredibly valuable experience.

With the SLDR event past, this officially marks the end of our fall semester. In the spring, we’re looking forward to visiting Freedom Scientific’s office in Clearwater and presenting our work! We also have big plans in the works for our app, with simulated braille input/output and mobile development on the horizon. Over the break, we will probably be cleaning up our code to make sure that it’s all fairly readable and easier to build off of, but we’ll also make sure to get plenty of rest before the next semester starts. Until then!

Week 13: SLDR Drafts and Gaffes

A word cloud of the 100 most common words in SyncAssist’s SLDR draft

We’ve shifted gears back into IPPD mode this week, with the SLDR (System Level Design Review) Draft and peer review presentation both due this Tuesday! At 8300 words long, the SLDR report was definitely a beast to tackle, but we managed to have a workable draft and got good feedback on how we can improve the formatting and content of the paper.

We also got great pointers on how to make our presentation better during the SLDR peer review event β€”some of the major points included making the use cases for the app a bit more clear, and emphasizing the human side of our project. This advice was great because as engineers, we can sometimes get caught up with the technical aspects of projects, so it’s nice to be reminded to that our projects are really human-centered, and thus our end users should be a big part of our process.

Since next week is fall break, we’ll be slowing down a bit, but after the break we’ll have our final SLDR presentation and draft due, so stay tuned for more updates to come!

Week 12: Prototype Presentations

SyncAssist and Coach Grant at Prototype Inspection Day!

Lots of exciting things for Team 5 this week!

First, we presented our project progress to judges at Prototype Inspection Day (as seen above)! We showed off all of the video conferencing features that we’ve implemented, like screensharing, chat messaging, and the star of the show β€” remote control. Though we had some hiccups here and there with the technology and were a little bit shaky at first, overall the presentation went well! The judges gave us great feedback and seemed impressed with all the work that we’ve been able to accomplish in such a short amount of time.

SyncAssist with our first round of judges at PID

The other exciting thing is that we may finally be able to slow down our development work and rest a bit πŸ˜…. During our liaison meeting this week, we were encouraged to work mostly on improving the features that we’ve already built, rather than implementing new ones. This is fantastic news for us, since the speed of our work has made our codebase a bit unwieldy, and having the time to work on refactoring and improving will help us have a solid foundation to build off of starting next semester.

Since we’re having our SLDR peer review event next week, we’ll be working on preparing for that presentation over the next few days, so stay tuned for updates on how it goes!

Week 11: PID Planning, Feature Frenzy

Team 5 coding together

This week, Team 5 has been working nonstop! We were able to implement screenshare in the Electron version of our app, have working mute and unmute buttons, display a participants list, and more! Here’s a snippet of what it looks like now:

Since our Prototype Inspection Day is coming up, we’ve been in a rush to also implement remote control. There are a lot of different flows that we’ve been considering how to implement, like the initial agreement between the controller and the desktop being controlled, as well as how to end the remote control session. During our liaison meeting this week, we also got a bit of feedback on which features to prioritize, since our main audience is blind or visually impaired people and the core of the app should be remote control.

Overall, though, we’re making good progress and we’re excited to see how this app develops!

Week 10: New Beginnings

Team 5 at our weekly planning meeting

You might recall from last week’s post that we planned to pivot our app development to modifying the source code of Jitsi Meet. About an hour after the post was published, we made another pivot back to developing our app from scratch using the API provided by lib-jitsi-meet. Although it might seem like we can’t stop changing our minds, we’ve made some huge progress in app development over the past week so it seems like this plan is here to stay.

Through lib-jitsi-meet, we were able to implement camera and audio sharing, screen sharing, chat functionality, and basic routing for different conference rooms in only a few days! Here’s a sneak peek of what our app looks like right now:

Meeting room page of our app

Over the next week, we’ll be continuing to add features to our app like remote control, as well as debugging existing features. One of the only downsides of starting a new app from scratch like this is that the codebase changes very frequently, and often drastically so. To get around this, we’ll be meeting to code together and keep everyone in the loop!