Blog Posts

Week 4 Update

New concatenation of plug ID for pin models

The team has been working hard to finish collecting our new dataset and further improve our two pin classification approaches. We plan to have the new dataset completed by the weekend. So far, we have 11 out of 15 plugs in a temporary dataset which has been used for preliminary testing. We have noticed sizeable improvements in performance in the deep learning plug and pin classifiers with this new dataset. Along with the new dataset, the deep learning models can now be configured to take additional inputs other than images. There is opportunity to take information from the feature extraction portion and provide it as new inputs to the deep neural networks.

For the feature extraction approach for pin classification, most work has been completed. Fine-tuning will continue along with fixing small issues.

Week 3 Update

Pin mapping step of feature extraction

In the past week we have made some more progress on our pin identification feature extraction technique. We have completed the first phase and have moved on to pin mapping. This is a step where we create a map of pin locations to be applied on a detected plug during live use. Relative location of the probe tip to these mapped pin locations can yield our pin identification.

For our deep learning approach, we have improved the training process by logging more data to see how our results change as we tune the model.

We are pushing through our new data collection. As of today, we have five out of fifteen plugs collected. Someone from the team has been going into the lab every day to collect a plug.

Semester 2, Week 2 Update

Newly marked plug
Feature extraction prototype

We began making more progress on our pin classification using both deep learning and feature extraction. These two approaches are being experimented with and in the next two weeks we will choose one to develop for rest of the semester.

For the feature extraction method, we initially colored the central ring of the plug. This bright color of green allowed us to match pixels by color in order to reliably find the ring in software. From there, an ellipsis is matched to the ring to find our boundary for pin identification. Additional dots were made that identify the center and top of ring.

The deep learning approach has been implemented as a branch of our original plug classifier. We need to perform further testing and tuning on this part. Initial results show that the model may have difficult classifying fine-grain differences between pins.

We are excited to present for QRB 1 next week and hear feedback about our approaches.

Semester 2, Week 1 Update

Plug view without LEDs
Plug view with new LEDs

This week the team began working again for the beginning of the second semester of IPPD. During the break, we made critical model improvements that helped us gain a lot of confidence in our approach for plug classification. We are very close to our plug classification performance targets.

Moving into the next semester, we have coordinated our efforts and developed a plan to work on the pin identification portion of our project. The team has spent time with our liaisons and coach to find unique ways of solving this second problem. A notable improvement we have made in the last week was the addition of LEDs to our multimeter probe. This allows us to have a more uniform image brightness which can help us. With this, however, we must spend a lot of time to collect new data for our models.

Along with the continuous development of our product, we are preparing for QRB 1 in two weeks. Overall, we are pleased with the status of our project and are looking forward to push through this final semester!

Week 15 Update

Picture of the team after presenting our SLDR.

This week the team was able to present our System Level Design Review to a couple of our liaisons as well as some industry guest from different IPPD sponsors. Overall the event went great and everyone performed well. During the event the team received some valuable feedback from the attendees at the presentation that the team will use to improve the project. We updated our sponsors with our plans for next semester where we hope to continue to make progress for our project!

Week 14 Update

Title card for the team’s SLDR presentation.

The team has been preparing for the System Level Design Review coming up on Tuesday, December 7th. At the presentation the team plans to show the current progress that has been made on the project, as well as the plans for the upcoming spring semester.

Besides the SLDR, after receiving some great feedback from Prototype Inspection Day the team was able to make adjustments to different aspects of the project. Additional testing has been done on the type of architecture that the dataset will be used for the CNN. There has been discussion on adding additional light to the current hardware design as well to create a better dataset.

Week 12 Update

Photo of our team after finishing presenting our prototype.

This week our team presented our prototype where we receive a lot of useful and positive feed back from the inspection committee that attended the event. Using the comments received, the team now has some new topics to research as well as work on any problems that showed up during the prototype presentation. After identifying some differences in the results from the testing and live implementation, we have decided to revisit data collection to improve the overall quality of the data set.

The next major event that the team is looking forward to is the System Level Design Review where we hope to see our sponsors here at UF! At this event the team intends to show off all the progress that has been made this semester starting from the beginning. Plans for the next semester are also being made to ensure that the team stays on track for the second half of the project.

Week 11 Update

Training (orange) and Validation (red) graph for the spare categorical accuracy.

With our plans for the prototype inspection day completed last week, the team worked on preparing the material needed to present for the upcoming day. The team was able to train the model over 30 epochs for 5 different plugs and a script was created for the presentation that uses the model to predict the class for a single frame captured from a live video feed. The graph pictured above is from the same model that we will be using for prototype inspection day and can be considered a sneak peek at what will be presented!

Besides the prototype the team has identified different areas that still need to be worked on before we can really go full force into testing different models to use for the vision system. Additional data needs to collected and sampled for some of the plugs that have lower pin counts. Different metrics are being researched to see what is the best way to view the results for each test.

Week 10 Update

Simple diagram outlining the team’s plan for prototype inspection day.

This week the team completed the plan for the presentation on prototype inspection day. Work that still needs to be completed before then includes: completing the script to run through the different steps and training the model with the data set that was recently completed. The team aims to test the reliability of the model trained when used by users other than those within the team as well as see how a third party with limited knowledge might approach using the prototype created.

For the training process the team is working on organizing the data set in a way that it can easily be utilized to train the CNN through batch training. Different file structures were explored in order to efficiently store the data and be able to modify the data easily in the future.

Week 9 Update

The final plug that was needed to complete the data set.

This week the team continued to work on completing the data set as the main focus. We are pleased with the progress made as we now have the initial set completed! What’s left of the process will be to continue to expand the data set but through data augmentation techniques. With the majority of the work being completed for the data collection, the focus for the team will now be to test different CNN parameters to see which one results in the best plug classification accuracy.

For the upcoming prototype inspection day, we have a general idea of what we want to show off and what kind of data we want to collect from this presentation. The goal will be to show the data flow from the frame capturing to the image classification. We are looking forward to making the first step in a functional design of the project and we hope that we will gain some positive feedback during prototype inspection day!