Week by Week Breakdown
Week 1: Introduction and project Management
An introductory lecture was given to us on the graduation project’s organisation and planning, and it provided some useful insights.
Week 2: User Research
We had an outstanding interactive session on user research and the required steps to conduct good problem framing, stakeholder mapping, and user interviews in order to develop the concept of our projects. Additionally, we discovered who our supervisors are.
Week 3: Online persona workshop
This class highlighted the impact and use of our online persona, which promotes professionalism and employment. Additionally, it was useful to understand how to use our graduation projects as a stepping stone to the future.
Week 4: Is my project idea ok? – Idea discussion
I had my first meeting with my supervisor in which I shared my idea and possible ways forward. My idea got accepted so I started setting my own deadlines for our future meetings.
Week 5: Supervision week– Research
At the following meeting, I was tasked with presenting my first blog post, furthering this we discussed the intended project research, including the development process, potential issues and solutions, user research, and time management.
Spring break: Design modification + User Survey
During the week that we were off for spring break, I worked on the design that needed to be altered in order to be compatible with the new method of functioning of my app. A user survey, which was necessary for my research, was also something that I worked on at this period.
Week 6: Supervision week– Research of ML models and datasets
After presenting my findings from the previous week and the tasks I did doing the break, we analysed the image datasets that best suited the machine learning model I had to train. We observed several datasets, but the one that caught my attention required a request from a supervisor; what a fortunate coincidence that I had a great responsive supervisor to make the request on my behalf.
Week 7: Presentation week
This week, as a formative task, we were required to present our ideas, which went well. The fact that I had not completed a consent form for my survey was brought to my attention, so I proceeded to do so along with other suggestions from my peers and tutors.
Week 8: Captions database set up + machine learning model
During this week, I was advised to use a pre-trained model rather than build my own, as doing so would have taken a very long time and slowed down my development process. As the necessary dataset was finally accessible, I proceeded to discover appropriate models trained using that dataset and evaluate their accuracy in Google Colaboratory. In the meantime, I was working on the database of captions by extracting, categorising, and filtering them out.
Week 9: Swift UI design
This week was devoted on converting my design prototype from Figma to Xcode and implementing navigation, buttons, and essential functionality. This was the simplest portion of the coding, as it was rather uncomplicated and primarily concerned with design.
Week 10: Camera and gallery feature
This week, I focused on the camera implementation, which proved to be more difficult than anticipated and was riddled with errors over which I had no control. After numerous sessions with the technical team and a great deal of reading, I identified the problem and modified the way of installing the camera. It took far longer than anticipated, which significantly slowed down the rest of the tasks.
Week 11: Database implementation
After the database implementation also failed to work, I began to worry throughout this week. In the end, I was able to access the csv file via the CCI server and display it in the text fields, but I was held down somewhat by the dynamic filtering and sorting.
Week 12: Machine learning model integration + making it all work
Following the completion of the database’s activation and functionality, I had one week to integrate the machine learning model in the application and connect it to the database. After discovering a resource that helped me identify the coding method for all of this, I felt at ease and my task became easier to accomplish.
Problems Faced
- Expected problem: The machine learning model might include biases
Solution: Proper data collecting and training of the model with said data could be beneficial in reducing the amount of human bias that is present in the code. The model might be trained more effectively by adopting an image dataset that is both comprehensive and diverse. Nevertheless, a model will always have its own biases, and while these can never be eradicated completely, we should constantly attempt to minimise them as much as possible.
- Expected Problem: Data extracted from sites could have copyright
Solution: Conduct research, check to see whether any data extractions violate copyright laws, and make use of open-source websites. Include reference in the database, specifying which captions were retrieved from which sites.
- Actual Problem: A large training dataset for the Machine Learning Model might take days to train properly, special equipment needed
Solution: I came to the conclusion that utilising a pre-trained model would be more practical and time-saving than building my own model from scratch. Ensuring that the model is likewise available for free usage and has an open source licence.
- Actual Problem: Implementing a method of categorising the captions in the database
Solution: Due to the fact that the way in which I planned to categorise the captions—which was algorithmically by employing another training model—did not work, I was compelled to manually add categories to each caption, which resulted in a higher amount of precision but required a significant amount of additional time investment.
- Actual Problem: Image classification of the model predicting only one facial expression
Solution: Due to a fault with the facial expression labelling, every image was predicted as “neutral”. To address the issue, I had to examine the code where the images are fetched and processed, where I discovered that the problem was their format. The model expected jpg, but some files were in an incompatible format, therefore I had to alter the required file type to any file format.
- Actual Problem: Inaccurate machine learning model
Solution: Once the problem with the labelling was handled, it became clear that the model was inaccurate, a problem that could not be resolved because it is pre-trained and so beyond my control.
How the process went
In the beginning, everything went according to plan, I was able to meet my targets and timeframes, and even get a head start on other tasks. Until the coding part provided several challenges and issues, particularly with regard to camera integration and dynamic database filtering and sorting. This slowed down the machine learning model integration which is the most important part of the whole project. Once the facial analysis was operational, I discovered inaccuracies in the model, but it was too late to integrate another model or sort out the inaccuracies because the deadline was approaching, and this would have been a risky move at this point in the project timeline.
Overall, the path was challenging, but essential to my development since it taught me patience, time management, and a great deal of new coding skills. It was enjoyable to work on such a large project for the first time, as it allowed for the use of a great deal of imagination, as well as research both online and from CCI professionals, which was very beneficial when learning new methods and approaches.