|
| 1 | +--- |
| 2 | +title: "GSoC '25 Week 9 Update by Elwin Li" |
| 3 | +excerpt: "Improvements on the musicblocks generation RAG pipeline and a new MIDI file uploading widget" |
| 4 | +category: "DEVELOPER NEWS" |
| 5 | +date: "2025-08-02" |
| 6 | +slug: "2025-08-02-gsoc-25-Elwin-Li-week09" |
| 7 | +author: "@/constants/MarkdownFiles/authors/elwin-li.md" |
| 8 | +tags: "gsoc25,sugarlabs,week9,RAG pipeline,MIDI widget" |
| 9 | +image: "assets/Images/GSOC.png" |
| 10 | +--- |
| 11 | + |
| 12 | +<!-- markdownlint-disable --> |
| 13 | + |
| 14 | +# Week 9 Progress Report by Elwin Li |
| 15 | + |
| 16 | +**Project:** MusicBlocks Generation Model |
| 17 | + |
| 18 | +**Mentors:** [Walter Bender](https://github.com/walterbender), [Anindya Kundu](https://github.com/meganindya), [Devin Ulibarri](https://github.com/pikurasa) |
| 19 | + |
| 20 | +**Reporting Period:** 2025-07-27 - 2025-08-02 |
| 21 | + |
| 22 | +--- |
| 23 | + |
| 24 | +## Goals for This Week |
| 25 | + |
| 26 | +- **Goal:** Complete the MIDI generation RAG pipeline |
| 27 | + |
| 28 | +--- |
| 29 | + |
| 30 | +## This Week's Achievements |
| 31 | + |
| 32 | +This week I upgraded the RAG pipeline with the Gemini embedding model, which resulted in very accurate retrieval. I also used Gemini for the generation component, which also showed much better results compared to previous approaches. |
| 33 | + |
| 34 | +The biggest update for this week is the creation of a MIDI upload widget. This widget allows users to upload any MIDI file and it will automatically generate the corresponding MusicBlocks. In the video demo below, I demonstrate using the widget to upload a MIDI file that was generated using the RAG pipeline with the query "guitar solo with similar style to hotel california". The widget also includes the option to directly generate a MIDI file and convert through a prompt, but it currently does not work, and will be next weeks plan. |
| 35 | + |
| 36 | +[youtube: B0lPJwHc4pI] |
| 37 | + |
| 38 | +--- |
| 39 | + |
| 40 | +## Challenges & How I Overcame Them |
| 41 | + |
| 42 | +- **Challenge:** The RAG pipeline was not able to generate music directly to MusicBlocks format. |
| 43 | + |
| 44 | + **Solution:** Used MIDI as an intermediate format, creating a MIDI upload widget that converts MIDI files to MusicBlocks, effectively bridging the gap between the generation pipeline and the MusicBlocks format. |
| 45 | + |
| 46 | +--- |
| 47 | + |
| 48 | +## Key Learnings |
| 49 | + |
| 50 | +- Gained deep understanding of RAG (Retrieval-Augmented Generation) systems and their application to music generation. |
| 51 | +- Learned about the importance of better embedding models (Gemini) for improved retrieval accuracy. |
| 52 | + |
| 53 | +--- |
| 54 | + |
| 55 | +## Next Week's Roadmap |
| 56 | + |
| 57 | +Connect the MIDI generation pipeline with the MIDI upload widget, so that the generated music will automatically be in MusicBlocks format. |
| 58 | + |
| 59 | +--- |
| 60 | + |
| 61 | +## Acknowledgments |
| 62 | + |
| 63 | +Thank you to my mentors, the Sugar Labs community, and fellow GSoC contributors for ongoing support. |
| 64 | + |
| 65 | +--- |
0 commit comments