Skip to content

Commit d87dd79

Browse files
authored
GSoC '25 week09 update by Elwin Li (sugarlabs#355)
1 parent e7028ed commit d87dd79

File tree

1 file changed

+65
-0
lines changed

1 file changed

+65
-0
lines changed
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: "GSoC '25 Week 9 Update by Elwin Li"
3+
excerpt: "Improvements on the musicblocks generation RAG pipeline and a new MIDI file uploading widget"
4+
category: "DEVELOPER NEWS"
5+
date: "2025-08-02"
6+
slug: "2025-08-02-gsoc-25-Elwin-Li-week09"
7+
author: "@/constants/MarkdownFiles/authors/elwin-li.md"
8+
tags: "gsoc25,sugarlabs,week9,RAG pipeline,MIDI widget"
9+
image: "assets/Images/GSOC.png"
10+
---
11+
12+
<!-- markdownlint-disable -->
13+
14+
# Week 9 Progress Report by Elwin Li
15+
16+
**Project:** MusicBlocks Generation Model
17+
18+
**Mentors:** [Walter Bender](https://github.com/walterbender), [Anindya Kundu](https://github.com/meganindya), [Devin Ulibarri](https://github.com/pikurasa)
19+
20+
**Reporting Period:** 2025-07-27 - 2025-08-02
21+
22+
---
23+
24+
## Goals for This Week
25+
26+
- **Goal:** Complete the MIDI generation RAG pipeline
27+
28+
---
29+
30+
## This Week's Achievements
31+
32+
This week I upgraded the RAG pipeline with the Gemini embedding model, which resulted in very accurate retrieval. I also used Gemini for the generation component, which also showed much better results compared to previous approaches.
33+
34+
The biggest update for this week is the creation of a MIDI upload widget. This widget allows users to upload any MIDI file and it will automatically generate the corresponding MusicBlocks. In the video demo below, I demonstrate using the widget to upload a MIDI file that was generated using the RAG pipeline with the query "guitar solo with similar style to hotel california". The widget also includes the option to directly generate a MIDI file and convert through a prompt, but it currently does not work, and will be next weeks plan.
35+
36+
[youtube: B0lPJwHc4pI]
37+
38+
---
39+
40+
## Challenges & How I Overcame Them
41+
42+
- **Challenge:** The RAG pipeline was not able to generate music directly to MusicBlocks format.
43+
44+
**Solution:** Used MIDI as an intermediate format, creating a MIDI upload widget that converts MIDI files to MusicBlocks, effectively bridging the gap between the generation pipeline and the MusicBlocks format.
45+
46+
---
47+
48+
## Key Learnings
49+
50+
- Gained deep understanding of RAG (Retrieval-Augmented Generation) systems and their application to music generation.
51+
- Learned about the importance of better embedding models (Gemini) for improved retrieval accuracy.
52+
53+
---
54+
55+
## Next Week's Roadmap
56+
57+
Connect the MIDI generation pipeline with the MIDI upload widget, so that the generated music will automatically be in MusicBlocks format.
58+
59+
---
60+
61+
## Acknowledgments
62+
63+
Thank you to my mentors, the Sugar Labs community, and fellow GSoC contributors for ongoing support.
64+
65+
---

0 commit comments

Comments
 (0)