Reflection Learning Widget

[Commanderk3]

Backend Code: Fast API code

Technical Implementation

Phase 1: Project Setup

• The first step was deciding which LLM to use, since it forms the core of the chatbot. I initially experimented with Llama-3 running locally, but later switched to Gemini API.
• I explored various sentence transformers available on HuggingFace. These play a crucial role in building a reliable RAG system. I ultimately chose the sentence-transformers/all-MiniLM-L6-v2 model because it is both lightweight and accurate.
• I learned and implemented a basic RAG pipeline. At first, I used ChromaDB as the vector database, but eventually migrated to a Qdrant cluster, a cloud-based vector database.
• The chatbot was hosted on Streamlit to allow mentors to test it.
• A FastAPI backend was developed with /chat, /summary, and /analysis endpoints.

Phase 2: Adding Features

Multi-Agent Chatbot

• To make reflection more engaging, three persona-based AI agents were introduced:
  • Rohan – for general reflection practice.
  • Alan Kay – for logical and coding-related reflections.
  • Ludwig van Beethoven – for reflections on musical aspects.

Enhanced Context Passing

• To better understand the user’s project, the chatbot now leverages the JSON conversion function originally developed for the AI Debugger. The resulting flowchart is transformed into an algorithmic summary, which helps the LLM interpret the project’s structure.
• In addition, the descriptions of the blocks used in the project are also passed to the LLM.

Automated Periodic Summaries

• Previously, users had to click a separate button to generate summaries. However, based on feedback from Streamlit app testing, the system was updated to generate summaries periodically on its own, improving the overall user experience.

Reasoning Model Integration

• I experimented with reasoning models, which employ a chain-of-thought approach to simulate logical reasoning. This significantly improved the LLM’s ability to understand user projects.
• As a result, the algorithmic summary is now generated using a reasoning model, while the rest of the conversation is handled by a base model.

Phase 3: Fast API Backend

• Initially, FastAPI was not part of the setup. Later, it was upgraded to include new API endpoints such as /chat and /analysis. Over time, additional endpoints like /projectcode and /updatecode were also introduced.

You can learn more about this here.

Phase 4: Native Widget Integration

• Developed a JavaScript widget and integrated it into the Music Blocks application.
• Added real-time project analysis that continuously evaluates the user’s project state, produces an algorithmic summary, and initiates reflection with the user.
• Built a conversational interface with session-aware tracking, allowing seamless dialogue that carries over context from previous exchanges.
• Implemented localStorage support to store analysis reports, which can be reused for generating updated insights.
• Introduced a Download as Text button for easy report export.
• Added a Refresh button to ensure the AI receives the latest code whenever the user makes changes.

Phase 5: Deploy to EC2

• The FastAPI server was deployed on an AWS EC2 t3.micro instance running Ubuntu 22.04 LTS.
• Security groups were configured to allow HTTP, HTTPS, SSH, and FastAPI’s custom port.
• Since the default storage and memory were limited, the disk was resized and a swap file was created to prevent crashes.
• The project repository was cloned, dependencies installed in a virtual environment, and environment variables configured. - - - Finally, the FastAPI app was set up as a systemd service, allowing it to run in the background, restart on failure, and start automatically on boot.

---

Data Flow Diagram

---

Tech Stack and Tools

Frontend

• Technologies: HTML, CSS, JavaScript
• Component: Music Blocks widget (chat-based interface)

Backend

• Framework: Python FastAPI with Uvicorn server, LangChain
• LLM Services: Google Gemini API
• Sentence-transformers (embedding generation)

Vector Database

• Qdrant cluster for semantic search and storage

Infrastructure

• AWS EC2 (deployment)
• systemd (service management)

---

Project Code

• Backend Code: Fast API server
• Frontend Code: Reflection Widget

Pull Requests

• Music Blocks Reflection Widget (Frontend)
• Documentation for Developers
• Documentation for Users

Weekly Blogs

• Week 1
• Week 2
• Week 3
• Week 4
• Week 5
• Week 6
• Week 7
• Week 8
• Week 9
• Week 10
• Week 11
• Week 12

---

Key Learnings

Frameworks and Tools

• Gained hands-on experience with FastAPI and Streamlit, including building prototypes and integrating frontend widgets with backend services.
• Learned to work with LangChain as the primary development framework.
• Explored and deepened understanding of Gemini and LangChain documentation, covering advanced LLM concepts like structured output, function calling, reasoning, and prompting techniques.

APIs and Backend Development

• Learned how API endpoints function and how to manage API keys securely during deployment.
• Improved understanding of structuring API requests and responses for a smoother user experience.

Prompt Engineering

• Discovered the importance of refining prompts and using few-shot prompting to achieve more accurate and context-aware LLM outputs.

Deployment and Infrastructure

• Learned how to deploy an EC2 instance on AWS.

Learning Approach

• Realized the importance of relying on official documentation over tutorials or blog posts, since it provides the most reliable and up-to-date information.

Future Development Goals

• Integrate Sugar-AI to enhance capabilities in the near future.
• Add multi-language support to enable international educational use.
• Implement offline functionality using small language models for resource-constrained environments.
• Extend the reflection learning interface to additional Sugar apps like Sugar Journal.

---

Acknowledgments

This project was carried out under the mentorship of Walter Bender, Sumit Srivastava, and Devin Ulibarri as part of Google Summer of Code 2025 with Sugar Labs. Its successful implementation highlights the collaborative spirit of open-source educational technology and the dedication of the Sugar Labs community to creating accessible learning tools.

Special thanks to the wider Music Blocks community for their valuable feedback during development and testing, and to Google Summer of Code for making this contribution to educational technology possible.

---

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[walterbender]

This is working great. Except one small issue. If you hit enter instead of click on the SEND button, the MB project runs. We probably want to disable all keyboard shortcuts while the widget is open.

[Commanderk3]

This is working great. Except one small issue. If you hit enter instead of click on the SEND button, the MB project runs. We probably want to disable all keyboard shortcuts while the widget is open.

working on it

[Commanderk3]

@walterbender Bug fixed

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.

[github-actions]

✅ All Jest tests passed! This PR is ready to merge.