This system provides AI-powered real-time environmental understanding, navigation instructions, speech interaction, and gesture communication. Designed to assist blind and low-vision users, it uses a combination of:

• YOLOv8n Object Detection
• Smart Narration Engine (priority-based)
• Groq LLaMA language model
• On-device Speech Synthesis
• Whisper Speech-to-Text
• Sign Language Detection Models

All wrapped inside a clean Streamlit mobile-friendly interface.

Demo Scenarios

We designed scenarios that highlight the system's strength:

Blindfold Walk-through

Place chairs/tables unpredictably → system guides user.

Sudden Obstacle / Danger

Simulate approaching object → triggers urgent warning.

Describe Scene

Upload a room photo → full semantic description.

Voice Commands

Use STT to navigate or interact.

Sign Language

Demonstrate gesture → text translation.

Speech → Sign

Say a sentence → system outputs animated signs.

How to set it up

1. run the frontend via streamlit run frontend/app.py
2. navigate to the backend folder and run uvicorn main:app --host 0.0.0.0 --port 8000 --reload
3. make sure to set up your venv and .env file