Small scaffold for an event-driven research assistant backend (Python, FastAPI, Kafka, Vector DB).
Key highlights
- Developed event-driven, distributed backend services for processing unstructured data across multiple sources.
- Built streaming pipelines with Kafka, reducing latency by 30%.
- Designed scalable APIs and backend services to support concurrent processing workloads.
- Containerized microservices, increasing concurrent request handling by 4×.
Quickstart
- Create a virtual environment and install dependencies:
python -m venv .venv
source .venv/Scripts/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt- Run the FastAPI app:
uvicorn src.main:app --reload --port 8000- POST text to
/ingestto exercise the ingest endpoint (placeholder for Kafka publish).
Project layout
src/— application code (FastAPI app, Kafka skeleton, vector db placeholder)requirements.txt— Python depsDockerfile— container image for the app.env.example— example environment variables
This is a minimal scaffold. You can iterate with me to convert it to production grade.