Text Analyzer API

A FastAPI-based text analysis service that uses Ollama for AI-powered text processing and PostgreSQL for data storage. The API extracts structured information from unstructured text including summaries, titles, topics, sentiment, and keywords.

Features

• Text Summarization: Generate 1-2 line summaries using Ollama
• Title Generation: Create concise, descriptive titles
• Topic Extraction: Identify main topics from text
• Sentiment Analysis: Determine positive, negative, or neutral sentiment
• Keyword Extraction: Extract the 3 most relevant keywords using YAKE
• Database Storage: PostgreSQL integration for storing analysis results
• Docker Support: Complete containerization with Docker Compose

Prerequisites

• Docker and Docker Compose
• Ollama installed and running locally

Setting Up

1. Install Ollama (if not already installed):

   # Visit https://ollama.ai/ to download and install Ollama
   # Or use the installation script for your OS

2. Pull and run a model in Ollama:

   # Download a model (default is llama3.2:3b)
   ollama pull llama3.2:3b
   
   # Start Ollama server
   ollama serve

3. Clone and setup the project:

   git clone https://github.com/Robin-07/text-analyzer.git
   cd text-analyzer

4. Start the application:

   docker compose up --build

API Usage

POST /analyze

Analyze unstructured text and extract structured information.

Request Body:

{
  "text": "Your unstructured text here..."
}

Response:

{
  "id": 1,
  "summary": "Brief 1-2 line summary of the text",
  "title": "Generated title for the text",
  "topics": ["topic1", "topic2", "topic3"],
  "sentiment": "positive|negative|neutral",
  "keywords": ["keyword1", "keyword2", "keyword3"],
  "created_at": "2024-01-01T12:00:00Z"
}

Example using curl:

curl -X POST "http://localhost:8000/analyze" \
     -H "Content-Type: application/json" \
     -d '{"text": "The weather today is absolutely beautiful. The sun is shining brightly and there are no clouds in the sky. I think I will go for a walk in the park this afternoon."}'

GET /search

Search and filter previously analyzed texts by various criteria.

Query Parameters:

• title: Filter by title (partial match, case-insensitive)
• topics: Comma-separated list of topics to filter by
• sentiment: Filter by sentiment (positive, negative, neutral)
• keywords: Comma-separated list of keywords to filter by
• limit: Number of results to return (max 100, default 10)
• offset: Number of results to skip (default 0)

Response:

{
  "results": [
    {
      "id": 1,
      "summary": "Brief summary...",
      "title": "Generated title...",
      "topics": ["topic1", "topic2"],
      "sentiment": "positive",
      "keywords": ["keyword1", "keyword2"],
      "created_at": "2024-01-01T12:00:00Z"
    }
  ],
  "total": 25,
  "limit": 10,
  "offset": 0
}

Examples using curl:

Search by sentiment:

curl "http://localhost:8000/search?sentiment=positive&limit=5"

Search by title and topics:

curl "http://localhost:8000/search?title=weather&topics=nature,outdoor"

Search with multiple filters:

curl "http://localhost:8000/search?sentiment=positive&topics=weather,nature&limit=5"

Testing

The project includes basic tests written using pytest:

# Run tests (inside virtual env)
python run_tests.py

Database Schema

The application uses the following database table to store analysis results:

CREATE TABLE analysis_results (
    id SERIAL PRIMARY KEY,
    original_text TEXT NOT NULL,
    summary TEXT,
    title VARCHAR(500),
    topics TEXT,
    sentiment VARCHAR(50),
    keywords TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Design Components

Ollama was chosen as the LLM server because it provides local, privacy-preserving AI inference without requiring cloud API keys or internet connectivity, while supporting a wide range of open-source models that can run efficiently on consumer hardware. YAKE was selected for keyword extraction because it's language-independent, requires no pre-trained models, and delivers consistent results with minimal computational overhead. FastAPI was chosen for the web framework to handle concurrent requests efficiently, while PostgreSQL with asyncpg provides robust data persistence and complex querying capabilities for the search functionality. Docker Compose ensures consistent and simplified deployment across environments, making the entire stack portable and easy to set up with a single command.

Pending Improvements

• OpenAI/Claude API integration for powerful models.
• Production-grade inference engine like vLLM/TensorRT-LLM
• Asynchronous/Background Analysis processing
• Robust and Detailed Error Handling
• Authentication/Rate Limits
• Metrics Integration (Prometheus/Grafana)