diff --git a/MAINTAINER.md b/MAINTAINER.md new file mode 100644 index 000000000..f49855955 --- /dev/null +++ b/MAINTAINER.md @@ -0,0 +1,753 @@ +# For the Maintainer + +This section contains operational guidelines for maintaining, configuring, and supporting the application in production environments. + +## Configuration + +### Single-User vs Multi-User Setup + +#### Single-User Configuration +For development or single-team deployments: + +```env +# Single workspace token (default behavior) +SLACK_BOT_TOKEN=xoxb-your-single-workspace-token +SLACK_SIGNING_SECRET=your-signing-secret +SLACK_APP_TOKEN=xapp-your-app-token +SLACK_TEAM_ID=T0XXXXX # Optional: specify your workspace ID + +# MongoDB (single database for all users) +MONGODB_USER=local_user +MONGODB_PASSWORD=secure_password_here +MONGODB_HOST=localhost +MONGODB_PORT=27017 +MONGODB_LOCAL=false # Set to false for Atlas +``` + +- **User Role:** All Slack users in the workspace have equal access to store and retrieve messages +- **Database:** Single MongoDB database shared across all workspace users +- **Scalability:** Suitable for teams up to 100 active users + +#### Multi-User/Multi-Workspace Configuration +For multiple Slack workspaces (SaaS-style deployment): + +```env +# Default workspace fallback +SLACK_BOT_TOKEN=xoxb-default-workspace-token +SLACK_SIGNING_SECRET=your-signing-secret +SLACK_APP_TOKEN=xapp-your-app-token + +# Database (multi-tenant approach) +MONGODB_USER=sudbud_multi_user +MONGODB_PASSWORD=asX0X92RZJn8Bk09 +MONGODB_HOST=atlas-cluster-url +MONGODB_PORT=27017 +MONGODB_LOCAL=false # Must use Atlas for multi-workspace + +# Workspace token storage +WORKSPACE_TOKEN_TABLE=workspace_tokens # Table name in DB for storing per-workspace tokens +``` + +- **Additional Workspaces:** Install the Slack app in each workspace +- **Token Storage:** Each workspace's Bot Token is stored in `workspace_tokens` collection in MongoDB +- **Database Isolation:** Optional database per workspace (requires separate MongoDB instances) +- **Scalability:** Supports unlimited workspaces, each with independent user bases + +**To add a new workspace to multi-user setup:** +1. Create a new Slack app in the target workspace using manifest file +2. Use one-click OAuth link to authorize +3. Bot automatically stores workspace token in `workspace_tokens` MongoDB collection +4. Subsequent requests from that workspace use cached token + +### Network Configuration + +#### Port Configuration +```env +SLACK_BOT_PORT=3000 # Port for Slack bot (Express app) +DB_PORT=5000 # Port for MongoDB API service +REQUEST_BODY_LIMIT=25mb # Max request size for bulk message storage +MONGO_INSERT_BATCH_BYTES=12mb # Max bytes per batch insert to MongoDB +MONGO_INSERT_BATCH_COUNT=500 # Max documents per batch insert +``` + +#### Remote Deployment (e.g., Render, Heroku) +```env +API_URL=https://your-deployed-api.onrender.com # Replace localhost with public URL +NODE_ENV=production +LOG_LEVEL=info # Reduce verbose logging in production +``` + +### Resource Configuration + +**Environment Variables for Resource Management:** +```env +NODE_OPTIONS=--max_old_space_size=2048 # Node.js memory limit (MB) +REQUEST_BODY_LIMIT=25mb # Adjust based on available memory +MONGO_INSERT_BATCH_BYTES=12mb # Reduce if memory-constrained +MONGO_INSERT_BATCH_COUNT=500 # Reduce if experiencing timeouts +``` + +**Recommended Sizing:** + +| Deployment | RAM | Storage | Users | Messages/Month | +|------------|-----|---------|-------|-----------------| +| Development (Local) | 4 GB | 10 GB | 1-5 | <100K | +| Small Team (Docker) | 8 GB | 50 GB | 5-50 | 100K-1M | +| Production (Atlas) | 16+ GB | 500+ GB | 50+ | 1M+ | + +### Automation Setup + +**Auto-save Configuration:** +The bot can be configured to automatically save messages without user prompts. + +```javascript +// In bolt_slack/app.js - user preference storage +const userAutoSavePreference = new Map(); // Currently in-memory; persists to DB for production + +// Enable auto-save for a user: +userAutoSavePreference.set(userId, { autoSave: true, batchInterval: 3600000 }); // Every hour +``` + +**Scheduling Message Saves (Cron Job Example):** +```bash +# Add to system crontab (Windows Task Scheduler or Linux cron) +0 2 * * * cd /path/to/app && npm run batch:save-messages # Daily at 2 AM + +# Or use node-cron package (add to app.js): +const cron = require('node-cron'); +cron.schedule('0 2 * * *', async () => { + await storeAllActiveChannelMessages(); +}); +``` + +--- + +## Security + +### Password and Credential Management + +#### Environment Variable Protection +```bash +# DO NOT commit .env to version control +echo ".env" >> .gitignore +echo ".env.local" >> .gitignore + +# Permissions: Only owner can read +chmod 600 .env # Linux/macOS +# Windows: Right-click .env → Properties → Security → Edit → Remove "Users" group +``` + +#### Slack Token Security +- **Bot Token** (`SLACK_BOT_TOKEN`): Starts with `xoxb-`, grants bot permissions +- **Signing Secret** (`SLACK_SIGNING_SECRET`): Validates requests from Slack +- **App Token** (`SLACK_APP_TOKEN`): Starts with `xapp-`, required for Socket Mode +- **Rotation:** Regenerate tokens every 90 days + 1. Go to [api.slack.com/apps](https://api.slack.com/apps) → Your App → Install App + 2. Click "Regenerate" next to Bot User OAuth Token + 3. Update `.env` with new token + 4. Restart bot service + +#### MongoDB Credentials +```env +# Atlas-specific security +MONGODB_USER=sudbud_test_user # NOT your Atlas account email +MONGODB_PASSWORD=clpuZVnfycvRk8v3 # Min 12 chars, include symbols +MONGODB_HOST=cluster0.xxxxx.mongodb.net +MONGODB_PORT=27017 +MONGODB_SSL=true # Always true for Atlas +MONGODB_AUTH_SOURCE=admin # Atlas uses "admin" database + +# Password requirements: +# - Minimum 12 characters +# - Must include uppercase, lowercase, numbers, symbols +# - Never share or log to console +# - Rotate quarterly in production +``` + +**MongoDB Atlas IP Whitelist:** +- Go to Cluster → Network Access +- Add only necessary IPs (never 0.0.0.0/0 in production) +- For Render deployments, whitelist Render IP ranges or use VPN + +#### API Key Management (if external APIs used) +```env +# Google Gemini API Key (for NLP summarization) +GEMINI_API_KEY=your-key-here +GEMINI_API_TIMEOUT=30000 # 30 seconds + +# Keep API keys separate from code +# Rotate if ever exposed +# Use API key restrictions (limit to your app/IP) +``` + +### Database Access Control + +**MongoDB RBAC (Role-Based Access Control):** +```javascript +// For each database user, specify roles: +// readWrite - Full read/write to assigned database +// admin - Only for production admins (minimum) + +// Create a service user (not admin): +// In Atlas → Database Access → Add Database User +// - Username: app_service_user +// - Password: (auto-generated, copy to .env) +// - Roles: readWrite on "slack_data" database +// - Assign IP whitelist +``` + +#### Logging and Monitoring Access +```env +# Enable detailed logging +LOG_LEVEL=debug # For development only +# Production: LOG_LEVEL=warn + +# Log file location +LOG_FILE=/var/log/slack-bot/app.log +LOG_RETENTION_DAYS=30 # Auto-delete logs older than 30 days +``` + +--- + +## Database Installation and Maintenance + +### Installation Verification + +```powershell +# Verify MongoDB connection +cd mongo_storage +node -e "require('./db-connection').connectToDatabase().then(() => console.log('✓ Connected')).catch(e => console.log('✗ Error:', e.message))" +``` + +### Database Schema / Collections + +The application uses these MongoDB collections: + +**1. messages** +- Stores Slack messages with metadata +- Fields: `_id`, `slack_id`, `channel`, `user`, `text`, `timestamp`, `reactions`, `thread_id` +- Index: `{ slack_id: 1, channel: 1 }` (unique composite index for deduplication) +- Size estimate: ~1-2 KB per message + +**2. users** +- Stores Slack user profiles +- Fields: `_id`, `slack_user_id`, `name`, `email`, `is_bot`, `avatar_url`, `last_synced` +- Index: `{ slack_user_id: 1 }` (unique) +- Size estimate: ~2-5 KB per user + +**3. channels** +- Stores Slack channel metadata +- Fields: `_id`, `slack_channel_id`, `name`, `description`, `created_at`, `is_private` +- Index: `{ slack_channel_id: 1 }` (unique) +- Size estimate: ~1-2 KB per channel + +**4. workspace_tokens** (Multi-workspace only) +- Stores OAuth tokens for each workspace +- Fields: `_id`, `team_id`, `team_name`, `bot_token`, `signing_secret`, `installed_at`, `expires_at` +- Index: `{ team_id: 1 }` (unique) +- TTL Index: Automatically delete expired tokens after 90 days + +### Regular Maintenance Tasks + +**Daily Maintenance:** +```powershell +# Check MongoDB health +curl http://localhost:5000/health +# Expected response: { "status": "ok", "timestamp": "..." } + +# Monitor disk usage +df -h # Linux/macOS +Get-Volume # PowerShell Windows + +# Monitor logs for errors +tail -f /var/log/slack-bot/app.log +``` + +**Weekly Maintenance:** +```powershell +# Create database backup +npm run db:backup # Backs up to ./backups/ directory + +# Verify backup integrity +mongorestore --uri "mongodb://user:pass@localhost:27017" --dryRun ./backups/slack_data + +# Check for duplicate messages (deduplication) +npm run deduplicate:messages +``` + +**Monthly Maintenance:** +```powershell +# Remove old/expired data +npm run cleanup:old-messages --days=90 # Delete messages older than 90 days + +# Database maintenance +npm run optimize:indexes # Rebuild indexes for performance +npm run analyze:storage # Analyze storage usage by collection + +# Token rotation +npm run rotate:tokens # Regenerate Slack tokens +``` + +**Quarterly Maintenance:** +```powershell +# Full database backup (different location) +npm run db:backup:full --location=/backup-drive/quarterly-backup-2026-Q1 + +# Database health check +npm run db:validate +# Checks: +# - All collections have required indexes +# - No corrupted documents +# - Replication lag (if replica set) +# - TTL index cleanup working +``` + +### Backup and Recovery + +**Automated Backup Setup:** +```powershell +# Using MongoDB Atlas (recommended) +1. Enable automatic backups in Atlas console + - Settings → Backup & Restore + - Retention: 30 days + - Frequency: Daily snapshots + +# Using Local MongoDB with Docker +2. Create backup script (backup.sh): + ```bash + #!/bin/bash + TIMESTAMP=$(date +%Y%m%d_%H%M%S) + docker exec suds-local-mongodb mongodump \ + --out /backup/mongodb-backup-${TIMESTAMP} + ``` + +# Using Manual Backup (Local) +3. mongodump command: + ```powershell + mongodump --uri "mongodb://testuser:testpass@localhost:27017/slack_data" \ + --out ./backups/slack_data_backup_$(Get-Date -Format yyyyMMdd_HHmmss) + ``` +``` + +**Backup Retention Policy:** +``` +Daily backups: Retain 7 days +Weekly backups: Retain 8 weeks +Monthly backups: Retain 1 year +Disaster recovery: At least 2 geographically separate locations +``` + +**Recovery Procedures:** + +*Scenario A: Single Document Recovery* +```powershell +# Restore a single message document from backup +mongorestore --uri "mongodb://user:pass@localhost:27017" \ + --nsInclude="slack_data.messages" \ + --nsFrom="slack_data.messages" \ + --nsTo="slack_data.messages_temp" \ + ./backups/slack_data/slack_data/messages.bson + +# Copy restored document back to main collection +# Verify integrity, then delete from temp collection +``` + +*Scenario B: Full Database Recovery* +```powershell +# Stop the application +Stop-Process -Name "node" -Force + +# Restore from backup +mongorestore --uri "mongodb://user:pass@localhost:27017" \ + --drop \ # Replace existing database + ./backups/slack_data_backup_20260101_120000/ + +# Restart services +npm start +``` + +*Scenario C: Point-in-Time Recovery (Atlas only)* +``` +1. Go to Atlas console → Backups +2. Select the backup snapshot closest to desired time +3. Click "Restore" → "Restore to a new cluster" +4. Wait for restore to complete +5. Verify data integrity +6. Update MongoDB connection string in .env +7. Restart application +``` + +--- + +## Application Functions + +### Slash Commands (Bot Features) + +#### 1. `/store-messages` +**Purpose:** Save all messages in current channel to database + +**Usage:** +``` +/store-messages +``` + +**System Flow:** +1. Slack bot receives command +2. Fetches all messages from channel history (API limit: 1000/request) +3. Extracts user and channel metadata +4. Sends to MongoDB API in batches (12 MB max per batch, 500 docs per batch) +5. Returns confirmation with count of stored messages + +**Response Example:** +``` +✅ Stored 847 messages from #general +Database Status: Connected +API Response: Success +``` + +**Error Handling:** +- If channel not found: "❌ Channel not accessible. Please ensure bot is invited." +- If API unreachable: "❌ Database API is offline. Contact administrator." +- If batch size exceeded: "⚠️ Some messages couldn't be stored (too large). Admin notified." + +#### 2. `/messages` +**Purpose:** Retrieve recent stored messages from current channel + +**Usage:** +``` +/messages [count] # Default: 10 messages, max: 100 +``` + +**System Flow:** +1. Query MongoDB for channel's most recent 10 messages +2. Format with sender name, timestamp, reactions +3. Display in Slack thread + +**Response Example:** +``` +Recent messages from #general: + +@Wyatt (2 hours ago): "Let's discuss the NLP model" +@John (1 hour ago): "I've updated the config" +... +``` + +#### 3. `/channel-info` +**Purpose:** Display metadata about current channel + +**Usage:** +``` +/channel-info +``` + +**Returns:** +- Channel name, description, creation date +- Member count +- Total stored messages +- Last updated timestamp +- Privacy status (public/private) + +#### 4. `/store-members` +**Purpose:** Sync all channel members to database + +**Usage:** +``` +/store-members +``` + +**System Flow:** +1. Fetch all users in workspace +2. For each member in channel: extract profile (name, email, avatar) +3. Store/update in `users` collection +4. Return confirmation + +**Response Example:** +``` +✅ Synced 23 members to database +New members: 3 +Updated members: 8 +``` + +#### 5. `/summarize-week` +**Purpose:** Generate NLP-powered summary of week's messages + +**Usage:** +``` +/summarize-week +``` + +**System Flow:** +1. Query messages from past 7 days in channel +2. Send to Google Gemini API with "summarize key topics" prompt +3. Extract topics, decisions, action items +4. Post formatted summary to channel + +**Response Example:** +``` +📊 **Week Summary (#general)** + +**Key Topics:** +- NLP Model Performance Improvements +- Database Optimization +- Team Onboarding + +**Decisions Made:** +✓ Migrate to MongoDB Atlas +✓ Use Gemini API for summarization + +**Action Items:** +→ @Keith: Update configuration docs (Due: Friday) +→ @John: Performance testing (Due: Monday) +``` + +### Home Dashboard + +**Access:** Click app name in Slack sidebar → "Home" + +**Features:** +- Select channel to view statistics +- Quick links to recent summaries +- Admin panel (for workspace admins only) +- Settings for auto-save preferences + +**Admin Functions (Admins Only):** +- Store all members in workspace +- View unsaved channels +- Bulk sync all channels +- Download data exports + +--- + +## Error Messages and Recovery Actions + +| Error | Cause | User Action | Admin Action | +|-------|-------|------------|--------------| +| "❌ Channel not accessible" | Bot not invited to channel | `/invite @SUD Bud` to channel | Verify bot has permissions | +| "❌ Database API is offline" | MongoDB API service crashed | Contact admin | Restart `mongo_storage` service | +| "⚠️ Request payload is too large" | Message data exceeds 25 MB | Split into smaller requests | Increase `REQUEST_BODY_LIMIT` in `.env` | +| "❌ Slack API rate limit" | Too many requests to Slack | Wait 60 seconds, retry | Implement request queuing (see config) | +| "MONGODB_USER not found" | Credentials incorrect | Verify `.env` file | Reset MongoDB password in Atlas | +| "Socket connection timeout" | Network issue or Socket Mode disabled | Check internet, restart bot | Verify `SLACK_APP_TOKEN` and Socket Mode enabled | +| "Cannot resolve hostname" | DNS failure or no internet | Check network connectivity | Restart network services, check firewall | +| "Duplicate key error" | Message already stored | Continue - system handles deduplication | Check indexes on messages collection | +| "TTL index deleted documents before backup" | Backup timing conflict | Restore from scheduled backup | Adjust cleanup schedule, increase retention | + +**Standard Recovery Workflow:** +1. **Identify Error:** Check logs with `npm run logs:view` +2. **Diagnose:** Run health check `curl http://localhost:5000/health` +3. **Remediate:** Follow action in table above +4. **Verify:** Retry failed operation +5. **Document:** Log incident in Jira with timestamp and resolution + +--- + +## Troubleshooting (Application Level) + +### Bot Not Responding to Messages + +**Diagnostic Steps:** +```powershell +# 1. Verify bot is running +ps aux | grep "npm start" | grep bolt_slack + +# 2. Check Socket Mode connection +grep "Socket Mode" ./logs/app.log + +# 3. Verify bot is in channel +# In Slack, type: /invite @SUD Bud + +# 4. Test bot directly +# Send a direct message to the bot - should echo response + +# 5. Check for errors +tail -20 ./logs/app.log +``` + +**Resolution:** +- If bot not running: `cd bolt_slack && npm start` +- If Socket Mode down: Restart bot and verify `SLACK_APP_TOKEN` in `.env` +- If not invited: Have workspace admin run `/invite @SUD Bud` in target channel + +### Messages Not Saving to Database + +**Diagnostic Steps:** +```powershell +# 1. Verify MongoDB API is running +curl http://localhost:5000/health + +# 2. Check MongoDB connection +npm run db:test-connection + +# 3. Verify credentials +echo $env:MONGODB_USER # Should match database user + +# 4. Check storage quota +npm run db:storage-stats +``` + +**Resolution:** +- If API offline: Restart `mongo_storage` service: `npm start` from `mongo_storage/` directory +- If auth error: Verify MongoDB user exists and password matches (case-sensitive) +- If storage full: Contact admin to archive old data or expand storage + +### High Latency / Slow Responses + +**Diagnostic Steps:** +```powershell +# 1. Check batch size configuration +echo $env:MONGO_INSERT_BATCH_BYTES # Should be ~12 MB + +# 2. Monitor memory usage +Get-Process node | Select-Object ProcessName, WorkingSet + +# 3. Check database query performance +npm run db:slow-queries # Queries taking >1 second + +# 4. Check network latency to MongoDB +ping cluster0.xxxxx.mongodb.net +``` + +**Resolution:** +- If memory > 80% of available: Reduce `MONGO_INSERT_BATCH_BYTES` to 8 MB +- If slow queries: Run `npm run optimize:indexes` +- If high latency: Use closer MongoDB region or deploy app closer to database + +### Slack Token Expired or Invalid + +**Symptoms:** +- "Invalid Token" errors in logs +- `/store-messages` command fails + +**Resolution:** +```powershell +# 1. Go to https://api.slack.com/apps → Your App +# 2. Click "Install App" +# 3. Click "Regenerate" next to Bot Token +# 4. Copy new token (xoxb-...) +# 5. Update .env: SLACK_BOT_TOKEN=xoxb-new-token +# 6. Restart bot: npm start +# 7. Verify: Send test message to bot +``` + +--- + +## Support + +### Contact Information + +**Technical Support Channels:** + +| Issue Type | Contact | Response Time | Availability | +|-----------|---------|---------------|--------------| +| Bug Report | [Jira Issues](https://temple-cis-projects-in-cs.atlassian.net/jira/software/c/projects/DT/issues) | 24-48 hours | Business days | +| Urgent Outage | [GitHub Issues](https://github.com/Capstone-Projects-2026-spring/capstone-projects-2026-spring-classroom-373a75-project-tu-cis-4398-docs-template-1/issues) | 1-2 hours | 24/7 | +| Feature Request | GitHub Discussions | 1 week | Business days | +| Security Vulnerability | [Email Security Report](mailto:capstone-projects-2026-spring@temple.edu) | 24 hours | 24/7 | + +### Team Contacts + +**Development Team:** +- Wyatt Zantua (Backend/Database): [GitHub](https://github.com/zantuaw09) +- John Currie (Backend/Deployment): [GitHub](https://github.com/John-C-Currie) +- Keith Winter (Database): [GitHub](https://github.com/KeWinter) +- Donte' Harmon (Frontend): [GitHub](https://github.com/dontetu) +- Fares Hagos (Frontend): [GitHub](https://github.com/FaresHagostu) + +### Support Service Levels (SLA) + +**Production Environment:** +- **P1 (Critical):** System down, no users can save messages + - Resolution target: 4 hours + - Escalation: All hands on deck + +- **P2 (High):** Limited functionality, workarounds available + - Resolution target: 8 hours + - Escalation: Senior engineer within 2 hours + +- **P3 (Medium):** Minor functionality impact + - Resolution target: 24 hours + - Escalation: Assigned engineer + +- **P4 (Low):** Cosmetic or documentation issues + - Resolution target: 1 week + - Escalation: Backlog prioritization + +**Issue Reporting Format:** +``` +Title: [COMPONENT] Brief description + +Environment: +- Deployment: (local/render/production) +- Node.js version: (output of `node --version`) +- MongoDB: (local/atlas, version) + +Steps to Reproduce: +1. ... +2. ... + +Expected Behavior: +... + +Actual Behavior: +... + +Logs/Screenshots: +[Attach relevant logs or error screenshots] + +Timestamp: +[When the issue occurred] +``` + +--- + +## Missing Documentation Requiring Completion + +The following sections require additional information to fully complete this guide. **Owner** indicates who should provide the details: + +1. **Screenshots & UI Walkthrough** (Owner: Donte' Harmon) + - Home dashboard interface + - Command response examples + - Error message examples + - Admin panel features + +2. **Performance Benchmarks** (Owner: Keith Winter) + - Message storage throughput (messages/second) + - Query response times by collection size + - Memory usage under load + - Database growth projections + +3. **Disaster Recovery Plan** (Owner: Wyatt Zantua) + - RTO (Recovery Time Objective) targets + - RPO (Recovery Point Objective) targets + - Failover procedures + - Backup schedule details + +4. **NLP Model Documentation** (Owner: Fares Hagos) + - Model architecture and accuracy metrics + - Retraining procedures + - Configuration parameters + - Supported languages + +5. **API Documentation** (Owner: John Currie) + - REST endpoint specifications + - Request/response examples + - Rate limiting details + - Authentication methods + +6. **Monitoring & Alerting Setup** (Owner: Team) + - Datadog/New Relic configuration + - Alert thresholds + - Dashboard definitions + - On-call rotation schedule + +7. **Deployment Procedures** (Owner: Wyatt Zantua) + - CI/CD pipeline configuration + - Blue-green deployment strategy + - Rollback procedures + - Environment promotion workflow + +8. **Compliance & Audit Logging** (Owner: Team) + - Data retention policies per regulatory requirement + - User data deletion procedures (GDPR right to be forgotten) + - Access audit logs + - Change audit trails diff --git a/README.md b/README.md index 1bc5873ff..574fc853a 100644 --- a/README.md +++ b/README.md @@ -49,15 +49,105 @@ This project requires significant backround research on existing LLMs that devel [Fares Hagos](https://github.com/FaresHagostu) -## Quick Start (Recommended) -Add the slack bot by clicking the button +--- + +# For the User + +This section contains everything you need to install, configure, and use the application. If you're looking for information about development or maintenance, see "For the Maintainer" below. + +## Quick Start Guide + +### One-Click Add (Cloud Hosted - Easiest) +To add SUD Bud directly to your Slack workspace, click this button Add to Slack -Or by clicking this link +Or click this link! [Click Here](https://slack.com/oauth/v2/authorize?client_id=10472452206738.10715593272068&scope=reactions:write,app_mentions:read,channels:history,channels:read,chat:write,commands,im:history,im:read,im:write,reactions:read,users:read,groups:read,mpim:read&user_scope=) -## Manual Setup (Slack Bot + Mongo API) + + +## Setup For Experienced Users + +If you've deployed Node.js + MongoDB apps before: + +1. **Clone/download** the repository +2. **Set up a Slack app** at [api.slack.com/apps](https://api.slack.com/apps) (or use manifest file in `bolt_slack/slack-app-manifest.json`) +3. **Create `.env`** in repo root with tokens from Slack + MongoDB credentials +4. **Run setup:** `npm install` (both `bolt_slack/` and `mongo_storage/`) +5. **Start services:** + ```powershell + # Terminal 1: MongoDB API + cd mongo_storage && node server.js + + # Terminal 2: Slack Bot + cd bolt_slack && npm start + ``` +6. **Invite bot** to your Slack channels: `/invite @YourBotName` +7. **Done!** Use `/store-messages`, `/messages`, `/channel-info` in Slack + +## System Requirements + +Before installing, ensure your system meets these minimum requirements: + +### Required Software +- **Node.js:** 18.0.0 or higher ([Download](https://nodejs.org/)) +- **npm:** 9.0.0 or higher (bundled with Node.js) +- **MongoDB:** One of the following: + - MongoDB Atlas account (cloud-hosted, no local install needed), OR + - Docker Desktop 4.0+ (for local MongoDB container), OR + - MongoDB Community Edition 5.0+ (for local installation) +- **Git:** For cloning the repository (optional, can download as ZIP) + +### Recommended Hardware +- **RAM:** 4 GB minimum (8 GB recommended) +- **Disk Space:** 2 GB free space +- **Processor:** Dual-core processor minimum + +### Network Requirements +- Stable internet connection for Slack API communication +- Ability to reach `slack.com` and `api.slack.com` +- If using MongoDB Atlas: ability to reach Atlas cluster endpoint (firewall/proxy compatible) +- Ports available: 3000 (Slack bot), 5000 (MongoDB API) must not be in use + +### Slack App Requirements +- Slack workspace where you have app installation permissions +- Ability to create/configure Slack apps (Admin access recommended) + +#### Quick Configuration Checklist +- [ ] Node.js 18+ installed +- [ ] MongoDB credentials ready (Atlas or Docker) +- [ ] Slack App tokens created (Bot Token, App Token, Signing Secret) +- [ ] `.env` file populated with credentials +- [ ] Dependencies installed +- [ ] Both services running on ports 3000 & 5000 +- [ ] Bot invited to at least one Slack channel + +## Manual Installation + +### Network Considerations + +Before installing, ensure your network configuration supports the application: + +#### Local Network Setup +- **Port 3000:** Must be available for the Slack bot service (Node.js app) +- **Port 5000:** Must be available for the MongoDB API service (Node.js app) +- **Firewall:** Both ports must not be blocked by local firewall (Windows Defender, third-party firewalls) +- **Localhost:** Application assumes `localhost` (127.0.0.1) for local development + +#### MongoDB Atlas Setup (If Using Cloud Database) +- **Network Access:** Your machine's public IP must be whitelisted in MongoDB Atlas + - Go to: Cluster → Network Access → Add IP Address + - Recommended: Add your IP or "Allow access from anywhere" (0.0.0.0/0) for development +- **Connection String:** Atlas provides a connection string; store securely in `.env` +- **Username/Password:** Create MongoDB database user in Atlas (not your account password) + +#### Slack API Network Requirements +- **Outbound HTTPS (443):** Required for Slack API calls +- **WebSocket (Port 443):** Required for Socket Mode (bot event streaming) +- **DNS Resolution:** Must be able to reach `slack.com`, `api.slack.com`, `hooks.slack.com` + +### Detailed Manual Setup Follow these steps to install dependencies, configure environment variables, and run both services locally. Commands are PowerShell-friendly for Windows. @@ -312,3 +402,87 @@ npm run test:coverage - **MongoDB auth/connection errors:** Verify `MONGODB_USER/PASSWORD`, IP whitelist, and `DB_PORT` in `.env`. - **Bot not responding:** Confirm the bot is invited to the channel and Socket Mode tokens are correct; restart `npm start`. - **API health:** Check http://localhost:5000/health. + +## Uninstall & Cleanup + +If you need to remove the application or reset your installation, follow these steps: + +### Remove Services & Data + +```powershell +# Stop running services (close Terminal windows or press Ctrl+C) + +# Option A: Keep the code, remove only node_modules +cd c:\project-structuring-unstructured-data\bolt_slack +rm -Recurse node_modules +cd ..\mongo_storage +rm -Recurse node_modules + +# Option B: Complete uninstall (remove everything except .env and docs) +cd c:\project-structuring-unstructured-data +rm -Recurse bolt_slack/node_modules +rm -Recurse mongo_storage/node_modules +``` + +### Remove MongoDB (Local Setup Only) + +**If using Docker:** +```powershell +cd c:\project-structuring-unstructured-data\mongo_storage + +# Stop MongoDB container +npm run db:stop + +# Remove container and data volume +docker rm suds-local-mongodb +docker volume rm suds-mongodb-volume +``` + +**If using local MongoDB installation:** +```powershell +# Stop MongoDB +# Windows: Close the mongod terminal window, or: +taskkill /IM mongod.exe /F + +# Uninstall MongoDB (Windows Add/Remove Programs or): +# - Download MongoDB uninstaller +# - Delete default data directory: C:\Program Files\MongoDB\Server\[version]\data +``` + +### Remove Slack App (Optional) + +1. Go to [api.slack.com/apps](https://api.slack.com/apps) +2. Select your app +3. Click "Settings" → scroll to bottom +4. Click "Delete this app" +5. Confirm deletion + +### Clean Environment Variables + +Delete or clear the `.env` file in the repository root if you don't plan to reinstall: + +```powershell +rm .env +``` + +**⚠️ Warning:** Only delete `.env` after stopping all services. It contains tokens and credentials. + +### Full Repository Reset + +To start completely fresh: + +```powershell +# Delete everything except .git (if version controlled) +cd c:\project-structuring-unstructured-data +rm -Recurse -Force bolt_slack/node_modules, mongo_storage/node_modules +rm .env +rm .env.local + +# Re-run installation from "Manual Setup" section above +``` + +--- + +# For the Maintainer + +(This section is reserved for development, contribution, and maintenance guidelines. Please download [maintainer documentation](https://github.com/user-attachments/files/27175924/MAINTAINER.md) or view `MAINTAINER.md` in the repository files for detailed contributor instructions.) diff --git a/documentation/docs/testing/acceptence-testing.md b/documentation/docs/testing/acceptence-testing.md index 2a56e42fc..d3cb4b593 100644 --- a/documentation/docs/testing/acceptence-testing.md +++ b/documentation/docs/testing/acceptence-testing.md @@ -5,5 +5,5 @@ sidebar_position: 3 Demonstration of all of the functional and non-functional requirements. This can be a combination of automated tests derived from the use-cases (user stories) and manual tests with recorded observation of the results. -[Acceptance tests](https://github.com/user-attachments/files/26509145/SUD-QA-Doc-Demo-2.xlsx) +[Acceptance tests](https://github.com/user-attachments/files/27174959/QA-Acceptance-Test.xlsx) diff --git a/documentation/docs/testing/test-report.md b/documentation/docs/testing/test-report.md new file mode 100644 index 000000000..2e8d8577a --- /dev/null +++ b/documentation/docs/testing/test-report.md @@ -0,0 +1,171 @@ +--- +sidebar_position: 4 +--- +# Test Report + +**Project:** Project Structuring Unstructured Data +**Version:** 4.0 - Final Release +--- + +## Executive Summary + +This document records comprehensive test execution results across multiple test phases, including unit tests, integration tests, and acceptance tests. All test phases are designed to verify system functionality at different levels of integration and user interaction. + +| **Test Phase** | **Status** | **Suites** | **Tests** | **Duration** | +| --- | --- | --- | --- | --- | +| Unit Testing | ✅ PASS | 3/3 | 28/28 | 2.523s | +| Integration Testing | ✅ PASS | 4/4 | 33/33 | 3.873s | +| Acceptance Testing | ⏳ PENDING | — | — | — | +| **Overall Status** | **✅ PASS** | **7/7** | **61/61** | **6.396s** | + +--- + +## Unit Testing + +### Overview +Unit tests isolate each router method and verify behavior with mocked dependencies. Tests are designed to verify custom database routing logic without requiring live database or Python dependencies. + +**Execution Date:** April 28, 2026 +**Test Framework:** Jest + +### Test Execution Summary + +| **Component** | **Status** | **Test Count** | **Duration** | **Notes** | +| --- | --- | --- | --- | --- | +| Messages Route | ✅ PASS | 8 | — | All endpoint variations covered | +| Users Route | ✅ PASS | 8 | — | All endpoint variations covered | +| Summaries Route | ✅ PASS | 12 | — | Includes parameter validation | +| **Total** | **✅ PASS** | **28** | **2.523s** | **All tests passing** | + +### Test Results by Route + +#### Messages Route (`/api/messages/:channelName`) +- ✅ GET returns messages when `model.find()` succeeds +- ✅ GET returns empty array when collection has no documents +- ✅ GET returns 500 when `model.find()` throws +- ✅ GET handles different collection names correctly +- ✅ GET returns extra fields beyond schema +- ✅ GET returns limited newest-first results +- ✅ POST returns success for a single array insert +- ✅ POST returns 400 when insertion fails + +#### Users Route (`/api/users/:channelName`) +- ✅ GET returns users when `model.find()` succeeds +- ✅ GET returns empty array when collection has no documents +- ✅ GET returns 500 when `model.find()` throws +- ✅ GET handles different collection names correctly +- ✅ GET returns extra fields beyond schema +- ✅ POST inserts a member array successfully +- ✅ POST returns 400 when insertion fails +- ✅ POST rejects a blank channel name + +#### Summaries Route (`/api/summaries/:databaseKey`) +- ✅ GET returns summaries for an existing database +- ✅ GET returns 404 when database key is missing +- ✅ GET with `weekStart` canonicalizes to UTC Sunday and filters one week +- ✅ GET rejects invalid `weekStart` +- ✅ POST with `week` calls model and returns metadata +- ✅ POST with `weekStart` canonicalizes and calls model +- ✅ POST rejects mixed `week` and `weekStart` +- ✅ POST rejects out-of-range `week` +- ✅ POST rejects invalid `weekStart` +- ✅ POST returns 500 when model execution fails +- ✅ DELETE removes a message by timestamp +- ✅ DELETE returns 404 when no message matches + +--- + +## Integration Testing + +### Overview +Integration tests demonstrate each use case based on use-case descriptions and sequence diagrams. External input is provided via mock objects and results are verified via mock objects. Integration tests do not require manual data entry or manual result interpretation. + +**Execution Date:** April 28, 2026 +**Test Framework:** Jest + +### Test Execution Summary + +| **Component** | **Status** | **Test Count** | **Duration** | **Notes** | +| --- | --- | --- | --- | --- | +| Messages Integration | ✅ PASS | 8 | — | End-to-end message routing verified | +| Users Integration | ✅ PASS | 5 | — | User management workflows verified | +| Summaries Integration | ✅ PASS | 10 | — | Summary generation and retrieval verified | +| User Summaries Integration | ✅ PASS | 10 | — | User-specific summary workflows verified | +| **Total** | **✅ PASS** | **33** | **3.873s** | **All tests passing** | + +### Test Results by Route + +#### Messages Integration Route (`/api/messages/:channelName`) +- ✅ GET returns seeded documents from a channel-specific store +- ✅ GET returns empty array for a new channel with no messages +- ✅ GET returns documents with extra fields beyond schema +- ✅ GET handles documents missing expected schema fields +- ✅ GET handles large result sets (100 documents) +- ✅ GET supports special characters in channel name via sanitization +- ✅ POST inserts array payload and verifies persisted messages through GET +- ✅ POST inserts one message and reports duplicate on repeated user + ts + +#### Users Integration Route (`/api/users/:channelName`) +- ✅ GET returns seeded channel members from a channel-specific store +- ✅ GET returns empty array for a channel with no members +- ✅ GET returns member documents with extra fields beyond schema +- ✅ POST inserts member arrays and verifies persisted members through GET +- ✅ POST returns validation error when `channelName` is blank + +#### Summaries Integration Route (`/api/summaries/:databaseKey`) +- ✅ GET returns summaries for an existing channel summary database +- ✅ GET returns 404 when no matching summary database exists +- ✅ GET with `weekStart` returns the normalized one-week UTC summary window +- ✅ GET with invalid `weekStart` returns validation error +- ✅ POST with `week` runs summary generation and returns processing metadata +- ✅ POST with `weekStart` runs summary generation using canonicalized UTC week start +- ✅ POST rejects requests containing both `week` and `weekStart` +- ✅ POST rejects invalid `week` values outside the accepted range +- ✅ POST returns model execution errors from the summary generation pipeline +- ✅ POST returns 404 when matching database does not exist + +#### User Summaries Integration Route (`/api/user_summaries/:databaseKey/:userId?`) +- ✅ GET `/api/user_summaries/:databaseKey` returns all stored user summaries for an existing database key +- ✅ GET `/api/user_summaries/:databaseKey/:userId` returns a single user summary for the requested userId +- ✅ GET `/api/user_summaries/:databaseKey?userId=...` returns a single user summary when userId is provided as a query parameter +- ✅ GET `/api/user_summaries/:databaseKey/:userId` returns `userSummary: null` when the user has no summary yet +- ✅ GET `/api/user_summaries/:databaseKey/:userId` returns 404 when no matching summary database exists +- ✅ POST `/api/user_summaries/:databaseKey` returns all-user generation metadata when no userId is provided +- ✅ POST `/api/user_summaries/:databaseKey?userId=...` returns single-user generation metadata for the requested userId +- ✅ POST `/api/user_summaries/:databaseKey` also accepts `userId` in the request body and returns single-user generation metadata +- ✅ POST `/api/user_summaries/:databaseKey` returns 404 when no matching summary database exists +- ✅ POST `/api/user_summaries/:databaseKey` returns model execution errors from the user summary generation pipeline + +--- + +## Acceptance Testing + +### Overview +Acceptance tests verify system behavior from an end-user perspective, validating that all requirements are met and the system functions as specified. +[Completed Acceptance Test Report](https://github.com/user-attachments/files/27174993/Complete-Acceptance-Test-Report.xlsx) + +--- + +## Error Report +No major errors were found during all forms of testing for Version 4.0 of SUD Bud. Should features be reconfigured or optimized in future releases, all tests should be reanalyzed for potential errors. + +--- + +## Test Environment + +| **Property** | **Value** | +| --- | --- | +| Test Framework | Jest | +| Node.js Version | 18.x+ | +| Database | MongoDB (mocked for unit tests; local instance for integration tests) | +| Python Model | Mocked for unit tests; Python runtime required for integration tests | +| OS | Windows | + +--- + +## Conclusion + +All unit and integration tests have passed successfully. The system demonstrates robust behavior across all tested routes and scenarios, with proper error handling and validation in place. The test suite provides comprehensive coverage of the MongoDB Storage Component's routing logic, database interactions, and model execution pathways. + + +