v2.10.0 — Prometheus observability and verification suite

Highlights

• Add production-grade Prometheus integration tests and benchmark suite.
• Validate canonical histogram buckets for policy latency monitoring.
• Proven sampling behavior and safe performance overhead under 10 percent.
• Opt-in metrics only, with SSRF protection for metrics sink URL.
• Documentation and pilot onboarding materials included.

Performance

• Sampling validated: 10k evals at 10 percent → 9.86 percent sampled.
• Runtime overhead: well under 10 percent in benchmarked workloads.
• Memory overhead: within expected gates.

Notes for operators

• Metrics are disabled by default. Enable using PUSHGATEWAY_URL environment variable.
• Pushgateway cleanup recommended: use the provided cleanup script or schedule deletion of stale jobs.

CrashLens v2.2.1

Release Date: August 10, 2025

Version: 2.2.1

Status: Production Ready

---

🎉 Introducing CrashLens v2.0

The complete open-source platform for LLM cost monitoring and policy enforcement is here!

CrashLens v2.0 delivers a production-ready solution for organizations to monitor, control, and optimize their AI/LLM usage across all major platforms.

---

⭐ What's New in v2.0

🔗 Multi-Source Plugin System

Connect to any LLM platform with one command:

crashlens scan --source=langfuse --simulate    # Langfuse traces
crashlens scan --source=helicone --hours-back=24  # Helicone analytics
crashlens scan --source=openai --org=org-123   # OpenAI usage API

📦 Community Rule Pack Library

Ready-to-deploy policy templates:

• Block expensive models for simple tasks
• Detect and prevent retry storms
• Enforce per-trace cost limits
• Monitor model fallback patterns
• CI/CD-friendly validation rules

🛡️ Production-Grade Policy Engine

• YAML-based rule configuration
• Environment scoping and inheritance
• Cost threshold enforcement
• Simulation mode for safe testing
• Slack/webhook notifications

---

🚀 Get Started in 5 Minutes

# Install CrashLens v2.0
pip install crashlens

# Quick start with simulation
crashlens scan logs.jsonl --simulate

# Use community templates
crashlens scan --source=langfuse --policy policies/langfuse/retry-loop-detector.yaml

# Generate custom policies
crashlens init --template basic-safety --output my-policy.yaml

---

📊 Complete Feature Set

✅ OSS v1 Features (100% Complete)

• YAML-based rule configuration
• Policy evaluation engine
• Schema-aware matching
• CLI with CI-friendly output
• Dry-run simulation mode

✅ OSS v2 Features (100% Complete)

• Rule scoping by environment/metadata
• Cost threshold and budget controls
• Rule inheritance and suppression
• Multi-source plugin architecture
• Slack/webhook integrations

🚀 Enterprise Foundation (75% Complete)

• Extensible plugin system
• Community template library
• Runtime enforcement SDK (coming in v2.1)
• Analytics dashboard (coming in v2.2)

---

🎯 Perfect For

👥 Development Teams - Prevent cost overruns and enforce best practices

🏢 Organizations - Governance and compliance across AI usage

🔧 DevOps Engineers - CI/CD integration and automated monitoring

📊 Platform Teams - Multi-cloud LLM cost consolidation

🌍 Open Source Community - Extensible platform for custom policies

---

📚 Resources

• Installation: pip install crashlens
• Documentation: [Complete Status Report](https://www.notion.so/CRASHLENS_STATUS_REPORT.md)
• Quick Start: [Usage Guide](https://www.notion.so/docs/USAGE.md)
• Community Rules: [Rule Pack Library](https://www.notion.so/policies/langfuse/)
• Examples: [CI/CD Integration](https://www.notion.so/examples/)

---

🌟 Community & Contributions

CrashLens v2.0 is built for community contribution:

• Plugin Development - Add support for new LLM platforms
• Rule Pack Creation - Share policy templates for common use cases
• Feature Requests - Shape the roadmap for enterprise features
• Bug Reports - Help improve stability and compatibility

---

🎉 Thank You

CrashLens v2.0 represents a major milestone in open-source AI cost monitoring. With complete OSS functionality, plugin ecosystem, and enterprise-ready architecture, we're excited to see how the community adopts and extends the platform.

Ready to optimize your LLM costs?

pip install crashlens
crashlens scan --source=langfuse --simulate

---

🚀 CrashLens v2.0 - Complete. Production-Ready. Community-Driven.

Full Changelog: v1.1.7...v2.2.1

CrashLens v1.1.7 — Grouped Reports, Cleaner Output, and Critical Fixes

🚀 CrashLens v1.1.7 is here with a powerful revamp to the --detailed reporting system and key bug fixes! Say goodbye to scattered trace files and hello to clean, grouped issue summaries ready for automation, collaboration, and real-time action.

---

✨ What's New

🗂️ Grouped Category Reports (NEW!)

The --detailed flag now generates one JSON file per detector type, each containing:

• ✅ A clear summary of affected traces and estimated waste
• 🔎 All relevant issues grouped by category (e.g., retry_loop.json)
• 💡 Targeted fix suggestions
• 📊 Embedded metadata for scripting/automation

detailed_output/
├── fallback_failure.json
├── retry_loop.json
├── fallback_storm.json
└── overkill_model.json

---

🐛 Bug Fixes

• Fixed: TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

  • 🔍 Cause: Missing/null cost values in logs
  • ✅ Fix: All formatters now safely handle None costs

---

🧹 Cleanup & Improvements

• 🧽 Removed the --include-empty flag to simplify CLI usage
• 📘 Updated README with full CLI reference and pro tips
• 🧪 Improved --demo guidance and usage examples

---

📋 Updated CLI Cheatsheet

# 🔍 Scan log file
crashlens scan logs.jsonl

# 🎯 Grouped report by waste type
crashlens scan --detailed

# 💰 Cost summaries
crashlens scan --summary
crashlens scan --summary-only

# 🎨 Output formats
crashlens scan --format json
crashlens scan --format markdown
crashlens scan --format human

# 🧾 Input sources
crashlens scan --stdin
crashlens scan --paste
crashlens scan --demo

---

🔮 Coming Soon

• Visual dashboards for grouped output
• More detectors (e.g., long prompt spam, unused response tokens)
• SDK + GitHub Action support

---

🧠 Tip

Use the grouped JSONs to create your own dashboards, auto-alerts, or cost heatmaps. One file per category = easy integrations!

---

📦 Install / Upgrade

pip install --upgrade crashlens
# or
pip install crashlens==1.1.7

---

💬 Feedback

Let us know how the new --detailed feels! We built it to make your GPT cost debugging actionable, team-friendly, and automatable.

[🔗 Full Changelog »](v1.1.6...v1.1.7)