Security Hardening, Insights Hub, Image Provider Overhaul, Analytics Dashboard & UI Redesign

[zach7036]

Security Hardening, Insights Hub, Image Provider Overhaul, Analytics Dashboard & UI Redesign

Summary

This PR introduces comprehensive security fixes, a new multi-view Insights page, a redesigned image provider system with fallback support, new analytics and species pages, five new API endpoints, a collapsible sidebar navigation, a rewritten weekly report, and several bug fixes.

---

Security

• SQL injection prevention: Replaced all raw string interpolation in SQL queries with parameterized bindings across PHP (bindValue with :param placeholders) and Python (? placeholders with param lists). Affected: fetch_species_array, fetch_best_detection, fetch_all_detections, get_todays_count_for, get_this_weeks_count_for, get_species_by, the filename lookup in play.php, and all 7+ queries in weekly_report.php.
• Shell command injection prevention: Wrapped user-controlled values in escapeshellarg() for rm, timedatectl set-timezone, echo | tee, and sudo date -s calls in play.php, species_tools.php, and config.php.
• PHP 8.2 compatibility: Replaced deprecated FILTER_SANITIZE_STRING with FILTER_SANITIZE_FULL_SPECIAL_CHARS in views.php and play.php.

Insights Page (NEW — 1,327 lines)

A comprehensive Insights hub at scripts/insights.php with 7 subviews, each loading only its own data for efficient queries:

• Dashboard: Lifetime species count, best day, longest detection streak, rare species list, milestones, and a composite Yard Health Score (0–100) calculated from stability, volume, rarity, and Shannon diversity — with dynamic actionable recommendations.
• Behavior: Dawn chorus order (which species start singing first), hourly activity chart, peak hour, nocturnal detections, and per-species activity windows.
• Migration: New arrivals (last 14 days), gone-quiet species, year-over-year arrival timing comparison, seasonal presence heatmap (Year-round / Seasonal / Transient classification), monthly diversity trend chart, and Shannon diversity index.
• Environmental: Temperature bracket analysis, weather condition impact on detections, per-species ideal temperature ranges, and a 30-day temperature-vs-detections dual-axis chart. Requires the weather table; gracefully shows empty state without it.
• Health: 30-day confidence trend chart, phantom species detection (low-confidence repeats likely to be misidentifications), burst days (1.5× above average), silent days (≤3 detections), and confidence tier breakdown.
• Trends & Forecasting: Monthly diversity/detection trends, year-over-year diversity comparison, "expected today" predictions based on historical day-of-year data, and top-5 species peak weeks.
• Report: Embeds the redesigned weekly report inline.

A standalone insights_logic.php (77 lines) provides the same Dashboard KPIs as a reusable module.

Weekly Report Redesign (rewritten, 204 → 425 lines)

• All SQL queries now use parameterized bindings.
• Fixed week boundary calculation for the Sunday edge case.
• Improved "first seen" logic to check for detections before the report week rather than outside it.
• New KPIs: daily average, busiest day of the week, peak activity hour.
• Card-based UI with week-over-week trend badges, expandable species lists, info tooltips, and CSS design system integration.
• Legacy ?ascii output preserved.

Image Provider System

• Fallback pattern: get_image() now accepts an optional $fallback_provider. If the primary provider (Wikipedia or Flickr) returns no image, it automatically tries the other. All call sites wire both providers.
• Wikipedia rewrite: Tries both scientific and common English names as page titles. Uses Wikimedia Commons thumburl at 1024px. Better metadata extraction with null-safe handling.
• Flickr improvements: Higher resolution (_b suffix, 1024px). Removed orientation filter. Switched to urlencode(). Fixed constructor to call parent::__construct(). Added null checks on all API responses.
• General: get_json() returns false on failure instead of crashing. get_com_en_name() uses a static cache. Debug logging to birdnet_img.log. Image cache DBs renamed to *_v4.db to force rebuild.

New Features

• Analytics page (scripts/analytics.php, 945 lines): KPI cards, hourly activity heatmap, species trends, confidence distribution, time-period filtering with Chart.js.
• Species page (scripts/species.php, 333 lines): Searchable, sortable species catalog with time-period filtering and summary KPIs.
• 5 new API endpoints:
  • GET /api/v1/analytics/activity?days=N — Hourly detection counts.
  • GET /api/v1/analytics/stats?days=N — Aggregate stats.
  • GET /api/v1/detections/recent?limit=N — Most recent detections.
  • GET /api/v1/detections/timeline?date=YYYY-MM-DD — Full day timeline with hourly grouping and 5-minute same-species clustering.
  • GET /api/v1/species/search?q=term — Autocomplete search (common + scientific names).
• Per-species charts (stats.php): Confidence distribution and seasonal presence bar charts.
• Weather integration (scripts/utils/weather.py): Hourly Open-Meteo sync storing temperature and condition codes in birds.db, with cron template.
• Extended stats: fetch_stats() now returns newspeciestally, topspecies, and topspeciescount.
• Overview AJAX endpoint: ?ajax_chart_data=true returns JSON species/hourly data for client-side chart rendering, with session-based image caching.

UI/UX

• Collapsible sidebar navigation replacing the top nav bar. Top-level items with emoji labels, plus an expandable Insights dropdown containing all 7 subviews. Hidden <input> passes the selected subview via query string.
• Live audio slide-out panel fixed to top-right corner, replacing the in-banner stream button.
• Full-viewport iframe layout — removed the banner from index.php.
• Timeline view for today's detections, replacing the legacy table and search input.
• CSS design system with custom properties (:root variables for colors, spacing, shadows, radii). Sidebar styles added. Light theme expanded (~900 → 2,000 lines). Dark theme rewritten from 972 lines to 118 lines using variable overrides.
• Chart colors: Light-mode background changed to soft teal (#e8f4f2), heatmap palette from Greens to Blues_r.
• Daily Charts simplified to an eBird CSV export button.
• Broken image handling: onerror fallback and null-safe image array access throughout.
• Best Recordings link added to the Recordings page.
• 404 response updated to structured JSON.

Bug Fixes

• advanced.php: Fixed two variable name bugs — freqshift_reconnect_delay comparison was using $freqshift_hi, and raw_spectrogram comparison was using undefined $RAW_SPECTROGRAM.
• spectrogram.php: Fixed delay calculation from DateInterval::format('%s') (partial seconds) to getTimestamp() subtraction (total seconds). Changed file discovery from glob() to scandir(). Wrapped parent audio access in try/catch for cross-origin safety. Handled autoplay policy blocks with .catch().
• helpers.py: Removed redundant os.path.join() double-prepending RECS_DIR. Settings path now checks BIRDNET_CONF env var before defaulting.

Infrastructure

• Removed php-mbstring from install dependencies.
• Added insights.php symlink to clear_all_data.sh.
• Updated all repo URLs from Nachtzuster/BirdNET-Pi to zach7036/BirdNET-Pi.

New Files

File	Lines	Purpose
scripts/insights.php	1,327	Insights hub with 7 subviews
scripts/insights_logic.php	77	Reusable insights logic module
scripts/analytics.php	945	Analytics dashboard
scripts/species.php	333	Species catalog
scripts/clear_image_cache.php	21	Cache cleanup utility
scripts/utils/weather.py	75	Hourly weather sync
templates/weather.cron	2	Weather cron schedule
homepage/static/dashboard-charts.js	370	Overview chart rendering
homepage/static/timeline-view.js	337	Today's detections timeline