From b695632a4ad3575377976c160aef626614d7a7ed Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:21:10 -0500 Subject: [PATCH 1/8] Init 2026 --- subteams/app/2025-26/full-explain.md | 284 ++++++++++++++++++ subteams/app/2025-26/modifyingMessages.md | 14 + subteams/app/{ => pre-2020}/app.md | 0 .../{ => pre-2020}/compakt_info/cloudflare.md | 0 4 files changed, 298 insertions(+) create mode 100644 subteams/app/2025-26/full-explain.md create mode 100644 subteams/app/2025-26/modifyingMessages.md rename subteams/app/{ => pre-2020}/app.md (100%) rename subteams/app/{ => pre-2020}/compakt_info/cloudflare.md (100%) diff --git a/subteams/app/2025-26/full-explain.md b/subteams/app/2025-26/full-explain.md new file mode 100644 index 0000000..e140c51 --- /dev/null +++ b/subteams/app/2025-26/full-explain.md @@ -0,0 +1,284 @@ +## Dev only - this is likley too much info to crawl through if you aren't actively developing this app or migrating it to a future challenge. + +================================================================================ + +--- main.dart --- +Entry point and UI layer of the app. + + main() + Calls runApp(Root()), which starts the Flutter widget tree. + + Root (StatelessWidget) + Creates two long-lived singletons and registers them as Providers so every + widget in the tree can access them without manual threading: + - iarc_state.State — GPS position and map coordinate data + - Networking — TCP connection to the server + - SpeechRecognition — microphone / voice command handler + - app_logger.Logger — in-app log store + + HomePage / _HomePageState (StatefulWidget) + The one and only screen. On initState() it: + 1. Calls checkrgbFile() — if RGBmap.dat does not exist on disk, creates a + blank 1000x1000 black RGBA file so the map widget has something to show. + 2. Calls initFileWatcher() — sets up a dart:io file watcher on RGBmap.dat. + Whenever the file changes on disk, loadImage() re-reads it and calls + setState() to redraw the map. + 3. Calls SpeechRecognition.initSpeech() to load the Vosk model. + 4. Calls Networking.initializeSocket() to connect to the server. + + loadImage() + Reads the raw RGBA bytes from RGBmap.dat into a ui.Image using + ui.decodeImageFromPixels. Guards against partial writes by checking the file + is exactly width*height*4 bytes (4 MB) before decoding. + + Normal UI (debugModeOn = false): + - AppBar: green background when connected, red when not. + - Map: a RawImage widget displaying the decoded ui.Image, overlaid with a + green CircleAvatar dot that tracks the phone's map coordinates from State. + - Text below map: shows "Awaiting input..." while mic is active, otherwise + the last command response from SpeechRecognition. + - SpeedDial (top button, green): + Arm — sends MessageTypes.ARM + Emergency Land — sends MessageTypes.EMERGENCY_LAND + Takeoff — sends MessageTypes.START_TAKEOFF + Debug — toggles debug view + - Mic / Connect FAB (bottom): if connected, toggles listening; if not + connected, retries initializeSocket(). + + Debug UI (debugModeOn = true): + Scrollable list of all Logger entries (info=black, debug=blue, error=red). + SpeedDial with Return (back to normal view) and Clear (wipe the log). + +================================================================================ + +--- networking.dart --- +All TCP communication between the app and the IARC-10 server. + + MessageTypes + Namespace of integer constants shared between the app and the server. + Each constant is an agreed-upon channel number for a specific action: + App-side: APP_CONFIG(401), SEND_PHONE_LOCATION(999), + REQUEST_DRONE_LOCATIONS(415), SET_SCAN_STATUS(410), etc. + Drone-side: ARM(520), DISARM(523), START_TAKEOFF(530), START_DEMO(535), + START_MISSION(540), EMERGENCY_LAND(555), LAND(556), etc. + Server→App: SEND_PATHS_TO_APP(420), SEND_DRONE_LOCATIONS(425), + ARM_ACK(521), ARM_NACK(522), START_TAKEOFF_ACK(531), etc. + + Networking (ChangeNotifier) + Fields: + - localIp / listenPort(5100): the phone's own server address. + - externalIp(10.106.93.77) / externalPort(6002): the IARC-10 server. + - hasConnection / attemptingConnection: drive the UI button state. + + getIp() + Scans the device's network interfaces for one named "wlan" with an IPv4 + address. Returns the first match (the phone's Wi-Fi IP). + + initializeSocket() + 1. Gets the local IP. + 2. Calls attemptConnection() to make a test TCP connection to the server. + 3. On success: closes the test socket, starts a listening ServerSocket on + listenPort via startServer(), sets hasConnection=true, then sends: + - APP_CONFIG with the phone's IP and port so the server knows where + to call back. + - REQUEST_DRONE_LOCATIONS to get reference points for the position + tracker. + - sendPhoneLocation() with the current GPS fix. + + sendData(dataToSend) + Opens a fresh TCP connection for every outbound message (connect → write + JSON line → flush → close). If the connection attempt fails, sets + hasConnection=false and notifies the UI. + + handleServerMessage(client) + The callback registered on the listening ServerSocket. Reads incoming bytes, + UTF-8 decodes and JSON-parses them into a Data object, then switches on + recievedData.id: + ARM_ACK/NACK, PING_ACK/NACK, TAKEOFF_ACK, DEMO_ACK/DONE, MISSION_ACK, + NEW_WAYPOINTS_ACK, REACHED_WAYPOINT, SCAN_ERROR + → update SpeechRecognition.commandResponse text. + SEND_PATHS_TO_APP + → calls map.processMapFromJson() to redraw the map file. + SEND_DRONE_LOCATIONS + → destructures drone1/drone2 lat-long + x/y pairs and calls + state.launchPhoneTracker() to initialize the coordinate transform. + + sendPhoneLocation() + Sends a SEND_PHONE_LOCATION message with state.latitude / state.longitude. + + constructDebugMessage(embeddedMessage) + Wraps any Data object inside an APP_DEBUG message so the server echoes it + back to the app — used only during development to test the inbound handler. + +================================================================================ + +--- speech.dart --- +Offline voice recognition using the Vosk speech engine. + + SpeechRecognition (ChangeNotifier) + Fields: + - isListening: drives the mic button icon in the UI. + - commandResponse: the text displayed below the map after a command. + - _lastProcessedText: deduplicates repeated Vosk results. + + initSpeech() + Loads the bundled Vosk model from assets (vosk-model-small-en-us-0.15.zip), + creates a Recognizer restricted to a fixed grammar: + 'arm drones', 'take off', 'start scanning', 'start demo', + 'start mission', 'stop', 'send location', 'hover', '[unk]' + Registers _onResult as the result callback. Sets networking.setSpeechRecog + so the Networking class can update commandResponse on incoming ACKs. + + startListening() + Starts the Vosk SpeechService, sets isListening=true, then sleeps 3 seconds + and calls stopListening(). Recognition is therefore always a fixed 3-second + window. + + stopListening() + Calls _speechService.stop() and clears isListening. + + _onResult(result) + JSON-decodes the Vosk result, extracts the "text" field, skips empty or + duplicate strings, then passes the text to messaging.setAppState(). Stores + the return value in commandResponse and notifies the UI. + +================================================================================ + +--- messaging.dart --- +Message serialization and voice command parsing. + + Data + The single message format used by every app↔server exchange: + { + "id": int, // MessageTypes constant + "dronesToSendData": [int, ...], // which drones to target + "data": { ... }, // payload, varies by message type + "senderId": 0 // always 0 for the app + } + Data.fromJson / toJson handle serialization. MessageTypes constants are + also defined here (duplicated from networking.dart for use by messaging code). + + setAppState(state, networking, lastWords) + Voice command dispatch table. Steps: + 1. Splits lastWords into tokens, lowercases all. + 2. Rejects if fewer than 2 tokens. + 3. Matches wordList[0] against a fixed set of control words, then + wordList[1] for sub-commands. + + Supported commands and the messages they send: + "arm drones" → ARM(520) + "take off" → START_TAKEOFF(530) + "start scanning" → SET_SCAN_STATUS(410) {Message: 'Start scan'} + "start demo" → START_DEMO(535) + "start mission" → START_MISSION(540) + "stop" → SET_SCAN_STATUS(410) {Message: 'Stop scanning'} + "send location" → updateLocation() then sendPhoneLocation() + "hover [m|meters]" + → SET_HOVER_STATUS(412) with height in feet + (auto-converts meters × 3.821 if unit given) + "debug map" → calls processMapFromJson() locally with hard-coded + test data; no network involved + "debug message" → sends an APP_DEBUG message to the server which + echoes it back to exercise handleServerMessage() + +================================================================================ + +--- state.dart --- +GPS position tracking and lat/long → map-coordinate conversion. + + State (ChangeNotifier) + Fields: + - _position: the current GPS Position from geolocator. + - latitude / longitude: getters that unwrap _position safely. + - _phoneCord [x, y]: the phone's position in the drone map's coordinate + system. Read by main.dart to place the green dot overlay. + - phoneTrackerInitialized: guards against drawing before calibration. + - transformMatrix, latLongMatrix, localCordMatrix: 2×2 matrices used for + the coordinate transform (from ml_linalg). + - _refLatLong, _refLocal: the reference point (drone 1's known position + in both coordinate systems). + + launchPhoneTracker(state, lat1, long1, x1, y1, lat2, long2, x2, y2) + Called by Networking when SEND_DRONE_LOCATIONS arrives. Uses the GPS + and map coordinates of two drones as calibration points, then: + 1. Calls initTransformMatrix() to compute the transform. + 2. Gets an initial GPS fix. + 3. Calls updatePhonePosition() once immediately. + 4. Starts a continuous GPS stream via initGeolocationStreamListener(). + + initTransformMatrix(...) + Builds a 2×2 affine transform from two control points. The matrix maps + Δlat/Δlong offsets to Δx/Δy map offsets. Computed as: + localCordMatrix × inverse(latLongMatrix) + where each matrix encodes the rotation/scale of its respective space. + + updatePhonePosition() + Converts the current GPS position to map coordinates: + phoneCoords = transformMatrix × (phoneLatLong − _refLatLong) + _refLocal + Stores the result in _phoneCord and notifies listeners. Also re-renders the + map by calling map.processMapFromJson(prevJson, this) from the GPS stream. + + updateLocation() + One-shot GPS fetch via Geolocator.getCurrentPosition(). Called by the + "send location" voice command before reporting position to the server. + +================================================================================ + +--- map.dart --- +Renders the 1000×1000 mission map image and writes it to disk. + + processMapFromJson(jsonIn, state) + Rebuilds the full map from scratch on every call. Input is a JSON map with: + "paths": list of waypoint objects {x, y, straight, lastInPath} + "mines": list of mine objects {x, y, radius} + + Drawing pipeline (using the 'image' dart package): + 1. Creates a blank 1000×1000 RGBA image with a black background. + 2. Iterates over the path list: + - If path[i].lastInPath == 1: skip (end-of-segment marker). + - If path[i].straight == 0: treat path[i], path[i+1], path[i+2] as + start/midpoint/end of a quadratic Bézier arc; calls _drawArc(). + Advances i by 2. + - Otherwise: draws a straight line from path[i] to path[i+1]. + Advances i by 1. + All path lines are green (0, 255, 0), thickness 3. + 3. For each mine, draws a solid red outline circle and a semi-transparent + red filled circle (alpha 80) at its (x, y) with the given radius. + 4. Converts the image to raw RGBA bytes (Uint8List). + 5. Writes bytes to RGBmap.dat.tmp, then atomically renames to RGBmap.dat. + The atomic rename prevents the file watcher in main.dart from reading + a partially written file. + + _drawArc(img, start, control, end, color, steps) + Approximates a quadratic Bézier curve by stepping t from 0→1 in `steps` + increments, computing the on-curve point at each t, and drawing a straight + line segment between consecutive points. + The control point is derived from the on-curve midpoint using the standard + inverse Bézier formula: ctrl = 2*mid − 0.5*start − 0.5*end. + + prevJson + Module-level variable that caches the last JSON received. Used by + state.dart's GPS stream to re-render the map with the latest path data + whenever the phone's position changes. + +================================================================================ + +--- logger.dart --- +In-app logging that feeds the debug UI in main.dart. + + Logger (ChangeNotifier) + Stores log entries as a List (Flutter rich-text widgets). The debug + UI in _HomePageState watches this list directly and rebuilds when it changes. + + Log levels: + i(str) — info: black text, prefix "I " + d(str) — debug: blue italic, prefix "D " + e(str) — error: red bold, prefix "E " + + clear() — empties the list and notifies listeners. + + logger + Module-level singleton (Logger logger = Logger()) imported by all other files + so every class writes to the same log store. + +================================================================================ diff --git a/subteams/app/2025-26/modifyingMessages.md b/subteams/app/2025-26/modifyingMessages.md new file mode 100644 index 0000000..028c41f --- /dev/null +++ b/subteams/app/2025-26/modifyingMessages.md @@ -0,0 +1,14 @@ +# Creating a New Message + +1. Add the message type to the `MessageTypes` class in `networking.dart` +2. To send the message, call: + ```dart + sendData( + Data( + MessageTypes., + [1], + { } + ).toJson() + ) + ``` +3. To receive the message, add it to the switch case in `handleServerMessage` (also in `networking.dart`) diff --git a/subteams/app/app.md b/subteams/app/pre-2020/app.md similarity index 100% rename from subteams/app/app.md rename to subteams/app/pre-2020/app.md diff --git a/subteams/app/compakt_info/cloudflare.md b/subteams/app/pre-2020/compakt_info/cloudflare.md similarity index 100% rename from subteams/app/compakt_info/cloudflare.md rename to subteams/app/pre-2020/compakt_info/cloudflare.md From 5848337d65450067c2a8dec71d6cfc78c1c15beb Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:25:49 -0500 Subject: [PATCH 2/8] Update --- subteams/app/2025-26/full-explain.md | 494 ++++++++---------- subteams/app/{pre-2020 => pre-2025}/app.md | 0 .../compakt_info/cloudflare.md | 0 3 files changed, 230 insertions(+), 264 deletions(-) rename subteams/app/{pre-2020 => pre-2025}/app.md (100%) rename subteams/app/{pre-2020 => pre-2025}/compakt_info/cloudflare.md (100%) diff --git a/subteams/app/2025-26/full-explain.md b/subteams/app/2025-26/full-explain.md index e140c51..6b20636 100644 --- a/subteams/app/2025-26/full-explain.md +++ b/subteams/app/2025-26/full-explain.md @@ -1,284 +1,250 @@ -## Dev only - this is likley too much info to crawl through if you aren't actively developing this app or migrating it to a future challenge. +> **Dev only** — This is likely too much detail to read through unless you are actively developing this app or migrating it to a future challenge. -================================================================================ +--- + +## main.dart ---- main.dart --- Entry point and UI layer of the app. - main() - Calls runApp(Root()), which starts the Flutter widget tree. - - Root (StatelessWidget) - Creates two long-lived singletons and registers them as Providers so every - widget in the tree can access them without manual threading: - - iarc_state.State — GPS position and map coordinate data - - Networking — TCP connection to the server - - SpeechRecognition — microphone / voice command handler - - app_logger.Logger — in-app log store - - HomePage / _HomePageState (StatefulWidget) - The one and only screen. On initState() it: - 1. Calls checkrgbFile() — if RGBmap.dat does not exist on disk, creates a - blank 1000x1000 black RGBA file so the map widget has something to show. - 2. Calls initFileWatcher() — sets up a dart:io file watcher on RGBmap.dat. - Whenever the file changes on disk, loadImage() re-reads it and calls - setState() to redraw the map. - 3. Calls SpeechRecognition.initSpeech() to load the Vosk model. - 4. Calls Networking.initializeSocket() to connect to the server. - - loadImage() - Reads the raw RGBA bytes from RGBmap.dat into a ui.Image using - ui.decodeImageFromPixels. Guards against partial writes by checking the file - is exactly width*height*4 bytes (4 MB) before decoding. - - Normal UI (debugModeOn = false): - - AppBar: green background when connected, red when not. - - Map: a RawImage widget displaying the decoded ui.Image, overlaid with a - green CircleAvatar dot that tracks the phone's map coordinates from State. - - Text below map: shows "Awaiting input..." while mic is active, otherwise - the last command response from SpeechRecognition. - - SpeedDial (top button, green): - Arm — sends MessageTypes.ARM - Emergency Land — sends MessageTypes.EMERGENCY_LAND - Takeoff — sends MessageTypes.START_TAKEOFF - Debug — toggles debug view - - Mic / Connect FAB (bottom): if connected, toggles listening; if not - connected, retries initializeSocket(). - - Debug UI (debugModeOn = true): - Scrollable list of all Logger entries (info=black, debug=blue, error=red). - SpeedDial with Return (back to normal view) and Clear (wipe the log). - -================================================================================ - ---- networking.dart --- +### `main()` +Calls `runApp(Root())`, which starts the Flutter widget tree. + +### `Root` (StatelessWidget) +Creates long-lived singletons and registers them as Providers so every widget in the tree can access them without manual threading: +- `iarc_state.State` — GPS position and map coordinate data +- `Networking` — TCP connection to the server +- `SpeechRecognition` — microphone / voice command handler +- `app_logger.Logger` — in-app log store + +### `HomePage` / `_HomePageState` (StatefulWidget) +The one and only screen. On `initState()` it: +1. Calls `checkrgbFile()` — if `RGBmap.dat` does not exist on disk, creates a blank 1000×1000 black RGBA file so the map widget has something to show. +2. Calls `initFileWatcher()` — sets up a `dart:io` file watcher on `RGBmap.dat`. Whenever the file changes on disk, `loadImage()` re-reads it and calls `setState()` to redraw the map. +3. Calls `SpeechRecognition.initSpeech()` to load the Vosk model. +4. Calls `Networking.initializeSocket()` to connect to the server. + +### `loadImage()` +Reads the raw RGBA bytes from `RGBmap.dat` into a `ui.Image` using `ui.decodeImageFromPixels`. Guards against partial writes by checking the file is exactly `width * height * 4` bytes (4 MB) before decoding. + +### Normal UI (`debugModeOn = false`) +- **AppBar:** green background when connected, red when not. +- **Map:** a `RawImage` widget displaying the decoded `ui.Image`, overlaid with a green `CircleAvatar` dot that tracks the phone's map coordinates from `State`. +- **Text below map:** shows "Awaiting input..." while mic is active, otherwise the last command response from `SpeechRecognition`. +- **SpeedDial** (top button, green): + - Arm — sends `MessageTypes.ARM` + - Emergency Land — sends `MessageTypes.EMERGENCY_LAND` + - Takeoff — sends `MessageTypes.START_TAKEOFF` + - Debug — toggles debug view +- **Mic / Connect FAB** (bottom): if connected, toggles listening; if not connected, retries `initializeSocket()`. + +### Debug UI (`debugModeOn = true`) +Scrollable list of all Logger entries (info = black, debug = blue, error = red). SpeedDial with Return (back to normal view) and Clear (wipe the log). + +--- + +## networking.dart + All TCP communication between the app and the IARC-10 server. - MessageTypes - Namespace of integer constants shared between the app and the server. - Each constant is an agreed-upon channel number for a specific action: - App-side: APP_CONFIG(401), SEND_PHONE_LOCATION(999), - REQUEST_DRONE_LOCATIONS(415), SET_SCAN_STATUS(410), etc. - Drone-side: ARM(520), DISARM(523), START_TAKEOFF(530), START_DEMO(535), - START_MISSION(540), EMERGENCY_LAND(555), LAND(556), etc. - Server→App: SEND_PATHS_TO_APP(420), SEND_DRONE_LOCATIONS(425), - ARM_ACK(521), ARM_NACK(522), START_TAKEOFF_ACK(531), etc. - - Networking (ChangeNotifier) - Fields: - - localIp / listenPort(5100): the phone's own server address. - - externalIp(10.106.93.77) / externalPort(6002): the IARC-10 server. - - hasConnection / attemptingConnection: drive the UI button state. - - getIp() - Scans the device's network interfaces for one named "wlan" with an IPv4 - address. Returns the first match (the phone's Wi-Fi IP). - - initializeSocket() - 1. Gets the local IP. - 2. Calls attemptConnection() to make a test TCP connection to the server. - 3. On success: closes the test socket, starts a listening ServerSocket on - listenPort via startServer(), sets hasConnection=true, then sends: - - APP_CONFIG with the phone's IP and port so the server knows where - to call back. - - REQUEST_DRONE_LOCATIONS to get reference points for the position - tracker. - - sendPhoneLocation() with the current GPS fix. - - sendData(dataToSend) - Opens a fresh TCP connection for every outbound message (connect → write - JSON line → flush → close). If the connection attempt fails, sets - hasConnection=false and notifies the UI. - - handleServerMessage(client) - The callback registered on the listening ServerSocket. Reads incoming bytes, - UTF-8 decodes and JSON-parses them into a Data object, then switches on - recievedData.id: - ARM_ACK/NACK, PING_ACK/NACK, TAKEOFF_ACK, DEMO_ACK/DONE, MISSION_ACK, - NEW_WAYPOINTS_ACK, REACHED_WAYPOINT, SCAN_ERROR - → update SpeechRecognition.commandResponse text. - SEND_PATHS_TO_APP - → calls map.processMapFromJson() to redraw the map file. - SEND_DRONE_LOCATIONS - → destructures drone1/drone2 lat-long + x/y pairs and calls - state.launchPhoneTracker() to initialize the coordinate transform. - - sendPhoneLocation() - Sends a SEND_PHONE_LOCATION message with state.latitude / state.longitude. - - constructDebugMessage(embeddedMessage) - Wraps any Data object inside an APP_DEBUG message so the server echoes it - back to the app — used only during development to test the inbound handler. - -================================================================================ - ---- speech.dart --- +### `MessageTypes` +Namespace of integer constants shared between the app and the server. Each constant is an agreed-upon channel number for a specific action: + +| Direction | Constants | +|---|---| +| App-side | `APP_CONFIG(401)`, `SEND_PHONE_LOCATION(999)`, `REQUEST_DRONE_LOCATIONS(415)`, `SET_SCAN_STATUS(410)`, … | +| Drone-side | `ARM(520)`, `DISARM(523)`, `START_TAKEOFF(530)`, `START_DEMO(535)`, `START_MISSION(540)`, `EMERGENCY_LAND(555)`, `LAND(556)`, … | +| Server→App | `SEND_PATHS_TO_APP(420)`, `SEND_DRONE_LOCATIONS(425)`, `ARM_ACK(521)`, `ARM_NACK(522)`, `START_TAKEOFF_ACK(531)`, … | + +### `Networking` (ChangeNotifier) +**Fields:** +- `localIp` / `listenPort(5100)` — the phone's own server address +- `externalIp(10.106.93.77)` / `externalPort(6002)` — the IARC-10 server +- `hasConnection` / `attemptingConnection` — drive the UI button state + +#### `getIp()` +Scans the device's network interfaces for one named `"wlan"` with an IPv4 address. Returns the first match (the phone's Wi-Fi IP). + +#### `initializeSocket()` +1. Gets the local IP. +2. Calls `attemptConnection()` to make a test TCP connection to the server. +3. On success: closes the test socket, starts a listening `ServerSocket` on `listenPort` via `startServer()`, sets `hasConnection = true`, then sends: + - `APP_CONFIG` with the phone's IP and port so the server knows where to call back. + - `REQUEST_DRONE_LOCATIONS` to get reference points for the position tracker. + - `sendPhoneLocation()` with the current GPS fix. + +#### `sendData(dataToSend)` +Opens a fresh TCP connection for every outbound message (connect → write JSON line → flush → close). If the connection attempt fails, sets `hasConnection = false` and notifies the UI. + +#### `handleServerMessage(client)` +The callback registered on the listening `ServerSocket`. Reads incoming bytes, UTF-8 decodes and JSON-parses them into a `Data` object, then switches on `recievedData.id`: +- `ARM_ACK/NACK`, `PING_ACK/NACK`, `TAKEOFF_ACK`, `DEMO_ACK/DONE`, `MISSION_ACK`, `NEW_WAYPOINTS_ACK`, `REACHED_WAYPOINT`, `SCAN_ERROR` → update `SpeechRecognition.commandResponse` text. +- `SEND_PATHS_TO_APP` → calls `map.processMapFromJson()` to redraw the map file. +- `SEND_DRONE_LOCATIONS` → destructures drone1/drone2 lat-long + x/y pairs and calls `state.launchPhoneTracker()` to initialize the coordinate transform. + +#### `sendPhoneLocation()` +Sends a `SEND_PHONE_LOCATION` message with `state.latitude` / `state.longitude`. + +#### `constructDebugMessage(embeddedMessage)` +Wraps any `Data` object inside an `APP_DEBUG` message so the server echoes it back to the app — used only during development to test the inbound handler. + +--- + +## speech.dart + Offline voice recognition using the Vosk speech engine. - SpeechRecognition (ChangeNotifier) - Fields: - - isListening: drives the mic button icon in the UI. - - commandResponse: the text displayed below the map after a command. - - _lastProcessedText: deduplicates repeated Vosk results. +### `SpeechRecognition` (ChangeNotifier) +**Fields:** +- `isListening` — drives the mic button icon in the UI +- `commandResponse` — the text displayed below the map after a command +- `_lastProcessedText` — deduplicates repeated Vosk results + +#### `initSpeech()` +Loads the bundled Vosk model from assets (`vosk-model-small-en-us-0.15.zip`), creates a `Recognizer` restricted to a fixed grammar: +``` +'arm drones', 'take off', 'start scanning', 'start demo', +'start mission', 'stop', 'send location', 'hover', '[unk]' +``` +Registers `_onResult` as the result callback. Sets `networking.setSpeechRecog` so the `Networking` class can update `commandResponse` on incoming ACKs. - initSpeech() - Loads the bundled Vosk model from assets (vosk-model-small-en-us-0.15.zip), - creates a Recognizer restricted to a fixed grammar: - 'arm drones', 'take off', 'start scanning', 'start demo', - 'start mission', 'stop', 'send location', 'hover', '[unk]' - Registers _onResult as the result callback. Sets networking.setSpeechRecog - so the Networking class can update commandResponse on incoming ACKs. +#### `startListening()` +Starts the Vosk `SpeechService`, sets `isListening = true`, then sleeps 3 seconds and calls `stopListening()`. Recognition is therefore always a fixed 3-second window. - startListening() - Starts the Vosk SpeechService, sets isListening=true, then sleeps 3 seconds - and calls stopListening(). Recognition is therefore always a fixed 3-second - window. +#### `stopListening()` +Calls `_speechService.stop()` and clears `isListening`. - stopListening() - Calls _speechService.stop() and clears isListening. +#### `_onResult(result)` +JSON-decodes the Vosk result, extracts the `"text"` field, skips empty or duplicate strings, then passes the text to `messaging.setAppState()`. Stores the return value in `commandResponse` and notifies the UI. - _onResult(result) - JSON-decodes the Vosk result, extracts the "text" field, skips empty or - duplicate strings, then passes the text to messaging.setAppState(). Stores - the return value in commandResponse and notifies the UI. +--- -================================================================================ +## messaging.dart ---- messaging.dart --- Message serialization and voice command parsing. - Data - The single message format used by every app↔server exchange: - { - "id": int, // MessageTypes constant - "dronesToSendData": [int, ...], // which drones to target - "data": { ... }, // payload, varies by message type - "senderId": 0 // always 0 for the app - } - Data.fromJson / toJson handle serialization. MessageTypes constants are - also defined here (duplicated from networking.dart for use by messaging code). - - setAppState(state, networking, lastWords) - Voice command dispatch table. Steps: - 1. Splits lastWords into tokens, lowercases all. - 2. Rejects if fewer than 2 tokens. - 3. Matches wordList[0] against a fixed set of control words, then - wordList[1] for sub-commands. - - Supported commands and the messages they send: - "arm drones" → ARM(520) - "take off" → START_TAKEOFF(530) - "start scanning" → SET_SCAN_STATUS(410) {Message: 'Start scan'} - "start demo" → START_DEMO(535) - "start mission" → START_MISSION(540) - "stop" → SET_SCAN_STATUS(410) {Message: 'Stop scanning'} - "send location" → updateLocation() then sendPhoneLocation() - "hover [m|meters]" - → SET_HOVER_STATUS(412) with height in feet - (auto-converts meters × 3.821 if unit given) - "debug map" → calls processMapFromJson() locally with hard-coded - test data; no network involved - "debug message" → sends an APP_DEBUG message to the server which - echoes it back to exercise handleServerMessage() - -================================================================================ - ---- state.dart --- +### `Data` +The single message format used by every app↔server exchange: +```json +{ + "id": int, + "dronesToSendData": [int, ...], + "data": { ... }, + "senderId": 0 +} +``` +`id` is a `MessageTypes` constant. `dronesToSendData` specifies which drones to target. `senderId` is always `0` for the app. `Data.fromJson` / `toJson` handle serialization. + +> Note: `MessageTypes` constants are also defined here, duplicated from `networking.dart` for use by messaging code. + +### `setAppState(state, networking, lastWords)` +Voice command dispatch table. + +**Steps:** +1. Splits `lastWords` into tokens, lowercases all. +2. Rejects if fewer than 2 tokens. +3. Matches `wordList[0]` against a fixed set of control words, then `wordList[1]` for sub-commands. + +**Supported commands:** + +| Voice command | Message sent | +|---|---| +| `"arm drones"` | `ARM(520)` | +| `"take off"` | `START_TAKEOFF(530)` | +| `"start scanning"` | `SET_SCAN_STATUS(410)` `{Message: 'Start scan'}` | +| `"start demo"` | `START_DEMO(535)` | +| `"start mission"` | `START_MISSION(540)` | +| `"stop"` | `SET_SCAN_STATUS(410)` `{Message: 'Stop scanning'}` | +| `"send location"` | `updateLocation()` then `sendPhoneLocation()` | +| `"hover [m\|meters]"` | `SET_HOVER_STATUS(412)` with height in feet (auto-converts meters × 3.821 if unit given) | +| `"debug map"` | Calls `processMapFromJson()` locally with hard-coded test data; no network involved | +| `"debug message"` | Sends an `APP_DEBUG` message to the server which echoes it back to exercise `handleServerMessage()` | + +--- + +## state.dart + GPS position tracking and lat/long → map-coordinate conversion. - State (ChangeNotifier) - Fields: - - _position: the current GPS Position from geolocator. - - latitude / longitude: getters that unwrap _position safely. - - _phoneCord [x, y]: the phone's position in the drone map's coordinate - system. Read by main.dart to place the green dot overlay. - - phoneTrackerInitialized: guards against drawing before calibration. - - transformMatrix, latLongMatrix, localCordMatrix: 2×2 matrices used for - the coordinate transform (from ml_linalg). - - _refLatLong, _refLocal: the reference point (drone 1's known position - in both coordinate systems). - - launchPhoneTracker(state, lat1, long1, x1, y1, lat2, long2, x2, y2) - Called by Networking when SEND_DRONE_LOCATIONS arrives. Uses the GPS - and map coordinates of two drones as calibration points, then: - 1. Calls initTransformMatrix() to compute the transform. - 2. Gets an initial GPS fix. - 3. Calls updatePhonePosition() once immediately. - 4. Starts a continuous GPS stream via initGeolocationStreamListener(). - - initTransformMatrix(...) - Builds a 2×2 affine transform from two control points. The matrix maps - Δlat/Δlong offsets to Δx/Δy map offsets. Computed as: - localCordMatrix × inverse(latLongMatrix) - where each matrix encodes the rotation/scale of its respective space. - - updatePhonePosition() - Converts the current GPS position to map coordinates: - phoneCoords = transformMatrix × (phoneLatLong − _refLatLong) + _refLocal - Stores the result in _phoneCord and notifies listeners. Also re-renders the - map by calling map.processMapFromJson(prevJson, this) from the GPS stream. - - updateLocation() - One-shot GPS fetch via Geolocator.getCurrentPosition(). Called by the - "send location" voice command before reporting position to the server. - -================================================================================ - ---- map.dart --- +### `State` (ChangeNotifier) +**Fields:** +- `_position` — the current GPS `Position` from geolocator +- `latitude` / `longitude` — getters that unwrap `_position` safely +- `_phoneCord [x, y]` — the phone's position in the drone map's coordinate system; read by `main.dart` to place the green dot overlay +- `phoneTrackerInitialized` — guards against drawing before calibration +- `transformMatrix`, `latLongMatrix`, `localCordMatrix` — 2×2 matrices used for the coordinate transform (from `ml_linalg`) +- `_refLatLong`, `_refLocal` — the reference point (drone 1's known position in both coordinate systems) + +#### `launchPhoneTracker(state, lat1, long1, x1, y1, lat2, long2, x2, y2)` +Called by `Networking` when `SEND_DRONE_LOCATIONS` arrives. Uses the GPS and map coordinates of two drones as calibration points, then: +1. Calls `initTransformMatrix()` to compute the transform. +2. Gets an initial GPS fix. +3. Calls `updatePhonePosition()` once immediately. +4. Starts a continuous GPS stream via `initGeolocationStreamListener()`. + +#### `initTransformMatrix(...)` +Builds a 2×2 affine transform from two control points. The matrix maps Δlat/Δlong offsets to Δx/Δy map offsets. Computed as: +``` +localCordMatrix × inverse(latLongMatrix) +``` +where each matrix encodes the rotation/scale of its respective space. + +#### `updatePhonePosition()` +Converts the current GPS position to map coordinates: +``` +phoneCoords = transformMatrix × (phoneLatLong − _refLatLong) + _refLocal +``` +Stores the result in `_phoneCord` and notifies listeners. Also re-renders the map by calling `map.processMapFromJson(prevJson, this)` from the GPS stream. + +#### `updateLocation()` +One-shot GPS fetch via `Geolocator.getCurrentPosition()`. Called by the `"send location"` voice command before reporting position to the server. + +--- + +## map.dart + Renders the 1000×1000 mission map image and writes it to disk. - processMapFromJson(jsonIn, state) - Rebuilds the full map from scratch on every call. Input is a JSON map with: - "paths": list of waypoint objects {x, y, straight, lastInPath} - "mines": list of mine objects {x, y, radius} - - Drawing pipeline (using the 'image' dart package): - 1. Creates a blank 1000×1000 RGBA image with a black background. - 2. Iterates over the path list: - - If path[i].lastInPath == 1: skip (end-of-segment marker). - - If path[i].straight == 0: treat path[i], path[i+1], path[i+2] as - start/midpoint/end of a quadratic Bézier arc; calls _drawArc(). - Advances i by 2. - - Otherwise: draws a straight line from path[i] to path[i+1]. - Advances i by 1. - All path lines are green (0, 255, 0), thickness 3. - 3. For each mine, draws a solid red outline circle and a semi-transparent - red filled circle (alpha 80) at its (x, y) with the given radius. - 4. Converts the image to raw RGBA bytes (Uint8List). - 5. Writes bytes to RGBmap.dat.tmp, then atomically renames to RGBmap.dat. - The atomic rename prevents the file watcher in main.dart from reading - a partially written file. - - _drawArc(img, start, control, end, color, steps) - Approximates a quadratic Bézier curve by stepping t from 0→1 in `steps` - increments, computing the on-curve point at each t, and drawing a straight - line segment between consecutive points. - The control point is derived from the on-curve midpoint using the standard - inverse Bézier formula: ctrl = 2*mid − 0.5*start − 0.5*end. - - prevJson - Module-level variable that caches the last JSON received. Used by - state.dart's GPS stream to re-render the map with the latest path data - whenever the phone's position changes. - -================================================================================ - ---- logger.dart --- -In-app logging that feeds the debug UI in main.dart. - - Logger (ChangeNotifier) - Stores log entries as a List (Flutter rich-text widgets). The debug - UI in _HomePageState watches this list directly and rebuilds when it changes. - - Log levels: - i(str) — info: black text, prefix "I " - d(str) — debug: blue italic, prefix "D " - e(str) — error: red bold, prefix "E " - - clear() — empties the list and notifies listeners. - - logger - Module-level singleton (Logger logger = Logger()) imported by all other files - so every class writes to the same log store. - -================================================================================ +### `processMapFromJson(jsonIn, state)` +Rebuilds the full map from scratch on every call. Input is a JSON map with: +- `"paths"` — list of waypoint objects `{x, y, straight, lastInPath}` +- `"mines"` — list of mine objects `{x, y, radius}` + +**Drawing pipeline** (using the `image` dart package): +1. Creates a blank 1000×1000 RGBA image with a black background. +2. Iterates over the path list: + - If `path[i].lastInPath == 1`: skip (end-of-segment marker). + - If `path[i].straight == 0`: treat `path[i]`, `path[i+1]`, `path[i+2]` as start/midpoint/end of a quadratic Bézier arc; calls `_drawArc()`. Advances `i` by 2. + - Otherwise: draws a straight line from `path[i]` to `path[i+1]`. Advances `i` by 1. + - All path lines are green `(0, 255, 0)`, thickness 3. +3. For each mine, draws a solid red outline circle and a semi-transparent red filled circle (alpha 80) at its `(x, y)` with the given radius. +4. Converts the image to raw RGBA bytes (`Uint8List`). +5. Writes bytes to `RGBmap.dat.tmp`, then atomically renames to `RGBmap.dat`. The atomic rename prevents the file watcher in `main.dart` from reading a partially written file. + +### `_drawArc(img, start, control, end, color, steps)` +Approximates a quadratic Bézier curve by stepping `t` from 0→1 in `steps` increments, computing the on-curve point at each `t`, and drawing a straight line segment between consecutive points. The control point is derived from the on-curve midpoint using the standard inverse Bézier formula: +``` +ctrl = 2*mid − 0.5*start − 0.5*end +``` + +### `prevJson` +Module-level variable that caches the last JSON received. Used by `state.dart`'s GPS stream to re-render the map with the latest path data whenever the phone's position changes. + +--- + +## logger.dart + +In-app logging that feeds the debug UI in `main.dart`. + +### `Logger` (ChangeNotifier) +Stores log entries as a `List` (Flutter rich-text widgets). The debug UI in `_HomePageState` watches this list directly and rebuilds when it changes. + +| Method | Level | Style | Prefix | +|---|---|---|---| +| `i(str)` | info | black | `I ` | +| `d(str)` | debug | blue italic | `D ` | +| `e(str)` | error | red bold | `E ` | + +`clear()` empties the list and notifies listeners. + +### `logger` +Module-level singleton (`Logger logger = Logger()`) imported by all other files so every class writes to the same log store. diff --git a/subteams/app/pre-2020/app.md b/subteams/app/pre-2025/app.md similarity index 100% rename from subteams/app/pre-2020/app.md rename to subteams/app/pre-2025/app.md diff --git a/subteams/app/pre-2020/compakt_info/cloudflare.md b/subteams/app/pre-2025/compakt_info/cloudflare.md similarity index 100% rename from subteams/app/pre-2020/compakt_info/cloudflare.md rename to subteams/app/pre-2025/compakt_info/cloudflare.md From d58c29671a05de731ac841a957b11a0271e3f734 Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:30:40 -0500 Subject: [PATCH 3/8] Added setup docs --- subteams/app/2025-26/setup.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 subteams/app/2025-26/setup.md diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md new file mode 100644 index 0000000..78e58c0 --- /dev/null +++ b/subteams/app/2025-26/setup.md @@ -0,0 +1,26 @@ +# Setup + +## Environment + +- [Install Flutter](https://docs.flutter.dev/install/custom) +- [Add Flutter to PATH](https://docs.flutter.dev/install/add-to-path) + +## Pushing Code to Phone + +Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). + +1. Pair your device: + ``` + adb pair : + ``` + > **Note:** This is not the IP/port shown on the wireless debugging screen. Tap "Pair with code" to get a separate pairing port. + +2. Connect to the device: + ``` + adb connect : + ``` + +3. Install the APK: + ``` + adb install /path/to/apk + ``` From f1e65d9245fa3846e0170b743376cf12c5b4cde3 Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:32:34 -0500 Subject: [PATCH 4/8] Added note --- subteams/app/2025-26/setup.md | 1 + 1 file changed, 1 insertion(+) diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md index 78e58c0..55dc0b3 100644 --- a/subteams/app/2025-26/setup.md +++ b/subteams/app/2025-26/setup.md @@ -24,3 +24,4 @@ Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). ``` adb install /path/to/apk ``` + > **Note:** on my computer the path was build/app/outputs/flutter-apk/app-release.apk From 57e451a112b4bd110912987a7057b4b47db1e70b Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:37:35 -0500 Subject: [PATCH 5/8] Changes --- subteams/app/2025-26/setup.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md index 55dc0b3..d075a6a 100644 --- a/subteams/app/2025-26/setup.md +++ b/subteams/app/2025-26/setup.md @@ -5,6 +5,7 @@ - [Install Flutter](https://docs.flutter.dev/install/custom) - [Add Flutter to PATH](https://docs.flutter.dev/install/add-to-path) + ## Pushing Code to Phone Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). @@ -25,3 +26,19 @@ Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). adb install /path/to/apk ``` > **Note:** on my computer the path was build/app/outputs/flutter-apk/app-release.apk + + +##Running in Debug Mode + +1. Pair and connect normally + +2. Run +``` +flutter devices +``` + To see currently connected devices +3. Run +``` +flutter run +``` + (Should simply be flutter run android) From bfaaeb26858ce4b4742e80f7860d7681f1c40094 Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:38:06 -0500 Subject: [PATCH 6/8] Typo --- subteams/app/2025-26/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md index d075a6a..7647d1a 100644 --- a/subteams/app/2025-26/setup.md +++ b/subteams/app/2025-26/setup.md @@ -28,7 +28,7 @@ Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). > **Note:** on my computer the path was build/app/outputs/flutter-apk/app-release.apk -##Running in Debug Mode +## Running in Debug Mode 1. Pair and connect normally From 7177f6c95a1819e15afdf3c89c492dc8a06fc83d Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:39:11 -0500 Subject: [PATCH 7/8] Typo --- subteams/app/2025-26/setup.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md index 7647d1a..c19811a 100644 --- a/subteams/app/2025-26/setup.md +++ b/subteams/app/2025-26/setup.md @@ -36,9 +36,12 @@ Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). ``` flutter devices ``` + To see currently connected devices + 3. Run ``` flutter run ``` + (Should simply be flutter run android) From bd2f8af3d08f6e215848da95aa892cc4f1a925ff Mon Sep 17 00:00:00 2001 From: Nolan Date: Mon, 4 May 2026 19:41:31 -0500 Subject: [PATCH 8/8] Typo --- subteams/app/2025-26/setup.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/subteams/app/2025-26/setup.md b/subteams/app/2025-26/setup.md index c19811a..5aa5725 100644 --- a/subteams/app/2025-26/setup.md +++ b/subteams/app/2025-26/setup.md @@ -37,11 +37,11 @@ Uses [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb). flutter devices ``` - To see currently connected devices +To see currently connected devices 3. Run ``` flutter run ``` - (Should simply be flutter run android) +(Should simply be flutter run android)