This file is automatically maintained by the reflexion system. When the AI discovers issues, workarounds, or corrections while building applications, they are logged here.
All v2 endpoint URLs MUST end with /. Without it, the server redirects (301),
causing two API calls to be counted against your rate limit.
Unlike v2, the v3 beta endpoints (/api/v3/beta/...) return 404 Resource not found if the URL ends with a trailing slash. Omit the slash entirely.
For example, use /api/v3/beta/modem_software_versions not
/api/v3/beta/modem_software_versions/. Non-beta v3 endpoints (e.g.
/api/v3/subscriptions/) still accept trailing slashes.
The configuration_managers endpoint has its own id field that may differ from the
router's id. Always use get_configuration_manager_id(router_id) to look up the
correct config manager ID before making PUT/PATCH calls.
PATCH only adds or updates. To remove config items, you must use PUT with the removals list in the diff.
When using arrays (not objects) in a PATCH body, the entire array is replaced. Use objects with string keys for partial array updates.
When using a UUID _id_ as the key for an array element, you must also include
the _id_ field inside the object body, or you get a validation error.
Password fields always return "*" on GET. You cannot read existing passwords.
Set new passwords by sending cleartext — they are encrypted automatically.
If a device applies a config but can't reach NCM within 15 minutes, it rolls back and suspends sync. Check the "Configuration Rejected" alert for details.
API v3 requires both Content-Type: application/vnd.api+json AND
Accept: application/vnd.api+json. Missing either causes errors.
This endpoint and the routers.overlay_network_binding field were deprecated 09/30/2024.
Remove all references.
This field still appears in responses but returns inaccurate data since 12/31/2023.
NcmClient() returns different class instances based on what keys you provide:
- Only v2 keys →
NcmClientv2 - Only v3 token →
NcmClientv3 - Both →
NcmClientv2v3
The SDK checks CP_BASE_URL for v2 and CP_BASE_URL_V3 for v3 base URLs.
Using the SDK's module-level method delegation (import ncm; ncm.get_routers(...))
auto-initializes a singleton via get_ncm_instance(). If only a v3 token is set
(no v2 API keys), the singleton is a NcmClientv3. Calling a v2-only method like
get_routers() on it will not raise an error — instead, the v2 __get_json
pagination loop sends requests to the v2 base URL without proper auth headers.
The SDK's retry adapter (5 retries, exponential backoff on 408/503/504) causes
the script to appear to hang indefinitely with no output.
Workaround: in v3-only scripts, do NOT use module-level delegation for v2 methods.
Either instantiate NcmClientv3 directly and use v3 equivalents (e.g.
get_asset_endpoints() instead of get_routers()), or ensure all four v2 API
keys are set in the environment.
The SDK's _return_handler method returns error information as strings
(e.g. "ERROR: 400: {...}") for 400, 401, 404, and 500 status codes instead
of raising exceptions. This means try/except blocks around SDK calls will
NOT catch API errors. Callers must inspect the return value to detect failures.
In v2's __get_json, non-2xx responses silently break out of the pagination
loop and return partial (possibly empty) results with no error indication at all.
In v3's __get_json, the error string is returned directly, so code expecting
a list will receive a string instead.
Workaround: check return values explicitly. For v3 methods, check if the result
is a string starting with "ERROR:" or is not a list. For v2 GET methods,
an unexpectedly empty list may indicate a silent error.
The regrade() method sends an atomic:operations payload but does not set
the required JSON:API atomic extension Content-Type header. Compare with
unlicense_devices(), which correctly sets:
Content-Type: application/vnd.api+json;ext="https://jsonapi.org/ext/atomic"
Accept: application/vnd.api+json;ext="https://jsonapi.org/ext/atomic"
The missing header can cause the API to reject the request with a 400 error
(e.g. "mac_address must be specified") even when the field is present in
the payload, because the server doesn't parse the atomic operations format
without the extension header.
The regrade() method normalizes MAC addresses by stripping colons only when
the input is exactly 17 characters (len(smac) == 17). This misses:
- Dash-separated MACs (
00-30-44-1A-2B-3C, 17 chars) — colons are stripped (finding none), dashes pass through - Dot-separated Cisco format (
0030.441A.2B3C, 14 chars) — passes through with dots intact - Lowercase MACs — passed through as-is (API may expect uppercase)
Workaround: always normalize MACs to bare uppercase hex before passing to
regrade(): mac.upper().replace(':', '').replace('-', '').replace('.', '')
The POST /asset_endpoints/regrades atomic operations endpoint rejects the
entire batch with a 400 Bad Request if the same mac_address value appears
more than once across operations in a single request. The error message is:
"mac_address values must only occur once". This applies per-request, not
globally — the same MAC can appear in separate requests. Always deduplicate
MAC addresses within each batch before sending.
The POST /asset_endpoints/regrades endpoint strictly validates that
mac_address is exactly 12 hexadecimal characters (uppercase or lowercase,
no separators). Any other format — including MACs with colons, dashes, dots,
or fewer/more than 12 characters — returns 400 Bad Request with:
"mac_address must be 12 digit hexadecimal with optional colons". Despite
the error message mentioning "optional colons", bare 12-digit hex is the
safest format. Validate with ^[0-9A-Fa-f]{12}$ before sending.
The NCM SDK and the scripts/utils/session.py utility use different key naming
conventions for credentials. Mixing them up causes silent auth failures.
- SDK (
NcmClient):{'X-CP-API-ID': '...', 'X-CP-API-KEY': '...', 'X-ECM-API-ID': '...', 'X-ECM-API-KEY': '...'} - Session utility (
APISession):cp_api_id='...', cp_api_key='...', ecm_api_id='...', ecm_api_key='...' credentials.py: uses snake_case keys matching the session utility, NOT the SDK
If using get_credentials() with the SDK, you must remap the keys to HTTP header names.
The standard env var name for the v3 bearer token in this project is NCM_API_TOKEN.
All scripts have been refactored to use this name. Historical names that may
appear in older code or external references:
TOKEN— legacy name (removed from scripts)V3_BEARER_TOKEN— Inventory Dashboard legacy (migrated to NCM_API_TOKEN)CP_API_TOKEN— never used by any actual script
When building new scripts/dashboards that need v3, use NCM_API_TOKEN.
The NCM API limits __in filter parameters to 100 comma-separated values per request.
The SDK auto-chunks these transparently, but if you're making raw API calls with
requests or the session utility, you must chunk manually or the API will error.
If the repo is cloned or moved to a different path than where the .venv was
originally created, the pip (and other scripts in .venv/bin/) will have a
stale shebang pointing to the old Python path. Running .venv/bin/pip install ...
fails with bad interpreter: no such file or directory.
Workaround: use .venv/bin/python -m pip install ... instead, or recreate the venv.
The requirements.txt lists flask but it may not be installed in the venv if
dependencies were never fully installed. Always run
.venv/bin/python -m pip install -r requirements.txt after cloning or setting up
the project for the first time.
Running python3 -m venv .venv --clear to fix a broken venv also deletes the
API credentials that setup_env.py injected into .venv/bin/activate. After
recreating the venv, you must re-run setup_env.py and then source .venv/bin/activate
to restore the environment variables. This affects all scripts that rely on check_env().
The NCM API v2 supports ?fields= on most endpoints for partial responses, but
the Python SDK only allows fields on methods where it's explicitly in the
allowed_params list. Before using fields with any SDK method, check its
allowed_params in the source.
Methods known to support fields: get_routers(), get_groups() (added 2026-03-31).
Methods known to NOT support fields: get_products(), get_firmwares().
When fetching routers without the fields parameter, relational fields like group,
product, actual_firmware, and target_firmware return full API URLs
(e.g. https://www.cradlepointecm.com/api/v2/groups/12345/), not names or IDs.
To get human-readable values, either use expand=group on the request, or build
a lookup dict from a separate get_groups() call. The full_product_name field
does return the model name directly (e.g. "AER1600").
On large accounts the /groups/ endpoint can return thousands of groups (4000+),
requiring many paginated requests that take minutes to complete. Using
expand=group on the /routers/ call is dramatically faster because the API
inlines the group object (with name, id, etc.) directly into each router
response, eliminating the separate fetch entirely. The same applies to
expand=account. Always prefer expand over a separate lookup fetch when you
only need the related resource's name or ID.
The v3 /subscriptions/ endpoint returns attributes.start_time and
attributes.end_time but no explicit status field. To determine if a
subscription is active or expired, compare end_time against the current UTC time.
The subscription type is in relationships.subscription_type.data.id.
When using the NCM SDK's get_routers(fields='...') inside a Flask application,
it returns an empty list, even though the same call works correctly in a standalone
Python script. The root cause is unclear but may relate to how Flask's request
context interacts with the SDK's session/urllib. Workaround: omit the fields
parameter and fetch full objects instead.
The v2 /routers/ endpoint returns MAC addresses in the mac field using
colon-separated format (e.g. 00:30:44:1A:2B:3C), while the v3
/asset_endpoints endpoint returns them in the mac_address field using
bare uppercase hex (e.g. 0030441A2B3C). When joining data across v2 and v3,
you must normalize both to the same format (e.g. strip colons/dashes and
upper-case) before matching.
Several v3 endpoints are served under /api/v3/beta/ rather than /api/v3/.
Requests without the /beta/ prefix may 404. Known beta endpoints include
exchange_sites, modem_software_versions, and modem_upgrades. Because
these are beta endpoints, their contracts may change without notice. Always
check the OpenAPI spec or release notes for the correct path prefix.
The v2 /net_devices/ endpoint supports an is_asset boolean
filter. Setting is_asset=true returns only the primary modem interfaces
(the physical cellular modems), filtering out virtual/logical interfaces.
This is useful when you only need modem-level details (IMEI, ICCID, carrier)
and want to avoid processing hundreds of non-modem net_device records.
The v2 /net_devices/ endpoint supports expand=router and expand=account.
When using expand=router, the router field becomes an inline object with
the full router record (id, name, state, mac, group URL, etc.) instead of a
URL. This eliminates the need for a separate /routers/ fetch when building
per-modem views. Note: if the net_device has no associated router, the field
remains null even with expand.
The /net_device_health/ endpoint returns health records for net_devices of
ALL modes (wan, lan, mdm), not just WAN or modem interfaces. When joining
health data to net_devices, do NOT filter by mode='wan' or you will silently
miss most records. Fetch all net_devices (no mode filter) and join by
net_device ID to get complete results.
The /net_device_health/ endpoint returns only:
cellular_health_category— string: "poor", "fair", "good", or "excellent"cellular_health_score— integer: 0–100net_device— URL reference to the net_deviceid,resource_url
It does NOT contain signal metrics (RSSI, RSRP, RSRQ, SINR, etc.). For actual
signal data, use /net_device_metrics/ filtered by the same net_device IDs.
The typical pattern for a cellular health dashboard is:
GET /net_device_health/→ health scores + net_device URLsGET /net_device_metrics/?net_device__in=id1,id2,...→ signal metricsGET /net_devices/?id__in=id1,id2,...→ carrier, model, router refGET /routers/?expand=group→ device names, groups, state
The /net_devices/ endpoint does NOT return signal strength fields (rssi,
rsrp, rsrq, sinr, signal_percent). These fields only exist on the
/net_device_metrics/ endpoint. The /net_devices/ response includes modem
hardware info: carrier, model, mfg_model, mfg_product, connection_state,
imei, iccid, apn, modem_fw, etc.
The router field on /net_devices/ can be null for orphaned or
unassigned modems. Code that assumes router is always a URL will fail.
Always check for null before extracting the router ID. For display purposes,
fall back to hostname or name from the net_device record.
Despite being numeric, v2 API responses return id fields as strings
(e.g. "67468693" not 67468693). When building lookup dictionaries for
joining across endpoints, always normalize IDs to strings. Using int() for
lookups will cause key mismatches if the dict was keyed with the raw string
from the API response.
The /net_device_metrics/ endpoint returns:
- Signal:
rsrp,rsrq,rssi,rssnr,sinr,dbm,cinr,ecio,signal_strength(integer 0–100, NOTsignal_percent) - Cell info:
cell_id,mcc,mnc,tac,lac,service_type(e.g. "LTE", "WiFi", "Ethernet", "Not Available") - Usage:
bytes_in,bytes_out - Meta:
update_ts,bmask_applied,net_device(URL),id,resource_url
Note: id matches the net_device ID (same value), making joins trivial.
Non-cellular interfaces (WiFi, Ethernet) have null signal fields.
When using expand=router on /net_devices/, the inline router object's
relational fields (like group, account, product, actual_firmware) are
still URLs, not expanded objects. Only one level of expansion is supported.
To resolve group names when using this pattern, you must still call
/groups/ separately and build a lookup by URL or ID.
When making concurrent requests to v3 endpoints (e.g. /asset_endpoints),
the API may return 409 Conflict with the message "Conflict with internal
rules. i.e. invalid app-key." instead of the expected 429 Too Many Requests.
This is effectively a rate limit response. Treat 409 as retryable with
exponential backoff, the same as 429. The NCM API enforces a limit of
approximately 500 calls per minute across all endpoints.
During heavy pagination of v2 endpoints (especially /net_devices/ and
/routers/ on large accounts), the API may return 500 Internal Server Error
with a body like "cp internal error shard-router-3". This is a transient
server-side error, not a client issue. Retry with backoff. Include 500, 502,
503, and 504 in your retryable status code set.
When using httpx (or requests) for pagination, passing params={} (empty
dict) to client.get(url, params={}) strips any existing query parameters
from the URL. If the meta.next URL is
https://...?limit=500&offset=500, passing params={} turns it into
https://.../routers/ — causing an infinite loop that re-fetches page 1
forever. Use params=None instead of params={} on subsequent pagination
requests where the query parameters are already embedded in the next URL.
The v2 and v3 APIs are architecturally independent interfaces to the same underlying data. They do NOT cross-reference each other:
- Auth: v2 uses 4 API key headers; v3 uses a single Bearer token.
- IDs: v2 uses integers; v3 uses strings. They are different ID spaces.
- Spec: v3 follows JSON:API; v2 does not.
- Relationships: v2 embeds full URLs to other v2 resources. v3 uses
JSON:API
relationshipsblocks with type/id references to other v3 resources only. - Expand:
?expand=is v2-only. v3 uses JSON:API?include=(where supported) or you follow the relationship type/id to the v3 endpoint.
The key implication: v3 relationship references can only be resolved through v3 endpoints. If a related resource doesn't have a v3 endpoint yet, that relationship either won't appear in the v3 response or the data will be flattened into attributes. You cannot use a v2 endpoint to resolve a v3 relationship reference (or vice versa). When migrating scripts from v2 to v3, all related resource lookups must also be migrated to their v3 equivalents.
The v2 /routers/ endpoint assigns numeric IDs (e.g. 1234567) to devices.
The v3 /asset_endpoints endpoint has its own id field that does NOT
correspond to the v2 router ID. Passing a v2 router ID to
get_asset_endpoints(id=...) (which becomes filter[id]=1234567) will
return the wrong device or no results. To cross-reference between v2 and v3,
use a shared natural key like mac_address or serial_number instead of
either system's internal ID.
The subscription_ids on v3 /asset_endpoints are per-device assignment IDs
(e.g. 4c03Hj1hALlWWvK), NOT the parent subscription IDs returned by an
unfiltered GET /subscriptions (e.g. 55050000002xYV9). These two ID spaces
do not overlap. To resolve assignment IDs to full subscription details (name,
start/end dates), use GET /subscriptions?filter[id]=<assignment_id>.
Multiple IDs can be comma-separated: filter[id]=id1,id2,id3.
Looking up an assignment-level subscription ID by path
(GET /subscriptions/4c03Hj1hALlWWvK) returns 404, even though the ID is
valid. Use the query filter instead: GET /subscriptions?filter[id]=4c03Hj1hALlWWvK.
This is a JSON:API quirk — the filter endpoint resolves IDs that the direct
path lookup does not.
check_env() from utils.env_check always validates v2 API keys
(CP_API_ID, CP_API_KEY, ECM_API_ID, ECM_API_KEY). In v3-only
scripts that don't need v2 keys, it prints a misleading "Missing required
environment variables" message to stderr before calling sys.exit(1).
Workaround: suppress stderr and catch the SystemExit:
import io, sys
from utils.env_check import check_env
try:
_stderr = sys.stderr
sys.stderr = io.StringIO()
check_env()
except SystemExit:
pass
finally:
sys.stderr = _stderrThen validate v3-specific vars (NCM_API_TOKEN, etc.) separately.
The OpenAPI spec for POST /api/v3/beta/modem_upgrades documents a 201 Created response, but the API actually returns 200 OK with the created
resource in the body. Code that checks for status_code == 201 to confirm
creation will incorrectly treat a successful POST as a failure. Check for
both 200 and 201.
The OpenAPI spec says the data.type field in POST/PUT request bodies for
/api/v3/beta/modem_upgrades should be "modem_upgrade_parent". The API
rejects this with a 409 Conflict: "The resource object's type
(modem_upgrade_parent) is not the type that constitute the collection
represented by the endpoint (modem_upgrades)." Use "modem_upgrades" in
the request body instead. Note: the response still returns
"modem_upgrade_parent" as the type — the asymmetry is intentional.
The existing documentation notes that v3 returns 409 Conflict as a rate
limit. However, 409 is also used for JSON:API validation errors (e.g. wrong
data.type). To distinguish: if the 409 response body contains an errors
array, it is a real validation error and should NOT be retried. If it does
not contain errors, treat it as a rate limit and retry with backoff.
Several v2 time-series endpoints return 409 Conflict if called without a
mandatory router or net_device filter. This is NOT a rate limit — it's a
validation error requiring you to scope the query to specific device(s).
| Endpoint | Required Filter |
|---|---|
historical_locations |
router |
net_device_signal_samples |
net_device or net_device__in |
net_device_usage_samples |
net_device or net_device__in |
router_logs |
router |
router_state_samples |
router or router__in |
router_stream_usage_samples |
router or router__in |
These endpoints use TimeUUID-based pagination and are optimized for per-device queries. Always provide the filter or you'll get a 409.
Cradlepoint router configuration .bin files are compressed JSON, but the
compression format varies. Some use standard zlib (wbits=15), others use raw
deflate (no header, wbits=-15), and others use gzip (wbits=31). When decoding
.bin files, try multiple zlib.decompress(data, wbits) values in sequence:
15, -15, 31, 47 (auto-detect). The first one that succeeds without a
zlib.error is the correct format. The decompressed content is always UTF-8
encoded JSON.
When decompressing a Cradlepoint .bin configuration file, the resulting JSON
is an array structured as [{"config": {...}, "state": {...}, "fw_info": {...}}, [[...], ...]].
The first element is an object containing "config" (the actual configuration),
"state" (runtime state data), and "fw_info" (firmware version). The second
element is an array of path references (e.g. ["config", "lan", 0, "devices", 2])
that appear to be deletion/override markers for the config tree.
When working with configs for templating or pushing to devices:
- Reading: Extract only
parsed[0]["config"] - Writing: Wrap config as
[{"config": <config_obj>}, []]then zlib compress
The .bin files use standard zlib compression (magic bytes 78 9c, wbits=15).
Config top-level keys include: certmgmt, container, ecm, identities, lan,
routing, security, stats, system, wan, wlan.
Cradlepoint configuration JSON (extracted from .bin files or via API) contains
string values with raw newline characters (e.g. embedded YAML in container
project configs, PEM certificates). Python's json.loads() in strict mode
rejects these as "Invalid control character". Use json.loads(content, strict=False)
when parsing Cradlepoint config JSON to allow control characters in strings.
The NCM SDK uses requests.Session() with default SSL verification. Behind
corporate proxies that use TLS interception (MITM with custom CA certificates),
all API calls fail with SSLCertVerificationError: CERTIFICATE_VERIFY_FAILED.
The SDK does not expose a verify parameter, so callers must patch the session
directly after constructing the client:
client = ncm.NcmClient(api_keys=api_keys)
client.session.verify = False # Disable SSL verificationTo also suppress the InsecureRequestWarning that urllib3 emits on every
request when verify is disabled:
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)Alternative (proper fix): set the REQUESTS_CA_BUNDLE env var to the path of
the proxy's CA certificate bundle:
export REQUESTS_CA_BUNDLE=/path/to/corporate-ca-bundle.crtThis affects ALL scripts and dashboards using the SDK, not just specific apps.
The /device_apps/ endpoint returns the id field as a string (e.g. "83")
rather than an integer. This differs from most other v2 endpoints (like
/routers/ and /groups/) which return integer IDs. When building lookup
dictionaries keyed by app ID — for example, to match device_app_versions
back to their parent app — use string keys or cast consistently. Attempting
int() conversion on the URL-extracted ID and comparing against the raw
response ID will fail silently (lookup miss) if one side is a string and the
other is an int.
The resource_url and relational URL fields in v2 API responses can use
different base hostnames depending on the account's regional shard (e.g.
https://www.us0.cradlepointecm.com/api/v2/accounts/123/ vs
https://www.cradlepointecm.com/api/v2/accounts/123/). This means matching
a group's account URL against an account's resource_url by exact string
comparison will fail if the hostnames differ. Always extract the numeric ID
from the URL path (the last path segment before the trailing slash) and match
on that instead of comparing full URLs.
The v2 /groups/ endpoint DOES support expand=account. When used, the
account field becomes an inline object with id, name, is_disabled,
resource_uri, and account (parent account URL). This eliminates the need
for a separate /accounts/ fetch to resolve group account names. Known
endpoints that support expand: /routers/ (group, account), /groups/
(account).
The /alerts/ endpoint returns 409 Conflict with "Invalid ordering field specified: created_at" when using order_by=-created_at or any order_by
parameter. The endpoint does not support ordering — results must be sorted
client-side. The endpoint does support created_at__gt for time filtering,
but requires full ISO 8601 format with microseconds and timezone offset
(e.g. 2026-06-26T15:00:37.703000+00:00). The alert type field for
custom alerts is custom_alert (not custom).