Additional Plugins Documentation

[jokob-sk]

Hi @adamoutler ,

Moving the discussion here as Discord has message length limits.
No one ever asked more than what's in the docs, but I'll try to answer your questions best I can.

Is there any kind of official documentation on plug-in development? I have seen the plug-in page. But it's not exactly developer documentation. It is more of an overview, and then a video. I'm looking for some more referential information. Like a definitive guide.

Q1: Output format in plain text with a description of the limits of each field, and how it links to other data. As an example, watched values versus unknown watched values versus extra data.

• A1: The interface contract between a plugin and the main application is described here: https://jokob-sk.github.io/NetAlertX/PLUGINS_DEV/#column-order-and-values-plugins-interface-contract Most of the fields can contain any data type and no strict limits or validation is applied. The important part is the target column to which these fields are mapped. That then specifies the actual target data format. These mappings are in the specified via the config.json manifest file via the mapped_to_table and database_column_definitions definitions. The actual fields watched, extra, helper - can be used for whatever data you need to bring into the main application. The only thing functionality-wise is that the watched column changes can trigger notifications on a plugin level. The helper fields were added when I relaized that I needed to pass more than just 7 values (main/secondary ID + watched+ extra) from a plugin to the main app.

Q2: Logging apis - for some reason it appears that certain plugins log differently than others. As an example, the dig scanner provides output directly to the log, whereas plug-in provides information to a single file which is then catted out to the log.

• A2: I know the logging apis are not ideal and they are on my long laundry list of things to improve. I liked how one of the contributors wrapped mylog into a class in the OMADAOPENAPI plugin https://github.com/jokob-sk/NetAlertX/blob/main/front/plugins/omada_sdn_openapi/script.py . As with everything, I started with an idea and slowly discovered that the mylog() function isn't idea without a logger object ads different plugins might have different execution context, so I had issues where plugins were not respecting the LOG_LEVEL setting. Ideally, all logs should go thru the logging api so logs are not lost.

Q3: It seems like there should be some plug-in master class that would just allow a log method based on information that python can determine about the class.

• A3: I think I partially answered this above - open to a refactor, it's just labor intensive

Q4: Is there some way to format data into tables for the extra data output of a plugin?

• A4: There is not a table formatter. I'm also currently delaying any significant front-end changes as a contributor is working on a complete front end rewrite which can take a couple of months, so mainly focusing on backend changes and improvements. I'm open to considering it in the future (there is already a DataTable setting formatter/form control, which might be reused), depending on the specific use cases.

Q5: Inputs and outputs per plug-in type. Documentation is kind of broad and not really specific. I find myself looking at other plugins continuously to figure out how can I make something somewhat sort of similar. It would be nice to say a data gathering or name update plug-in does this and outputs this. A tool page plug-in does this and outputs this.

• A5: In short adding a name resolver needs currently changes in the core app, it's not fully plugin only driven (again something to improve on). Then there are device discovery plugins which can supply any device field with data (at least IP + MAC at a minimum), device-specific plugins (data is displayed on the device itself mapped via a field containing the device mac and the data_filters in the config.json), then generic plugins showing data only in Integrations -> Plugins, and lastly Plugins not showing any data and only used for configuration purposes (MAINT, CSVBCKP).

Q6: Information about how the config.json is processed.

• A6: Attaching an AI assisted overview below.

Maybe you could sit down with an ai and ask it for some comprehensive developer documentation. There are some really great tools like deep research, canvas, and you can provide it information such as code snippets and ask it to describe it. And at the end you can take all the mixed up mumbo jumbo notes and what not. Drop it into a canvas and ask for a comprehensive developer documentation in markdown format, in a code box, that you can just edit and then drop into GitHub.

Happy to do that - what do you recommend from the above?

[jokob-sk]

🔧 How NetAlertX Uses config.json

Each plugin in NetAlertX must include a config.json. This file is the manifest that tells the core application how to register, configure, and execute your plugin.

Here’s the full lifecycle of how it is processed:

---

1. Manifest Discovery

• On startup, NetAlertX scans /front/plugins/ for subfolders.
• Each folder is expected to contain a config.json.
• If missing or invalid, the plugin is ignored and a warning is logged in app.log.

---

2. Validation

• The following fields are mandatory:

  • code_name → matches the plugin folder name.
  • unique_prefix → must be globally unique across all plugins.
  • data_source → defines how the plugin collects data.

• Invalid configs are skipped gracefully.

---

3. Settings Registration

• The settings array defines plugin-specific options (e.g., timeouts, IP ranges, API tokens).
• These are registered into app.conf on first load.
• Important: once a setting is saved to app.conf, it overrides the value in config.json. Removing or changing defaults in config.json will not affect saved values unless you manually delete them from app.conf.

---

4. Data Source Binding

The data_source field tells NetAlertX how the plugin runs:

• script → run a shell command (CMD) and parse the results.
• app-db-query → run SQL directly on app.db.
• sqlite-db-query → run SQL on an external SQLite DB mounted at DB_PATH.
• template → generate internal settings, without running external code.

---

5. UI Generation

• The database_column_definitions section defines which plugin columns appear in the UI and how they map to the database.
• params allow placeholders (like {urls}, {macs}) to be dynamically replaced at runtime using plugin settings or SQL queries.
• filters allow users to filter plugin output by custom inputs (e.g. by MAC address) in the UI.

---

6. Runtime Execution

• On scan, NetAlertX executes the plugin according to data_source and CMD.
• Results must conform to the plugin object contract (pipe-delimited values).
• If mapped_to_table is defined, results are transformed and stored in the specified table using the mapped_to_column rules.

---

7. Post-Processing

• Duplicate results are de-duplicated in the Plugins_Objects table by the DBCLNP plugin.
• Watch values (Watched_Value1–4) are compared against previous runs to trigger alerts.
• Finalized results are displayed in the UI according to config.json schema.

---

✅ In short:

• config.json defines how your plugin is installed, configured, executed, and displayed.
• It is read at startup, normalized, and registered into the system.
• Once saved, settings are stored in app.conf and override defaults in config.json.

[adamoutler]

Thank you.
So what I'm understanding is watchedValues are always there and may or may not be populated with data. The helpers are essentially non-watchedValues which will add additional information into the main plugin table.

What are the main page fields and how are they updated? Eg let's say I saw my device on a specific location and need to update that, or maybe I'm able to determine a device has a static IP from DHCP reservation. Is that a manual table update for devices?

What about "device scanners" versus "other scanners"?

One of the big problems I have is trying to format this attached treasure trove of data. I'd like to have a table of | service | times contacted| another one for ports and categories.
host-192.168.1.22.json

I found that if a config is missing, the backend actually crashes.

[jokob-sk]

Hi @adamoutler ,

watchedValues are always there and may or may not be populated with data.

Yes, it depends on the plugin and mappings in config.json if those values are needed.

The helpers are essentially non-watchedValues which will add additional information into the main plugin table

Yes, main plugin table and potentially mapped to the CurrentScan table to populate device fields

What are the main page fields and how are they updated? Eg let's say I saw my device on a specific location and need to update that, or maybe I'm able to determine a device has a static IP from DHCP reservation. Is that a manual table update for devices?

Good question - the only mandatory fields are Primary and Secondary ID for plugins and MAC and IP for Devices (as far as I remember). There is no specific hierarchy of priorities of what data overwrites which fields, however currently there is no 1:1 mapping between CurrentScan and Devices tables, which means there are some fields on the Devices table, which can't be passed via the CurrentScan flow (happy to extend it if needed, rather than circumventing it and writing directly to the DB). Back to which data takes priority - in general user defined (first time discovered) values are kept and not overwritten. Placeholder values (e.g. (unknown)) are trying to be replaced by the app until at least some value is available. This logic lives in this file: https://github.com/jokob-sk/NetAlertX/blob/main/server/scan/device_handling.py -(again not too happy with how this works, as it would be nice for this to be user defined, but covers currently most common use cases).

What about "device scanners" versus "other scanners"?

Just a classification - if a scanner doesn't fit into the device scanners category, I add it to the "other scanners".

One of the big problems I have is trying to format this attached treasure trove of data.

Hummm, maybe sketch out a table or a visualization on how you'd imagine this to be represented and I can think about if it can be represented with the current UI system or if we need a custom form control for it. There might be the NMAP plugin (not NMAPDEV) which might give some pointers, if relevant.

[adamoutler]

Here's some information I'd like to see in Extra Values.
Extra Values

Contacted Ports

Type	port	Service
udp	67	DHCP
udp	123	NTP
tcp	443	TLS.GMail
udp	443	QUIC.Google
tcp	5228	TLS.GoogleServices
tcp	80	HTTP.Google

NDPI Categories

Service	Breed	Duration	Bytes
GoogleServices	Acceptable	1320	86,339,285
TLS	Safe	1315	9,103,511
Google	Acceptable	1205	7,879,047,794
DataSaver	Fun	420	999,216,445
Unknown	Unrated	400	32,525
GMail	Acceptable	280	28,290,277
QUIC	Acceptable	185	268,388,931
MDNS	Acceptable	40	8,695
DHCP	Acceptable	15	2,082
ICMP	Acceptable	15	8,112
YouTube	Fun	15	46,329
Microsoft	Safe	10	88,834
Microsoft365	Acceptable	10	12,816
Telegram	Acceptable	10	623,596
ADS_Analytic_Track	Tracker/Ads	5	6,349
NTP	Acceptable	5	180

[adamoutler]

I pretty much get that my contract will be

 plugin_objects.add_object(
      primaryId=mac,
      secondaryId=ip,
      watched1=wv1_os,
      watched2=wv2_name,
      watched3=wv3_alerted_flows,
      watched4=wv4_contacts,
      extra=extra_data,
      foreignKey=foreign_key
)

I just have a lot of data to stick in the extras.
I'm also having problems getting data into the history/events after running. I've got a lot of information I'm working with, but at the end nothing seems to want to update. I think I'm delivering it properly.

[jokob-sk]

Hummm... you can still use and display helper values:

 plugin_objects.add_object(
      primaryId=mac,
      secondaryId=ip,
      watched1=wv1_os,
      watched2=wv2_name,
      watched3=wv3_alerted_flows,
      watched4=wv4_contacts,
      extra=extra_data,
      foreignKey=foreign_key,
      helpVal1="something",
      helpVal2="something"
      helpVal3="something",
      helpVal4="something",

)

These helper values can be displayed as well. I would start with the values that are the most important ones and then we can have a look how to display the rest in the e.g. extra field.

You could also thing of it as a collection of attributes - if this is device specific, e.g.:

Attribute Name	Attribute Type	Attribute Value	Additional Info	Other/Transfer
udp	port	67	DHCP	N/A
NDPI	Acceptable	GoogleServices	1320	86,339,285

This is only a rough example (column name/description TBD) on how we could join the data with the current UI system. Just an idea....