Not all integrations/datasets set correct hostname

[PhilKet]

Apparently not all integrations/datasets set the host.hostname field correctly (e.g. Custom logs, system.auth). The system.syslog dataset does it correctly.

Current behaviour:
The Custom Log integration and system.auth dataset sets the FQDN in host.hostname field

host.hostname: host1.example.com
host.name: host1.example.com

Expected behaviour:
Set the correct hostname (e.g. like the system.syslog dataset does) in host.hostname field

host.hostname: host1
host.name: host1.example.com

Setup:
Kibana version: 8.13.4
Elasticsearch version: 8.13.4
Elastic Agent version: 8.13.4
Custom log integration version: 2.3.1
System integration version: 1.58.2

[jvalente-salemstate]

@PhilKet Do you have any other examples where it's not working as expected?

The two listed here, make sense for why

These are populated by elastic agent and are usually the same value when that happens

but system.syslog overwrites host.hostname with what's in the log file

%{SYSLOGTIMESTAMP:system.syslog.timestamp} %{SYSLOGHOST:host.hostname} %{DATA:process.name}(?:\[%{POSINT:process.pid:long}\])?: %{GREEDYMULTILINE:system.syslog.message}

It does not work in system.auth because there is some issue with the grok pattern for some folks, see #9565

The custom logs have no default pipeline at all so it doesn't get overwritten.

[PhilKet]

Okay, I see. So it probably is the same for all integrations apart from the system integration and the problem lies on the elastic agent side, not the integration itself then?
That would match with my experience so far. Sorry, I am not an expert with the elastic agent and I had the impression that the agent just sets the agent.* fields and the host.* fields are set by the integrations somehow, hence the different values from different integrations. It is in my opinion just plainly wrong to set host1.example.com as hostname, because that is the FQDN of the host. The hostname could easily be abriviated from that though.

As this isn't an issue with any integration, I guess this can be closed.

[jvalente-salemstate]

I guess technically it's filebeat (standalone or spawned by elastic agent) that sets those two host fields, and not agent directly. The value there is set to whatever the hostname (there's some flag I think, but can't remember which) command returns.

You can disable that by adding publisher_pipeline.disable_host: true to the processors. If it's a forwarded event, tagged as such, this is also disabled.

Alternatively you can use a custom pipeline to add or remove the domain part, or change the host name on the host itself.

[botelastic]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!