[Cisco Nexus] Improve timezone and timestamp handling #14504
Conversation
P1llus
added
enhancement
…g even more formats on other date processors
|
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices) |
🚀 Benchmarks reportTo see the full report comment with |
|
💚 Build Succeeded
History
|
andrewkroh
added
the
documentation
There was a problem hiding this comment.
This looks pretty good, but I had a couple questions.
@taylor-swanson can you take a look at this as well, please?
| changes: | ||
| - description: Fix bug that did not recognize timestamps to use tz_map override. | ||
| type: bugfix | ||
| link: https://github.com/elastic/integrations/pull/14458 |
There was a problem hiding this comment.
PR link needs to be updated
| @@ -3,126 +3,101 @@ description: Pipeline for processing Cisco Nexus logs. | |||
| processors: | |||
| - set: | |||
| field: ecs.version | |||
| value: '8.17.0' | |||
| tag: 'set_ecs_version' | |||
| value: "8.17.0" | |||
There was a problem hiding this comment.
Curious why you're updating the single quotes to double quotes? I do prefer the doubles, but curious if there's some yaml-specific reason to do this.
| }, | ||
| "cisco_nexus": { | ||
| "log": { | ||
| "description": "EARL 0 NF ASIC: Uncorrectable Parity error in Netflow Table.", | ||
| "facility": "EARL", |
There was a problem hiding this comment.
Is there a reason why all of these fields are being deleted? Is this part of the processing changes in default.yml, or something else?
Proposed commit message
The PR introduces tz_map as an option in the configuration of the integration, similar to tz_map this is specifically for usecases where the vendor provides a timezone format in their logs which is not supported by Java, and is the only way for a user to map these unsupported formats to a proper IANA format on ingest time.
While implementing this there was also several unecessary additions between syslog and vendor timestamps, so I changed that up a bit and updated the system and pipeline tests. Older documents now get missing ECS fields and the dates are parsed correctly also on older logs (which is why older pipeline test output have changes in them).
Checklist
changelog.ymlfile.