diff --git a/rfcs/0000-observable-field-set.md b/rfcs/0000-observable-field-set.md new file mode 100644 index 0000000000..ac968f45fc --- /dev/null +++ b/rfcs/0000-observable-field-set.md @@ -0,0 +1,149 @@ +# 0000: Observable Field Set + + +- Stage: **0 (strawperson)** +- Date: **TBD** + + + + + +This RFC proposes the addition of a new field set named observable to the Elastic Common Schema (ECS). The goal of this field set is to provide a structured way to describe cyber observables that are observed in network and host events and alerts, and cyber threat intelligence. Cyber observables are raw artifacts that can be observed in network traffic, system logs, or other data sources. This field set closely aligns with the [`threat field set`](https://github.com/elastic/ecs/blob/main/rfcs/text/0008-threat-intel.md) but because not every observable is an indicator, or may be designated one +over the course of an investigation, its main goal is to aid in tracking and investigating them. When determining what +fields are required, using the [STIX Cyber Observable data model](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_mlbmudhl16lr) as guidance is recommneded as most security systems provide observables in this structure. + + + + + +## Fields + +The following fields were added as a starting point and to aid in the understanding of the field set. These fields were +closely modeled using the [STIX Cyber Observable data model](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_mlbmudhl16lr) and the existing [`threat.indicator.*`](https://github.com/elastic/ecs/blob/main/schemas/threat.yml#L345) fields. + +```text +observable.description +observable.domain +observable.email.address +observable.first_seen +observable.id +observable.ip +observable.last_seen +observable.modified_at +observable.name +observable.port +observable.provider +observable.reference +observable.spec_version +observable.type +``` + +Like the threat field set, the following field sets make sense for field reuse: + +``` +observable.as.* +observable.file.* +observable.geo.* +observable.registry.* +observable.url.* +observable.x509.* +``` + + + + + +## Usage + + + +## Source data + + + + + + + +## Scope of impact + + + +## Concerns + + + + + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @brett-fitz | author + + + +## References + + + +* [RFC 0008: Cyber Threat Intelligence Fields](https://github.com/elastic/ecs/blob/main/rfcs/text/0008-threat-intel.md) +* [OpenCTI Elastic threat intelligence integration with observable field set](https://docs.elastic.co/integrations/ti_opencti) + * NOTE: This field set can be greatly simplified by using existing ECS common fields as well as modeling the observable field set more closely with the indicator object from the threat field set. +* [STIX 2.1 Cyber observable objects](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_mlbmudhl16lr) + +### RFC Pull Requests + + + +* Stage 0: https://github.com/elastic/ecs/pull/2311 + + diff --git a/rfcs/text/0000/observable.yml b/rfcs/text/0000/observable.yml new file mode 100644 index 0000000000..f8439da517 --- /dev/null +++ b/rfcs/text/0000/observable.yml @@ -0,0 +1,143 @@ +- name: observable + title: Observable + short: Fields describing an observable. + description: >- + Observable fields capture details about a cyber observable. Cyber observables are + raw artifacts that can be observed in network traffic, system logs, or other data sources. + + fields: + + - name: description + level: custom + type: text + short: Description of the observable. + description: >- + A description of the observable. + example: IP x.x.x.x was observed in investigation y. + + - name: domain + level: custom + type: keyword + short: Domain name of the observable. + description: >- + Identifies an observable as a domain name. Only filled in when value is a domain. + example: elastic.co + + - name: email.address + level: custom + type: keyword + short: Email address of the observable. + description: >- + Identifies an observable as an email address. Only filled in when value is an email address. + example: phish@example.com + + - name: first_seen + level: custom + type: date + short: First time the observable was seen. + description: >- + Only filled in when value is a domain. + example: 2019-01-01T00:00:00.000Z + + - name: id + level: custom + type: keyword + short: Unique identifier of the observable. + description: >- + The unique identifier of the observable typically in STIX serialization format. + example: ipv4-addr--cbd67181-b9f8-595b-8bc3-3971e34fa1cc + + - name: ip + level: custom + type: ip + short: IP address of the observable. + description: >- + Identifies an observable as an IP address. Only filled in when value is an IP address. + example: 1.1.1.1 + + - name: last_seen + level: custom + type: date + short: Last time the observable was seen. + description: >- + Only filled in when value is a domain. + example: 2019-01-01T00:00:00.000Z + + - name: modified_at + level: custom + type: date + short: Last time the observable was modified. + description: >- + Only filled in when value is a domain. + example: 2019-01-01T00:00:00.000Z + + - name: name + level: custom + type: keyword + short: observable display name. + description: >- + The display name observable in an UI friendly format + + URL, IP address, email address, registry key, port number, hash value, + or other relevant name can serve as the display name. + example: 5.2.75.227 + + - name: port + level: custom + type: long + short: Port number of the observable. + description: >- + Identifies an observable as a port number (irrespective of direction). + example: 443 + + - name: provider + level: custom + type: keyword + short: Provider of the observable. + description: >- + The name of the observable's provider. + example: sensor + normalize: + - array + + - name: reference + level: custom + type: keyword + short: Reference to the observable. + description: >- + Reference URL linking to additional information about this observable. + example: https://system.example.com/observable/0001234 + + - name: spec_version + level: custom + type: keyword + short: STIX 2.x specification version. + description: >- + STIX 2.x specification version. + example: 2.1 + + - name: type + level: custom + type: keyword + short: observable type. + description: > + Type of observable as represented by Cyber Observable in STIX 2.0. + expected_values: + - autonomous-system + - artifact + - directory + - domain-name + - email-addr + - file + - ipv4-addr + - ipv6-addr + - mac-addr + - mutex + - port + - process + - software + - url + - user-account + - windows-registry-key + - x509-certificate + example: ipv4-addr