Skip to content

Fix up the 'Templates' section #2178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions manage-data/data-store/index-basics.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,7 @@ You’re now ready to create new indices using your index template.
GET /my-index-000001,my-index-000002
```

### Manage component templates
### Manage component templates [index-management-manage-component-templates]

Component templates are a type of [template](/manage-data/data-store/templates.md) used as reusable building blocks within index templates to configure index settings, mappings, and aliases.

Expand All @@ -303,7 +303,7 @@ Create, edit, clone, and delete your component templates in the **Component Temp
* To show details and perform operations, click the template name.
* To create new component templates, use the **Create component template** wizard.

### Manage enrich policies
### Manage enrich policies [manage-enrich-policies]

An [enrich policy](/manage-data/ingest/transform-enrich/data-enrichment.md#enrich-policy) is a set of configuration options used to add data from your existing indices to incoming documents during ingest. An enrich policy contains:

Expand Down
128 changes: 73 additions & 55 deletions manage-data/data-store/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,85 @@ products:
- id: elasticsearch
---

# Templates [index-templates]
# Templates [elasticsearch-templates]

::::{note}
This topic describes the composable index templates introduced in {{es}} 7.8. For information about how index templates worked previously, see the [legacy template documentation](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-template).
::::
Templates are the mechanism by which {{es}} applies settings, mappings, and other configurations when creating indices or data streams.

You configure templates prior to creating indices or data streams. When an index is created, either manually or by indexing a document, the matching template determines the settings, mappings, and other configurations to apply. When used with a [data stream](/manage-data/data-store/data-streams.md), a template also defines how each backing index is configured as it is created.

$$$getting$$$
An index template is a way to tell {{es}} how to configure an index when it is created. For data streams, the index template configures the stream’s backing indices as they are created. Templates are configured **prior to index creation**. When an index is created - either manually or through indexing a document - the template settings are used as a basis for creating the index.
There are two types of template:

There are two types of templates: index templates and [component templates](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-component-template). Component templates are reusable building blocks that configure mappings, settings, and aliases. While you can use component templates to construct index templates, they aren’t directly applied to a set of indices. Index templates can contain a collection of component templates, as well as directly specify settings, mappings, and aliases.
* An [**index template**](#index-templates) is the main configuration object applied when creating an index or data stream. It matches index names using `index_patterns` and resolves conflicts using a `priority` value. An index template can optionally define settings, mappings, and aliases directly, and refer to a list of component templates that provide reusable configuration blocks. It can also indicate whether it should create a data stream or a regular index.

The following conditions apply to index templates:
* A [**component template**](#component-templates) is a reusable building block that defines settings, mappings, and aliases. Component templates are not applied directly; they must be referenced by index templates.

* Composable templates take precedence over legacy templates. If no composable template matches a given index, a legacy template may still match and be applied.
Together, index templates and their referenced component templates form what is known as *composable templates*.

The following conditions apply to using templates:

* Composable index templates take precedence over any [legacy templates](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/indices-templates-v1.html), which were deprecated in {{es}} 7.8. If no composable template matches a given index, a legacy template may still match and be applied.
* If an index is created with explicit settings and also matches an index template, the settings from the [create index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create) request take precedence over settings specified in the index template and its component templates.
* Settings specified in the index template itself take precedence over the settings in its component templates.
* If a new data stream or index matches more than one index template, the index template with the highest priority is used.
* When you create an index template, be careful to avoid [naming pattern collisions](#avoid-index-pattern-collisions) with built-in {{es}} index templates.

:::{tip}
For a detailed exploration and examples of setting up composable templates, refer to the Elastic blog [Index templating in Elasticsearch: How to use composable templates](https://www.elastic.co/search-labs/blog/index-composable-templates).
:::

## Index templates [index-templates]

An **index template** is used to configure an index when it is created. [Mappings](/manage-data/data-store/mapping.md), [settings](elasticsearch://reference/elasticsearch/index-settings/index.md), and [aliases](/manage-data/data-store/aliases.md) specified in the index template are inherited by each created index. These can be specified directly in the index template, and they can be specified in one or more component templates used to construct the index template.

You can create and manage index templates on the **Index management** page in {{kib}} or by using the [index template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-index-template) API.

For the {{kib}} steps and a walk-through example, refer to [Manage index templates](/manage-data/data-store/index-basics.md#index-management-manage-index-templates).

Using the API, the following request creates an index template that is *composed of* the two component templates shown in the [component templates](#component-templates) example.

```console
PUT _index_template/template_1
{
"index_patterns": ["te*", "bar*"],
"template": {
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": true
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
},
"aliases": {
"mydata": { }
}
},
"priority": 501,
"composed_of": ["component_template1", "runtime_component_template"],
"version": 3,
"_meta": {
"description": "my custom"
}
}
```

:::{tip}
The following features can be useful when you're setting up index templates:

* You can test the effect of an index template before putting it into use. Refer to [Simulate multi-component templates](/manage-data/data-store/templates/simulate-multi-component-templates.md) to learn more.
* You can create an index template for a component template that does not yet exist. When doing so, you can use the `ignore_missing_component_templates` configuration option in an index template so that the missing component template is ignored. Refer to [Ignore missing component templates](/manage-data/data-store/templates/ignore-missing-component-templates.md) to learn more.
:::

::::{admonition} Avoid index pattern collisions
:name: avoid-index-pattern-collisions
### Avoid index pattern collisions [avoid-index-pattern-collisions]

{{es}} has built-in index templates, each with a priority of `100`, for the following index patterns:

Expand All @@ -49,15 +107,13 @@ If you use {{fleet}} or {{agent}}, assign your index templates a priority lower
* To avoid naming collisions with built-in and Fleet-managed index templates, avoid using `@` as part of the name of your own index templates.
* Beginning in {{stack}} version 9.1, {{fleet}} uses indices named `fleet-synced-integrations*` for a feature. Avoid using this name to avoid collisions with built-in indices.

::::
## Component templates [component-templates]

A **component template** is a reusable building block that defines mappings, settings, and aliases. Component templates are not applied directly to indices, but referenced by index templates and used when determining the final configuration of an index.

You can create and manage component templates on the **Index management** page in {{kib}} or by using the [component template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-component-template) API. For the {{kib}} steps, refer to [Manage component templates](/manage-data/data-store/index-basics.md#index-management-manage-component-templates).

## Create index template [create-index-templates]

Use the [put index template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-index-template) and [put component template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-component-template) APIs to create and update index templates. You can also [manage index templates](/manage-data/data-store/index-basics.md#index-management) from Stack Management in {{kib}}.

The following requests create two component templates.
Using the API, the following request creates the two component templates used in the previous index template example:

```console
PUT _component_template/component_template1
Expand Down Expand Up @@ -91,41 +147,3 @@ PUT _component_template/runtime_component_template
```

1. This component template adds a [runtime field](mapping/map-runtime-field.md) named `day_of_week` to the mappings when a new index matches the template.


The following request creates an index template that is *composed of* these component templates.

```console
PUT _index_template/template_1
{
"index_patterns": ["te*", "bar*"],
"template": {
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": true
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
},
"aliases": {
"mydata": { }
}
},
"priority": 501,
"composed_of": ["component_template1", "runtime_component_template"],
"version": 3,
"_meta": {
"description": "my custom"
}
}
```
122 changes: 1 addition & 121 deletions manage-data/data-store/templates/index-template-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,124 +12,4 @@ products:

# Manage index templates [manage-index-templates]

Create, edit, clone, and delete your index templates in the **Index Templates** view. Changes made to an index template do not affect existing indices.

:::{image} /manage-data/images/elasticsearch-reference-management-index-templates.png
:alt: Index templates
:screenshot:
:::

In {{serverless-full}}, the default **logs** template uses the logsDB index mode to create a [logs data stream](../data-streams/logs-data-stream.md).

If you don’t have any templates, you can create one using the **Create template** wizard.

### Try it: Create an index template [_try_it_create_an_index_template]

In this tutorial, you’ll create an index template and use it to configure two new indices.

**Step 1. Add a name and index pattern**

1. In the **Index Templates** view, open the **Create template** wizard.

:::{image} /manage-data/images/elasticsearch-reference-management_index_create_wizard.png
:alt: Create wizard
:screenshot:
:::

2. In the **Name** field, enter `my-index-template`.
3. Set **Index pattern** to `my-index-*` so the template matches any index with that index pattern.
4. Leave **Data Stream**, **Priority**, **Version**, and **_meta field** blank or as-is.

**Step 2. Add settings, mappings, and aliases**

1. Add [component templates](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-component-template) to your index template.

Component templates are pre-configured sets of mappings, index settings, and aliases you can reuse across multiple index templates. Badges indicate whether a component template contains mappings (**M**), index settings (**S**), aliases (**A**), or a combination of the three.

Component templates are optional. For this tutorial, do not add any component templates.

:::{image} /manage-data/images/elasticsearch-reference-management_index_component_template.png
:alt: Component templates page
:screenshot:
:::

2. Define index settings. These are optional. For this tutorial, leave this section blank.
3. Define a mapping that contains an [object](elasticsearch://reference/elasticsearch/mapping-reference/object.md) field named `geo` with a child [`geo_point`](elasticsearch://reference/elasticsearch/mapping-reference/geo-point.md) field named `coordinates`:

:::{image} /manage-data/images/elasticsearch-reference-management-index-templates-mappings.png
:alt: Mapped fields page
:screenshot:
:::

Alternatively, you can click the **Load JSON** link and define the mapping as JSON:

```js
{
"properties": {
"geo": {
"properties": {
"coordinates": {
"type": "geo_point"
}
}
}
}
}
```

You can create additional mapping configurations in the **Dynamic templates** and **Advanced options** tabs. For this tutorial, do not create any additional mappings.

4. Define an alias named `my-index`:

```js
{
"my-index": {}
}
```

5. On the review page, check the summary. If everything looks right, click **Create template**.

**Step 3. Create new indices**

You’re now ready to create new indices using your index template.

1. Index the following documents to create two indices: `my-index-000001` and `my-index-000002`.

```console
POST /my-index-000001/_doc
{
"@timestamp": "2019-05-18T15:57:27.541Z",
"ip": "225.44.217.191",
"extension": "jpg",
"response": "200",
"geo": {
"coordinates": {
"lat": 38.53146222,
"lon": -121.7864906
}
},
"url": "<example-url-01>"
}

POST /my-index-000002/_doc
{
"@timestamp": "2019-05-20T03:44:20.844Z",
"ip": "198.247.165.49",
"extension": "php",
"response": "200",
"geo": {
"coordinates": {
"lat": 37.13189556,
"lon": -76.4929875
}
},
"memory": 241720,
"url": "<example-url-02>"
}
```

2. Use the [get index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get) to view the configurations for the new indices. The indices were configured using the index template you created earlier.

```console
GET /my-index-000001,my-index-000002
```
This page has moved. Refer to [Manage index templates](/manage-data/data-store/index-basics.md#index-management-manage-index-templates) for all of the steps to create and manage index templates in {{kib}}.
2 changes: 1 addition & 1 deletion manage-data/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ toc:
children:
- file: data-store/templates/simulate-multi-component-templates.md
- file: data-store/templates/ignore-missing-component-templates.md
- file: data-store/templates/index-template-management.md
- hidden: data-store/templates/index-template-management.md
- file: data-store/aliases.md
- file: data-store/manage-data-from-the-command-line.md
- file: ingest.md
Expand Down
8 changes: 6 additions & 2 deletions redirects.yml
Original file line number Diff line number Diff line change
Expand Up @@ -284,11 +284,15 @@ redirects:
anchors:
'ece-ccs-ccr-traffic-filtering': 'ece-ccs-ccr-network-security'

# Related to https://github.com/elastic/docs-content/pull/2010
# Related to https://github.com/elastic/docs-content/pull/2010
'manage-data/lifecycle/index-lifecycle-management/index-management-in-kibana.md': 'manage-data/data-store/index-basics.md'

# Related to https://github.com/elastic/docs-content/pull/2097
'explore-analyze/machine-learning/machine-learning-in-kibana/xpack-ml-anomalies.md': 'explore-analyze/machine-learning/anomaly-detection.md'
'explore-analyze/machine-learning/machine-learning-in-kibana/xpack-ml-dfa-analytics.md': 'explore-analyze/machine-learning/data-frame-analytics.md'

'solutions/search/retrievers-examples.md': 'elasticsearch://reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md'
'solutions/search/retrievers-examples.md': 'elasticsearch://reference/elasticsearch/rest-apis/retrievers/retrievers-examples.md'

# Related to https://github.com/elastic/docs-content/pull/2010
'manage-data/data-store/templates/index-template-management.md': 'manage-data/data-store/index-basics.md'

Loading