You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/trino/pages/usage-guide/catalogs/delta-lake.adoc
+9-1Lines changed: 9 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
= Delta Lake
2
2
3
-
[Delta Lake](https://delta.io/) is an open-source storage framework that enables building a Lakehouse architecture with support for multiple compute engines.
3
+
https://delta.io/[Delta Lake] is an open-source storage framework that enables building a Lakehouse architecture with support for multiple compute engines.
4
4
It depends on a Hive metastore being present and makes use of the same metastore ConfigMap used by the Hive connector.
5
5
6
6
== Example Delta Lake catalog configuration
@@ -27,8 +27,16 @@ spec:
27
27
accessStyle: Path
28
28
credentials:
29
29
secretClass: minio-credentials
30
+
# TLS is required for S3 connections, see the note below
31
+
tls:
32
+
verification:
33
+
server:
34
+
caCert:
35
+
secretClass: minio-tls-certificates
30
36
----
31
37
38
+
IMPORTANT: The `tls` section is required, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3].
39
+
32
40
== Connect to S3 store or HDFS
33
41
34
42
The Delta Lake connector connects to S3 or HDFS in the same way the xref:usage-guide/catalogs/hive.adoc[] connector does.
Copy file name to clipboardExpand all lines: docs/modules/trino/pages/usage-guide/catalogs/hive.adoc
+15-2Lines changed: 15 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,13 +26,19 @@ spec:
26
26
accessStyle: Path
27
27
credentials:
28
28
secretClass: minio-credentials
29
-
configOverrides: # <4>
29
+
tls: # <4>
30
+
verification:
31
+
server:
32
+
caCert:
33
+
secretClass: minio-tls-certificates
34
+
configOverrides: # <5>
30
35
hive.metastore.username: trino
31
36
----
32
37
<1> The name of the catalog as it will appear in Trino
33
38
<2> TrinoCluster can use these labels to select which catalogs to include
34
39
<3> The name of your Hive Stacklet
35
-
<4> Use `configOverrides` to add arbitrary properties to the Trino catalog configuration
40
+
<4> TLS is required for S3 connections, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3]
41
+
<5> Use `configOverrides` to add arbitrary properties to the Trino catalog configuration
36
42
37
43
== Connect to S3 store
38
44
@@ -50,11 +56,18 @@ spec:
50
56
accessStyle: Path
51
57
credentials:
52
58
secretClass: minio-credentials
59
+
tls:
60
+
verification:
61
+
server:
62
+
caCert:
63
+
secretClass: minio-tls-certificates
53
64
# OR
54
65
s3:
55
66
reference: my-minio
56
67
----
57
68
69
+
IMPORTANT: The `tls` section is required, see xref:usage-guide/s3.adoc#tls-required[Connecting Trino to S3].
70
+
58
71
See xref:concepts:s3.adoc[] for details about S3 connections.
59
72
60
73
IMPORTANT: Make sure that the underlying Hive metastore also has access to the S3 store, because it will e.g. check if the directory exists when creating tables.
Copy file name to clipboardExpand all lines: docs/modules/trino/pages/usage-guide/s3.adoc
+20-10Lines changed: 20 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,10 @@
2
2
:description: Configure S3 connections in Trino either inline within the TrinoCatalog or via an external S3Connection resource for centralized management.
3
3
4
4
You can specify S3 connection details directly inside the TrinoCatalog specification or by referring to an external S3Connection custom resource.
5
-
This mechanism used used across the whole Stackable Data Platform, read the xref:concepts:s3.adoc[S3 concepts page] to learn more.
5
+
This mechanism is used across the whole Stackable Data Platform, read the xref:concepts:s3.adoc[S3 concepts page] to learn more.
6
+
7
+
[#tls-required]
8
+
IMPORTANT: Every S3 connection must contain a `tls` section, because Trino uses the native S3 client, which requires TLS.
6
9
7
10
== Inline
8
11
@@ -14,22 +17,24 @@ s3: # <1>
14
17
inline:
15
18
host: test-minio # <2>
16
19
port: 9000 # <3>
17
-
pathStyleAccess: true # <4>
18
-
secretClass: minio-credentials # <5>
19
-
tls:
20
+
accessStyle: Path # <4>
21
+
credentials:
22
+
secretClass: minio-credentials # <5>
23
+
tls: # <6>
20
24
verification:
21
25
server:
22
26
caCert:
23
-
secretClass: minio-tls-certificates #<6>
27
+
secretClass: minio-tls-certificates
24
28
----
25
29
<1> Entry point for the connection configuration
26
30
<2> Connection host
27
31
<3> Optional connection port
28
-
<4> Optional flag if path-style URLs should be used; This defaults to `false`
29
-
which means virtual hosted-style URLs are used.
30
-
<5> Name of the Secret object expected to contain the following keys:
31
-
`accessKey` and `secretKey`
32
-
<6> Optional TLS settings for encrypted traffic. The `secretClass` can be provided by the Secret Operator or yourself.
32
+
<4> Optional access style, either `Path` or `VirtualHosted`; This defaults to
33
+
`VirtualHosted`, which means virtual hosted-style URLs are used.
34
+
<5> Name of the SecretClass providing the credentials. The Secret behind it is expected
35
+
to contain the following keys: `accessKey` and `secretKey`
36
+
<6> TLS settings for encrypted traffic. The `secretClass` can be provided by the Secret Operator or yourself.
37
+
This section is required, see <<tls-required,the note above>>.
33
38
34
39
A self provided S3 TLS secret can be specified like this:
35
40
@@ -84,6 +89,11 @@ spec:
84
89
accessStyle: Path
85
90
credentials:
86
91
secretClass: minio-credentials
92
+
tls:
93
+
verification:
94
+
server:
95
+
caCert:
96
+
secretClass: minio-tls-certificates
87
97
----
88
98
89
99
This has the advantage that the connection configuration is configured in a single place and can be shared across applications, reducing the cost of updating these details.
0 commit comments