Skip to content

feat: support envFrom and envVars #1183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
+143 −3
Closed

feat: support envFrom and envVars #1183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 50 additions & 2 deletions config/charts/inferencepool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@ Note that the provider name is needed to deploy provider-specific resources. If

### Install with Custom Environment Variables

To set custom environment variables for the EndpointPicker deployment:
#### Simple Environment Variables

To set simple key-value environment variables for the EndpointPicker deployment:

```txt
$ helm install vllm-llama3-8b-instruct \
Expand All @@ -43,6 +45,50 @@ inferenceExtension:
FEATURE_FLAG_ENABLED: "true"
```

#### Advanced Environment Variables

For more complex environment variable configurations, including secrets, configmaps, and field references, use the `envVars` and `envFrom` fields:

```yaml
# values.yaml
inferenceExtension:
# Simple key-value environment variables
env:
Copy link
Contributor

@vMaroon vMaroon Jul 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have env accept any yaml? then it's up to the user to use it properly.

E.g., in the epp-deployment template, env can have:

{{- toYaml . | nindent X }} # X should probably be 8
        {{- end }}

The applying of a helm chart goes through Kubernetes validation in all cases. What do you think are the benefits of an explicit schema?

SIMPLE_VAR: "simple-value"
Comment on lines +55 to +57
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens in the same var is defined in both places?
this could get confusing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ Is it possible to fold env into envVars`?


# Advanced environment variables with valueFrom support
envVars:
- name: HF_TOKEN
valueFrom:
secretKeyRef:
name: hf-token
key: token
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CONFIG_VALUE
valueFrom:
configMapKeyRef:
name: my-config
key: config-key
- name: MEMORY_LIMIT
valueFrom:
resourceFieldRef:
resource: limits.memory

# Load all environment variables from secrets/configmaps
envFrom:
- secretRef:
name: my-secret
- configMapRef:
name: my-configmap
```

And apply it with:

```txt
Expand Down Expand Up @@ -84,7 +130,9 @@ The following table list the configurable parameters of the chart.
| `inferenceExtension.image.tag` | Image tag of the endpoint picker. |
| `inferenceExtension.image.pullPolicy` | Image pull policy for the container. Possible values: `Always`, `IfNotPresent`, or `Never`. Defaults to `Always`. |
| `inferenceExtension.extProcPort` | Port where the endpoint picker service is served for external processing. Defaults to `9002`. |
| `inferenceExtension.env` | Map of environment variables to set in the endpoint picker container. Defaults to `{}`. |
| `inferenceExtension.env` | Map of simple key-value environment variables to set in the endpoint picker container. Defaults to `{}`. |
| `inferenceExtension.envVars` | List of advanced environment variables with valueFrom support (secretKeyRef, configMapKeyRef, fieldRef, resourceFieldRef). Defaults to `[]`. |
| `inferenceExtension.envFrom` | List of sources to populate environment variables from (secretRef, configMapRef). Defaults to `[]`. |
| `provider.name` | Name of the Inference Gateway implementation being used. Possible values: `gke`. Defaults to `none`. |

## Notes
Expand Down
58 changes: 58 additions & 0 deletions config/charts/inferencepool/templates/epp-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,64 @@ spec:
- name: {{ $key }}
value: {{ $value | quote }}
{{- end }}
{{- if .Values.inferenceExtension.envVars }}
{{- range .Values.inferenceExtension.envVars }}
- name: {{ .name }}
{{- if .value }}
value: {{ .value | quote }}
{{- else if .valueFrom }}
valueFrom:
{{- if .valueFrom.secretKeyRef }}
secretKeyRef:
name: {{ .valueFrom.secretKeyRef.name }}
key: {{ .valueFrom.secretKeyRef.key }}
{{- if .valueFrom.secretKeyRef.optional }}
optional: {{ .valueFrom.secretKeyRef.optional }}
{{- end }}
{{- else if .valueFrom.configMapKeyRef }}
configMapKeyRef:
name: {{ .valueFrom.configMapKeyRef.name }}
key: {{ .valueFrom.configMapKeyRef.key }}
{{- if .valueFrom.configMapKeyRef.optional }}
optional: {{ .valueFrom.configMapKeyRef.optional }}
{{- end }}
{{- else if .valueFrom.fieldRef }}
fieldRef:
fieldPath: {{ .valueFrom.fieldRef.fieldPath }}
{{- if .valueFrom.fieldRef.apiVersion }}
apiVersion: {{ .valueFrom.fieldRef.apiVersion }}
{{- end }}
{{- else if .valueFrom.resourceFieldRef }}
resourceFieldRef:
resource: {{ .valueFrom.resourceFieldRef.resource }}
{{- if .valueFrom.resourceFieldRef.containerName }}
containerName: {{ .valueFrom.resourceFieldRef.containerName }}
{{- end }}
{{- if .valueFrom.resourceFieldRef.divisor }}
divisor: {{ .valueFrom.resourceFieldRef.divisor }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- if .Values.inferenceExtension.envFrom }}
envFrom:
{{- range .Values.inferenceExtension.envFrom }}
{{- if .secretRef }}
- secretRef:
name: {{ .secretRef.name }}
{{- if .secretRef.optional }}
optional: {{ .secretRef.optional }}
{{- end }}
{{- else if .configMapRef }}
- configMapRef:
name: {{ .configMapRef.name }}
{{- if .configMapRef.optional }}
optional: {{ .configMapRef.optional }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
volumeMounts:
- name: plugins-config-volume
mountPath: "/config"
Expand Down
36 changes: 35 additions & 1 deletion config/charts/inferencepool/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,44 @@ inferenceExtension:
# - pluginRef: max-score-picker
# weight: 1

# Example environment variables:
# Example environment variables (simple key-value pairs):
# env:
# KV_CACHE_SCORE_WEIGHT: "1"

# Example enhanced environment variables with valueFrom support:
# envVars:
# - name: HF_TOKEN
# valueFrom:
# secretKeyRef:
# name: hf-token
# key: token
# - name: POD_NAME
# valueFrom:
# fieldRef:
# fieldPath: metadata.name
# - name: POD_NAMESPACE
# valueFrom:
# fieldRef:
# fieldPath: metadata.namespace
# - name: CONFIG_VALUE
# valueFrom:
# configMapKeyRef:
# name: my-config
# key: config-key
# - name: MEMORY_LIMIT
# valueFrom:
# resourceFieldRef:
# resource: limits.memory
# - name: SIMPLE_VALUE
# value: "simple-string-value"

# Example envFrom for loading all keys from secrets/configmaps:
# envFrom:
# - secretRef:
# name: my-secret
# - configMapRef:
# name: my-configmap

inferencePool:
targetPortNumber: 8000
modelServerType: vllm # vllm, triton-tensorrt-llm
Expand Down