【Emergency Help】The issue regarding the usage method of the "thanos query --endpoint.sd-config-file" parameter

[cscowx]

Hello, everyone!

Container image version：thanosio/thanos:main-2025-10-07-be442a5

/bin $ ./thanos --version
thanos, version 0.40.0-dev (branch: main, revision: be442a5)
build user: root@76cd6c6b9001
build date: 20251007-14:11:33
go version: go1.25.1
platform: linux/amd64
tags: netgo,slicelabels

My CoreDNS does not support IPv6.

./thanos query --help

--endpoint.sd-config-file=<file-path>  
                                 Path to Config File with endpoint definitions

May I ask --endpoint.sd-config-file=
How should the format of this configuration file be written? Could you provide me with a specific example or an official document?

I found that there were the following errors in the operation log of the Thanos Query Pod.

kubectl logs -f pod/my-thanos-query-0 -n monitor

{"addr":"","caller":"endpointset.go:240","level":"warn","msg":"Duplicate endpoint address is provided","ts":"2025-10-21T05:21:18.04131647Z"}
{"address":"","caller":"endpointset.go:407","component":"endpointset","err":"getting metadata: failed to exit idle mode: dns resolver: missing address","level":"warn","msg":"update of endpoint failed","ts":"2025-10-21T05:21:18.041900188Z"}
{"addr":"","caller":"endpointset.go:240","level":"warn","msg":"Duplicate endpoint address is provided","ts":"2025-10-21T05:21:23.037603403Z"}
{"address":"","caller":"endpointset.go:407","component":"endpointset","err":"getting metadata: failed to exit idle mode: dns resolver: missing address","level":"warn","msg":"update of endpoint failed","ts":"2025-10-21T05:21:23.037929997Z"}
{"addr":"","caller":"endpointset.go:240","level":"warn","msg":"Duplicate endpoint address is provided","ts":"2025-10-21T05:21:28.04155388Z"}
{"address":"","caller":"endpointset.go:407","component":"endpointset","err":"getting metadata: failed to exit idle mode: dns resolver: missing address","level":"warn","msg":"update of endpoint failed","ts":"2025-10-21T05:21:28.041965737Z"}

Some DNS resolution tests were also conducted, as follows

If the parameter "type=" is not specified, the domain name resolution will encounter problems. The phenomenon is that sometimes it can be parsed successfully, and sometimes it returns "No answer".

kubectl exec -it pod/my-thanos-query-0 -n monitor -- nslookup my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local

Defaulted container "my-thanos-query" out of: my-thanos-query, config-render (init)
Server:         10.96.0.2
Address:        10.96.0.2:53

*** Can't find my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local: No answer

By using "type=A", the parsing can be successfully completed.

kubectl exec -it pod/my-thanos-query-0 -n monitor -- nslookup -type=A my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local

Defaulted container "my-thanos-query" out of: my-thanos-query, config-render (init)
Server:         10.96.0.2
Address:        10.96.0.2:53

Name:   my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local
Address: 10.244.156.200
Name:   my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local
Address: 10.244.218.134

I wonder if it's the DNS resolution issue that caused these error logs? How can we solve this problem? Or is it that my endpoints.json configuration file is written incorrectly?
I couldn't find any examples related to this parameter on the Thanos website either.

./thanos query --endpoint.sd-config-file=/etc/thanos/endpoints.json

cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-thanos-query-request-logging-config
  namespace: monitor
data:
  request-logging.yaml: |
    http:
      options:
        level: INFO
        decision:
          log_start: true
          log_end: true
    grpc:
      options:
        level: INFO
        decision:
          log_start: true
          log_end: true
EOF

cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-thanos-endpoints-config
  namespace: monitor
data:
  endpoints.json: |
    {
      "endpoints": [
        {
          "labels": {
            "group": "my-prometheus-thanos-sidecar",
            "env": "prod",
            "cluster": "nanjing-k8s"
          },
          "dns_sd_configs": [
            {
              "names": ["my-prometheus-thanos-sidecar-headless.monitor.svc.cluster.local"],
              "type": "A",
              "port": 10901
            }
          ]
        },
        {
          "labels": {
            "group": "my-thanos-store",
            "env": "prod",
            "cluster": "nanjing-k8s"
          },
          "dns_sd_configs": [
            {
              "names": ["my-thanos-store-headless.monitor.svc.cluster.local"],
              "type": "A",
              "port": 10901
            }
          ]
        }
      ]
    }
EOF