service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443" is ignored!

[aholbreich]

My config

    service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
    service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"

however Loadbalancer ignored 443 port settiong and is still listening on 80. Same shown on Digital Ocean Admin Console

[timoreimann]

Does this show all of the LB-specific annotations you have set? My guess is you missed to specify the certificate ID or TLS pass-through, one of which must be given when HTTPS is used.

See also https://github.com/digitalocean/digitalocean-cloud-controller-manager/blob/master/docs/controllers/services/annotations.md#servicebetakubernetesiodo-loadbalancer-tls-ports, especially the second paragraph.

[aholbreich]

Hi @timoreimann unfortunatley your guess is wrong.
cer't id was given all the time. here my current setup:

service.beta.kubernetes.io/do-loadbalancer-certificate-id: "cb7187e9-xxxxx"
service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
service.beta.kubernetes.io/do-loadbalancer-hostname: "a.b.c.com"
service.beta.kubernetes.io/do-loadbalancer-enable-backend-keepalive: "true"

all annotations work except port 443. i have to go to the UI and uopdate it from 80 to 443 :( after evy change

[aholbreich]

@timoreimann is there way to gather debug information or watch the logs?

[aholbreich]

Clarifying "ignored". It's even always set to "80" even if it was correcten in the UI manually as soon Kubernetes Service resource is touched.

[mman]

I have it setup as http with forced redirect and it seems to work, I created the LB several months ago and updated it to add the do-loadbalancer-name annotation after the last DO release.

    service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
    service.beta.kubernetes.io/do-loadbalancer-algorithm: "least_connections"
    service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
    service.beta.kubernetes.io/do-loadbalancer-certificate-id: "xyz"
    service.beta.kubernetes.io/do-loadbalancer-hostname: "a.b.c.d"
    service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true"
    service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "false"
    service.beta.kubernetes.io/do-loadbalancer-name: "xyz"

[timoreimann]

@aholbreich it almost sounds like you have a Service Port definition provided for port 80 but none for port 443. Could you please share your full Service manifest?

[timoreimann]

Also worth noting that errors are propagated as events in the namespace of the Service. Not sure if it'd help you here since it doesn't seem to be about a legitimate error.

And yeah, changes done to the LB in the DO control panel are reconciled away by design. All configurations should go through the Service object.

[aholbreich]

---
kind: Service
apiVersion: v1
metadata:
  name: abc
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-certificate-id: "cb7187e9-fb7f-4b91-b690-abc"
    service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
    service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
    service.beta.kubernetes.io/do-loadbalancer-hostname: "abc.me"
    service.beta.kubernetes.io/do-loadbalancer-enable-backend-keepalive: "true"
    service.beta.kubernetes.io/do-loadbalancer-name: "lb1"
spec:
  type: LoadBalancer
  selector:
    app: lyricsforme
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 2368

this is the last one:

P.S. this docu
https://www.digitalocean.com/docs/kubernetes/how-to/configure-load-balancers/#ssl-certificates
gives example with:
service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
shouldn't it be 'https'?

[timoreimann]

Your issue is that you have a Service Port definition for port 80 specified but none for port 443. Each specified port annotation needs to match up with a Service port or otherwise the corresponding annotations will be ignored silently. We could probably do better here by emitting a warning event.

The example is correct: the default protocol of http is required because we still want port 80 to be accessible via HTTP. If you also want unencrypted access, you'll need to the same. (All HTTPS ports are implicitly assumed to run on HTTPS, thereby overriding the default protocol in the example.)

This official example is perhaps what you want.

[aholbreich]

Sounds still confusing to me.

1)service.beta.kubernetes.io/do-loadbalancer-protocol: do definig the outer or Listening port of managed LB? please confirm.

Having that LB is able to terminate SSL on 443 and forward to any workers port. If i got you correct i have to provide corresponing 443 ports on the service?
why? i don't get it... what if i don't have anything on 443?

[aholbreich]

But i've got it working. The key is to have 443 as port on LB. I think it's a bit misleading, but maybe do the job

[timoreimann]

@aholbreich glad you got it to work. 🎉

service.beta.kubernetes.io/do-loadbalancer-protocol defines the default entry protocol (i.e., what the LB will be expecting for its incoming traffic). If the proxy terminates SSL requests (i.e., a certificate is used), the internal protocol (between the LB and the worker droplet) will be HTTP of course.

I hear what you're saying: you'd like accept HTTPS traffic on the LB's port 443 and forward the HTTP traffic to port 80. I think that's a reasonable request but unfortunately it is not possible today.

We have plans to fundamentally change and improve the way LBs can be configured, possibly leveraging a more structured and less error-prone CRD format. I think you discovered #256 already which is going to be part of the solution.