Logs flooded with `WARN: Container is not in same network as caddy`

[accforgithubtest]

I have

• A single shared network: default (resulting in <stack_name>_default).
• ipv4_address is manually assigned to every single container.
• Caddy attached to this network ipv4_address: 10.10.1.100.
• Multiple applications attached to ipv4_address: 10.10.1.xxx.
• Multiple Applications attached to ipv4_address: 10.10.5.xxx.
• Working reverse proxy: Caddy can route to the applications in both 10.10.1.xxx and 10.10.5.xxx.
• CADDY_INGRESS_NETWORKS is set to - CADDY_INGRESS_NETWORKS=<stack_name>_default.

Issue - Persistent warning which is flooding the logs : Container is not in same network as caddy, even though the reverse proxy is working and able to route to the applications in both 10.10.1.xxx and 10.10.5.xxx.

Important bits of the docker compose files -

Parent (stack) docker compose

networks:
  default:
    driver: bridge
    enable_ipv6: false
    ipam:
      config:
        - subnet: 10.10.0.0/16

...
...

include:
  - path: 'docker/caddy-docker-proxy/docker-compose.yaml'
  - path: 'docker/application_abc/docker-compose.yaml'
  - path: 'docker/application_def/docker-compose.yaml'
  - path: 'docker/application_xyz/docker-compose.yaml'

Caddy docker proxy compose file -

services:
  caddy:
    image:  ....
    ...
    networks:
      default:
        ipv4_address: 10.10.1.100

Many other application use this docker compose file, one for each application

services:
  application_abc:
    image:  ....
    ...
    networks:
      default:
        ipv4_address: 10.10.1.xxx

services:
  application_xyz:
    image:  ....
    ...
    networks:
      default:
        ipv4_address: 10.10.5.xxx

LLM suggests -

This confirms that the issue is not the missing configuration, but a subtle behavior or strict check within the caddy-docker-proxy source code itself, likely related to the manual static IP assignments falling into different subnets (10.10.1.x vs 10.10.5.x) within the larger $10.10.0.0/16$ supernet.

Since the configuration is correct and the routing works, the only remaining option to silence a persistent, but harmless, warning is to use the specific environment variable designed to disable all network checks when necessary.

For scenarios where the proxy incorrectly determines reachability (like yours), the documentation suggests using CADDY_DOCKER_INSECURE_NETWORKS.

I do not see a CADDY_DOCKER_INSECURE_NETWORKS in the readme. I assume LLM is not right here.

But is but a subtle behavior or strict check within the caddy-docker-proxy source code itself, the issue ?

[polarathene]

Persistent warning which is flooding the logs : Container is not in same network as caddy, even though the reverse proxy is working and able to route to the applications in both 10.10.1.xxx and 10.10.5.xxx.

If routing is working, then the warning isn't related to those containers. You must have another container being discovered that CDP is processing. I'm not sure how it's going about that, but perhaps check any other running containers? For network changes I think you may need docker compose up --force-recreate, or even a full docker compose down && docker compose up, I don't think any inactive containers with CDP labels would trigger anything but pretty sure in a clean reproduction environment the warning won't be logged.

---

The warning is logged here:

caddy-docker-proxy/generator/containers.go

Lines 22 to 39 in a955641

	for networkName, network := range container.NetworkSettings.Networks {
	include := false
	if !onlyIngressIps {
	include = true
	} else if overrideNetwork {
	include = networkName == ingressNetworkFromLabel
	} else {
	include = g.ingressNetworks[network.NetworkID]
	}
	if include {
	ips = append(ips, network.IPAddress)
	}
	}
	if len(ips) == 0 {
	logger.Warn("Container is not in same network as caddy", zap.String("container", container.ID), zap.String("container id", container.ID))

That iterates over the key/value output from a Docker API endpoint containers/json, but you can use the Docker CLI to see similar for a container like this:

$ docker inspect example-1 | jq .[].NetworkSettings.Networks

{
  "example_default": {
    "IPAMConfig": null,
    "Links": null,
    "Aliases": [
      "example-1",
      "example"
    ],
    "MacAddress": "52:d6:22:83:9c:1d",
    "DriverOpts": null,
    "GwPriority": 0,
    "NetworkID": "d01811fd8317d52797260c4ee2a993c0dd25a89e7879725aaa2ed03c44a04eef",
    "EndpointID": "04d763fa2ca584842bf5978ec8341263456d5b311acced30245d81f9fa2d5677",
    "Gateway": "172.23.0.1",
    "IPAddress": "172.23.0.2",
    "IPPrefixLen": 16,
    "IPv6Gateway": "",
    "GlobalIPv6Address": "",
    "GlobalIPv6PrefixLen": 0,
    "DNSNames": [
      "example-1",
      "example",
      "95a5a1a7ed05",
      "example.test"
    ]
  }
}

So in this case the NetworkSettings.Networks array of objects, only had the one network entry, example_default. One of those 3 conditions need to be true for CDP to then add the IPAddress from that network object to CDP's ips array, otherwise you'll get that warning log when no network for that container was found.

I do not see a CADDY_DOCKER_INSECURE_NETWORKS in the readme. I assume LLM is not right here.

That's just the LLM spitting lies (as they're well known to do). Just like it's assessment with the static IP assignment to different subnets, despite all being to the same common network/subnet.

I don't quite understand Go well enough to make sense of what's happening a little bit earlier in the same source code I referenced above:

caddy-docker-proxy/generator/containers.go

Line 20 in a955641

ingressNetworkFromLabel, overrideNetwork := container.Labels[IngressNetworkLabel]

But that seems to be relevant to the README entry here which permits a container/service to have a label caddy_ingress_network as a way to specify the network to share with CDP for IP selection associated to the container (which is similar to the CDP ENV CADDY_INGRESS_NETWORKS):

caddy-docker-proxy/README.md

Line 333 in a955641

:warning: caddy docker proxy does a best effort to automatically detect what are the ingress networks. But that logic fails on some scenarios: [#207](https://github.com/lucaslorentz/caddy-docker-proxy/issues/207). To have a more resilient solution, you can manually configure Caddy ingress network using CLI option `ingress-networks`, environment variable `CADDY_INGRESS_NETWORKS`. You can also specify the ingress network per container/service by adding to it a label `caddy_ingress_network` with the network name.

As per #207 if you're running with cgroups v2 (likely?) CADDY_INGRESS_NETWORKS may be required, but there was added fallback logic that landed since.