feat: adds support for nginx variables in service_name param

[undying]

Description

Added support for Nginx variables in the service_name parameter for service discovery. This allows dynamic service name resolution based on Nginx variables such as $host, $http_host, and others, working with all service discovery implementations (Consul, Eureka, Nacos, etc.).

Detailed Description

Problem

The current implementation of service discovery in APISIX does not support using Nginx variables in the service_name parameter. This limits configuration flexibility as the service name must be hardcoded in the configuration, regardless of which service discovery implementation is used.

Solution

Added support for Nginx variables in the service_name parameter at the upstream level. This change enables dynamic service name resolution across all service discovery implementations. For example:

Usage Example

1. Define a variable in Nginx configuration:

map $http_host $backend {
  default echo;
  "~([^.]+).domain.local" $1;
}

2. Define routes

routes:
  -
    uri: "/*"
    host: "*.domain.local"
    upstream:
      service_name: $backend  # Nginx variable
      type: "roundrobin"
      discovery_type: "consul"  # or any other discovery type

2. Use the variable in route configuration with any service discovery:

routes:
  -
    uri: "/*"
    host: "*.domain.local"
    upstream:
      service_name: $backend
      type: "roundrobin"
      discovery_type: "consul"  # or "eureka", "nacos", etc.

In this example, if a request comes to service1.domain.local, the $backend variable will be set to service1, and APISIX will look for a service with this name in the configured service discovery system.

Backward Compatibility

The change is fully backward compatible because:

1. Existing configurations will continue to work without changes
2. The new functionality is only activated when using Nginx variables in service_name
3. The change is implemented at the upstream level, making it available to all service discovery implementations

---

Checklist

• I have explained the need for this PR and the problem it solves
• I have explained the changes or the new features added to this PR
• I have added tests corresponding to this change
• I have updated the documentation to reflect this change
• I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

[moonming]

@undying please fix CI, thx

[undying]

I've pushed the linter fix, hope that will help.
Btw, how do you run tests locally to have a fast response loop while developing plugins?

[Baoyuantop]

I've pushed the linter fix, hope that will help. Btw, how do you run tests locally to have a fast response loop while developing plugins?

I don't get it. Can you describe it in more detail?

[undying]

Can you describe it in more detail?

I mean, there are a lot of GitHub actions in the repository, including running tests inside the build step. I tried to run this step locally to make debugging locally faster, but I couldn't, because the project has too many dependencies.

I thought maybe you have some tools for running tests locally?

[Baoyuantop]

Can you describe it in more detail?

I mean, there are a lot of GitHub actions in the repository, including running tests inside the build step. I tried to run this step locally to make debugging locally faster, but I couldn't, because the project has too many dependencies.

I thought maybe you have some tools for running tests locally?

How did you try? What were the mistakes you encountered?

Here's a document to refer to https://apisix.apache.org/docs/apisix/internal/testing-framework/

[undying]

Okay, it seems that everything is built successfully, hope it can be merged now.

[undying]

Hi @undying, please fix the failed CI

Finally fixed

[nic-6443]

please add service_name to this error log, since service_name is no longer a fixed value, we need to know the specific service_name when there is a no valid upstream node.

[undying]

I added logging service_name. This format is ok?

[membphis]

This new feature is not safe, we can wait for more voice from the community.

For me, one upstream work for one service, much safer.

Here is a bad example:

# bad example, which may cause APISIX unsafe, allow user to access any service of K8s or eureka

$ curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -i -d '
{
    "uri": "/*",
    "upstream": {
        "service_name": "${http_service}",
        "discovery_type": "eureka"
    }
}'

I sugguest: need to wait for more voice from community, don't merge this PR soon

[undying]

This new feature is not safe, we can wait for more voice from the community.

We can consider Nginx as an unsafe web server because it allows us to enable such features out of the box. I believe it's the user's choice how to use such a feature, so its presence is much better than its absence.

For me, one upstream work for one service, much safer.

It's safer, but it's not convenient when you have hundreds of microservices with simple domain/service mapping. The "safe" method requires you to manually copy and paste configurations for every such service, which results in a massive configuration that negatively affects performance and maintainability.

Here is a bad example:

Actually, it's a very good example because this is how the feature works. You create a single configuration with simple mapping. For additional security, you can further refine the configuration and extract the service name not directly from the header, but from a special map that serves as an allow list for services.

Example:

map $http_host $service {
    hostnames;
    default ""; # or some dummy upstream that returns a more valuable response
    service-a.* service_a;
    service-b.* service_b;
}

$ curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -i -d '
{
    "uri": "/*",
    "upstream": {
        "service_name": "${service}",
        "discovery_type": "eureka"
    }
}'

[membphis]

We can consider Nginx as an unsafe web server because it allows us to enable such features out of the box. I believe it's the user's choice how to use such a feature, so its presence is much better than its absence.

I don't know a good way to balance between safe and convenient for this new feature.

I love this PR, but we need to consider the unsafe thing. More discussion and suggestions from the community are needed, we may get another good way to resolve the issue