Webserver and fileserver parametric ports

[lorenzospano]

Motivation

This PR aims at making configurable all the ports that clearml-server uses.
Right now, some ports are hardcoded and not parametric ie webserver on port 80 and fileserver on port 8081.
Issue descrition - #293

Fileserver port parameterization

The listening port for the fileserver is hardcoded in fileserver/fileserver.py (when launched via flask) and in docker/build/internal_files/entrypoint.sh (when launched via gunicorn).

Flask setup

The code portion in fileserver/fileserver.py is reported below for simplicity.

    parser.add_argument(
        "--port", "-p", type=int, default=8081, help="Port (default %(default)d)"
   
    parser.add_argument(
        "--ip", "-i", type=str, default="0.0.0.0", help="Address (default %(default)s)"

    parser.add_argument("--debug", action="store_true", default=False)

I modified the fileserver to read the default values for ip, port and debug from the config and added the configurations inside of fileserver/config/default/fileserver.conf.
This behavior was guided by the existing mechanism in apiserver.

Gunicorn setup

Similarly, I replicated the existing behavior of the apiserver to also make gunicorn configurable via environment variables.
The default port is preserved, while now it is also possible to override it with FILESERVER_GUNICORN_BIND.
The intended behavior is the same as the one used for the apiserver.

Webserver port parameterization

The webserver port is hardcoded in docker/build/internal_files/clearml.conf.template.

    listen       80 default_server;
    ${COMMENT_IPV6_LISTEN}listen       [::]:80 default_server;

Per my understanding this is the conf that nginx uses to create the webserver UI, and it is not configurable via environment variables.
For this reason I modified the template to accepts an additional environment variable that allows to compile the webserver port via envsubst.
I also modified the envsubst command that is execute inside of docker/build/internal_files/entrypoint.sh to include such a value when the template is compiled.

Testing

I tested the startup and basic interactions among those services.
I could not test (yet) more advanced features such as agents and pipeline runs.

If there is any known component that features hardcoded ports please highlight it so that I can improve my contribution

[ainoam]

Thanks for this @lorenzospano - We'll try to review it as soon as we can.

[shyallegro]

Hi @lorenzospano
can you please explain why you need to change the internal docker webserver port?
I don't see any reason to do that, since you can expose it to any port you like through docker configuration