How to access docker server instance when tests are not in debug mode?

[bdw429s]

I'd like to be able to access a web-based monitor library in order to tune the performance of my framework WHILE the normal benchmarks are running on it.

When I start the tests in debug mode with this command (I'm using Docker desktop):

$ docker run -it --rm --network tfb -v /var/run/docker.sock:/var/run/docker.sock -v /D_DRIVE/docs/GitHub/FrameworkBenchmarks:/FrameworkBenchmarks techempower/tfb --mode debug --test cfml

I am able to hit my site the browser:

And when I run the tests normally with this command:

$ docker run -it --rm --network tfb -v /var/run/docker.sock:/var/run/docker.sock -v /D_DRIVE/docs/GitHub/FrameworkBenchmarks:/FrameworkBenchmarks techempower/tfb --test cfml

I get this in the browser (while the cfml container is up and the tests are running):

@msmith-techempower said on the Google group that

The current toolset should not discern between benchmark and debug modes

but that's not the behavior I'm experiencing.

[bdw429s]

Any insight on this?

[bdw429s]

I am on Windows running Docker via a docker machine VM. I will give that a try and report back.

[msmith-techempower]

Oof, okay well... since you're on Windows and the current toolset requires Docker-in-Docker, I am not sure that you can actually run it in host mode. I spent a LONG time trying, but I just could never get it working, which is one of the reasons I started writing the new toolset.

[bdw429s]

Right, so if I use just the docker --network host flag:

$ docker run -it --network host --rm -v /var/run/docker.sock:/var/run/docker.sock -v /D_DRIVE/docs/GitHub/FrameworkBenchmarks:/FrameworkBenchmarks te
chempower/tfb --test cfml

then I get the test seem to try to run, but the DB isn't accessible:

techempower/postgres:latest: Database was not ready after startup

and even though my CFML container starts, it isn't reachable.

Accessing URL http://tfb-server:8080/db.cfm:
Verifying test db for cfml caused an exception: HTTPConnectionPool(host='tfb-server', port=8080): Max retries exceeded with url: /db.cfm (Caused by Ne
wConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e56cff690>: Failed to establish a new connection: [Errno -3] Temporary failure in
name resolution',))
   FAIL for http://tfb-server:8080
     Server did not respond to request

And if I add the tool set flag of --network-mode host on the end, then regardless of my docker host flag, the tests immediately error out before they even start my CFML container with the following errors.

================================================================================
Running Tests...
================================================================================
wrk: Docker build failed; terminating
wrk: Traceback (most recent call last):
wrk:   File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 48, in __build
wrk:     decode=True
wrk:   File "/usr/local/lib/python2.7/dist-packages/docker/api/build.py", line 269, in build
wrk:     timeout=timeout,
wrk:   File "/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py", line 46, in inner
wrk:     return f(self, *args, **kwargs)
wrk:   File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 226, in _post
wrk:     return self.post(url, **self._set_request_timeout(kwargs))
wrk:   File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 590, in post
wrk:     return self.request('POST', url, data=data, json=json, **kwargs)
wrk:   File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 542, in request
wrk:     resp = self.send(prep, **send_kwargs)
wrk:   File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 655, in send
wrk:     r = adapter.send(request, **kwargs)
wrk:   File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 516, in send
wrk:     raise ConnectionError(e, request=request)
wrk: ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /v1.35/build?q=False&pull=True&t=techempower%2Ft
fb.wrk&nocache=False&forcerm=True&rm=False&dockerfile=wrk.dockerfile (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff
17012cd10>: Failed to establish a new connection: [Errno 111] Connection refused',))
wrk: Build time: 0s
A fatal error has occurred
Traceback (most recent call last):
  File "/FrameworkBenchmarks/toolset/run-tests.py", line 244, in main
    any_failed = benchmarker.run()
  File "/FrameworkBenchmarks/toolset/benchmark/benchmarker.py", line 57, in run
    self.docker_helper.build_wrk()
  File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 364, in build_wrk
    tag="techempower/tfb.wrk")
  File "/FrameworkBenchmarks/toolset/utils/docker_helper.py", line 48, in __build
    decode=True
  File "/usr/local/lib/python2.7/dist-packages/docker/api/build.py", line 269, in build
    timeout=timeout,
  File "/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 590, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=2375): Max retries exceeded with url: /v1.35/build?q=False&pull=True&t=techempower%2Ftfb.wr
k&nocache=False&forcerm=True&rm=False&dockerfile=wrk.dockerfile (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff17012
cd10>: Failed to establish a new connection: [Errno 111] Connection refused',))
Shutting down (may take a moment)

[msmith-techempower]

And if I add the tool set flag of --network-mode host on the end, then regardless of my docker host flag, the tests immediately error out before they even start my CFML container with the following errors.

Right, this is expected on Windows.

then I get the test seem to try to run, but the DB isn't accessible:

techempower/postgres:latest: Database was not ready after startup

Right, this is also expected when you only have --network host set on Docker. Essentially, that is telling Docker to do a few things, but largely it puts the started containers (application and database) on the host network. However, because you are missing the --network-mode host on the toolset, it is expecting the hostname tfb-database to resolve, but it doesn't because your host network knows nothing about that.

[bdw429s]

Thanks for trying to help. It sounds like the final take away from this is that it's not possible. I don't know much about the toolset, but that's a shame since the services are accessible in debug mode so it does seem like it should be possible. I'll look forward to a future update of the toolset that allows this and in the mean time I suppose I'll have to use a JMeter test to try and simulate what the toolset is doing in debug mode.