Proposal to increase the severity of the ConnSvc startup timeout message

[bieryAtFnal]

Modified the logging of the message that says that the ConnectivityServer startup has timed out to be an error message instead of a debug messages (in client.py).

Description

While I was working on the integtest in daqsystemtest that demonstrates the running of a small test system on multiple computers (sample_ehn1_multihost_test.py), I noticed that the integtest would fail the first time that I tried it with a new base release. I traced this to the slow loading of the new nightly release on the remote computers, but I needed to enable DEBUG messages in drunc in order to discover what was happening.

It would have been really great if there had been an error message that told me that the startup of the ConnectivityServer was timing out, and that was why drunc was aborting the DAQ session.

This PR suggests a change in message severity that I believe would help.

As an aside, I see that there is a DruncSetupException thrown when the ConnSvc startup times out, but I'm not seeing any console output that tells me that an exceptional condition was encountered in drunc. It simply closes down the DAQ session.

Another side comment is that there doesn't seem to be a way to increase the ConnSvc startup timeout in the drunc code. That is probably not critical, but it may be nice to have.

To test this change, I locally modified the drunc process_manager_driver to use an artificially short timeout, as shown below. Without the change to the message severity covered in this PR, drunc simply closes down the session without telling us why it did so. With the change in message severity, it prints out a helpful message.

diff --git a/src/drunc/process_manager/process_manager_driver.py b/src/drunc/process_manager/process_manager_driver.py
index e5b5633f..edff7b1a 100644
--- a/src/drunc/process_manager/process_manager_driver.py
+++ b/src/drunc/process_manager/process_manager_driver.py
@@ -133,7 +133,7 @@ class ProcessManagerDriver:
                 self.log.debug(
                     f"Checking connectivity service readiness before booting application {request.process_description.metadata.name}"
                 )
-                if csc and not csc.is_ready(timeout=10):
+                if csc and not csc.is_ready(timeout=0):
                     raise DruncSetupException(
                         "Connectivity service did not respond within timeout."
                     )

Type of change

• Optimization (non-breaking, back-end change that speeds up the code)

Further checks

• Code is commented, particularly in hard-to-understand areas