Skip to content

[Bug]: PilotSync Agent doesn't upload files to https servers #8640

Description

@Loxeris

Search before creating an issue

  • I have searched existing issues and confirmed this is not a duplicate

Bug Description

When using a web server's https address in uploadLocation of the PilotSync Agent, the pilot files are not uploaded and only file paths are provided by the PUT requests.

Steps to Reproduce

  1. run an external web server handling put requests, i.e. nginx with the following config:
    http {

        access_log /dev/stdout;
        error_log /dev/stdout;


        server {
          listen 8080;
          listen 8443 ssl;
          server_name localhost;

          ssl_certificate    hostcert.pem;
          ssl_certificate_key hostkey.pem;

          location /pilot/ {
            autoindex on;
            root /opt/dirac/webRoot/www/;
            dav_methods PUT;
            create_full_put_path  on;
            expires 1h;
            break;
          }
        }
  1. set UploadLocations = https://your-server:8443/pilot in the Systems/WorkloadManagement/Agents/PilotSyncAgent config

Expected Behavior

Pilot files are synced on the nginx server.
dirac-pilot.py, pilot.json, ... are created under /opt/dirac/webRoot/www/pilot/

Actual Behavior

A single pilot file is created at /opt/dirac/webRoot/www/pilot, its text content being the path of the file that should be uploaded last.

Environment

  • DIRAC Version: v9.1.9
  • OS: AlmaLinux

Relevant Log Output

Additional Context

Additionally PilotSync only expects 200 and 202 responses, while webDAV servers typically use 201 on creation and 204 or 200 on updates, causing unnecessary error logs.

I believe the put request should be updated with the file basename added to the url, and use file content instead of the file path in the request data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions