Allow POST requests from public user for specific URLs

[knabar]

With the introduction of the table slice call in #564, the option to use POST requests (in addition to GET) was included to allow for large number of rows to be fetched in one request. GET requests are limited by the maximum length of URIs.

An issue not previously considered is when omero.web.public.enabled is enabled in combination with omero.web.public.get_only. In that case there is no way to use POST for the table slice call. All POST requests would have to be allowed.

This PR proposes a new setting to selectively allow POST for some URLs, similar to how a set of URLs can be enabled for public mode.

With the latest commits, this PR adds a new setting to allow flexible configuration of URLs per request method, without impacting the functionality of the current settings, i.e. if this new setting is not configured, the behavior remains unchanged.

Example:

omero config set omero.web.public.url_filter_by_method '{"POST": "^/webgateway/table/\d+/slice/"}'

This may not be the best or simplest approach, any thoughts or ideas on how else we could approach this?

Some tests for the old and new settings have been added via ome/openmicroscopy#6460

cc: @will-moore @sbesson @jburel

[sbesson]

To demonstrate the original issue, the following requests can be executed after authenticating into OMERO.web using the Console tab of the Developer tools. They should return the first four cells (2 rows, 2 columns) of a table:

• using the GET request

$.get("/webgateway/table/2060/slice/?rows=0,1&columns=0,1", rsp=>console.log(rsp))

• or a POST request

$.post("/webgateway/table/2060/slice/", {rows: "0,1", columns: "0,1"}, rsp=>console.log(rsp))

Executing the same request against a table in a public OMERO.web instance deployed with the default omero.web.public.get_only configuration e.g. https://idr.openmicroscopy.org/webclient/?experimenter=-1 currently succeeds when using GET but fail with a 403 error when using POST.

The ability to slice the table using POST request is critical when slicing large table using large numbers of rows/columns. While GET requests are typically restricted by the maximum URL length, the body sent via a POST request has a larger upper limit. With this code, the expectation is to support the GET and POST requests to the /webgateway/table/<id>/slice endpoint either authenticated or anonymously (i.e. using the public user).

[will-moore]

This looks like a pretty clean solution to the issue.

The only thought I had was since only GET requests are allowed via the settings.PUBLIC_GET_ONLY, if you wanted to allow other methods (PUT, DELETE, PATCH, OPTIONS, and HEAD) then we probably don't want a setting for each one. So, maybe this new setting should allow all of those too, not just POST?

[knabar]

The only thought I had was since only GET requests are allowed via the settings.PUBLIC_GET_ONLY, if you wanted to allow other methods (PUT, DELETE, PATCH, OPTIONS, and HEAD) then we probably don't want a setting for each one. So, maybe this new setting should allow all of those too, not just POST?

I agree that we don't want a separate setting for each method, but I'm also hesitant to make it so you have to allow all methods on a URL if you just need one.

For maximum flexibility we could have a setting that takes a dict of methods and regular expressions:

omero config set omero.web.public.url_method_filter '{"POST": "^/webgateway/table/\d+/slice/", "HEAD": "/some-other-regex/"}'

Is that going too far?

[knabar]

Suggestion to bundle all read (GET, OPTIONS, HEAD) and all write (POST, PUT, DELETE, PATCH) requests together in two settings

[will-moore]

I think we can leave the omero.web.public.get_only setting named the same, since renaming is probably more trouble than it's worth. But this would also allow OPTIONS and HEAD requests.

For the new setting, I wonder if we should simply allow ALL http methods for the specified URLs?
Currently, if there's a URL where you want to allow GET and POST, you have to add that URL to both the omero.web.public.url_filter AND the new filter.
Can we imagine a scenario where you'd want to allow POST but not allow GET for a URL?

If not, then a new setting could be named omero.web.public.url_filter_all. This is a nicer name than omero.web.public.url_filter_unsafe, which would be the correct term for methods that are not "safe" https://developer.mozilla.org/en-US/docs/Glossary/Safe/HTTP

[sbesson]

For the new setting, I wonder if we should simply allow ALL http methods for the specified URLs?
Currently, if there's a URL where you want to allow GET and POST, you have to add that URL to both the omero.web.public.url_filter AND the new filter.
Can we imagine a scenario where you'd want to allow POST but not allow GET for a URL?

I don't have any scenario in mind but it would be useful for the sake of this conversation to compile the list of non-GET endpoints available via OMERO.web.

I think some level of redundancy/consistency between these settings will be necessary. For instance, if you decide to block some URLs in omero.web.public.url_filter as described in https://omero.readthedocs.io/en/stable/sysadmins/public.html but declare them in the new settings, what is the expected behavior?

I think we can leave the omero.web.public.get_only setting named the same, since renaming is probably more trouble than it's worth. But this would also allow OPTIONS and HEAD requests.

In all cases, what is proposed here is either deprecating or overriding the behavior of omero.web.public.get_only and we should also update the documentation of this property accordingly.