plone.api.content.find index validation is not appropriate in some implementations

[ewohnlich]

https://github.com/plone/plone.api/blob/2.5.1/src/plone/api/content.py#L706 This section attempts to validate the query by making sure at least one ZCatalog index is present. However, some implementations remove the ZCatalog index in cases where querying on that index is only meant to be sent to a third party service like Elasticsearch. For example, wildcard.hps sends portal.portal_catalog(SearchableText="foo") to ES and encourages you to remove the SearchableText index from the ZCatalog as it would only take up space and increase index time.

But if that index is removed, api.content.find(SearchableText="foo") always returns an empty list.

What is the best solution here? Maybe an environment variable that can be set to skip the validate indexes step? Personally I do not see the benefit in the validation here in any case. If I call api.content.find() I would expect to get all content on the site, just as if I called portal.portal_catalog().

[davisagli]

The validation is useful in case a developer puts a typo in the name of an index. For example if the developer wrote api.content.find(SerchableText="foo"), it's better to raise an exception (because they meant SearchableText) rather than silently return all content.

The ZCatalog IPluggableIndex interface, which is implemented by indexes, has a method to get the query names that are handled by that index: getIndexQueryNames. This validation should collect the result of that from all indexes, and validate the query against that list, rather than against the list of index ids.

[ewohnlich]

In the case of a developer typo it doesn't currently raise an exception though, it just silently returns an empty list.

I don't follow you on IPluggableIndex, unfortunately. Or perhaps I wasn't clear on what wildcard.hps is doing, as it's a bit odd - the SearchableText index (for example) is completely removed from the catalog. There is no index to call either getIndexQueryNames or getId. wildcard.hps makes a bunch of monkey patches so that most of the Plone catalog API still works, recognize that these are special ES-only "indexes", and make a request to ES instead.

I guess it's probably best for wildcard.hps to just patch the entire api.content.find function, but I would have liked to be a bit more surgical.

[davisagli]

In the case of a developer typo it doesn't currently raise an exception though, it just silently returns an empty list.

Oh, I see. That's dumb.

I was assuming that wildcard.hps added its own fake index to the catalog Indexes, which would handle queries that need to be dispatched to ES. If that's not the case and it's just monkeypatching specific query functions, then yeah, my suggestion to use getIndexQueryNames doesn't help.