AttributeError: 'NoneType' object has no attribute 'element_identifier'

[bgruening]

Found this in our logs, on 24.0.

Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254906]: uvicorn.access INFO 2024-04-17 17:02:24,752 [pN:main.4,p:2379960,tN:MainThread] 138.51.65.224:0 - "GET /api/notifications/status?since=2024-04-17T15%3A02%3A19.366 HTTP/1.0" 200
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]: galaxy.web.framework.decorators ERROR 2024-04-17 17:02:24,754 [pN:main.4,p:2379949,tN:WSGI_1] Uncaught exception in exposed API method:
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]: Traceback (most recent call last):
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/web/framework/decorators.py", line 346, in decorator
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     rval = func(self, trans, *args, **kwargs)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/webapps/galaxy/api/tools.py", line 593, in create
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     return self.service._create(trans, payload, **kwd)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/webapps/galaxy/services/tools.py", line 171, in _create
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     vars = tool.handle_input(
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:            ^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/tools/__init__.py", line 1885, in handle_input
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     all_params, all_errors, rerun_remap_job_id, collection_info = self.expand_incoming(
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:                                                                   ^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/tools/__init__.py", line 1809, in expand_incoming
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     expanded_incomings, collection_info = expand_meta_parameters(trans, self, incoming)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/tools/parameters/meta.py", line 220, in expand_meta_parameters
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     expanded_incomings = permutations.expand_multi_inputs(incoming_template, classifier)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/util/permutations.py", line 36, in expand_multi_inputs
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     single_inputs, matched_multi_inputs, multiplied_multi_inputs = __split_inputs(inputs, classifier, key_filter)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/util/permutations.py", line 53, in __split_inputs
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     input_type, expanded_val = classifier(input_key)
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:                                ^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/tools/parameters/meta.py", line 204, in classifier
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     values = __expand_collection_parameter(
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:   File "/opt/galaxy/server/lib/galaxy/tools/parameters/meta.py", line 253, in __expand_collection_parameter
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     hda.element_identifier = element.element_identifier
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]:     ^^^^^^^^^^^^^^^^^^^^^^
Apr 17 17:02:24 sn06.galaxyproject.eu gunicorn[2254901]: AttributeError: 'NoneType' object has no attribute 'element_identifier'

[mvdbeek]

Can you check what elements you have in history_dataset_collection_association 2521822 ?

[mvdbeek]

(assuming this is https://sentry.galaxyproject.org/share/issue/778fc821c07643a6aa609316271f0a3a/)

[sanjaysrikakulam]

I am not sure what exactly I should look for, maybe can you give us an SQL query to run?

galaxy=> select * from history_dataset_collection_association where id=2521822;
   id    | collection_id | history_id |                     name                     | hid | visible | deleted | copied_from_history_dataset_collection_association_id | implicit_output_name | implicit_collection_jobs_id | job_id |        create_time         |        update_time
---------+---------------+------------+----------------------------------------------+-----+---------+---------+-------------------------------------------------------+----------------------+-----------------------------+--------+----------------------------+----------------------------
 2521822 |       8471925 |    3134291 | HISAT2 on collection 46: aligned reads (BAM) |  66 | t       | f       |                                                       | output_alignments    |                      745211 |        | 2024-04-15 14:59:11.610975 | 2024-04-15 14:59:11.610978
(1 row)

galaxy=> select * from dataset_collection where id=8471925;
   id    | collection_type |        create_time         |        update_time         | populated_state | populated_state_message | element_count
---------+-----------------+----------------------------+----------------------------+-----------------+-------------------------+---------------
 8471925 | list            | 2024-04-15 14:59:11.603993 | 2024-04-15 14:59:11.603997 | new             |                         |             2
(1 row)

[mvdbeek]

Thanks, that's all I need. I think populated_state = new is the likely problem here. Might be interesting to know why that collection hasn't been populated yet, but from the code side we shouldn't even consider these as possible inputs.

[mvdbeek]

Alright, I think the only way users can submit a collection like this is if they drag it from the history panel into the input field, which bypasses a lot of validation (a hack we should probably disallow by also validating dragged input ?). I see that the dataset collection matcher correctly filters out collections that are not populated, so #18023 is just going to raise an appropriate exception if a user tries to submit such a collection as input.

Again though, you probably want to figure out what happened to that collection.

[mvdbeek]

Should be fixed with #18023 ... we just can't work with collections in this state if we don't persist the request, which we're going to do in #17393, so that this will behave a little more like in workflows, where this works because we simply try again at the next scheduling iteration.

That's of course also not going to be the silver bullet, since the output of that job won't be created until the request can be processed, but I don't really see a way around this. It should be a fairly rare situation fortunately.

[bgruening]

Thanks a lot @mvdbeek !