Skip to content

[Bulk] Add _index, _id, status to ERROR object #10015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

[Bulk] Add _index, _id, status to ERROR object #10015

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3c46ea4
[Bulk] Add _index, _id, status to ERROR object
aswath86 Sep 13, 2023
cbc0a90
Merge remote-tracking branch 'origin/main' into main
aswath86 Sep 13, 2023
693cea5
Merge branch 'opensearch-project:main' into main
aswath86 Jun 28, 2024
140d25d
Merge branch 'opensearch-project:main' into main
aswath86 Jul 1, 2024
620165f
Merge branch 'opensearch-project:main' into main
aswath86 Jul 1, 2024
0a71128
Merge branch 'opensearch-project:main' into main
aswath86 Oct 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions server/src/main/java/org/opensearch/action/bulk/BulkItemResponse.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws
builder.field(_ID, failure.getId());
builder.field(STATUS, failure.getStatus().getStatus());
builder.startObject(ERROR);
builder.field(_INDEX, failure.getIndex());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add tests for this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding these new fields will add to the b/w usage for existing users who are not using filter path. I would suggest that we rather include this in the error reason in a way that this is backward compatible. You can also consider adding a new field that can be controlled by query parameter similar to what we have in _cat/nodes api where we can control which fields are returned.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default we should not be increasing the response size and it should be controlled by the user that they need the additional information that you have added here.

builder.field(_ID, failure.getId());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this always generated when the error is passed? What if the error was encountered even before the document id could be generated?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the behaviour would be the same as in for builder.field(_ID, failure.getId()); that is above line builder.startObject(ERROR);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, I see that users who are not providing filter_path, they will get the _id and _index field twice in case of errors and this adds additional payload by default. Wondering if there is a better way to solve this

Copy link
Member

@mgodwan mgodwan Jul 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if filter_path only contains error, how are users able to determine which document actually failed since successful docs won't return any response, and with auto-generated id, it becomes difficult for clients to know which document failed. (Applicable only for auto generated ids)

builder.field(STATUS, failure.getStatus().getStatus());
OpenSearchException.generateThrowableXContent(builder, params, failure.getCause());
builder.endObject();
}
Expand Down