Improve Library Loading of many DYFs #16255
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
If you install the top 10 packages from the package manager, you'll end up with over 1300 DYFs in total. Every single one of those gets loaded in memory and processed not once but twice when reading its
CustomNodeInfo
. That can be avoided completely by creating a file stream and reading the json content line by line. The DYNs are usually pretty small in size, but when there's thousands of them, this can quickly add up. The optimization proposed here is very similar to the one in #15759 and brings the exact same benefits:Current performance:
A lot of time is spent on reading and parsing the JSON data.
Proposed performance:
By streaming the files, we can improve the performance of the DYF scan function more than twice.
Declarations
Check these if you believe they are true
*.resx
filesRelease Notes
(FILL ME IN) Brief description of the fix / enhancement. Use N/A to indicate that the changes in this pull request do not apply to Release Notes. Mandatory section
Reviewers
(FILL ME IN) Reviewer 1 (If possible, assign the Reviewer for the PR)
(FILL ME IN, optional) Any additional notes to reviewers or testers.
FYIs
(FILL ME IN, Optional) Names of anyone else you wish to be notified of