iron-list (v2) crashes chrome (out-of-memory exception) when working with large lists

[gopherkhan]

Description

I'm using iron-list (v2) to render lists of log data over time. Depending on how noisy the entity being monitored is, the list can grow quite large.

I've found after supplying two lists of 30k+ elements to the iron list, the browser crashes.

This is likely due to the list's use of array-splice.html, which uses a Levenshtein edit distance function (calcEditDistances) to determine which elements need replacing.

The function creates an NxN array as it workspace, which for a 30,000 element list becomes a 900,000,000 entity grid, and consumes 1gig of heap.

Relevant code in array-splice.html:

function calcEditDistances(current, currentStart, currentEnd,
                              old, oldStart, oldEnd) {
    // "Deletion" columns
    let rowCount = oldEnd - oldStart + 1;
    let columnCount = currentEnd - currentStart + 1;
    let distances = new Array(rowCount);

    // "Addition" rows. Initialize null column.
    for (let i = 0; i < rowCount; i++) { // crashes reliably here
      distances[i] = new Array(columnCount);
      distances[i][0] = i;
    }

Expected outcome

Since the list was working in v1.x, the iron list should still take whatever I throw at it.

Actual outcome

Browser crash.

Live Demo

I'm not at liberty to share

Steps to reproduce

1. Put a sized iron-list element in the page, with a really simple template, a la:

iron-list {
        overflow: auto;
        height: calc(100vh - 286px);
      }
...
<iron-list id="ironList" items="[[data]]">
      <template>
        <div style="height: 24px">Hey hey hey!</div>
      </template>

Note: You can omit any template bindings in the row div.

2. Add a button to supply 30k elements on click
3. Click the button twice

Browsers Affected

• Chrome
• Firefox
• Safari 9
• Safari 8
• Safari 7
• Edge
• IE 11
• IE 10

[gopherkhan]

The list will reliably crash with 16k elements.

[gopherkhan]

My current workaround is to check if an incoming list is large (> 15k) and then blank out the list before supplying the new data.

[Westbrook]

I don't seem to suffer from this via this JSBin: https://jsbin.com/lolekifale/4/edit?html,console,output It splices 50K items onto the list with no problem as many times as I like. Have I missed something in your issue description?

Is it possible it has something to do with the elements you're posting into the iron-list? Or that for some reason it's not respecting the height: calc(100vh - 286px); declaration?

[gopherkhan]

It could be due to the list elements being serialized by a protobuf. I'll test again and report back.

[gopherkhan]

Okay, so the object size shouldn't matter (mine are about 230Bytes)

In your jsbin, change the click listener from:
list.splice('items', 49999, 0, ...items);

to

list.items = items;

jsbin will crash on click

[Westbrook]

Very interesting. I'd definitely say that's an issue to bring up with the main Polymer repo, though I'm not sure you'll get much of an answer as it seems like an issue that can be worked around by not doing that.

In the realm of iron-list it seems the fix for that is to instead do:

     list.splice('items', 0, 50000, ...items);

or

      list.items = [];
      list.items = items;

To achieve what you're noting here.

[gopherkhan]

Gotcha. In our app we are just assigning server-returned data into a reference that is bound to the list (via the template). You'd recommend not directly assigning the object in, but instead binding the list to a display array, and then using the splice syntax to clear and replace its content when the server-data comes in. Correct?

I think two fixes might be in order.

1. The iron-list doesn't bother calculate list differences if the list sizes are huge.
2. The polymer side fixes the Levenshtein edit distance to not explode with large array data.I can I'll file a bug for this part in the main polymer repo.

[Westbrook]

I'd suggest in that case you use the

  this.items = [];
  this.items = items;

approach as opposed to splice personally, seems like more computation work than it's worth, and it doesn't have any transpile artifacts. But, that's just me.

Good call on number 2 with the issue up to Polymer, that's most likely the same issue you'd need for number 1 as iron-list doesn't contain any of its own array management code.

[gopherkhan]

Awesome. I'll go with that for now. I filed a parallel bug here:
Polymer/polymer#5195