Empty overflowing polymorphic caches

[tekknolagi]

Right now we have no notion of a megamorphic cache, so when a
polymorphic cache is filled, any new receiver LayoutIds fall off the end
into the slow full attribute lookup.

With this change, polymorphic cache updates will remove the
existing cache entries and add the new entry.

Add tests.

[tekknolagi]

I found this by running sudo bpftrace ./util/probes/ic-miss.bt, then running the bench_pickle benchmark. I noticed that the FOR_ITER_POLYMORPHIC cache miss rate was enormous. I added a print to see what the key was and what the cache values were and noticed that most of the time it was one key that was never in the values. I then realized we (maybe intentionally) never updated the polymorphic cache. I decided to measure this change (hopefully this would lead to a faster steady-state) and here we are.

[tekknolagi]

{
  "django_minimal_requests": {
    "benchmark": "django_minimal_requests",
    "cg_instructions before": 707812,
    "cg_instructions now": 706229,
    "cg_instructions ∆": "-0.2%",
    "interpreter_args": [],
    "interpreter_name": "pyro",
    "version before": "6d57ee64d871ce6d629e6475e7135ca5b17c3fef",
    "version now": "a22b1dcde16deb463afb6cc70d1741c0a7c67350"
  }
}

[tekknolagi]

TODO: Do we need to delete old dependency links? Can we even do that?

[tekknolagi]

Ah, the least recently filled is only the first entry one time. Might be worth figuring out another strategy.

[github-actions]

Summary

Metric	Average	Best	Worst	Notes
cg_instructions	-1.3%	deltablue -5.2%	bench_base64 0.0%	typically < 0.2% noise

Benchmark details

Base vs. New

benchmark	cg_instructions
2to3	-1.3%
bench_base64	0.0%
bench_compile	-5.0%
bench_pickle	-1.6%
deltablue	-5.2%
fannkuch	-0.0%
go	-1.1%
loadproperty	-0.0%
nbody	-0.0%
nqueens	-0.0%
pyflate	-2.1%
pystone	-0.0%
richards	-0.0%

CPython vs New

benchmark	cg_instructions
2to3	-9.5%
bench_base64	-38.6%
bench_compile	1058.0%
bench_pickle	-20.5%
deltablue	-66.3%
fannkuch	-1.7%
go	-64.3%
loadproperty	-75.1%
nbody	22.1%
nqueens	42.6%
pyflate	-32.7%
pystone	-71.8%
richards	-80.0%

Base

benchmark	cg_instructions
2to3	2,351,267,772
bench_base64	2,960,312,113
bench_compile	3,014,276,224
bench_pickle	3,349,512,536
deltablue	1,462,412,056
fannkuch	5,500,949,325
go	1,901,331,692
loadproperty	435,669,337
nbody	9,515,559,818
nqueens	3,156,649,921
pyflate	9,942,551,724
pystone	1,124,783,513
richards	966,161,795

New

benchmark	cg_instructions
2to3	2,319,702,304
bench_base64	2,960,314,854
bench_compile	2,865,014,408
bench_pickle	3,295,853,148
deltablue	1,386,361,201
fannkuch	5,500,931,392
go	1,880,840,426
loadproperty	435,656,394
nbody	9,515,544,854
nqueens	3,156,617,557
pyflate	9,734,615,525
pystone	1,124,762,949
richards	966,138,042

CPython

benchmark	cg_instructions
2to3	2,563,513,726
bench_base64	4,820,261,715
bench_compile	247,405,181
bench_pickle	4,145,253,004
deltablue	4,109,391,416
fannkuch	5,594,128,593
go	5,264,669,023
loadproperty	1,747,793,767
nbody	7,793,839,180
nqueens	2,213,204,953
pyflate	14,468,095,431
pystone	3,981,563,969
richards	4,821,538,938