-
-
Notifications
You must be signed in to change notification settings - Fork 32.9k
benchmark: calibrate util.* #59655
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
benchmark: calibrate util.* #59655
Conversation
Review requested:
|
I see some benchmarks using syntax like 1e3 and others using 1000. Which one is preferable, or does it not matter? |
According to nodejs/performance#186 this benchmark takes 42 minutes to run a single run. So using calibrate-n script it suggests reduce it from 1e6 to 1e3 this should improve it.
7c95d23
to
5cd6f8a
Compare
According to nodejs/performance#186 this benchmark takes 1 minute to run a single run. So using calibrate-n script it suggests reduce it from 1e5 to 1e3
1e3 is shorter than 1000, thats all. |
dbc113d
to
3bfb15a
Compare
According to nodejs/performance#186 this benchmark takes 2 minute to run a single run. So using calibrate-n script it suggests reduce it from 1e6 to 1e2
According to nodejs/performance#186 this benchmark takes 5.5 minutes to run a single run. So using calibrate-n script it suggests reduce it from 1e6 to 1e2
According to nodejs/performance#186 this benchmark takes 1.3 minutes to run a single run. So using calibrate-n script it suggests reduce it from 5e3 to 10
According to nodejs/performance#186 this benchmark takes 0.47 minute to run a single run. So using calibrate-n script it suggests reduce it from 1e5 to 1e2
According to nodejs/performance#186 this benchmark takes 5.14 minutes to run a single run. So using calibrate-n script it suggests reduce it from 8e4 to 1e3
According to nodejs/performance#186 this benchmark takes 0.34 minute to run a single run. So using calibrate-n script it suggests reduce it from 3e4 to 10
According to nodejs/performance#186 this benchmark takes 0.04 minute to run a single run. So using calibrate-n script it suggests reduce it from 1e5 to 100
According to nodejs/performance#186 this benchmark takes 2.12 minutes to run a single run. So using calibrate-n script it suggests reduce it from 5e6 to 1e3
According to nodejs/performance#186 this benchmark takes 0.48 minutes to run a single run. So using calibrate-n script it suggests reduce it from 1e3 to 1e2
According to nodejs/performance#186 this benchmark takes 0.28 minute to run a single run. So using calibrate-n script it suggests reduce it from 1e6 to 1e3
3bfb15a
to
96f2e16
Compare
@brunocroh can you share the machine you used to run If your machine is at least twice as good as the one I mentioned, it might get a different CV% on these machines. If that's the case, I suggest you increase by one |
Nice, I ran it on a Mac M2 with 12 CPU cores and 16 GB of RAM. |
I set up a 4 vCPU, 16 GB dedicated machine to test these benchmarks again, and I reran the ones that took more than 5 minutes (text-encoder, inspect and get-callsite) to see if we could reduce n somehow, but unfortunately I couldn’t find any window for it. So I think the best option is to close this PR without any changes for now. |
According to nodejs/performance#186
Update all benchmarks in the util namespace by setting the
n
value to the result fromcalibrate-n
.Util Calibration Benchmarks