BM-1843: add expected probability for secondary fulfillment, calculate and convert total reward and prioritize#1633
Conversation
…dated for too long
…h-and-zkc-dynamically-priced-by-usd
…h-and-zkc-dynamically-priced-by-usd
…h-and-zkc-dynamically-priced-by-usd
…-expected-value-of-winning-proof-race-for-secondary
…ent all provers racing for the same orders
capossele
left a comment
There was a problem hiding this comment.
Approved, just noting that the breaking config change should be called out in release notes.
willpote
left a comment
There was a problem hiding this comment.
This looks good!
One thought is that we are making a few number decisions based on intuition in this change, specifically 50% default and the 20%-100% adjustment factor.
I think this is good as getting something to prod and observing how it performs will ultimately give us the best data, but I'm not confident we have the logs/metrics to evaluate it.
I think we want a way to see how secondary fulfillment orders are being ranked/adjusted, just so that we can get a sense if 50% is too aggressive/not aggressive enough, or that 20% penalty isn't too steep etc.
Tricky thing is being able to do this without spamming the logs, will leave it to you to come up with something clever :D
| @@ -1,7 +1,7 @@ | |||
| [market] | |||
| min_mcycle_price = "0.000000025" | |||
| expected_probability_win_secondary_fulfillment = 100 # assume we win all secondary fulfillments -> min_mcycle_price is used to judge whether to accept a request or not | |||
There was a problem hiding this comment.
I actually think we should leave this at the default for 'nightly'. Intention with nightly is that it runs a very default/vanilla config, similar to what a new prover would run if just using the defaults, so we can test how the defaults perform in the wild
Aligned to make it 100 for the release cluster though!
| @@ -1,7 +1,7 @@ | |||
| [market] | |||
| min_mcycle_price = "0.000000025" | |||
| expected_probability_win_secondary_fulfillment = 100 # assume we win all secondary fulfillments -> min_mcycle_price is used to judge whether to accept a request or not | |||
There was a problem hiding this comment.
Is it possible to go > 100? With this cluster, given its intention is to backstop, it would be good to have it always prioritize secondaries asap
…e and convert total reward and prioritize (#1633) - Adds a configurable `expected_probability_win_secondary_fulfillment parameter` (0-any), code default 50%) that scales the expected reward of secondary (lock-expired) orders by the probability of winning the proof race - Removes the redundant `min_mcycle_price_collateral_token` config. `min_mcycle_price` is now dynamically converted to ZKC via the price oracle for secondary order evaluation - Unifies primary (lock and fulfill) and secondary order prioritization into a single sort by converting all (discounted) rewards to ETH, so high-value secondary orders can rank above low-value primary orders - Applies a random factor (0.2x–1.0x) to secondary order sort keys to spread provers across different orders and prevent all provers starting to compete in the same proof race if there's a secondary order with a high collateral value - Internal/production provers use `expected_probability_win_secondary_fulfillment = 100` (no discount), since we want aggressive secondary fulfillment. The broker template and `generate-config` command recommends 50% for external provers.
expected_probability_win_secondary_fulfillment parameter(0-any), code default 50%) that scales the expected reward of secondary (lock-expired) orders by the probability of winning the proof racemin_mcycle_price_collateral_tokenconfig.min_mcycle_priceis now dynamically converted to ZKC via the price oracle for secondary order evaluationexpected_probability_win_secondary_fulfillment = 100(no discount), since we want aggressive secondary fulfillment. The broker template andgenerate-configcommand recommends 50% for external provers.