Fast Recovery From App Hash Mismatch

Context

cronosd rollback provide a way to rollback and recover from app-hash mismatch failure, but it takes hours to run, here is a faster way to do it.

Fast Rollback

NOTICE: python-iavl only support rocksdb for now.

1. Disable iavl fastnode fastnode reindex itself is slow enough, to make it faster, disable it. Set iavl-disable-fastnode = true in app.toml
2. Rollback tendermint state

   $ tendermint rollback --home /chain/.cronosd/
   

   Using the standalone tendermint cli, could be installed with:

   $ go install github.com/tendermint/tendermint/cmd/[email protected]
   

3. Fast rollback app state, replace $VER to the version number shown in the above command.

   $ nix run github:yihuang/python-iavl/v0.0.1 -- fast-rollback --db /path/to/data/application.db --target $VER