Phase2: use WAL for log to boost log performance

[liuluheng]

Plan/Roadmap

3 stages

1. hack sqlite (wal/parser) and NPLRuntime to get the wal data(may be frame) as the raft log.(35 days)

   • wal_page_hook. actordb uses tcp socket to get the wal data, we can try to use stack in NPL to get the data.
   • checkponit hook.
   • options that enable these hooks to make sqlite independent. This is at database level, may use PRAGMA or modify NPLRuntime.
   • this stage may works in NPLRuntime repo.

2. reorganize the Raft implementation to adapt to the wal, include deal with the sqlite checkpoint.(40 days)

   • replicate the wal data, may reconsider RPC.
   • deal with term and logIndex
   • deal with checkpoint. like packLog in the current implementation.
   • much work in this stage

3. test the throuput and fix.(15 days)