Optimising Archive Node Storage for Tezos by Tarides

@samoht thanks for the blog post, I have a few questions:

  1. when will this land in octez?
  2. I am looking for ways to improve performance and reduce cost of serving octez archive RPC data. The introduction of lower storage appears to make it practical to have a read-only lower storage that serves archive RPC up to a certain level. This would allow a single lower storage to be shared among many nodes, reducing cost. Could this be supported?
  3. what are you working on next? May I recommend support for warp-sync? This would allow a node to download its storage from other nodes with p2p, getting as quickly as possible to a state where it can validate new blocks, and then filling in backwards. This would eliminate the need for snapshot services. Some competing blockchains offer this.

Hey, thanks for your questions! I can try to answer briefly.

  1. This is planned to be release in octez v18. You can follow the integration on this MR
  2. The new design will indeed allow to store the lower and upper layer on different disks so in theory you could put the lower layer on a shared disk. However, the architecture you suggest is a bit tricky to support, as the current design is “single-writer, multiple-writes”, ie. only one node should be allowed to write in the lower, but at the same time, you want the upper layers to be in sync. I guess it’s possible to find a scheme to make this work but we’ll need to think a bit more about it. In any case, thanks for the idea, it’s an interesting scenario!
  3. Our next focus is on improving performance (in terms of disk space, and I/O access). We planning to start benchmarking and experimenting with various potential improvements very soon. I’ll add Warp-Sync-like to our todo list :slight_smile:
1 Like