Tezos Tarballs are live from Oxhead Alpha
Get them at xtz-shots.io. Read about them on our Medium post.
What is a Tezos Tarball?
A Tezos tarball is a an LZ4 and tarred node directory of our running Tezos nodes here at Oxhead Alpha. We run archive and rolling nodes for mainnet, hangzhounet, and ithacanet.
What do I do with it?
Syncing from a Tezos tarball will get your node (especially an archive node) synced much quicker than from a full or rolling snapshot.
A 12-hour old mainnet archive tarball will have your archive node fully synced in 40 minutes.
A 2-hour old mainnet rolling tarball can have your rolling node fully synced in 9 minutes.
Expect even less time for hangzhounet and ithacanet due to smaller chain size.
How it works
We take an EBS snapshot of an EC2 volume while the Tezos node is running.
This is equivalent to pulling the plug on the server, but the running node seems to handle it fine.
However this creates some problems on the storage occasionally-
Error One, Error Two, Error Three, Error Four, Error Five
We sanity check the storage by starting a tezos-node against it and wait for the RPC endpoint to go live. This sanity check is timed to two minutes, if it cant repair itself (one of the above errors occurs) in that time the storage is tossed and a new snapshot is taken and restored.
For a rolling tarball we export a traditional rolling Tezos snapshot from this validated storage with a headless node and then restore it to a new volume.
For both archive and rolling tarballs we selectively tarball and LZ4 (level 1 fastest and default) compress the validated storage minus peers.json and version.json.
Level 1 LZ4 compression adds a trivial amount of time to expansion versus raw tarball expansion but saves a considerable amount of storage space.
Garbage Collection
By periodically restarting a rolling node from rolling tarball, you are effectively garbage collecting the node. By preserving everything in the node folder and tossing out the rest of the storage one can keep storage needs to a minimum.
Restoring from a tarball makes it easier to leverage local instance storage in container orchestration environments such as tezos-k8s. This will allow temporary NVME storage directly attached to an EC2 instance to be used as chain storage. The storage is ephemeral, limited in quantity, but has the highest I/O of any storage. It can be tossed and resynced from a tarball at any time leading to pronounced performance increases.
Open Sourcing
We are working on open sourcing the code for this engine right now. We will release several helm charts in the tezos-k8s repository. The goal is that anyone can create their own tarballs from a node in their control on any cloud or local Kubernetes installation.
Disclaimer about trust.
By using our artifacts you’re trusting us as the source of your Tezos data.
Tezos snapshots are structured packages of Tezos data. When importing a Tezos snapshot, Octez rebuilds the context and verifies its soundness block by block. Tarballs in contrast are a raw representation of the node storage.
Restoring a node from a tarball is much faster than a Tezos snapshot, but this comes as the expense of safety. You must trust us as a reliable tarball source. Please consider your risk profile when opting for one or the other.
