How to deploy a kernel bigger than 24kB

By Pierre-Louis Dubois

If you started developing some kernels as explain in the previous blog post, you may reach the size limit of 24kB and have been facing a wall: your Tezos operation is too big to be executed. In this blog post, we will demonstrate how to deploy a kernel that is bigger than 24kB.

There are four steps to achieve it:

  • reducing the size of your kernel,
  • splitting your kernel into chunks of 4kB,
  • setting up the kernel installer developed by the Tezos core developer,
  • deploying the installer and your kernel

Prerequisites :pencil:

In this tutorial, I will assume you have read the previous tutorial and have all the prerequisites of it.

To sum up, you must know how to:

Let’s reduce the size of your kernel!

This step is not required, but highly recommended. There is a tool named wasm-strip that can drastically reduce the size of your kernel.

In this kernel.wasm example, assume its size is 2MB and store in the current directory.

$ wasm-strip kernel.wasm

Be careful, the wasm-strip command does not create a new file but replaces the original one.

Just after this command, your kernel should be smaller (32kB). You can verify the size of your kernel with ls:

ls -lh

If your kernel is smaller than 24kB, you are ready to go and you can originate your kernel as a normal operation. Otherwise, you should continue reading this post.

Let’s split your kernel

In the previous post, we have learned about the durable storage and what it is. Interestingly some files of the durable storage are also used for the correct execution of the kernel. One file that is important in our case /kernel/boot.wasm, this file contains the Wasm file of your kernel. If you update this file, your kernel will be updated.

That’s exactly what the kernel installer does. It’s a kernel that will read some data from the data reveal directory and update the /kernel/boot.wasm file.

To put data in the reveal data directory, the data must have a maximum size of 4kB, so that the refutation game can still work.

In this section, we will see how to split your kernel into many 4kB chunks. To split your kernel, we will use the Data Availability Comitee (DAC) with the legacy execution mode. The purpose of the DAC is to store data that is bigger than 4kB and make them available to the kernel into chunks of 4kB. To store them, the DAC will split the data into chunks of 4kB and then the kernel will have to merge these chunks to get back its data.

The legacy mode of the DAC is a HTTP server which accept data, split them into chunks, save them to a directory, and gives you a reveal hash.

If you have the Tezos source compiled, you will find the octez-dac-node binary in the Tezos root directory.

Let’s configure the DAC before using it:

$ mkdir dac
$ octez-dac-node configure as legacy with threshold 0 and data availability committee members --reveal-data-dir dac # The directory where your splitted kernel will be

Run the DAC node by running this command:

$ octez-dac-node run

The DAC will expose an endpoint http://localhost:10832/store_preimage to split bytes and store them as files under the reveal data directory.

When we have the DAC started, next step is to submit some bytes to it. The DAC only accepts bytes as hexadecimal, but currently, our kernel.wasm is not a hexadecimal value. We can use the command xxd to convert its format:

$ KERNEL=$(xxd -ps -c 0 kernel.wasm | tr -d '\n')

If you display this variable you should see a hexadecimal expression.

echo "{\"payload\": \"$KERNEL\", \"pagination_scheme\":\"Merkle_tree_V0\"}" | curl --silent --header "Content-Type: application/json" -X POST -d @- http://localhost:10832/store_preimage
> { "root_hash": "000389b989126c02e8c87fa101c4ccfd476a01896e43aa5206c94da9c7776d834d", "external_message": "..."}

With the command above, the DAC will return a root_hash. This root_hash will be used later, please save it. We have used the DAC to split our data into many 4kB chunks, converted it into a Merkle tree, and have the root_hash. You can stop the DAC execution and move to the next step.

If you want to learn more about this tutorial, please read our blog post on Marigold website :point_right: How to deploy a kernel bigger than 24kB

3 Likes

Great, thank you for this guide! I will try it myself and report back :slight_smile: looking for more!