Questions about Bootstrap Peers

[jcb]

In bin_node there are several default peers listed for clients to bootstrap their peer lists:

mainnet:

boot.tzbeta.net

delphinet:

      [ "delphinet.tezos.co.il";
        "delphinet.smartpy.io";
        "delphinet.kaml.fr";
        "13.53.41.201" ]

edonet:

      [ "51.75.246.56:9733";
        "edonet.tezos.co.il";
        "46.245.179.161:9733";
        "edonet.smartpy.io";
        "188.40.128.216:29732";
        "51.79.165.131";
        "edonet.boot.tezostaquito.io" ]

This article by Serokell on the TQTezos Medium provides a good summary of what these do:

Peer discovery

Peer discovery is the process of discovering other peers in a network. BitTorrent protocol use tracker servers to maintain a list of peers in a swarm. So, when you start to download a file using a torrent, the client can immediately get a list of peers from the tracker information in the torrent file.

Similarly, a Tezos node also needs to know where to start its connection to the P2P network. When you start a Tezos node, you can pass an explicit list of peers to the node via the --peer argument. For example, you can use the following foundation nodes to bootstrap your node:

dubnodes.tzbeta.net
franodes.tzbeta.net
sinnodes.tzbeta.net
nrtnodes.tzbeta.net
pdxnodes.tzbeta.net

But if this argument was not provided, the node will try to connect to boot.tzbeta.net. This is a domain name that has many nodes behind a load balancer.

Tezos node also has another technique to discover local peers, that is, peers in the local network. You can use the --discovery-addr option to provide the broadcast address of a network, and the node will send periodic UDP broadcast packets that contain information like peer id and the listening port. Any node that receives this can try to connect using the contained information and the source IP address in the UDP packets.

I have a few questions:

1. If mainnet has only one default bootstrap peer, boot.tzbeta.net, what happens in the event of:
   a. DDOS attack. Can an attacker stop new nodes from joining mainnet (without using --peer)? If so, how/what do we estimate is the cost per hour for an adversary to launch such an attack?
   b. An adversary doing e.g. social engineering attack and gaining root access to this box. Can they route all new nodes to a partitioned network they control?
   c. Same as b. except with with a compromise of the DNS records of tzbeta.net rather than the underlying box.
   d. Same as b except with a compromise of the hosting provider (unlikely, but it’s not paranoia if they really are out to get you).

2. The above article says that the backend of boot.tzbeta.net is a load balancer in front of a bunch of different peers. Can this backend be open-sourced (or is it already?) so that e.g. bakers can easily run their own bootstrap peers if need be?

3. Suppose that the answer to my question #2 is affirmative, and now many bakers are running their own bootstrap nodes. What are the consequences if a new nodes boots with a list of peers, some of which may be malicious? Is this an possible use case for a Web of Trust between the bootstrap peers?

4. Bitcoin previously used IRC to bootstrap its peer list, but this was dropped in favor of a DNS based solution because of Sybil attack surface. Tezos currently uses Proof of Work to prevent Sybil attacks in the peer to peer layer. Could a similar Proof of Work be used (perhaps in conjunction with Web of Trust or evidence of rolls) to have the bootstrap peers postulated in #3 (not ordinary peers), announce themselves on an IRC or Matrix channel?

[krixt]

1. boot.tzbeta.net resolves to 4 different IPs. A) Someone would have to DDOS them all simultanously. No, an attacker could not stop someone from joining mainnet because you can manually add peers to a running node. B) I guess this is possible.

2. How do you open-source a physical load balancer? Assuming it is software-based, the most popular load balancer is HAProxy and it’s been open source for decades.

3. The beauty behind blockchain is that every block includes the signature of the previous block. A malicious node could send invalid blocks to you, but you would just see them as invalid and discard. As you connect to other nodes via P2P, you’ll eventually get the right blocks.

4. A bootstrap peer is just an archive node. Anyone can start one of these and publish themselves as a bootstrapper. Any list posted anywhere is subject to the same issues you pointed out above.

[jcb]

1. What is the cost per hour to DDOS 4 different IPs simultaneously? Presumably it’s at most 4 times the cost of DDOSing one address? This site suggests order-of-magnitude $1000 per day.
2. Supposing boot.tzbeta.net becomes unavailable, how fast can the ecosystem deploy alternatives? If there is a potential future issue, can we anticipate it in the present by taking actions to reduce such deployment time? Perhaps all that would be required is a tutorial document: “Break Glass in case of DDOS: How to Replicate the Default Bootstrap Peer.”
3. Suppose you have a node which connects only to malicious peers and is thus isolated from the main honest network (i.e. an eclipse attack). What is Tezos’ exposure to eclipse attacks, and could a malicious boot.tzbeta.net perform one on a new node (which knows no peers)? I suspect eclipse attacks could potentially be more serious in Tezos given the Nothing-At-Stake-Problem and our use of over-the-air updates.
4. I don’t understand the comment about a bootstrap peer being an archive node. AFAIU, an archive node stores a full copy of the chain, whereas a bootstrap peer only has to maintain a current peer list. That is, can’t the bootstrap peer merely point to one or many archive nodes, rather than be one itself?

[krixt]

1. So, if it costs so much to DDOS, why would you do it to begin with?
2. You can bootstrap from any node, really. Every node shares it’s peers list with every other node to which it connects. This is P2P basics. So if boot.tzbeta is down, you can manually add any tezos peer to yours and when it connects, you’ll receive 50-300 other peers.
3. What do you gain here, as the attacker? Ok, so my node connected to a malicious network and I’m baking on that network. I get rewards, but then if I want to cash out, I can’t because my transactions won’t make it to the “good network”. Even if someone steals funds on the ‘bad network’, they only exist on that network and nowhere else. This bad network is essentially a separate chain/fork.
4. Yes, any node, whether archive, full, or rolling can function as a bootstrap node in this sense of just providing IPs to other P2P members.

[jcb]

1. Expensive? $1000 per day is very cheap to attack a network with a billion dollar market cap. Suppose for the sake of the argument that there is a viable attack based on DDOSing boot.tzbeta.net which causes significant price decline in Tezos. An attacker can profit by levering up and buying XTZBEAR or an equivalent instrument. That’s the simplest mechanism I can think of, but we should assume there are other ways to profit, particularly when targeted attacks on specific nodes are considered (more below).

2. I do understand that all you need to do to bootstrap is to pass the --peer <address> argument to your node. (There is also --discovery-addr, which is probably only situationally useful unless we start talking about things like https://www.zerotier.com/) . The question is simply where you get <address> from, and how you can trust it. Saying “every node shares its peer list with every other node” is true, but only for honest nodes. A malicious node is fully capable of sending you whatever peer-list it likes.

3. An eclipse attack doesn’t fork the chain (or at least, not necessarily). From the Heilman paper I linked to in my previous post:

In an eclipse attack, the attacker monopolizes all of the victim’s incoming and outgoing connections, thus isolating the victim from the rest of its peers in the network. The attacker can then filter the victim’s view of the blockchain, force the victim to waste compute power on obsolete views of the blockchain, or coopt the victim’s compute power for its own nefarious purposes.

This paper describes various attacks on Bitcoin, but there has also been work done on Ethereum vulnerabilities. Here’s a few examples of what an attacker might be able to do you if you’re eclipsed (i.e. your node connects to the rest of the network only through the attacker’s peers):

• The attacker can silently front run or delay all interactions between you and any public smart contract.
• The attacker can 0-confirmation double-spend to you by sending a transaction to your node, but not to the rest of the network.
• If you are a baker, the attacker can force you to to bake on top of a different history than the main network, i.e. they can enlist into supporting their fork.
• If the attacker controls bakers, they can perform an N-confirmation double-spend on you. This can have consequences outside the “bad chain”, if e.g. you execute a transaction on a separate network or IRL based on the Tezos blocks you see. It’s like a targeted malicious reorg.

I said “might be able”, because I am not entirely sure whether/how each of these hold in the context of Tezos specifically. I found some research on selfish-mining/baking in Tezos, which referred to this paper describing how eclipse attacks can be combined with a selfish-mining attack, but would love to be pointed at more specific research if it exists.

However, unless someone shows conclusively why Tezos is not vulnerable to certain consequences of eclipse attacks, we should prudentially assume it is vulnerable. To paraphrase an old saying: Attackers only have to win once. Tezos has to win always.

4. Any node can provide peerlists, but the whole premise of my inquiry here is that in practice all mainnet peer-list discovery goes through a single point of centralized infrastructure. We don’t think this infrastructure is compromised, but how do we really know?

The solution, in my opinion, is to figure out new bootstrapping mechanism where new nodes connect to many independently operated bootstrap servers. If you have multiple sources of truth for your initial peer-list, it becomes much more difficult for an attacker to compromise you during bootstrapping.

There are many different ways to achieve this, most of which are not mutually exclusive. My initial thought was a public IRC/Matrix channel (similar to the old Bitcoin approach) where bootstrap peers announce themselves with a PoW and perhaps web-of-trust endorsements, but this adds complexity, so perhaps someone else has a better idea. Or I could be totally off-base and there’s some good reason for the current boot.tzbeta.net bootstrapping to exist in the way it is, in which case I’m hoping someone can explain it.