You operate under the assumption that all inflation that doesn’t go to bakers is taken from them. There is no reason to think that. Inflation is taken from token holders. The reason bakers receive a reward is to help secure the network, that’s also something paid for by token holders. The fraction of the reward that bakers typically keep for themselves is related to the cost of capital and operations that they incur, not to the total amount of the block reward.
