Block Finalization Rate

What are Block Finalization Rate alerts?

The block finalization rate is defined as the number of blocks that your node has finalized divided by the number of blocks that your node has proposed in a given time period. Ideally, all blocks that your node proposes will become finalized and added to the Flow blockchain. Those that aren’t finalized become ‘orphan’ blocks and never make it to the chain. In the future, if too many of your proposed blocks get orphaned, you could be liable to slashing.

A proposed block can be finalized when the leader node receives votes for the proposed block from at least 66% of the participating nodes, and 3 further ‘ProtoBlocks’ are also built on top of it. You can read more about the proposing and finalizing of blocks in Flow’s technical papers found here, in particular section 4 (Block Formation) of technical paper 2.

In the future, we will also be monitoring whether you propose a block when you are the leader node, if you fail to do so, you could be liable to slashing.

How are Block Finalization Rates triggered?

Every thirty minutes we calculate the block finalization rate for your node. We count the number of OnFinalizedBlock log events and divide this by the count of OnProposingBlock log events. If this rate is lower than 90% then we trigger a block finalization rate alert.

What notifications will I receive?

You will receive a notification when your node’s finalization rate drops below 90% for a given thirty-minute period, it will remain firing as long as the rate for follow-on periods remains below 90%. If the rate goes back above 90%, you will receive a clearing alert.

Note: You may need to wait until the next thirty-minute checkpoint to see this alert.

How do I resolve it?

The process of proposing and finalizing a block is both CPU-heavy and requires both I/O throughput and low latency on the networking side.

The incorporation of votes received from other participating nodes into an ‘Aggregated Consensus Signature’ in order to build the block is very CPU-intensive. A lot of orphaned blocks occur when a node can’t process the work required to build a block within the timeframe of the Flow protocol. This process takes place over a very short period of time (~200-400ms), so our node agent’s native CPU monitoring may not always capture these ‘spikes’ in CPU usage. In the first instance of this alert firing, unless something else is plainly obvious, we suggest you increase the CPU capacity of your machine.

The act of broadcasting a proposed block and receiving returning block votes is intuitively an I/O bounded procedure which is also impacted by network latency, it may be the case that your node’s internet connectivity is insufficient to support the finalization of proposed blocks.

If efforts to remedy both of the above issues don’t result in a sufficiently uplifted block finalization rate, we recommend you run through our standard checks:

• Check that the clock of your node has not drifted.

• Check that your node has not fallen behind the rest of the network.

• Check that key system-level metrics (e.g., CPU usage, RAM usage, Disk Space) of your node exhibit normal behavior.

• Check that your node is configured to receive requests and respond to other nodes on the network.

• Check that your node meets the minimum requirements for hardware outlined here.

• Check that your node meets the minimum requirements for networking outlined here.