Node Agent

The Metrika Node Agent (“Agent”) is open source software (available on GitHub here) that runs on a Node Operator's host and powers Metrika’s Node Monitoring capabilities. The agent works out-of-the-box with zero configuration required to install and get started.

The Agent runs on modern GNU/Linux systems that utilizes systemd, and is capable of automatically detecting nodes present on the system, even when containerized (docker). Node Operators can install and configure the agent using configuration management tools like Chef, Puppet, or Ansible.

The Agent collects up to 75 metrics:

• System metrics from hosts

• Prometheus metrics & logs from Flow nodes

This data is sent to the Metrika Platform where it powers a unified view into System, Node & Network health. Providing Node Operators with an end-to-end view into node operations.

This page provides more information about the Agent and how you can use it to send data to the Metrika platform.

Agent Overhead

An example of the Agent resource consumption is below.

• Agent Test Version: 0.7.3

• CPU: ~2% of the CPU used on average *

• Memory: ~25MB of RAM used (RSS memory) **

• Network Bandwidth: ~40 KB/s ▼ | 100 KB/s ▲ *

• Disk:

  • Linux standalone binary (amd64): 15MB

*CPU utilization and Network bandwidth may vary depending on the blockchain node type being monitored and rate of events produced.

** The agent will additionally buffer up to 50MB of data in memory when there is an issue with the connection to the Metrika platform.

Prerequisites

• Operating System: GNU/Linux with systemd

• CPU Architecture: x86_64

• Linux Kernel: Version 2.6.23 or later

• Command Line Tool: cURL (only required if the agent is running as a standalone binary)

• Flow Client Name: flow-go

• Flow Log Level: Info

  • See the Flow knowledge base article here to set loglevel to info

  • See log rotation section below for more information on managing disk space utilization

• Permissions: The Metrika Agent only needs permissions to the blockchain node configuration and log files.

  • It does not need read access to privileged files, e.g. caches, files containing private cryptographic tokens, keys, or other information that might be deemed confidential

There may be additional prerequisites if the node or agent is containerized. Please see the sections below for more information.

Node Software Containerized by Docker

Since the node software is containerized, we assume that Docker runtime is already available on the host OS. In cases when the agent is run by Docker, Docker runtime doesn’t pose an additional requirement.

Additional Prerequisites

• Docker API: v1.41 support

• Permissions: The Agent needs access to certain read operations of the docker daemon such as container ls, logs, & event. To manage these operations, you may choose to deploy the agent using a Docker reverse proxy. Included below are the prerequisites based on whether or not a proxy is being used:

  • Running the agent as a standalone binary with a proxy (recommended):

    • Run a reverse proxy with access to Docker daemon’s socket. Instructions about a proposed setup based on Caddy can be found here.

  • Running the agent as a standalone binary without a proxy:

    • Add the Agent to the docker group on the host so it can read agent logs & configurations from containerized node clients.

      • Please note, this implicitly gives the Metrika Agent permissions that are tantamount to having root permissions on the host.

  • Running the agent as a Docker container with a proxy:

    • Run a reverse proxy** **with access to Docker daemon’s socket. Instructions about a proposed setup based on Caddy can be found here.

  • Running the agent as a Docker container without a proxy:

    • No additional prerequisites

Node Software as Standalone Binary

Additional Prerequisites

• Running the agent as a standalone binary without a proxy:

  • No additional prerequisites

• Running the agent as a Docker container without a proxy:

  • Docker runtime v20.10.0

Installation

In the Metrika Platform, go to the Install Agents page by clicking on Install Agents in the sidebar.

To install the Agent, choose the one-line install command that aligns with your node, agent & proxy combination.

Included below is an example of the one-line install command for a Node & Agent running as a standalone binary:

MA_PLATFORM={platform_endpoint} MA_BLOCKCHAIN=flow MA_API_KEY={api_key} bash -c "$(curl -L https://raw.githubusercontent.com/Metrika-Inc/agent/main/install.sh)"

Please reference the GitHub page here for the most up-to-date installation instructions for your node, agent & proxy combination.

Installation Flags

• There are several other commands available to manage the installation of your Metrika Node Agent. To run these commands, simply enter the installation 1-liner with one of the following:

  • --upgrade Upgrade the Metrika Node agent to the latest version

  • --reinstall Reinstall / refresh the Metrika Agent installation

  • --uninstall Stop and remove the Metrika Agent

  • --purge Stop and remove the Metrika Agent, including any agent configuration/data.

  • --no-docker-grp Do NOT add to the system docker group (requires docker proxy for containerized nodes)

  • --no-systemd-journal-grp Do NOT add to the systemd-journal group. Use this to disable systemd node discovery

Configuration & Node Discovery

More information on how the agent loads its configuration and detects a blockchain node can be found on the GitHub page here.

Validation

• There are 3 validations to ensure the Agent is running and sending data to the Metrika Platform

  • Agent Installed & Running

    • After running the one-line install command, you will receive a success message in the terminal informing you the Agent is installed and running

  • Node Recognized By Platform

    • To validate the node has been recognized by the Metrika Platform, navigate to the Node Overview Dashboard. In the Node Overview table, you’ll should see the recently added nodes.

  • Agent Sending Data to the Metrika Platform

    • Once the data has been received and processed, the columns in the Node Overview table will be populated, the node will be tagged as New (for 24 hours) and the Node Details view will be accessible by clicking on the Host Name

    • In the Node Details view, all of the statistics and charts should be populated with data

Troubleshooting

If you're experiencing issues with the Agent, tips on troubleshooting the installation can be found on the GitHub page here.

Monitoring Agent Performance

• There is instrumentation in-place to monitor agent performance. The metrics are exposed under port :9999 by default

  • Note - This port is configurable through the runtime.metrics_addr parameter in the agent configuration file (normally located under /etc/metrikad/configs/agent.yml). The Agent will need to be restarted for changes to take effect.

• To view these metrics, run the following in your browser on the machine where the agent is installed http://localhost:9999/metrics

• The raw metrics can be continuously collected from this endpoint and sent to your monitoring system.

Network Support

The Agent and Metrika Platform support the following Flow networks: Mainnet, Testnet, Canarynet & Localnet. The same one liner and API key can be used across each network. To learn more about accessing and exploring data between networks on the Metrika Platform, visit the Knowledge Base page here.

Security

Built in line with industry-wide standards and processes to keep customer data confidential. We take a proactive approach to customer security and are committed to keeping your data safe:

• Open Source: The Agent code is publicly accessible and available to anyone who wants to review or contribute to it

• Audited: Regularly audited by independent security auditors. Results are made publicly available.

• System Dependencies: The Agent is written in Go and distributed as a statically linked binary. There are no additional runtime dependencies on local systems

• Third Parties: Best practices during development designed to minimize the use (and risk) of 3rd party packages

• Data Access: If needed, resource limits can be set using Linux cgroups. The Agent has limited disk access and restricted access to node logs and configuration files

• Data Customization: Configuration options to control and customize the kinds of data the Agent parses and transmits

• Encryption: Data is secured in transit and protected using Transport Layer Security (TLS) v1.3

• Authentication: The Metrika Platform currently supports Google & Github’s OpenID services

Log Rotation

To avoid any disk space utilization issues with Flow’s info logs, we recommend:

1. Setting up log rotation for docker daemon logs (example for json-file logging driver here)

2. If possible, mount /var/lib/docker to a separate disk

Uninstalling

To uninstall the agent, follow the 2-step process below:

1. Based on the data you want to preserve, run the 1-liner with one of the following commands:

   • --uninstall Stop and remove the Metrika Agent.

   • --purge Stop and remove the Metrika Agent, including any agent configuration/data.

2. Navigate to the Node Overview table on the Metrika Platform, find the relevant node(s), and click the Remove option in the menu dropdown