Skip to main content

Archives

An IOTA archive instance stores the full IOTA transaction history since genesis in a database agnostic format. This includes information about transactions (with client authentication), effects, events, and checkpoints. As such, archival storage can be used for data auditing and for replaying historic transactions.

info

The current archival storage format doesn't include historic object versions.

As a full node operator, you can enable archival fallback for your full node by specifying the URL to upload archival data. To ensure a healthy network, we encourage the IOTA community to set up additional archives to ensure archival data availability across the network. In a typical configuration, an archive trails behind the latest checkpoint by approximately 10 minutes.

A Full Node that starts from scratch can replay (and thus re-verify) transactions that occurred since IOTA genesis from the given archive via configuring Archival Fallback in the fullnode.yaml configuration file to point to the S3 bucket that stores the archive.

An IOTA full node that fails to retrieve checkpoints from its peers via state sync protocol falls back to downloading the missing checkpoints from its pre-configured archive. This fallback enables a full node to catch up with the rest of the system regardless of the pruning policies of its peers.

IOTA full node key-value store backup​

To enable historic data queries for the IOTA full nodes that prune old transactional data, full node RPC implementation is configured to fallback for querying missing transactional data from a remote store.

If the information about the transaction digest, effects, events, or checkpoints is not available locally, a full node automatically retrieves the historical data from a cloud-based key-value store (currently managed by the IOTA Foundation). Note that the current key-value store implementation keeps historic transactional data only: we plan to provide support for a similar setup for retrieving the historic object versions in a future release.

An IOTA archive is a history of all transaction data on IOTA, trailing behind the latest checkpoint by 10 minutes. As an IOTA node operator, you can write this history to an object store like S3, GCS, or similar for safe keeping. Saving this archive data is considered a best practice because IOTA prunes transactions on full nodes to remove historical transactions and their effects. Peer nodes, for example, might not catch up with all transactions and effects through synchronization if they lag behind the current epoch by more than the latest few epochs. Instead of relying on synchronization, peer nodes can fallback to downloading this information from an archive.

IOTA Archival nodes (full nodes that write to an archive) don't store historical state on local storage and don't help query historical data. They serve the purpose of enabling peer nodes to catch up to the latest checkpoint and are useful for auditing and verifying the complete history of all transactions on the network.

Create an Archival node​

To start storing transaction history as an archive, you need to modify your node configuration. Open your fullnode.yaml file and add the following configuration.

state-archive-write-config:
  object-store-config:
    object-store: "S3"
    bucket: "<bucket_name>"
    aws-access-key-id: "<AWS_ACCESS_KEY_ID>"
    aws-secret-access-key: "<AWS_SECRET_ACCESS_KEY>"
    aws-region: "<aws_region>"
    object-store-connection-limit: 20
  concurrency: 5
  use-for-pruning-watermark: false
state-archive-read-config:
  - object-store-config:
      object-store: "S3"
      # Use the same bucket which is being used in `state-archive-write-config`
      bucket: "<bucket_name>"
      aws-access-key-id: "<AWS_ACCESS_KEY_ID>"
      aws-secret-access-key: "<AWS_SECRET_ACCESS_KEY>"
      aws-region: "<aws_region>"
      object-store-connection-limit: 20
    concurrency: 5
    # This should be set to true in this case. Setting this to true
    # would prevent pruning of local transaction data until it is archived
    # in the bucket
    use-for-pruning-watermark: true

Set up archival fallback​

To enable your node to fallback to an archive in case of lag, add this to your fullnode.yaml file:

state-archive-read-config:
  - object-store-config:
      object-store: "S3"
      # Use iota-testnet-archive for testnet 
      # Use iota-mainnet-archive for mainnet
      bucket: "iota-<testnet|mainnet>-archive"
      aws-access-key-id: "<AWS-ACCESS-KEY-ID>"
      aws-secret-access-key: "<AWS-SECRET-ACCESS-KEY>"
      aws-region: "<AWS-REGION>"
      object-store-connection-limit: 20
    # How many objects to read ahead when catching up  
    concurrency: 5
    # Whether to prune local state based on latest checkpoint in archive.
    # This should stay false for most use cases
    use-for-pruning-watermark: false