Export
Export data to SST
This document describes how to use BR Raw KV backup capability to export KV data to remote storages as SST files.
Basic Usage
br backup raw --pd ⟨pd address⟩ \
-s ⟨storage url⟩ \
--start ⟨start key⟩ \
--end ⟨end key⟩ \
--format ⟨key format⟩ \
--ratelimit ⟨in MiB/s⟩
This will export all KV data in the range [start key, end key)
to the specified storage in the format of SST.
Supported storage
The storage URL support following schemes:
Service | Scheme | Example |
---|---|---|
Local filesystem, distributed on every node | local | local:///path/to/dest/ |
Hadoop HDFS and other compatible services | hdfs | hdfs:///prefix/of/dest/ |
Amazon S3 and other compatible services | s3 | s3://bucket-name/prefix/of/dest/ |
GCS | gcs, gs | gcs://bucket-name/prefix/of/dest/ |
Write to nowhere(for benchmark only) | noop | noop:// |
S3 and GCS can be configured using URL and command line parameters, see the BR documentation External Storage for more information.
HDFS configuration
To use HDFS storage, Apache Hadoop or compatible client should be installed and currectly configured on all BR and TiKV machines. The bin/hdfs
binary in hadoop installation will be used by BR and TiKV.
Various configuration should be provided for HDFS storage to work, see the following table.
Component | Configuration | Environment variable | Configuration file item |
---|---|---|---|
BR | hadoop installation directory | HADOOP_HOME | (None) |
TiKV | hadoop installation directory | HADOOP_HOME | backup.hadoop.home |
TiKV | linux user to use when calling hadoop | HADOOP_LINUX_USER | backup.hadoop.linux-user |
For TiKV, the configuration file have higher priority than environment variables.
Parse SST file
Java Client
The exported SST file can be parsed using Java Client.