Performance Benchmarking with TPC-C

On this page

This page shows you how to reproduce CockroachDB's TPC-C performance benchmarking results on commodity AWS hardware. Across all scales, CockroachDB can process tpmC (new order transactions per minute) at near maximum efficiency. Start by choosing the scale you're interested in:

Before you begin

Review TPC-C concepts

TPC-C provides the most realistic and objective measure for OLTP performance at various scale factors. Before you get started, consider reviewing what TPC-C is and how it is measured.

Step 1. Set up the environment

• Provision VMs
• Configure your network

Provision VMs

1. Create 4 VM instances, 3 for CockroachDB nodes and 1 for the TPC-C workload.

   • Create all instances in the same region and the same security group.
   • Use the c5d.4xlarge machine type.
   • Use local SSD instance store volumes. Local SSDs are low latency disks attached to each VM, which maximizes performance. This configuration best resembles what a bare metal deployment would look like, with machines directly connected to one physical disk each. We do not recommend using network-attached block storage.

2. Note the internal IP address of each instance. You'll need these addresses when starting the CockroachDB nodes.

Configure your network

CockroachDB requires TCP communication on two ports:

• 26257 for inter-node communication (i.e., working as a cluster) and for the TPC-C workload to connect to nodes
• 8080 for exposing your DB Console

Create inbound rules for your security group:

Inter-node and TPCC-to-node communication

DB Console

Step 2. Start CockroachDB

1. SSH to the first VM where you want to run a CockroachDB node.

2. Download the CockroachDB archive for Linux, extract the binary, and copy it into the PATH:

   $ curl https://binaries.cockroachdb.com/cockroach-v21.1.21.linux-amd64.tgz \
   | tar -xz
   

   $ cp -i cockroach-v21.1.21.linux-amd64/cockroach /usr/local/bin/
   

   If you get a permissions error, prefix the command with sudo.

3. Run the cockroach start command:

   $ cockroach start \
   --insecure \
   --advertise-addr=<node1 internal address> \
   --join=<node1 internal address>,<node2 internal address>,<node3 internal address> \
   --cache=.25 \
   --background
   

4. Repeat steps 1 - 3 for the other 2 VMs for CockroachDB nodes. Each time, be sure to adjust the --advertise-addr flag.

5. On any of the VMs with the cockroach binary, run the one-time cockroach init command to join the first nodes into a cluster:

   $ cockroach init --insecure --host=<address of any node on --join list>
   

Step 3. Import the TPC-C dataset

CockroachDB comes with a number of built-in workloads for simulating client traffic. This step features CockroachDB's version of the TPC-C workload.

1. SSH to the VM where you want to run TPC-C.

2. Download the CockroachDB archive for Linux, extract the binary, and copy it into the PATH:

   $ curl https://binaries.cockroachdb.com/cockroach-v21.1.21.linux-amd64.tgz \
   | tar -xz
   

   $ cp -i cockroach-v21.1.21.linux-amd64/cockroach /usr/local/bin/
   

   If you get a permissions error, prefix the command with sudo.

3. Import the TPC-C dataset:

   $ cockroach workload fixtures import tpcc \
   --warehouses=2500 \
   'postgres://root@<address of any CockroachDB node>:26257?sslmode=disable'
   

   This will load 200 GB of data for 2500 "warehouses". This can take a while to complete.

   You can monitor progress on the Jobs screen of the DB Console. Open the DB Console by pointing a browser to the address in the admin field in the standard output of any node on startup.

Step 4. Run the benchmark

1. Still on the same VM, create an addrs file containing connection strings to the 3 CockroachDB nodes:

   postgres://root@<node 1 internal address>:26257?sslmode=disable postgres://root@<node 2 internal address>:26257?sslmode=disable postgres://root@<node 3 internal address>:26257?sslmode=disable
   

2. Run TPC-C for 30 minutes:

   $ cockroach workload run tpcc \
   --warehouses=2500 \
   --ramp=1m \
   --duration=30m \
   $(cat addrs)
   

Step 5. Interpret the results

Once the workload has finished running, you will see a result similar to the following. The efficiency and latency can be combined to determine whether this was a passing run. You should expect to see an efficiency number above 95%, well above the required minimum of 85%, and p95 latencies well below the required maximum of 10 seconds.

_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
 1800.0s    31064.6  96.6%    107.4     88.1    243.3    302.0    402.7    973.1

See also

• Performance Overview

• Hardware

  CockroachDB works well on commodity hardware in public cloud, private cloud, on-prem, and hybrid environments. For hardware recommendations, see our Production Checklist.

  Also note that CockroachDB creates a yearly cloud report focused on evaluating hardware performance. For more information, see the 2020 Cloud Report.

• Performance Tuning

  For guidance on tuning a real workload's performance, see SQL Best Practices, and for guidance on techniques to minimize network latency in multi-region or global clusters, see Multi-Region Overview.

Was this helpful?