New in v2.0 The Node Map visualizes the geographical configuration of a multi-regional cluster by plotting the node localities on a world map. The Node Map also provides real-time cluster metrics, with the ability to drill down to individual nodes to monitor and troubleshoot the cluster health and performance.
This page walks you through the process of setting up and enabling the Node Map.
Set Up and Enable the Node Map
To enable the Node Map, you need to start the cluster with the correct --locality
flags and assign the latitudes and longitudes for each locality.
--locality
flags and all localities are assigned the corresponding latitudes and longitudes. Consider a scenario of a four-node geo-distributed cluster with the following configuration:
Node | Region | Datacenter |
---|---|---|
Node1 | us-east-1 | us-east-1a |
Node2 | us-east-1 | us-east-1b |
Node3 | us-west-1 | us-west-1a |
Node4 | eu-west-1 | eu-west-1a |
Step 1. Ensure the CockroachDB Version is 2.0 or Higher
$ cockroach version
Build Tag: v2.0.7
Build Time: 2018-12-10 00:00:00
Distribution: CCL
Platform: darwin amd64 (x86_64-apple-darwin13)
Go Version: go1.10
C Compiler: 4.2.1 Compatible Clang 3.8.0 (tags/RELEASE_380/final)
Build SHA-1: 367ad4f673b33694df06caaa2d7fc63afaaf3053
Build Type: release
If any node is running an earlier version, upgrade it to CockroachDB v2.0.
Step 2. Start the Nodes with the Correct --locality
Flags
To start a new cluster with the correct --locality
flags:
Start Node 1:
$ cockroach start \
--insecure \
--locality=region=us-east-1,datacenter=us-east-1a \
--host=<node1 address> \
--cache=.25 \
--max-sql-memory=.25 \
--join=<node1 address>:26257,<node2 address>:26257,<node3 address>:26257,<node4 address>:26257
Start Node 2:
$ cockroach start \
--insecure \
--locality=region=us-east-1,datacenter=us-east-1b \
--host=<node2 address> \
--cache=.25 \
--max-sql-memory=.25 \
--join=<node1 address>:26257,<node2 address>:26257,<node3 address>:26257,<node4 address>:26257
Start Node 3:
$ cockroach start \
--insecure \
--locality=region=us-west-1,datacenter=us-west-1a \
--host=<node3 address> \
--cache=.25 \
--max-sql-memory=.25 \
--join=<node1 address>:26257,<node2 address>:26257,<node3 address>:26257,<node4 address>:26257
Start Node 4:
$ cockroach start \
--insecure \
--locality=region=eu-west-1,datacenter=eu-west-1a \
--host=<node4 address> \
--cache=.25 \
--max-sql-memory=.25 \
--join=<node1 address>:26257,<node2 address>:26257,<node3 address>:26257,<node4 address>:26257
Use the cockroach init
command to perform a one-time initialization of the cluster:
$ cockroach init --insecure
Access the Admin UI. The following page is displayed:
Step 3. Set the Enterprise License and refresh the Admin UI
The following page should be displayed:
Step 4. Set the Latitudes and Longitudes for the Localities
Launch the built-in SQL client:
$ cockroach sql --insecure --host=<address of any node>
Insert the approximate latitudes and longitudes of each region into the system.locations
table:
> INSERT INTO system.locations VALUES
('region', 'us-east-1', 37.478397, -76.453077),
('region', 'us-west-1', 38.837522, -120.895824),
('region', 'eu-west-1', 53.142367, -7.692054);
For the latitudes and longitudes of AWS, Azure, and Google Cloud regions, see Location Coordinates for Reference.
Step 5. View the Node Map
Open the Overview page and select Node Map from the View drop-down menu. The Node Map will be displayed:
Step 6. Navigate the Node Map
Let's say you want to navigate to Node 2, which is in datacenter us-east-1a
in the us-east-1
region:
- Click on the map component marked as region=us-east-1 on the Node Map. The datacenter view is displayed.
- Click on the datacenter component marked as datacenter=us-east-1a. The individual node components are displayed.
- To navigate back to the cluster view, either click on Cluster in the bread-crumb trail at the top of the Node Map, or click Up to region=us-east-1 and then click Up to Cluster in the lower left-hand side of the Node Map.
Troubleshoot the Node Map
Node Map Not Displayed
The Node Map will not be displayed until all nodes have localities and are assigned the corresponding latitudes and longitudes. To verify if you have assigned localities as well as latitude and longitudes assigned to all nodes, navigate to the Localities debug page (https://<address of any node>:8080/#/reports/localities
) in the Admin UI.
The Localities debug page displays the following:
- Localities configuration that you set up while starting the nodes with the
--locality
flags. - Nodes corresponding to each locality.
- Latitude and longitude coordinates for each locality/node.
On the page, ensure that every node has a locality as well as latitude/longitude coordinates assigned to them.
Node Map Not Displayed for All Locality Levels
The Node Map is displayed only for the locality levels that have latitude/longitude coordinates assigned to them:
- If you assign the latitude/longitude coordinates at the region level, the Node Map shows the regions on the world map. However, when you drill down to the datacenter and further to the individual nodes, the world map disappears and the datacenters/nodes are plotted in a circular layout.
- If you assign the latitude/longitude coordinates at the datacenter level, the Node Map shows the regions with single datacenters at the same location assigned to the datacenter, while regions with multiple datacenters are shown at the center of the datacenter coordinates in the region. When you drill down to the datacenter levels, the Node Map shows the datacenter at their assigned coordinates. Further drilling down to individual nodes shows the nodes in a circular layout.
Assign latitude/longitude coordinates at the locality level that you want to view on the Node Map.
Known Limitations
Unable to Assign Latitude/Longitude Coordinates to Localities
You cannot assign latitude/longitude coordinates to localities if the components of your localities have the same name. For example, consider the following partial configuration:
Node | Region | Datacenter |
---|---|---|
Node1 | us-east | datacenter-1 |
Node2 | us-west | datacenter-1 |
In this case, if you try to set the latitude/longitude coordinates to the datacenter level of the localities, you will get the "primary key exists" error and the Node Map will not be displayed. You can, however, set the latitude/longitude coordinates to the region components of the localities, and the Node Map will be displayed.
Capacity Used Value Displayed is More Than Configured Capacity
If you are running multiple nodes on a single machine (not recommended in production) and didn't specify the maximum allocated storage capacity for each node using the --store
flag, the capacity metrics in the Admin UI are incorrect. This is because when multiple nodes are running on a single machine, the machine's hard disk is treated as an available store for each node, while in reality, only one hard disk is available for all nodes. The total available capacity is then calculated as the hard disk size multiplied by the number of nodes on the machine.
Location Coordinates for Reference
AWS locations
Location | SQL Statement |
---|---|
US East (N. Virginia) | INSERT into system.locations VALUES ('region', 'us-east-1', 37.478397, -76.453077) |
US East (Ohio) | INSERT into system.locations VALUES ('region', 'us-east-2', 40.417287, -76.453077) |
US West (N. California) | INSERT into system.locations VALUES ('region', 'us-west-1', 38.837522, -120.895824) |
US West (Oregon) | INSERT into system.locations VALUES ('region', 'us-west-2', 43.804133, -120.554201) |
Canada (Central) | INSERT into system.locations VALUES ('region', 'ca-central-1', 56.130366, -106.346771) |
EU (Frankfurt) | INSERT into system.locations VALUES ('region', 'eu-central-1', 50.110922, 8.682127) |
EU (Ireland) | INSERT into system.locations VALUES ('region', 'eu-west-1', 53.142367, -7.692054) |
EU (London) | INSERT into system.locations VALUES ('region', 'eu-west-2', 51.507351, -0.127758) |
EU (Paris) | INSERT into system.locations VALUES ('region', 'eu-west-3', 48.856614, 2.352222) |
Asia Pacific (Tokyo) | INSERT into system.locations VALUES ('region', 'ap-northeast-1', 35.689487, 139.691706) |
Asia Pacific (Seoul) | INSERT into system.locations VALUES ('region', 'ap-northeast-2', 37.566535, 126.977969) |
Asia Pacific (Osaka-Local) | INSERT into system.locations VALUES ('region', 'ap-northeast-3', 34.693738, 135.502165) |
Asia Pacific (Singapore) | INSERT into system.locations VALUES ('region', 'ap-southeast-1', 1.352083, 103.819836) |
Asia Pacific (Sydney) | INSERT into system.locations VALUES ('region', 'ap-southeast-2', -33.86882, 151.209296) |
Asia Pacific (Mumbai) | INSERT into system.locations VALUES ('region', 'ap-south-1', 19.075984, 72.877656) |
South America (São Paulo) | INSERT into system.locations VALUES ('region', 'sa-east-1', -23.55052, -46.633309) |
Azure locations
Location | SQL Statement |
---|---|
eastasia (East Asia) | INSERT into system.locations VALUES ('region', 'eastasia', 22.267, 114.188) |
southeastasia (Southeast Asia) | INSERT into system.locations VALUES ('region', 'southeastasia', 1.283, 103.833) |
centralus (Central US) | INSERT into system.locations VALUES ('region', 'centralus', 41.5908, -93.6208) |
eastus (East US) | INSERT into system.locations VALUES ('region', 'eastus', 37.3719, -79.8164) |
eastus2 (East US 2) | INSERT into system.locations VALUES ('region', 'eastus2', 36.6681, -78.3889) |
westus (West US) | INSERT into system.locations VALUES ('region', 'westus', 37.783, -122.417) |
northcentralus (North Central US) | INSERT into system.locations VALUES ('region', 'northcentralus', 41.8819, -87.6278) |
southcentralus (South Central US) | INSERT into system.locations VALUES ('region', 'southcentralus', 29.4167, -98.5) |
northeurope (North Europe) | INSERT into system.locations VALUES ('region', 'northeurope', 53.3478, -6.2597) |
westeurope (West Europe) | INSERT into system.locations VALUES ('region', 'westeurope', 52.3667, 4.9) |
japanwest (Japan West) | INSERT into system.locations VALUES ('region', 'japanwest', 34.6939, 135.5022) |
japaneast (Japan East) | INSERT into system.locations VALUES ('region', 'japaneast', 35.68, 139.77) |
brazilsouth (Brazil South) | INSERT into system.locations VALUES ('region', 'brazilsouth', -23.55, -46.633) |
australiaeast (Australia East) | INSERT into system.locations VALUES ('region', 'australiaeast', -33.86, 151.2094) |
australiasoutheast (Australia Southeast) | INSERT into system.locations VALUES ('region', 'australiasoutheast', -37.8136, 144.9631) |
southindia (South India) | INSERT into system.locations VALUES ('region', 'southindia', 12.9822, 80.1636) |
centralindia (Central India) | INSERT into system.locations VALUES ('region', 'centralindia', 18.5822, 73.9197) |
westindia (West India) | INSERT into system.locations VALUES ('region', 'westindia', 19.088, 72.868) |
canadacentral (Canada Central) | INSERT into system.locations VALUES ('region', 'canadacentral', 43.653, -79.383) |
canadaeast (Canada East) | INSERT into system.locations VALUES ('region', 'canadaeast', 46.817, -71.217) |
uksouth (UK South) | INSERT into system.locations VALUES ('region', 'uksouth', 50.941, -0.799) |
ukwest (UK West) | INSERT into system.locations VALUES ('region', 'ukwest', 53.427, -3.084) |
westcentralus (West Central US) | INSERT into system.locations VALUES ('region', 'westcentralus', 40.890, -110.234) |
westus2 (West US 2) | INSERT into system.locations VALUES ('region', 'westus2', 47.233, -119.852) |
koreacentral (Korea Central) | INSERT into system.locations VALUES ('region', 'koreacentral', 37.5665, 126.9780) |
koreasouth (Korea South) | INSERT into system.locations VALUES ('region', 'koreasouth', 35.1796, 129.0756) |
francecentral (France Central) | INSERT into system.locations VALUES ('region', 'francecentral', 46.3772, 2.3730) |
francesouth (France South) | INSERT into system.locations VALUES ('region', 'francesouth', 43.8345, 2.1972) |
Google Cloud locations
Location | SQL Statement |
---|---|
us-east1 (South Carolina) | INSERT into system.locations VALUES ('region', 'us-east1', 33.836082, -81.163727) |
us-east4 (N. Virginia) | INSERT into system.locations VALUES ('region', 'us-east4', 37.478397, -76.453077) |
us-central1 (Iowa) | INSERT into system.locations VALUES ('region', 'us-central1', 42.032974, -93.581543) |
us-west1 (Oregon) | INSERT into system.locations VALUES ('region', 'us-west1', 43.804133, -120.554201) |
northamerica-northeast1 (Montreal) | INSERT into system.locations VALUES ('region', 'northamerica-northeast1', 56.130366, -106.346771) |
europe-west1 (Belgium) | INSERT into system.locations VALUES ('region', 'europe-west1', 50.44816, 3.81886) |
europe-west3 (Frankfurt) | INSERT into system.locations VALUES ('region', 'europe-west3', 50.110922, 8.682127) |
europe-west4 (Netherlands) | INSERT into system.locations VALUES ('region', 'europe-west4', 53.4386, 6.8355) |
europe-west2 (London) | INSERT into system.locations VALUES ('region', 'europe-west2', 51.507351, -0.127758) |
asia-east1 (Taiwan) | INSERT into system.locations VALUES ('region', 'asia-east1', 24.0717, 120.5624) |
asia-northeast1 (Tokyo) | INSERT into system.locations VALUES ('region', 'asia-northeast1', 35.689487, 139.691706) |
asia-southeast1 (Singapore) | INSERT into system.locations VALUES ('region', 'asia-southeast1', 1.352083, 103.819836) |
australia-southeast1 (Sydney) | INSERT into system.locations VALUES ('region', 'australia-southeast1', -33.86882, 151.209296) |
asia-south1 (Mumbai) | INSERT into system.locations VALUES ('region', 'asia-south1', 19.075984, 72.877656) |
southamerica-east1 (São Paulo) | INSERT into system.locations VALUES ('region', 'southamerica-east1', -23.55052, -46.633309) |