Cluster Upgrade

2020-02-16

Before you begin

In this lab, you'll start with a fresh cluster, so make sure you've stopped and cleaned up the cluster from the previous labs.

Step 1. Install CockroachDB v2.0

  1. Download the CockroachDB v2.0 archive for your OS, and extract the binary:

    {% include copy-clipboard.html %} ~~~ shell $ curl https://binaries.cockroachdb.com/cockroach-v2.0.7.darwin-10.9-amd64.tgz \ | tar -xJ ~~~
    {% include copy-clipboard.html %} ~~~ shell $ wget -qO- https://binaries.cockroachdb.com/cockroach-v2.0.7.linux-amd64.tgz \ | tar xvz ~~~
  2. Move the v2.0 binary into the parent cockroachdb-training directory:

    {% include copy-clipboard.html %} ~~~ shell $ mv cockroach-v2.0.7.darwin-10.9-amd64/cockroach ./cockroach-v2.0 \ ; rm -rf cockroach-v2.0.7.darwin-10.9-amd64 ~~~
    {% include copy-clipboard.html %} ~~~ shell $ mv cockroach-v2.0.7.linux-amd64/cockroach ./cockroach-v2.0 \ ; rm -rf cockroach-v2.0.7.linux-amd64 ~~~

Step 2. Start a cluster running v2.0

Start and initialize a cluster like you did in previous modules, but this time using the v2.0 binary.

  1. In a new terminal, start node 1:

    $ ./cockroach-v2.0 start \
    --insecure \
    --store=node1 \
    --host=localhost \
    --port=26257 \
    --http-port=8080 \
    --join=localhost:26257,localhost:26258,localhost:26259
    
  2. In a new terminal, start node 2:

    $ ./cockroach-v2.0 start \
    --insecure \
    --store=node2 \
    --host=localhost \
    --port=26258 \
    --http-port=8081 \
    --join=localhost:26257,localhost:26258,localhost:26259
    
  3. In a new terminal, start node 3:

    $ ./cockroach-v2.0 start \
    --insecure \
    --store=node3 \
    --host=localhost \
    --port=26259 \
    --http-port=8082 \
    --join=localhost:26257,localhost:26258,localhost:26259
    
  4. In a new terminal, perform a one-time initialization of the cluster:

    $ ./cockroach-v2.0 init --insecure
    

{{site.data.alerts.callout_info}} You can disable a (manual or automatic) cluster version upgrade from the specified version until you reset your cluster by using the cluster.preserve_downgrade_option cluster setting. See the full Cluster Upgrade documentation for details. {{site.data.alerts.end}}

Step 3. Upgrade the first node to v2.1

  1. In node 1's terminal, press CTRL-C to stop the cockroach process.

  2. Verify that node 1 has stopped:

    $ ps | grep cockroach
    

    You should not see a cockroach process with --store=node1 and --port=26257.

    49659 ttys001    0:02.43 ./cockroach-v2.0 start --insecure --store=node2 --host=localhost --port=26258 --http-port=8081 --join=localhost:26257,localhost:26258,localhost:26259
    49671 ttys002    0:02.32 ./cockroach-v2.0 start --insecure --store=node3 --host=localhost --port=26259 --http-port=8082 --join=localhost:26257,localhost:26258,localhost:26259
    49705 ttys015    0:00.00 grep cockroach
    
  3. In node 1's terminal, restart the node using the v2.1 binary:

    $ ./cockroach start \
    --insecure \
    --store=node1 \
    --listen-addr=localhost:26257 \
    --http-addr=localhost:8080 \
    --join=localhost:26257,localhost:26258,localhost:26259
    
  4. Go to the Admin UI at http://localhost:8081 to view the Node List and then verify that the node has rejoined the cluster using the new version of the binary:

    CockroachDB Admin UI

Step 4. Upgrade the rest of the nodes to v2.1

  1. In node 2's terminal, press CTRL-C to stop the cockroach process.

  2. Verify that node 2 has stopped:

    $ ps | grep cockroach
    

    You should not see a cockroach process with --store=node2 and --port=26258.

    49659 ttys001    0:07.05 ./cockroach-v2.0 start --insecure --store=node3 --host=localhost --port=26259 --http-port=8082 --join=localhost:26257,localhost:26258,localhost:26259
    49824 ttys002    0:00.00 grep cockroach
    49717 ttys015    0:05.76 ./cockroach start --insecure --store=node1 --listen-addr=localhost:26257 --http-addr=localhost:8080 --join=localhost:26257,localhost:26258,localhost:26259
    
  3. Restart the node using the v2.1 binary:

    $ ./cockroach start \
    --insecure \
    --store=node2 \
    --listen-addr=localhost:26258 \
    --http-addr=localhost:8081 \
    --join=localhost:26257,localhost:26258,localhost:26259
    
  4. Wait 1 minute.

  5. In node 3's terminal, press CTRL-C to stop the cockroach process.

  6. Verify that node 3 has stopped:

    $ ps | grep cockroach
    

    You should not see a cockroach process with --store=node3 and --port=26259.

    49869 ttys001    0:00.01 grep cockroach
    49849 ttys002    0:02.38 ./cockroach start --insecure --store=node2 --listen-addr=localhost:26258 --http-addr=localhost:8081 --join=localhost:26257,localhost:26258,localhost:26259
    49717 ttys015    0:10.88 ./cockroach start --insecure --store=node1 --listen-addr=localhost:26257 --http-addr=localhost:8080 --join=localhost:26257,localhost:26258,localhost:26259
    
  7. Restart the node using the v2.1 binary:

    $ ./cockroach start \
    --insecure \
    --store=node3 \
    --listen-addr=localhost:26259 \
    --http-addr=localhost:8082 \
    --join=localhost:26257,localhost:26258,localhost:26259
    

Step 5. Check your cluster's versions

Back in the Admin UI, you'll see that all 3 nodes now have the same, upgraded version:

CockroachDB Admin UI

You can also use the cockroach node status command to check each node's version:

$ ./cockroach node status \
--insecure
id |    address      |        build         |            started_at            |            updated_at            | is_available | is_live  
+--+-----------------+----------------------+----------------------------------+----------------------------------+--------------+---------+
 1 | localhost:26257 | v2.1.0-beta.20180910 | 2018-09-13 22:27:29.290335+00:00 | 2018-09-13 22:37:32.314011+00:00 | true         | true     
 2 | localhost:26259 | v2.1.0-beta.20180910 | 2018-09-13 22:34:17.468613+00:00 | 2018-09-13 22:37:35.491432+00:00 | true         | true     
 3 | localhost:26258 | v2.1.0-beta.20180910 | 2018-09-13 22:36:06.252257+00:00 | 2018-09-13 22:37:31.770426+00:00 | true         | true
(3 rows)

Step 6. Clean up

This is the last module of the training, so feel free to stop you cluster and clean things up.

  1. Stop all CockroachDB nodes:

    $ pkill -9 cockroach
    
  2. Remove the nodes' data directories:

    $ rm -rf node1 node2 node3