Skip to main content
Ask Question
OpenStack
Asked a question recently

How do I extend an OpenStack cluster with extra controller/network nodes?

Where am I?

In Bright Computing, Inc. you can ask and answer questions and share your experience with others!

How do I extend an OpenStack cluster with extra controller/network nodes?

Scenario:

You have installed OpenStack with one controller. Now you want to add two more controllers to expand the database cluster.

Adding one more controller/network to your OpenStack cluster can be done by following these steps:

Decide on the nodes you want to use as controller/network nodes

Make sure the interfaces are configured on the network nodes that you want to add

For example :

If you have a node001 with two interfaces -- one interface for the external OpenStack network, and the other interface for the internal network -- then make sure that you match the interfaces configuration on your new nodes.

Now, append both the  OpenStackControllers and OpenStackNetworkNodes configuration overlay to the new nodes:

[am-c-02-08-t-c7u2-1->configurationoverlay]% append openstackcontrollers nodes node002,node003
[am-c-02-08-t-c7u2-1->configurationoverlay*]% append openstacknetworknodes nodes node002,node003
[am-c-02-08-t-c7u2-1->configurationoverlay*]% commit

Make sure you restart both of your new nodes after you commit:

[am-c-02-08-t-c7u2-1->device]% reboot -n node002,node003

While you wait for your two nodes to be rebooted, log into your first controller/network node.

Then install xtradbackup,

https://www.percona.com/doc/percona-xtrabackup/2.2/installation/yum_repo.html8

on the controller/network node.

After installation,  execute the following :

On your head node :

#cat /root/.galerarc
#ssh controller-1

[root@node001 ~]# mkdir -p /cm/shared/apps/openstack/backup/
[root@node001 ~]# innobackupex --galera-info --user=root --password=password --socket=/var/lib/mysql/mysql.sock /cm/shared/apps/openstack/backup/

The previous command will create a backup directory under /cm/shared/apps/openstack/backup/

with a name derived from the date and time of the day.

[root@node001 ~]# innobackupex -apply-log --galera-info --user=root --password=2O1711H542KV24Kg --socket=/var/lib/mysql/mysql.sock /cm/shared/apps/openstack/backup/BACKUP DIRECTORY/

First we will add a file to this backup before we stream it to the new nodes

[root@node001 2017-02-08_12-03-20]# cat xtrabackup_galera_info
#touch grastate.dat
#cat  xtrabackup_galera_info 
7c6112b1-ede7-11e6-9afb-fb7093d22da5:6336

The content of xtrabackup_galera_info should be used to construct the content of grastate.dat, which should look like this:

# GALERA saved state
version: 2.1
uuid:    7c6112b1-ede7-11e6-9afb-fb7093d22da5
seqno: 6336
safe_to_bootstrap: 0

Notice that 6336 is the number after “:” in xtrabackup_galera_info
Now we need to copy this backup to /var/lib/mysql to both of your new nodes

#ssh node002
#cd /var/lib/mysql
# rm -rf *
#rsync -avzp --progress /cm/shared/apps/openstack/backup/2017-02-08_12-03-20/ .
#chown -R mysql.mysql /var/lib/mysql

Repeat the previous steps to also rsync it over to your third node

When done, go back to your head node and execute the following :

[root@am-c-02-08-t-c7u2-1 ~]# cmsh
[am-c-02-08-t-c7u2-1]% configurationoverlay
[am-c-02-08-t-c7u2-1->configurationoverlay]% append galeranodes nodes node002..node003
[am-c-02-08-t-c7u2-1->configurationoverlay*]% commit

To confirm the controllers are there, please log into any of your new controllers and execute the following:

[root@node001 ~]# mysql -uroot -ppassword
MariaDB [(none)]> show status like 'wsrep_cluster_size';

Your output should be :

+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.00 sec)