Deploying Kolla-Ansible with External Ceph

When deploying Kolla-Ansible, although it is possible to deploy a Ceph cluster simultaneously, in some cases, operations personnel may prefer to manage Ceph and OpenStack separately while still using Ceph as the storage backend. In such cases, there is no need to worry; Kolla-Ansible can utilize the previously mentioned config override feature to use an external Ceph cluster. This article will introduce how to use Kolla-Ansible and its config override feature to deploy OpenStack using external Ceph.

Contents

Preparation

Since an external Ceph cluster is to be used, a Ceph cluster must first be deployed. There are many options for deploying Ceph; you can also refer to the previously introduced Ceph-Ansible for deployment.

Additionally, some basic knowledge and related information about using Kolla-Ansible are required, which can be found in this article:Deploying OpenStack via Kolla-Ansible and Containers

Ceph Configuration

RBD Pool

OpenStack Nova, Cinder (Cinder Backup), and Glance can all use RBD as a storage backend, so RBD pools need to be created for these three services.

sudo ceph osd pool create images 128
sudo ceph osd pool create vms 128
sudo ceph osd pool create volumes 128

Ceph Keyring Configuration

Ceph uses keyrings for authentication, so corresponding keyrings must also be created for the three services, with permissions granted to their respective pools.

sudo ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rdb_children, allow rwx pool=images' -o /etc/ceph/ceph.client.glance.keyring

sudo ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rx pool=images' -o /etc/ceph/ceph.client.cinder.keyring

sudo ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups -o /etc/ceph/ceph.client.cinder-backup.keyring

sudo ceph auth get-or-create client.nova mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=vms, allow rx pool=images' -o /etc/ceph/ceph.client.nova.keyring

Kolla-Ansible Configuration

`globals.yml`

在 globals.yml , we must set the option for deploying Ceph via Kolla-Ansible to 'no', while setting the options for Nova, Cinder, and Glance to use Ceph as the storage backend to 'yes'.

enable_ceph: "no"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"

Glance Configuration

Configuring Glance to use external Ceph basically involves three steps:

在 glance-api.conf Configure RBD Backend
在 /etc/ceph/ceph.conf Add Ceph Configuration
Add /etc/ceph/ceph.client.images.keyring

The first step is achieved through config override. First, create /etc/kolla/config/glance/glance-api.conf and add the following content:

[glance_store]
stores = rbd
default_store = rbd
rbd_store_pool = images
rbd_store_user = glance
rbd_store_ceph_conf = /etc/ceph/ceph.conf

Next, in the second step, also in /etc/kolla/config/glance/ceph.conf add the Ceph configuration

[global]
fsid = 88a8ea91-df1d-4f67-b78b-52bb2f04df4d
mon_initial_members = ceph01, ceph02, ceph03
mon_host = 192.168.113.10,192.168.113.11,192.168.113.10
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

Finally, place the keyring created above into /etc/kolla/config/glance/ceph.client.glance.keyring

sudo cp /etc/ceph/ceph.client.glance.keyring /etc/kolla/config/glance/ceph.client.glance.keyring

Kolla-Ansible will place all files ceph* within the folder into /etc/ceph inside the container.

Cinder Configuration

The configuration steps for Cinder are similar to Glance; first, create /etc/kolla/config/cinder/cinder-volume.conf and add the following content:

[DEFAULT]
enabled_backends=rbd-1

[rbd-1]
rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_user=cinder
backend_host=rbd:volumes
rbd_pool=volumes
volume_backend_name=rbd-1
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_secret_uuid = {{ cinder_rbd_secret_uuid }}

Next, configure Cinder-Backup by creating /etc/kolla/config/cinder/cinder-backup.conf and adding the following content

[DEFAULT]
backup_ceph_conf=/etc/ceph/ceph.conf
backup_ceph_user=cinder-backup
backup_ceph_chunk_size = 134217728
backup_ceph_pool=backups
backup_driver = cinder.backup.drivers.ceph
backup_ceph_stripe_unit = 0
backup_ceph_stripe_count = 0
restore_discard_excess_bytes = true

Next, similarly, place ceph.conf<code> 放到 </code> /etc/kolla/config/cinder/ 下

Finally, you need to copy all the keyrings.

sudo cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-backup/ceph.client.cinder.keyring
sudo cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-volume/ceph.client.cinder.keyring
sudo cp /etc/ceph/ceph.client.cinder-backup.keyring /etc/kolla/config/cinder/cinder-backup/ceph.client.cinder-backup.keyring

Nova Configuration

The Nova configuration is also very similar, following basically the same steps

Create based on the following content /etc/kolla/config/nova/nova-compute.conf

[libvirt]
images_rbd_pool=vms
images_type=rbd
images_rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_user=nova

將 ceph.conf<code> 放到 </code> /etc/kolla/config/nova/ 下

Finally, copy the keyring

sudo cp /etc/ceph/ceph.client.nova.keyring /etc/kolla/config/nova/ceph.client.nova.keyring

Deployment

To learn more about the actual deployment commands, you can refer to Deploying OpenStack via Kolla-Ansible and Containers

cd kolla-ansible
tools/generate_passwords.py
tools/kolla-ansible -i ansible/inventory/multinode bootstrap-servers
tools/kolla-ansible -i ansible/inventory/multinode prechecks
tools/kolla-ansible -i ansible/inventory/multinode deploy

After the Playbook runs successfully, you can try using each service to ensure they are functioning correctly.