OpenStack Swift - Container Sync (New & Old ways)

OpenStack Swift Container-Sync 

Swift 內建功能利用背景工作, 讓 Container to Container 如同鏡像一樣同步所有資料. 不僅僅是自己的Container, 還可以跟其他人Account內的Container相互同步資料. 這個功能也被部分企業用來將資料從舊的Swift Cluster轉移到新的環境上.

提示
如果User request是對object做POST的操作, 操作內容不一定會被同步, 除非object_post_as_copy = true 在 Proxy server 的設定, 目前這個值預設為開啟 true.

同步的設定可以有幾種模式 :

1. One-way sync : 單向同步 containerA --> containerB
2. Two-way sync : 雙向同步 containerA <--> containerB
3. Chain : 鏈狀同步 containerA --> containerB --> containerC --> containerA

首先你必須要知道目前有兩種方法(新/舊)來設定 Container Sync
First of all, you need to know there are two styles for setting container-sync currently. 

• OLD-style(舊): Supported since Swift's initial release. 任何版本,包含新版本
• NEW-style(新): Implemented in Swift 1.12 or later. 在1.12版本後才釋出

不論新舊styles, 都有一個重要的前提, 原資料所在的Container Server, 必須要能跟遠端目標Container 的 Proxy Server 透過網路連結的到.

NEW-Style - 多了realm(域)的觀念.
新的做法需要先建立一個設定檔 ( /etc/swift/container-sync-realms.conf )在兩個叢集的"所有"節點. 具體一點來說, 只要會涉及到這個同步過程的所有Proxy Server與Container Server 上面都必須要有這個設定檔案.

範例

[DEFAULT]

[realm1]
key = realm1key
key2 = realm1key2
cluster_name1 = https://host1/v1/
cluster_name2 = https://host2/v1/

[realm2]
key = realm2key
key2 = realm2key2
cluster_name3 = https://host3/v1/
cluster_name4 = https://host4/v1/

realm1 的名稱是可以自行定義, 用來配置一組Container-Sync域. 這個新的觀念可以提高Container-Sync功能的安全性還有彈性. Key則是用來做Request驗證的數位簽章. 送出請求的Container-Sync含有這個key, 接收到的 Proxy server 會驗證是否符合他這邊對應的資料, 如果送過來的key. 這個key並不等同于x-sync-key這個header.


[NEW-Style]

On both clusters (在兩邊的叢集都需要設定)
1. Create /etc/swift/container-sync-realms.conf on all nodes 所有節點都需要
    (https://github.com/openstack/swift/blob/master/etc/container-sync-realms.conf-sample)

      

2. Adding [container-sync] section in /etc/swift/container-server/1.conf on Container Nodes 
   所有container-server運作的節點都需要設定.
   (https://github.com/openstack/swift/blob/master/etc/container-server.conf-sample#L148)

3. Setting container_sync middleware in proxy’s pipline
    To add the container_sync middleware to your proxy pipeline. It needs to be after any memcache middleware     and before any auth middleware.     
    The container_sync section only needs the “use” item. For example:

               [pipeline:main]
               pipeline = healthcheck proxy-logging cache container_sync tempauth proxy-logging proxy-server

               [filter:container_sync]
               use = egg:swift#container_sync


Efforts on Cluster1

     Set the sync-to and sync-key headers on container con1
          $> swift post -t "//realm1/name1/AUTH_test/con2" -k "key" con1


Efforts on Cluster2

     Set the sync-to and sync-key headers on container con1
          $> swift post -t "//realm1/name2/AUTH_test/con1" -k "key" con2


[ OLD-style ]
     
     One-Way sync - Sync data from c1con to c2con 
     Two-Way sync - Sync data from c1con to c2con, also c2con to c1con

Efforts on Cluster1
     Configure container-sync in /etc/swift/container-server/1.conf on all SwiftStack Nodes running container servers in Cluster1. 

     1. Adding allowed_sync_hosts = $host1,$host2,$host3 in [DEFAULT] 
        section. $host can use either IP or HOSTNAME of the another 
        cluster's API entry point.  

        Example: (192.168.30.1 is the API IP of Cluster2)
        

     2. Adding [container-sync] section in 
        /etc/swift/container-server/1.conf on all Nodes running container servers in Cluster1.

        Example: 

     REF: https://github.com/openstack/swift/blob/master/etc/container-server.conf-sample#L148-L164

     3. Restart Container-server to apply the changes

        $> sudo -i swift-init all restart

     4. Create and add sync information to a container by Swift CLI

        * Create a container (skip it if it's already created)

          $> swift post c1con

        * Retrieve the account URI of c2con on Cluster2.
          It's supposed to look like:

          http://192.168.30.1/v1/AUTH_c2user2/c2con
          
        * Set container-sync required headers on c1con

          $> swift post -t 'http://192.168.30.1/v1/AUTH_c2user2/c2con' 
             -k ‘secret-key-must-same-on-both-container’ c1con 

        After the above operations, the container c1con should have two 
        headers now. 

        

Efforts on Cluster2
     Configure container-sync in /etc/swift/container-server/1.conf on all Nodes running container servers in Cluster2.

     If you don’t want to setup Two-Way sync, please skip this step. 
     1. Adding allowed_sync_hosts = $host1,$host2,$host3 in [DEFAULT] 
        section. $host can use either IP or HOSTNAME of the another 
        cluster's API entry point.  

        Example: (192.168.20.1 is the API IP of Cluster1)

        

     2. Adding [container-sync] section in /etc/swift/container-server/1.conf on all Nodes 
        running container servers in Cluster2.

        Example: 

        REF: https://github.com/openstack/swift/blob/master/etc/container-server.conf-sample#L148-L164

     3. Restart Container-server to apply the changes

        $> sudo -i swift-init all restart 

     4. Create and add sync information to a container by Swift CLI

        * Create a container (skip it if it's already created) 
 
          $> swift post c1con
          
        # For Two-Way sync

        * Retrieve the account URI of c1con on Cluster1.
          It's supposed to look like:

          http://192.168.20.1/v1/AUTH_c1user1/c1con
               
        * Set container-sync headers required on c1con

        # For Two-Way sync

        $> swift post -t 'http://192.168.20.1/v1/AUTH_c1user1/c1con' 
           -k ‘secret-key-must-same-on-both-container’ c2con 

        # For One-Way sync, the secret key on c2con is enough.

        $> swift post -k ‘secret-key-must-same-on-both-container’ c2con

        After the above operations, the container c2con should have two
        headers now.  

        

[驗證]
The swift-container-sync daemons will perform the sync periodically. You can upload some objects into c1con on Cluster1. Wait 5 minutes and check the container c2con on Cluster2 for updates. Also, you can find the PUT logs in /var/log/swift/proxy_access.log in Cluster2. 

[Important] Those PUT requests are issued by the container server on Cluster1. So, the Cluster2 API IP must be reachable by the container servers (nodes) on Cluster1.

If you don't want to wait for 5 minutes, you can stop the container-sync worker and execute the worker on Cluster1.

 $> sudo -i swift-init container-sync stop
 $> sudo -i swift-container-sync -o /etc/swift/container-server/1.conf