Sorry, you need to enable JavaScript to visit this website.

Feedback

Your feedback is important to keep improving our website and offer you a more reliable experience.

PyPy Enabling Guide for OpenStack Swift

This guide covers two deployment scenarios for OpenStack Swift - an existing one and a fresh installation. Both of them are configured to run by-default with PyPy as the default Python interpreter.

The information here is intended to clarify for a various types of users the general context in which they can enable PyPy instead of CPython. Also, the configuration used here is a small test scenario, and for simplicity we opted to assume that there are no other Python workloads running on the machines beside OpenStack Liberty, and we updated the standard Python interpreter link with the PyPy one.

In order to avoid reaching OS limits for open file descriptors, please set the PYPY_GC_MIN to a low enough value, such as 15 MB or lower, until the problem is solved, using the command bellow.

export PYPY_GC_MIN=15M

To consult default values for PyPy garbage collection parameters, please see http://doc.pypy.org/en/latest/gc_info.html

 

PyPy as the default Python interpreter

In this section you can see the steps needed to be made on every server to switch from CPython to PyPy. We will use the latest version of PyPy, together with a pip version for PyPy. The latest x86-64 binary (64 bit) compatible can be downloaded from http://pypy.org/download.html. Use the following commands to enable the usage of PyPy as the default Python interpreter:

wget http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-linux64.tar.bz2
tar -xf pypy-5.0.1-linux64.tar.bz2
mv pypy-5.0.1-linux64 pypy
rm /usr/bin/python
ln -s /opt/pypy/bin/pypy /usr/bin/python

Install pip as a dependency for PyPy. This step is needed for download and install any other 3rd party library.

cd /opt/pypy/bin
./pypy -m ensurepip
rm /usr/bin/pip
ln -s /opt/pypy/bin/pip /usr/bin/pip

pip might not be the latest one and you might get warning or errors during future installation using it. In order to mitigate this error run:

cd /opt/pypy/bin
./pip install --upgrade pip

Upgrading an existing OpenStack Swift installation

In this part, the steps needed to upgrade an existing installation are presented. After that, start the application as normal, no further code change is needed

Keystone

  1. Clone the official GitHub repository
    cd /opt
    git clone https://github.com/openstack/keystone.git
                    
  2. Install needed dependencies, using the pip for PyPy
    cd /opt/keystone
    pip install -r requirements.txt
    pip install PyMySQL
    pip install python-memcached
    pip install python-openstackclient
  3. Install the Keystone service
    cd /opt/keystone
    python setup.py install
  4. Upgrade the Identity service database
    su -s /bin/sh -c "/opt/pypy/bin/keystone-manage db_sync"
  5. Run the Keystone service
    /opt/pypy/bin/keystone-all &

Swift Proxy and Swift Object Server

  1. Clone the official GitHub repository
    cd /opt
    git clone https://github.com/openstack/swift
  2. Install needed dependencies, using pip for PyPy
    cd /opt/swift
    pip install -r requirements.txt
    pip install netaddr
    pip install pbr 
    pip install positional 
    pip install iso8601 
    pip install stevedore 
    pip install requests 
    pip install debtcollector 
    pip install webob
    pip install netifaces
    pip install xattr
    In case an error occurs during the installation of PyECLib, you need to install liberasurecode and then try again.
    wget https://bitbucket.org/tsg-/liberasurecode/downloads/liberasurecode-1.1.0.tar.gz 
    tar -xvf liberasurecode-1.1.0.tar.gz  
    cd liberasurecode-1.1.0 
    ./autogen.sh  
    ./configure  
    make 
    make install
  3. For Keystone support, the following modules need to be added
    cd /opt
    git clone https://github.com/openstack/keystonemiddleware.git 
    cd keystonemiddleware/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/keystoneauth.git 
    cd keystoneauth/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/python-keystoneclient.git 
    cd python-keystoneclient/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.i18n.git 
    cd oslo.i18n/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.config.git 
    cd oslo.config/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.serialization.git 
    cd oslo.serialization/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.utils.git 
    cd oslo.utils/ 
    python setup.py install
  4. Install the Swift Proxy service
    cd /opt/swift
    python setup.py install

Installing OpenStack Swift from scratch

In this part, the OpenStack Swift setup is presented for 3 dedicated servers, each running Ubuntu Server 14.04.4 64 bit, in a small case scenario. For a complete guide, please see the official OpenStack documentation available at http://docs.openstack.org

In the table below you can see for each of them the hostname and the role.

   
controllerpypy Keystone Identity Service
proxypypy Swift Proxy Server
objectpypy Swift Object Server

On each server, prior to installing everything, the bellow commands are given. You will need to reboot the machine after the commands finish execution.

    apt-get update
    apt-get upgrade
    apt-get install git gcc g++ libssl-dev memcached

Make sure that you have the hostname configured properly for each machine and that you have network connectivity before going further.

Keystone

Prerequisites

  1. Setup the Database
    • Install the packages. Choose a suitable password for the database root account
      apt-get install mariadb-server
    • Create and edit /etc/mysql/conf.d/mysqld_openstack.cnf with the following content:
      [mysqld]
      
      bind-address = 0.0.0.0
      default-storage-engine = innodb
      innodb_file_per_table
      collation-server = utf8_general_ci
      init-connect = 'SET NAMES utf8'
      character-set-server = utf8
    • Restart service
      service mysql restart
  2. Setup the Message queue
    • Install the package
      apt-get install rabbitmq-server
    • Add the openstack user. Replace RABBITMQ_PASSWORD with a suitable password
      rabbitmqctl add_user openstack RABBITMQ_PASSWORD
    • Add write and read access to the openstack user
      rabbitmqctl set_permissions openstack ".*" ".*" ".*"

Setting up the Identity Service

  1. Setup the database
    • Use the database access client to connect to the database server as the root user
      mysql -u root -p
    • Create a database called keystone
      CREATE DATABASE keystone;
    • Grant access to the keystone database. Replace KEYSTONE_DBPASS with a suitable password
      GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'KEYSTONE_DBPASS';
      GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'KEYSTONE_DBPASS';
    • Exit the database client
      quit
  2. Clone the official GitHub repository
    cd /opt
    git clone https://github.com/openstack/keystone.git
  3. Install needed dependencies, using the pip for PyPy
    cd /opt/keystone
    pip install -r requirements.txt
    pip install PyMySQL
    pip install python-memcached
    pip install python-openstackclient
  4. Install the Keystone service
    cd /opt/keystone
    python setup.py install
  5. Setup the Keystone configuration files
    mkdir /etc/keystone
    cp /opt/keystone/etc/keystone.conf.sample /etc/keystone/keystone.conf
    cp /opt/keystone/etc/keystone-paste.ini /etc/keystone/keystone-paste.ini
    Edit the /etc/keystone/keystone.conf file and complete the following:
    • Generate a random value to use as the administration token during configuration
      openssl rand -hex 16
    • In the [DEFAULT] section, define the value of the initial administration token, obtained from the previous step
      [DEFAULT]
      ...
      admin_token = ADMIN_TOKEN
    • Enable verbose logging in the [DEFAULT] section
      [DEFAULT]
      ...
      verbose = true
    • In the [database] section, configure database access. Replace KEYSTONE_DBPASS with the password you chose on database setup step.
      [database]
      ...
      connection = mysql+pymysql://keystone:KEYSTONE_DBPASS@0.0.0.0/keystone
    • In the [memcache] section, configure the Memcached service
      [memcache]
      ...
      servers = localhost:11211
    • In the [token] section, configure the UUID token provider and Memcached driver
      [token]
      ...
      provider = uuid
      driver = memcache
    • In the [revoke] section, configure the SQL revocation driver
      [revoke]
      ...
      driver = sql
  6. Populate the Identity service database
    su -s /bin/sh -c "/opt/pypy/bin/keystone-manage db_sync"
  7. Run the Keystone service
    /opt/pypy/bin/keystone-all &
  8. Create the service entity and API endpoints.
    • Export OpenStack variables. Replace ADMIN_TOKEN with the authentication token that you generated in the previous section
      export OS_TOKEN=ADMIN_TOKEN
      export OS_URL=http://controllerpypy:35357/v3
      export OS_IDENTITY_API_VERSION=3
    • Create the openstack executable. Create the file /usr/bin/openstack file with the following content:
      #!/usr/bin/python
      
      import sys
      
      from openstackclient.shell import main
      
      
      if __name__ == "__main__":
      sys.exit(main())
      Run the command
      chmod +x /usr/bin/openstack
      to make the file executable.
    • Create the service entry and API endpoints
      openstack service create --name keystone --description "OpenStack Identity" identity
      openstack endpoint create --region RegionOne identity public http://controllerpypy:5000/v2.0
      openstack endpoint create --region RegionOne identity internal http://controllerpypy:5000/v2.0
      openstack endpoint create --region RegionOne identity admin http://controllerpypy:35357/v2.0
    • Create projects, users and roles. Use a suitable password where prompted.
      openstack project create --23 default --description "Admin Project" admin
      openstack user create --domain default --password-prompt admin
      openstack role create admin
      openstack role add --project admin --user admin admin
      openstack project create --domain default --description "Service Project" service
      openstack project create --domain default --description "Demo Project" demo
      openstack user create --domain default --password-prompt demo
      openstack role create user
      openstack role add --project demo --user demo user

Swift Proxy Server

Prerequisites

  1. On the Keystone server and add a new user, called swift. Also we want to make sure that it has admin rights.
    openstack user create --domain default --password-prompt swift
    openstack role add --project service --user swift admin
    openstack service create --name swift --description "OpenStack Object Storage" object-store
  2. On the Keystone server, create the endpoints
    openstack endpoint create --region RegionOne object-store public http://controllerpypy:8080/v1/AUTH_%\(tenant_id\)s
    openstack endpoint create --region RegionOne object-store internal http://controllerpypy:8080/v1/AUTH_%\(tenant_id\)s
    openstack endpoint create --region RegionOne object-store admin http://controllerpypy:8080/v1

Install and configure components

  1. Clone the official GitHub repository
    cd /opt
    git clone https://github.com/openstack/swift
  2. Install needed dependencies, using pip for PyPy
    cd /opt/swift
    pip install -r requirements.txt
    pip install netaddr
    pip install pbr 
    pip install positional 
    pip install iso8601 
    pip install stevedore 
    pip install requests 
    pip install debtcollector 
    pip install webob
    pip install netifaces
    pip install xattr
    In case an error occurs during the installation of PyECLib, you need to install liberasurecode and then try again.
    wget https://bitbucket.org/tsg-/liberasurecode/downloads/liberasurecode-1.1.0.tar.gz 
    tar -xvf liberasurecode-1.1.0.tar.gz  
    cd liberasurecode-1.1.0 
    ./autogen.sh  
    ./configure  
    make 
    make install
  3. For Keystone support, the following modules need to be added
    cd /opt
    git clone https://github.com/openstack/keystonemiddleware.git 
    cd keystonemiddleware/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/keystoneauth.git 
    cd keystoneauth/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/python-keystoneclient.git 
    cd python-keystoneclient/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.i18n.git 
    cd oslo.i18n/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.config.git 
    cd oslo.config/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.serialization.git 
    cd oslo.serialization/ 
    python setup.py install
    cd /opt
    git clone https://github.com/openstack/oslo.utils.git 
    cd oslo.utils/ 
    python setup.py install
  4. Install the Swift Proxy service
    cd /opt/swift
    python setup.py install
  5. Setup the Swift Proxy configuration files
    mkdir /etc/swift
    curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/liberty
    Edit the /etc/swift/proxy-server.conf and complete the following:
    • In the [DEFAULT] section, configure the bind port, user, and configuration directory
      [DEFAULT]
      ...
      bind_port = 8080
      user = swift
      swift_dir = /etc/swift
    • In the [pipeline:main] section, enable the appropriate modules
      [pipeline:main]
      pipeline = catch_errors gatekeeper healthcheck proxy-logging cache
      container_sync bulk ratelimit authtoken keystoneauth container-quotas
      account-quotas slo dlo versioned_writes proxy-logging proxy-server
    • In the [app:proxy-server] section, enable automatic account creation
      [app:proxy-server]
      use = egg:swift#proxy
      ...
      account_autocreate = true
    • In the [filter:keystoneauth] section, configure the operator roles
      [filter:keystoneauth]
      use = egg:swift#keystoneauth
      ...
      operator_roles = admin,user
    • In the [filter:authtoken] section, configure Identity service access. Replace SWIFT_PASS with the password you chose for the swift user in the Identity service
      [filter:authtoken]
      paste.filter_factory = keystonemiddleware.auth_token:filter_factory
      ...
      auth_uri = http://controllerpypy:5000
      auth_url = http://controllerpypy:35357
      auth_plugin = password
      project_domain_id = default
      user_domain_id = default
      project_name = service
      username = swift
      password = SWIFT_PASS
      delay_auth_decision = true
    • In the [filter:cache] section, configure the memcached location
      [filter:cache]
      use = egg:swift#memcache
      ...
      memcache_servers = 127.0.0.1:11211

Swift Object Server

For simplicity, this configuration assumes that two dedicated storage units are used: /dev/sdb and /dev/sdc

Prerequisites

  1. Install the needed packages
    apt-get install xfsprogs rsync
  2. Edit the /etc/fstab file and add the following to it:
    /dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
    /dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
  3. Prepare /dev/sdb and /dev/sdc
    mkfs.xfs /dev/sdb
    mkfs.xfs /dev/sdc
    mkdir -p /srv/node/sdb
    mkdir -p /srv/node/sdc
    mount /srv/node/sdb
    mount /srv/node/sdc
  4. Edit the /etc/rsyncd.conf file and add the following to it. Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node.
    uid = swift
    gid = swift
    log file = /var/log/rsyncd.log
    pid file = /var/run/rsyncd.pid
    address = MANAGEMENT_INTERFACE_IP_ADDRESS
    
    [account]
    max connections = 2
    path = /srv/node/
    read only = false
    lock file = /var/lock/account.lock
    
    [container]
    max connections = 2
    path = /srv/node/
    read only = false
    lock file = /var/lock/container.lock
    
    [object]
    max connections = 2
    path = /srv/node/
    read only = false
    lock file = /var/lock/object.lock
  5. Edit the /etc/default/rsync file and enable the rsync service
    RSYNC_ENABLE=true
  6. Start the rsync service
    service rsync start

Install and configure components

  1. Clone the official GitHub repository
    cd /opt
    git clone https://github.com/openstack/swift
  2. Install needed dependencies, using pip for PyPy
    cd /opt/swift
    pip install -r requirements.txt
    pip install netaddr
    pip install pbr 
    pip install positional 
    pip install iso8601 
    pip install stevedore 
    pip install requests 
    pip install debtcollector 
    pip install webob
    pip install netifaces
    pip install xattr
    In case an error occurs during the installation of PyECLib, you need to install liberasurecode and then try again.
    wget https://bitbucket.org/tsg-/liberasurecode/downloads/liberasurecode-1.1.0.tar.gz 
    tar -xvf liberasurecode-1.1.0.tar.gz  
    cd liberasurecode-1.1.0 
    ./autogen.sh  
    ./configure  
    make 
    make install
  3. Install the Swift Object service
    cd /opt/swift
    python setup.py install
  4. Setup the Swift Object configuration files
    mkdir /etc/swift
    curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/liberty
    curl -o /etc/swift/container-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample?h=stable/liberty
    curl -o /etc/swift/object-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample?h=stable/liberty
    Edit the /etc/swift/account-server.conf and complete the following:
    • In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory. Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node.
      [DEFAULT]
      ...
      bind_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
      bind_port = 6002
      user = swift
      swift_dir = /etc/swift
      devices = /srv/node
      mount_check = true
    • In the [pipeline:main] section, enable the appropriate modules
      [pipeline:main]
      pipeline = healthcheck recon account-server
    • In the [filter:recon] section, configure the recon (meters) cache directory
      [filter:recon]
      use = egg:swift#recon
      ...
      recon_cache_path = /var/cache/swift
    Edit the /etc/swift/container-server.conf and complete the following:
    • In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory. Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node.
      [DEFAULT]
      ...
      bind_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
      bind_port = 6001
      user = swift
      swift_dir = /etc/swift
      devices = /srv/node
      mount_check = true
    • In the [pipeline:main] section, enable the appropriate modules
      [pipeline:main]
      pipeline = healthcheck recon container-server
    • In the [filter:recon] section, configure the recon (meters) cache directory
      [filter:recon]
      use = egg:swift#recon
      ...
      recon_cache_path = /var/cache/swift
    Edit the /etc/swift/object-server.conf and complete the following:
    • In the [DEFAULT] section, configure the bind IP address, bind port, user, configuration directory, and mount point directory. Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node.
      [DEFAULT]
      ...
      bind_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
      bind_port = 6000
      user = swift
      swift_dir = /etc/swift
      devices = /srv/node
      mount_check = true
    • In the [pipeline:main] section, enable the appropriate modules
      [pipeline:main]
      pipeline = healthcheck recon object-server
    • In the [filter:recon] section, configure the recon (meters) cache directory
      [filter:recon]
      use = egg:swift#recon
      ...
      recon_cache_path = /var/cache/swift
      recon_lock_path = /var/lock
  5. Ensure proper ownership of the mount point directory structure. Create the recon directory and ensure proper ownership of it
    chown -R swift:swift /srv/node
    mkdir -p /var/cache/swift
    chown -R root:swift /var/cache/swift
  6. Create and distribute initial rings
    • Create account ring. Replace STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node
      cd /etc/swift
      swift-ring-builder account.builder create 10 3 1
      swift-ring-builder account.builder add --region 1 --zone 1 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6002 --device sdb --weight 100
      swift-ring-builder account.builder add --region 1 --zone 2 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6002 --device sdc --weight 100
      swift-ring-builder account.builder
      swift-ring-builder account.builder rebalance
    • Create container ring. Replace STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node
      cd /etc/swift
      swift-ring-builder container.builder create 10 3 1
      swift-ring-builder container.builder add --region 1 --zone 1 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6001 --device sdb --weight 100
      swift-ring-builder container.builder add --region 1 --zone 2 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6001 --device sdc --weight 100
      swift-ring-builder container.builder
      swift-ring-builder container.builder rebalance
    • Create object ring. Replace STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network on the storage node
      cd /etc/swift
      swift-ring-builder object.builder create 10 3 1
      swift-ring-builder object.builder add --region 1 --zone 1 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6000 --device sdb --weight 100
      swift-ring-builder object.builder add --region 1 --zone 2 --ip STORAGE_NODE_MANAGEMENT_INTERFACE_IP_ADDRESS --port 6000 --device sdc --weight 100
      swift-ring-builder object.builder
      swift-ring-builder object.builder rebalance
    • Distribute ring configuration files. Copy the account.ring.gz, container.ring.gz, and object.ring.gz files to the /etc/swift directory on each storage node and any additional nodes running the proxy service.
  7. Finalize the installation
    • Obtain the /etc/swift/swift.conf file from the Object Storage source repository
      curl -o /etc/swift/swift.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample?h=stable/liberty
    • Edit the /etc/swift/swift.conf file and complete the following actions
      • In the [swift-hash] section, configure the hash path prefix and suffix for your environment. Replace HASH_PATH_PREFIX and HASH_PATH_SUFFIX with unique values
        [swift-hash]
        ...
        swift_hash_path_suffix = HASH_PATH_SUFFIX
        swift_hash_path_prefix = HASH_PATH_PREFIX
      • In the [storage-policy:0] section, configure the default storage policy
        [storage-policy:0]
        ...
        name = Policy-0
        default = yes
    • Copy the swift.conf file to the /etc/swift directory on each storage node and any additional nodes running the proxy service.
    • On all nodes, ensure proper ownership of the configuration directory
      chown -R root:swift /etc/swift
    • On the controller node and any other nodes running the proxy service, restart the Object Storage proxy service including its dependencies
      service memcached restart
      service swift-proxy restart
    • On the storage nodes, start the Object Storage services
      swift-init all start