[NOVA]Complete Installation of Multi Servers ---in detail

Multi Servers installation  

ubuntu
HOSTSNICNetworksIPgateway
ubuntu1eth0172.16.2.0/24172.16.2.79172.16.2.1
ubuntu1eth1192.168.1.0/24192.168.1.1192.168.1.2
ubuntu2eth0172.16.2.0/24172.16.2.78172.16.2.1
ubuntu2eth1192.168.1.0/24192.168.1.2none
ubuntu3~6eth1192.168.1.0/24192.168.1.3192.168.1.2

Match to this topology



HOST
ubuntu1 nova-api nova-objectstore mysql RabbitMQ
ubuntu2 nova-network nova-scheduler
ubuntu3 nova-compute
Add Nova repository on Ubuntu , We can use Release-Version or Latest-Trunk by replace "release" to "trunk" 

sudo apt-get update
sudo apt-get install python-software-properties -y
sudo add-apt-repository ppa:nova-core/trunk
sudo apt-get update

Packages
Ubuntu1 nova-api nova-objectstore mysql-server RabbitMQ phpmyadmin
Ubuntu2 nova-network nova-scheduler *dnsmasq *bridge-utils
ubuntu3 nova-compute dnamasq bridge-utils
Install Packages

Ubuntu1

Install RabbitMQ

apt-get install -y rabbitmq-server

Nova API & Objectstore services

apt-get install nova-api nova-objectstore


The following extra packages will be installed:
  libjs-jquery libmysqlclient16 mysql-common nova-common python-amqplib
  python-anyjson python-argparse python-boto python-carrot python-cheetah
  python-cjson python-daemon python-decorator python-dingus python-eventlet
  python-formencode python-gflags python-glance python-greenlet python-httplib2
  python-ipy python-ldap python-libxml2 python-lockfile python-m2crypto
  python-migrate python-mysqldb python-netaddr python-nose python-nova
  python-novaclient python-openid python-openssl python-pam python-paste
  python-pastedeploy python-pastescript python-pkg-resources python-prettytable
  python-pycurl python-routes python-scgi python-serial python-setuptools
  python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita
  python-twisted-bin python-twisted-core python-twisted-web python-webob
  python-zope.interface
Suggested packages:
  javascript-common python-argparse-doc python-cjson-dbg python-egenix-mxdatetime
  python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg
  python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
  python-coverage python-suds python-openssl-doc python-openssl-dbg python-pam-dbg
  python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi
  python-pgsql libjs-mochikit python-cherrypy3 python-cherrypy python-flup
  python-distribute python-distribute-doc libcurl4-gnutls-dev python-pycurl-dbg
  python-wxgtk2.8 python-wxgtk2.6 python-wxgtk python-sqlalchemy-doc
  python-psycopg2 python-kinterbasdb python-pymssql python-twisted-bin-dbg
  python-tk python-gtk2 python-glade2 python-qt3 python-profiler
  python-z3c.recipe.sphinxdoc
The following NEW packages will be installed:
  libjs-jquery libmysqlclient16 mysql-common nova-api nova-common nova-objectstore
  python-amqplib python-anyjson python-argparse python-boto python-carrot
  python-cheetah python-cjson python-daemon python-decorator python-dingus
  python-eventlet python-formencode python-gflags python-glance python-greenlet
  python-httplib2 python-ipy python-ldap python-libxml2 python-lockfile
  python-m2crypto python-migrate python-mysqldb python-netaddr python-nose
  python-nova python-novaclient python-openid python-openssl python-pam
  python-paste python-pastedeploy python-pastescript python-pkg-resources
  python-prettytable python-pycurl python-routes python-scgi python-serial
  python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy
  python-tempita python-twisted-bin python-twisted-core python-twisted-web
  python-webob python-zope.interface

Actually , I think python-suds should be installed too.
Install MySQL , plz remember the password which will set in nova.conf later.

sudo apt-get install mysql-server
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
service mysql restart

Install PHPmyadmin

sudo apt-get install phpmyadmin

Install other tools

sudo apt-get install euca2ools unzip

Ubuntu2


sudo apt-get install nova-network nova-scheduler dnsmasq bridge-utils



The following extra packages will be installed:
  libmysqlclient16 libreadline5 mysql-common nova-common python-amqplib python-anyjson python-boto python-carrot python-cheetah python-daemon python-decorator python-dingus
  python-eventlet python-formencode python-gflags python-glance python-greenlet python-ipy python-ldap python-lockfile python-m2crypto python-migrate python-mysqldb
  python-netaddr python-nose python-nova python-novaclient python-openid python-paste python-pastedeploy python-pastescript python-prettytable python-routes python-scgi
  python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-webob socat vlan
Suggested packages:
  resolvconf radvd python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
  python-coverage python-suds python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi python-pgsql libjs-mochikit python-cherrypy3 python-cherrypy
  python-flup python-sqlalchemy-doc python-psycopg2 python-kinterbasdb python-pymssql
The following NEW packages will be installed:
  bridge-utils dnsmasq libmysqlclient16 libreadline5 mysql-common nova-common nova-network nova-scheduler python-amqplib python-anyjson python-boto python-carrot
  python-cheetah python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags python-glance python-greenlet python-ipy python-ldap
  python-lockfile python-m2crypto python-migrate python-mysqldb python-netaddr python-nose python-nova python-novaclient python-openid python-paste python-pastedeploy
  python-pastescript python-prettytable python-routes python-scgi python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-webob socat
  vlan
0 upgraded, 46 newly installed, 0 to remove and 289 not upgraded.
Need to get 8,008kB of archives.

Install other tools

sudo apt-get install unzip

Ubuntu3


sudo apt-get install nova-compute dnsmasq bridge-utils python-suds


Reading state information... Done
The following extra packages will be installed:
  ajaxterm apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common cpu-checker curl dnsmasq-base ebtables gawk kpartx kvm libaio1 libapr1 libaprutil1
  libaprutil1-dbd-sqlite3 libaprutil1-ldap libasound2 libavahi-client3 libavahi-common-data libavahi-common3 libcurl3 libflac8 libice6 libjs-jquery libmysqlclient16 libnl1
  libogg0 libpciaccess0 libpulse0 libpython2.6 libsdl1.2debian libsdl1.2debian-alsa libsm6 libsndfile1 libvirt-bin libvirt0 libvorbis0a libvorbisenc2 libx11-xcb1 libxcb-atom1
  libxen3 libxi6 libxml2-utils libxtst6 mysql-common nova-common open-iscsi open-iscsi-utils python-amqplib python-anyjson python-argparse python-boto python-carrot
  python-cheetah python-cjson python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags python-glance python-greenlet python-httplib2
  python-ipy python-ldap python-libvirt python-libxml2 python-lockfile python-m2crypto python-migrate python-mysqldb python-netaddr python-nose python-nova python-novaclient
  python-openid python-openssl python-pam python-paste python-pastedeploy python-pastescript python-pkg-resources python-prettytable python-pycurl python-routes python-scgi
  python-serial python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-twisted-bin python-twisted-core python-twisted-web python-webob
  python-zope.interface qemu-common qemu-kvm seabios ssl-cert vgabios x11-common
Suggested packages:
  apache2-doc apache2-suexec apache2-suexec-custom resolvconf libasound2-plugins javascript-common pulseaudio policykit-1 lvm2 python-argparse-doc python-cjson-dbg
  python-egenix-mxdatetime python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
  python-coverage python-openssl-doc python-openssl-dbg python-pam-dbg python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi python-pgsql
  libjs-mochikit python-cherrypy3 python-cherrypy python-flup python-distribute python-distribute-doc libcurl4-gnutls-dev python-pycurl-dbg python-wxgtk2.8 python-wxgtk2.6
  python-wxgtk python-sqlalchemy-doc python-psycopg2 python-kinterbasdb python-pymssql python-twisted-bin-dbg python-tk python-gtk2 python-glade2 python-qt3 python-profiler
  python-z3c.recipe.sphinxdoc kvm-pxe mol-drivers-macosx openbios-sparc ubuntu-vm-builder uml-utilities openssl-blacklist
Recommended packages:
  python-psyco
The following NEW packages will be installed:
  ajaxterm apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common bridge-utils cpu-checker curl dnsmasq dnsmasq-base ebtables gawk kpartx kvm libaio1 libapr1
  libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libasound2 libavahi-client3 libavahi-common-data libavahi-common3 libcurl3 libflac8 libice6 libjs-jquery
  libmysqlclient16 libnl1 libogg0 libpciaccess0 libpulse0 libpython2.6 libsdl1.2debian libsdl1.2debian-alsa libsm6 libsndfile1 libvirt-bin libvirt0 libvorbis0a libvorbisenc2
  libx11-xcb1 libxcb-atom1 libxen3 libxi6 libxml2-utils libxtst6 mysql-common nova-common nova-compute open-iscsi open-iscsi-utils python-amqplib python-anyjson
  python-argparse python-boto python-carrot python-cheetah python-cjson python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags
  python-glance python-greenlet python-httplib2 python-ipy python-ldap python-libvirt python-libxml2 python-lockfile python-m2crypto python-migrate python-mysqldb
  python-netaddr python-nose python-nova python-novaclient python-openid python-openssl python-pam python-paste python-pastedeploy python-pastescript python-pkg-resources
  python-prettytable python-pycurl python-routes python-scgi python-serial python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-suds python-tempita
  python-twisted-bin python-twisted-core python-twisted-web python-webob python-zope.interface qemu-common qemu-kvm seabios ssl-cert vgabios x11-common
0 upgraded, 110 newly installed, 0 to remove and 41 not upgraded.
Need to get 24.4MB of archives.
After this operation, 99.4MB of additional disk space will be used.

Install other tools

sudo apt-get install unzip

Configuration

The most important stuff is Network mode
There are three modes support in Nova , depends on your environment.
*Flat
*FlatDHCP
*Vlan
/etc/nova/nova.conf

--sql_connection=mysql://root:nova@<$mysql_server_IP>/nova
--s3_host=<$Objectstore_IP>
--rabbit_host=<$RabbitMQ_IP>
--cc_host=<API_Server_IP>
--ec2_url=http://<$API_Server_IP>:8773/services/Cloud
--daemonize=1
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--FAKE_subdomain=ec2
--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets
--libvirt_type=<$type_of_Hypervisor>
--network_manager=nova.network.manager.<$Network_mode> # FlatManager / FlatDHCPManager / VlanManager
--flat_interface=<$flat_network_NIC>                   # What if there's only one NIC?
--logdir=/var/log/nova
--verbose
--fixed_range=<All_project_Network>
--network_size=<Number of IPs in this Network>

An Example as below

--sql_connection=mysql://root:nova@192.168.1.1/nova
--s3_host=192.168.1.1
--rabbit_host=192.168.1.1
--cc_host=192.168.1.1
--ec2_url=http://192.168.1.1:8773/services/Cloud
--daemonize=1
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--FAKE_subdomain=ec2
--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets
--libvirt_type=kvm
--network_manager=nova.network.manager.FlatManager
--logdir=/var/log/nova
--verbose
--fixed_range=192.168.1.0/24
--network_size=256

Copy nova.conf to all service hosts...
scp nova.conf ubuntu1:/ubuntu2:/ubunt3:

Configure MySQL DB on ubuntu1

Create nova database

mysql -uroot -p$MYSQL_PASS -e 'CREATE DATABASE nova;'

Update the DB to include user 'root'@'%' with super user privileges:

mysql -uroot -p$MYSQL_PASS -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;"

Set mySQL root password:
mysql -uroot -p$MYSQL_PASS -e "SET PASSWORD FOR 'root'@'%' = PASSWORD('$MYSQL_PASS');" 
service mysql restart

SETUP NOVA ENVIRONMENT (with root user)

on Ubuntu1

nova-manage db sync
nova-manage user admin <user_name>
nova-manage project create <project_name> <user_name>
nova-manage network create <project-network> <number-of-networks-in-project> <IPs in project>

Create Nova certifications


nova-manage project zipfile $NOVA_PROJECT $NOVA_PROJECT_USER <$PATH>/novacreds.zip

ex.
nova-manage project zipfile hugopro hugo ~/novacreds.zip

unzip novacreds.zip
source ~/novarc

Source it when bootup

cat ~/novarc >> ~/.bashrc

After all components is ready , restart all service
service libvirt-bin restart; service nova-network restart; service nova-compute restart; service nova-api restart; service nova-objectstore restart; service nova-scheduler restart;
On ubuntu1

service nova-api restart;service nova-objectstore restart;

On ubuntu2

service nova-network restart;service nova-scheduler restart;

On ubuntu3

service libvirt-bin restart;service nova-compute restart;

Closing steps, and cleaning up

Allow ping & SSH to instance

euca-authorize -P icmp -t -1:-1 default
euca-authorize -P tcp -p 22 default

Download image or from http://uec-images.ubuntu.com/

image="ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz" 
wget http://smoser.brickies.net/ubuntu/ttylinux-uec/$image
uec-publish-tarball $image mybucket

Add keypair

euca-add-keypair hugokey > hugokey.priv
chmod 600 hugokey.priv

Troubleshooting

1. While nova-network service can not be ran up , if there's a error about dnsmasq?
A. 

killall dnsmasq
service nova-network restart

2. While the instance failed to spawn
A.

killall dnsmasq
service nova-compute restart

3. Don't forget to add ipv4 forward on NOVA-NETWORK

vi /etc/sysctl.conf   #turn on ipv4forward
sysctl -p /etc/sysctl.conf

try other image first
check those issue
https://answers.launchpad.net/nova/+question/151725
Using Glance service or NFS share image from fronend
https://answers.launchpad.net/nova/+question/151125
sudo modprobe kvm
sudo chgroup kvm /dev/kvm
sudo chmod g+w /dev/kvm

Comments

  1. Manoj K SamtaniAugust 27, 2011 at 3:55 AM

    after reading, i feel it looks gr8...

    And also let me know if I have installed nova-compute on both cloud controller n compute node then on both the machines an br100 will be created......what will be the IP in case am using FlatDHCPManager with 10.0.0.0/24 network series.......

    In my case cloud controller br100 getting 10.0.0.1 andd on compute node its showing error that 10.0.0.1 is already in use in nova-compute.log file

    can plz help me on this..after reading, i feel it looks gr8...

    And also let me know if I have installed nova-compute on both cloud controller n compute node then on both the machines an br100 will be created......what will be the IP in case am using FlatDHCPManager with 10.0.0.0/24 network series.......

    In my case cloud controller br100 getting 10.0.0.1 andd on compute node its showing error that 10.0.0.1 is already in use in nova-compute.log file

    can plz help me on this..

    ReplyDelete
  2. Hugo KuoAugust 27, 2011 at 4:17 PM

    The br100 IP will be 10.0.0.1 in your case , the first ip of a network is set to GW for instance.

    In FlatDHCP , plz don't setup ip for br100 manually. linux_net.py will help admin to setup it up. plz check the IP 10.0.0.1 on compute-node with "ip addr show".

    ReplyDelete

Post a Comment

Popular posts from this blog

The method of Web Service HA+LB in OPENSTACK NOVA for private cloud only

Image
Cloud Server HA+LB  in NOVA Our purpose is to provide uninterrupted   web services  with low cost in OPENSTACK. Due to all instances traffic through Nova-Network host in a basic deployment of NOVA . And it's hard to do HA+LB for Nova-Network. As I know it will be very hard to implement.  After read "Bootstrapping OpenStack Clouds" doc writed by DELL....The fault zone concept it much easier and cheaper.  There're three diagrams .... I. Two zones topology , share nothing between each zones. But combine two instance network in same network segment. Zone1 using 192.168.1.0/25,   Zone2 using 192.168.1.128/25 . We do  not using floating ip.  Let instance route from Core Network  Switch directly.
Read more