Multi Servers installation
ubuntu
| HOSTS | NIC | Networks | IP | gateway |
| ubuntu1 | eth0 | 172.16.2.0/24 | 172.16.2.79 | 172.16.2.1 |
| ubuntu1 | eth1 | 192.168.1.0/24 | 192.168.1.1 | 192.168.1.2 |
| ubuntu2 | eth0 | 172.16.2.0/24 | 172.16.2.78 | 172.16.2.1 |
| ubuntu2 | eth1 | 192.168.1.0/24 | 192.168.1.2 | none |
| ubuntu3~6 | eth1 | 192.168.1.0/24 | 192.168.1.3 | 192.168.1.2 |
Match to this topology
HOST
ubuntu1 nova-api nova-objectstore mysql RabbitMQ
ubuntu2 nova-network nova-scheduler
ubuntu3 nova-compute
Add Nova repository on Ubuntu , We can use Release-Version or Latest-Trunk by replace "release" to "trunk"
sudo apt-get update
sudo apt-get install python-software-properties -y
sudo add-apt-repository ppa:nova-core/trunk
sudo apt-get update
Packages
Ubuntu1 nova-api nova-objectstore mysql-server RabbitMQ phpmyadmin
Ubuntu2 nova-network nova-scheduler *dnsmasq *bridge-utils
ubuntu3 nova-compute dnamasq bridge-utils
Install Packages
Ubuntu1
Install RabbitMQ
apt-get install -y rabbitmq-server
Nova API & Objectstore services
apt-get install nova-api nova-objectstore
The following extra packages will be installed:
libjs-jquery libmysqlclient16 mysql-common nova-common python-amqplib
python-anyjson python-argparse python-boto python-carrot python-cheetah
python-cjson python-daemon python-decorator python-dingus python-eventlet
python-formencode python-gflags python-glance python-greenlet python-httplib2
python-ipy python-ldap python-libxml2 python-lockfile python-m2crypto
python-migrate python-mysqldb python-netaddr python-nose python-nova
python-novaclient python-openid python-openssl python-pam python-paste
python-pastedeploy python-pastescript python-pkg-resources python-prettytable
python-pycurl python-routes python-scgi python-serial python-setuptools
python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita
python-twisted-bin python-twisted-core python-twisted-web python-webob
python-zope.interface
Suggested packages:
javascript-common python-argparse-doc python-cjson-dbg python-egenix-mxdatetime
python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg
python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
python-coverage python-suds python-openssl-doc python-openssl-dbg python-pam-dbg
python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi
python-pgsql libjs-mochikit python-cherrypy3 python-cherrypy python-flup
python-distribute python-distribute-doc libcurl4-gnutls-dev python-pycurl-dbg
python-wxgtk2.8 python-wxgtk2.6 python-wxgtk python-sqlalchemy-doc
python-psycopg2 python-kinterbasdb python-pymssql python-twisted-bin-dbg
python-tk python-gtk2 python-glade2 python-qt3 python-profiler
python-z3c.recipe.sphinxdoc
The following NEW packages will be installed:
libjs-jquery libmysqlclient16 mysql-common nova-api nova-common nova-objectstore
python-amqplib python-anyjson python-argparse python-boto python-carrot
python-cheetah python-cjson python-daemon python-decorator python-dingus
python-eventlet python-formencode python-gflags python-glance python-greenlet
python-httplib2 python-ipy python-ldap python-libxml2 python-lockfile
python-m2crypto python-migrate python-mysqldb python-netaddr python-nose
python-nova python-novaclient python-openid python-openssl python-pam
python-paste python-pastedeploy python-pastescript python-pkg-resources
python-prettytable python-pycurl python-routes python-scgi python-serial
python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy
python-tempita python-twisted-bin python-twisted-core python-twisted-web
python-webob python-zope.interface
Actually , I think python-suds should be installed too.
Install MySQL , plz remember the password which will set in nova.conf later.
sudo apt-get install mysql-server
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
service mysql restart
Install PHPmyadmin
sudo apt-get install phpmyadmin
Install other tools
sudo apt-get install euca2ools unzip
Ubuntu2
sudo apt-get install nova-network nova-scheduler dnsmasq bridge-utils
The following extra packages will be installed:
libmysqlclient16 libreadline5 mysql-common nova-common python-amqplib python-anyjson python-boto python-carrot python-cheetah python-daemon python-decorator python-dingus
python-eventlet python-formencode python-gflags python-glance python-greenlet python-ipy python-ldap python-lockfile python-m2crypto python-migrate python-mysqldb
python-netaddr python-nose python-nova python-novaclient python-openid python-paste python-pastedeploy python-pastescript python-prettytable python-routes python-scgi
python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-webob socat vlan
Suggested packages:
resolvconf radvd python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
python-coverage python-suds python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi python-pgsql libjs-mochikit python-cherrypy3 python-cherrypy
python-flup python-sqlalchemy-doc python-psycopg2 python-kinterbasdb python-pymssql
The following NEW packages will be installed:
bridge-utils dnsmasq libmysqlclient16 libreadline5 mysql-common nova-common nova-network nova-scheduler python-amqplib python-anyjson python-boto python-carrot
python-cheetah python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags python-glance python-greenlet python-ipy python-ldap
python-lockfile python-m2crypto python-migrate python-mysqldb python-netaddr python-nose python-nova python-novaclient python-openid python-paste python-pastedeploy
python-pastescript python-prettytable python-routes python-scgi python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-webob socat
vlan
0 upgraded, 46 newly installed, 0 to remove and 289 not upgraded.
Need to get 8,008kB of archives.
Install other tools
sudo apt-get install unzip
Ubuntu3
sudo apt-get install nova-compute dnsmasq bridge-utils python-suds
Reading state information... Done
The following extra packages will be installed:
ajaxterm apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common cpu-checker curl dnsmasq-base ebtables gawk kpartx kvm libaio1 libapr1 libaprutil1
libaprutil1-dbd-sqlite3 libaprutil1-ldap libasound2 libavahi-client3 libavahi-common-data libavahi-common3 libcurl3 libflac8 libice6 libjs-jquery libmysqlclient16 libnl1
libogg0 libpciaccess0 libpulse0 libpython2.6 libsdl1.2debian libsdl1.2debian-alsa libsm6 libsndfile1 libvirt-bin libvirt0 libvorbis0a libvorbisenc2 libx11-xcb1 libxcb-atom1
libxen3 libxi6 libxml2-utils libxtst6 mysql-common nova-common open-iscsi open-iscsi-utils python-amqplib python-anyjson python-argparse python-boto python-carrot
python-cheetah python-cjson python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags python-glance python-greenlet python-httplib2
python-ipy python-ldap python-libvirt python-libxml2 python-lockfile python-m2crypto python-migrate python-mysqldb python-netaddr python-nose python-nova python-novaclient
python-openid python-openssl python-pam python-paste python-pastedeploy python-pastescript python-pkg-resources python-prettytable python-pycurl python-routes python-scgi
python-serial python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-tempita python-twisted-bin python-twisted-core python-twisted-web python-webob
python-zope.interface qemu-common qemu-kvm seabios ssl-cert vgabios x11-common
Suggested packages:
apache2-doc apache2-suexec apache2-suexec-custom resolvconf libasound2-plugins javascript-common pulseaudio policykit-1 lvm2 python-argparse-doc python-cjson-dbg
python-egenix-mxdatetime python-dns python-greenlet-doc python-greenlet-dev python-greenlet-dbg python-ldap-doc mysql-server-5.1 mysql-server python-mysqldb-dbg ipython
python-coverage python-openssl-doc python-openssl-dbg python-pam-dbg python-pastewebkit libapache2-mod-wsgi libapache2-mod-python libapache2-mod-scgi python-pgsql
libjs-mochikit python-cherrypy3 python-cherrypy python-flup python-distribute python-distribute-doc libcurl4-gnutls-dev python-pycurl-dbg python-wxgtk2.8 python-wxgtk2.6
python-wxgtk python-sqlalchemy-doc python-psycopg2 python-kinterbasdb python-pymssql python-twisted-bin-dbg python-tk python-gtk2 python-glade2 python-qt3 python-profiler
python-z3c.recipe.sphinxdoc kvm-pxe mol-drivers-macosx openbios-sparc ubuntu-vm-builder uml-utilities openssl-blacklist
Recommended packages:
python-psyco
The following NEW packages will be installed:
ajaxterm apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common bridge-utils cpu-checker curl dnsmasq dnsmasq-base ebtables gawk kpartx kvm libaio1 libapr1
libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libasound2 libavahi-client3 libavahi-common-data libavahi-common3 libcurl3 libflac8 libice6 libjs-jquery
libmysqlclient16 libnl1 libogg0 libpciaccess0 libpulse0 libpython2.6 libsdl1.2debian libsdl1.2debian-alsa libsm6 libsndfile1 libvirt-bin libvirt0 libvorbis0a libvorbisenc2
libx11-xcb1 libxcb-atom1 libxen3 libxi6 libxml2-utils libxtst6 mysql-common nova-common nova-compute open-iscsi open-iscsi-utils python-amqplib python-anyjson
python-argparse python-boto python-carrot python-cheetah python-cjson python-daemon python-decorator python-dingus python-eventlet python-formencode python-gflags
python-glance python-greenlet python-httplib2 python-ipy python-ldap python-libvirt python-libxml2 python-lockfile python-m2crypto python-migrate python-mysqldb
python-netaddr python-nose python-nova python-novaclient python-openid python-openssl python-pam python-paste python-pastedeploy python-pastescript python-pkg-resources
python-prettytable python-pycurl python-routes python-scgi python-serial python-setuptools python-sqlalchemy python-sqlalchemy-ext python-stompy python-suds python-tempita
python-twisted-bin python-twisted-core python-twisted-web python-webob python-zope.interface qemu-common qemu-kvm seabios ssl-cert vgabios x11-common
0 upgraded, 110 newly installed, 0 to remove and 41 not upgraded.
Need to get 24.4MB of archives.
After this operation, 99.4MB of additional disk space will be used.
Install other tools
sudo apt-get install unzip
Configuration
The most important stuff is Network mode
There are three modes support in Nova , depends on your environment.
*Flat
*FlatDHCP
*Vlan
/etc/nova/nova.conf
--sql_connection=mysql://root:nova@<$mysql_server_IP>/nova
--s3_host=<$Objectstore_IP>
--rabbit_host=<$RabbitMQ_IP>
--cc_host=<API_Server_IP>
--ec2_url=http://<$API_Server_IP>:8773/services/Cloud
--daemonize=1
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--FAKE_subdomain=ec2
--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets
--libvirt_type=<$type_of_Hypervisor>
--network_manager=nova.network.manager.<$Network_mode> # FlatManager / FlatDHCPManager / VlanManager
--flat_interface=<$flat_network_NIC> # What if there's only one NIC?
--logdir=/var/log/nova
--verbose
--fixed_range=<All_project_Network>
--network_size=<Number of IPs in this Network>
An Example as below
--sql_connection=mysql://root:nova@192.168.1.1/nova
--s3_host=192.168.1.1
--rabbit_host=192.168.1.1
--cc_host=192.168.1.1
--ec2_url=http://192.168.1.1:8773/services/Cloud
--daemonize=1
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--FAKE_subdomain=ec2
--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets
--libvirt_type=kvm
--network_manager=nova.network.manager.FlatManager
--logdir=/var/log/nova
--verbose
--fixed_range=192.168.1.0/24
--network_size=256
Copy nova.conf to all service hosts...
scp nova.conf ubuntu1:/ubuntu2:/ubunt3:
Create nova database
mysql -uroot -p$MYSQL_PASS -e 'CREATE DATABASE nova;'
Update the DB to include user 'root'@'%' with super user privileges:
mysql -uroot -p$MYSQL_PASS -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;"
Set mySQL root password:
mysql -uroot -p$MYSQL_PASS -e "SET PASSWORD FOR 'root'@'%' = PASSWORD('$MYSQL_PASS');"
service mysql restart
SETUP NOVA ENVIRONMENT (with root user)
on Ubuntu1
nova-manage db sync
nova-manage user admin <user_name>
nova-manage project create <project_name> <user_name>
nova-manage network create <project-network> <number-of-networks-in-project> <IPs in project>
Create Nova certifications
nova-manage project zipfile $NOVA_PROJECT $NOVA_PROJECT_USER <$PATH>/novacreds.zip
ex.
nova-manage project zipfile hugopro hugo ~/novacreds.zip
unzip novacreds.zip
source ~/novarc
Source it when bootup
cat ~/novarc >> ~/.bashrc
After all components is ready , restart all service
service libvirt-bin restart; service nova-network restart; service nova-compute restart; service nova-api restart; service nova-objectstore restart; service nova-scheduler restart;
On ubuntu1
service nova-api restart;service nova-objectstore restart;
On ubuntu2
service nova-network restart;service nova-scheduler restart;
On ubuntu3
service libvirt-bin restart;service nova-compute restart;
Closing steps, and cleaning up
Allow ping & SSH to instance
euca-authorize -P icmp -t -1:-1 default
euca-authorize -P tcp -p 22 default
image="ttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz"
wget http://smoser.brickies.net/ubuntu/ttylinux-uec/$image
uec-publish-tarball $image mybucket
Add keypair
euca-add-keypair hugokey > hugokey.priv
chmod 600 hugokey.priv
Troubleshooting
1. While nova-network service can not be ran up , if there's a error about dnsmasq?
A.
killall dnsmasq
service nova-network restart
2. While the instance failed to spawn
A.
killall dnsmasq
service nova-compute restart
3. Don't forget to add ipv4 forward on NOVA-NETWORK
vi /etc/sysctl.conf #turn on ipv4forward
sysctl -p /etc/sysctl.conf
try other image first
after reading, i feel it looks gr8...
ReplyDeleteAnd also let me know if I have installed nova-compute on both cloud controller n compute node then on both the machines an br100 will be created......what will be the IP in case am using FlatDHCPManager with 10.0.0.0/24 network series.......
In my case cloud controller br100 getting 10.0.0.1 andd on compute node its showing error that 10.0.0.1 is already in use in nova-compute.log file
can plz help me on this..after reading, i feel it looks gr8...
And also let me know if I have installed nova-compute on both cloud controller n compute node then on both the machines an br100 will be created......what will be the IP in case am using FlatDHCPManager with 10.0.0.0/24 network series.......
In my case cloud controller br100 getting 10.0.0.1 andd on compute node its showing error that 10.0.0.1 is already in use in nova-compute.log file
can plz help me on this..
The br100 IP will be 10.0.0.1 in your case , the first ip of a network is set to GW for instance.
ReplyDeleteIn FlatDHCP , plz don't setup ip for br100 manually. linux_net.py will help admin to setup it up. plz check the IP 10.0.0.1 on compute-node with "ip addr show".