Show me the code! – By Davanum Srinivas

February 6, 2015

Deploy a Centos container in Docker for running OpenStack Nova tests

Filed under: Uncategorized — Davanum Srinivas @ 1:26 pm

First start a centos container using docker command line and drop down to the bash shell

dims@dims-ubuntu:~$ sudo docker run -i -t centos /bin/bash
[root@218b625b2529 nova]# rpm -q centos-release

Install the EPEL repository

[root@218b625b2529 /]# yum -y install epel-release

Install a few things needed before we can run Nova tests as documented here:

[root@218b625b2529 /]# yum -y install git python-devel openssl-devel python-pip git gcc libxslt-devel mysql-devel postgresql-devel libffi-devel libvirt-devel graphviz sqlite-devel

We need tox as well.

[root@218b625b2529 /]# pip install tox

Get the latest nova trunk

[root@218b625b2529 /]# git clone
[root@218b625b2529 /]# cd nova
[root@218b625b2529 nova]#

Run the tests as usual

[root@218b625b2529 nova]# tox -e py27 nova.tests.unit.test_crypto

January 13, 2015

Quickly running a single OpenStack Nova test

Filed under: Nova, openstack — Davanum Srinivas @ 10:41 am

Here’s how we usually run a single test

dims@dims-mac:~/openstack/nova$ time tox -e py27 nova.tests.unit.test_versions
py27 develop-inst-noop: /Users/dims/openstack/nova
py27 runtests: PYTHONHASHSEED='0'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | bash tools/ nova.tests.unit.test_versions
{1} nova.tests.unit.test_versions.VersionTestCase.test_version_string_with_package_is_good [0.180036s] ... ok
{0} nova.tests.unit.test_versions.VersionTestCase.test_release_file [0.184115s] ... ok

Ran: 2 tests in 13.0000 sec.
 - Passed: 2
 - Skipped: 0
 - Failed: 0
Sum of execute time for each test: 0.3642 sec.

Worker Balance
 - Worker 0 (1 tests) => 0:00:00.184115s
 - Worker 1 (1 tests) => 0:00:00.180036s
________________________________________________________________________________________________________________________ summary _________________________________________________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

real	0m14.452s
user	0m16.392s
sys	0m2.354s

Sometimes the usual way is not very helpful, especially when you are working on some new code and say running into issues importing code. Then, here’s what you do.

First, activate the py27 virtualenv

dims@dims-mac:~/openstack/nova$ . .tox/py27/bin/activate

Then use testtools

(py27)dims@dims-mac:~/openstack/nova$ python -m nova.tests.unit.test_versions
Tests running...

Ran 2 tests in 0.090s

Or you can install pytest

(py27)dims@dims-mac:~/openstack/nova$ pip install pytest
Collecting pytest
  Downloading pytest-2.6.4.tar.gz (512kB)
    100% |################################| 516kB 877kB/s
Collecting py>=1.4.25 (from pytest)
  Downloading py-1.4.26.tar.gz (190kB)
    100% |################################| 192kB 4.3MB/s
Installing collected packages: py, pytest
  Running install for py
  Running install for pytest
    Installing py.test-2.7 script to /Users/dims/openstack/nova/.tox/py27/bin
    Installing py.test script to /Users/dims/openstack/nova/.tox/py27/bin
Successfully installed py-1.4.26 pytest-2.6.4

And then run the same test using py.test

(py27)dims@dims-mac:~/openstack/nova$ find . -name py.test

(py27)dims@dims-mac:~/openstack/nova$ .tox/py27/bin/py.test -svx nova/tests/unit/
================================================================================================================== test session starts ===================================================================================================================
platform darwin -- Python 2.7.8 -- py-1.4.26 -- pytest-2.6.4 -- /Users/dims/openstack/nova/.tox/py27/bin/python2.7
collected 2 items

nova/tests/unit/ PASSED
nova/tests/unit/ PASSED

================================================================================================================ 2 passed in 1.69 seconds ================================================================================================================

These tips are based on the openstack-dev mailing list discussion:

June 29, 2014

CFv2 Deployment on latest DevStack using MicroBOSH

Filed under: Uncategorized — Davanum Srinivas @ 9:32 pm

As promised, here’s a follow up to the 2 previous posts:

Let’s now deploy a full CFv2 instance using microbosh, instructions are from here:
Install Cloud Foundry on OpenStack

Here’s the edited flavor(s) that i used:

Here’s the micro_bosh.yml for completeness:

Here’s the cf-173-openstack.yml originally from @ferdy with minor tweaks:

Things to tweak:
In my cf-173-openstack.yml, i had set static_ip to So once the MicroBOSH finished deploying CFv2, i had to find the correct vm for the router and set its floating ip to You can do this by running “bosh vms”, look for the ip address for “router/0”, then find the correct vm in horizon or using nova list and then set it’s ip address. You need to do this before you try to use the cf command line API. Be sure to download the latest and greatest CLI from

Flaky Stuff:
postgres vm ran into trouble multiple times, i figured out how to stop/start the shell script by hand but since other vms like cloud_controller, clock_global, cloud_controller_worker had issues it was better to whack a big hammer and run “bosh delete deployment cf” and re-instantiate the vms. (yes, i tried combinations of bosh start/recreate/restart commands as well)

bosh-lite’s is very helpful about how to build the cf release. Andy’s blogpost helped quite a bit to peel the onion for debugging as did Dr Nic’s posts. Thanks Folks!

June 24, 2014

Deploying BOSH with Micro BOSH on latest DevStack

Filed under: Uncategorized — Davanum Srinivas @ 1:33 pm

Follow up to Running Cloud Foundry’s Micro BOSH On Latest DevStack, I had to bump the VOLUME_BACKING_FILE_SIZE to 200GB in devstack as 100GB was not enough. Instructions from were handy as usual. Here’s my bosh-openstack.yml

name: bosh-openstack
director_uuid: 80a0b9cc-a7e4-4134-81ee-a186e8bebff8

  name: bosh
  version: latest

  workers: 3
  network: default
  reuse_compilation_vms: true
    instance_type: m1.small

  canaries: 1
  canary_watch_time: 3000-120000
  update_watch_time: 3000-120000
  max_in_flight: 4

  - name: floating
    type: vip
    cloud_properties: {}
  - name: default
    type: dynamic
    cloud_properties: {}

  - name: common
    network: default
    size: 8
      name: bosh-openstack-kvm-ubuntu
      version: latest
      instance_type: m1.small

  - name: nats
    template: nats
    instances: 1
    resource_pool: common
      - name: default
        default: [dns, gateway]

  - name: redis
    template: redis
    instances: 1
    resource_pool: common
      - name: default
        default: [dns, gateway]

  - name: postgres
    template: postgres
    instances: 1
    resource_pool: common
    persistent_disk: 16384
      - name: default
        default: [dns, gateway]

  - name: powerdns
    template: powerdns
    instances: 1
    resource_pool: common
      - name: default
        default: [dns, gateway]
      - name: floating

  - name: blobstore
    template: blobstore
    instances: 1
    resource_pool: common
    persistent_disk: 51200
      - name: default
        default: [dns, gateway]

  - name: director
    template: director
    instances: 1
    resource_pool: common
    persistent_disk: 16384
      - name: default
        default: [dns, gateway]
      - name: floating

  - name: registry
    template: registry
    instances: 1
    resource_pool: common
      - name: default
        default: [dns, gateway]

  - name: health_monitor
    template: health_monitor
    instances: 1
    resource_pool: common
      - name: default
        default: [dns, gateway]

    address: 0.nats.default.bosh-openstack.microbosh
    user: nats
    password: nats

    address: 0.redis.default.bosh-openstack.microbosh
    password: redis

  postgres: &bosh_db
    host: 0.postgres.default.bosh-openstack.microbosh
    user: postgres
    password: postgres
    database: bosh

    db: *bosh_db

    address: 0.blobstore.default.bosh-openstack.microbosh
      user: agent
      password: agent
      user: director
      password: director

    name: bosh
    address: 0.director.default.bosh-openstack.microbosh
    db: *bosh_db

    address: 0.registry.default.bosh-openstack.microbosh
    db: *bosh_db
      user: registry
      password: registry

      user: hm
      password: hm
      user: admin
      password: admin
    resurrector_enabled: true


    username: admin
    api_key: passw0rd
    tenant: admin
    default_security_groups: ["default", "ssh", "bosh"]
    default_key_name: microbosh

Running Cloud Foundry’s Micro BOSH on latest DevStack

Filed under: cloud foundry, openstack — Tags: , , , — Davanum Srinivas @ 7:45 am

The Cloud Foundry docs are excellent. Here’s where i started from:

I provisioned a big beefy bare metal box with Ubuntu 14.04 LTS on SoftLayer and installed DevStack on it as usual. Here’s the super simple local.conf that i used. Note the 100GB volume needed later for Micro BOSH deployment.


Once you deploy DevStack, follow the steps in the CF docs url above, Here’s the ~/.fog file that i used in step #2. Note that i am just using the “admin” credentials and the “admin” tenant for all openstack operations

  :openstack_api_key:   passw0rd
  :openstack_username:  admin
  :openstack_tenant: admin

Here’s a script to create all the security groups and floating ips needed in step #3.

nova secgroup-add-rule default icmp -1 -1 
nova secgroup-add-rule default tcp 22 22

nova secgroup-create ssh ssh
nova secgroup-add-rule ssh udp 68 68
nova secgroup-add-rule ssh tcp 22 22
nova secgroup-add-rule ssh icmp -1 -1

nova secgroup-create bosh bosh
nova secgroup-add-group-rule bosh bosh tcp 1 65535
nova secgroup-add-rule bosh tcp 4222 4222
nova secgroup-add-rule bosh tcp 6868 6868
nova secgroup-add-rule bosh tcp 25250 25250
nova secgroup-add-rule bosh tcp 25555 25555
nova secgroup-add-rule bosh tcp 25777 25777
nova secgroup-add-rule bosh tcp 53 53
nova secgroup-add-rule bosh udp 68 68
nova secgroup-add-rule bosh udp 53 53

nova secgroup-create cf-public cf-public
nova secgroup-add-rule cf-public udp 68 68
nova secgroup-add-rule cf-public tcp 80 80
nova secgroup-add-rule cf-public tcp 443 443

nova secgroup-create cf-private cf-private
nova secgroup-add-rule cf-private udp 68 68
nova secgroup-add-group-rule cf-private cf-private tcp 1 65535

nova floating-ip-create

For Step #4, i used this stem cell – bosh-stemcell-2611-openstack-kvm-ubuntu-lucid.tgz and the following microbosh.yml

name: microbosh-openstack

  level: DEBUG

  type: dynamic

  persistent_disk: 16384
    instance_type: m1.small

  plugin: openstack
      username: admin
      api_key: passw0rd
      tenant: admin
      default_security_groups: ["ssh", "bosh"]
      default_key_name: microbosh
      private_key: /opt/stack/bosh-workspace/microbosh.pem

      max_threads: 3
      resurrector_enabled: true

That was it! Final check on status.

stack@bigblue:~/bosh-workspace/deployments$ bosh micro status
Stemcell CID   6203baa8-d64f-4701-952e-a33ea0aabdb0
Stemcell name  bosh-stemcell-2611-openstack-kvm-ubuntu-lucid
VM CID         55fc5d01-e56a-4120-bd8c-6ec1c7d295ea
Disk CID       a9eb53a9-74a1-4210-8afe-d04ba68536ac
Micro BOSH CID bm-c84e8442-016a-499f-aa31-19a8a9c58a9e
Deployment     /opt/stack/bosh-workspace/deployments/microbosh-openstack/micro_bosh.yml

stack@bigblue:~/bosh-workspace/deployments$ bosh status

  Name       microbosh-openstack
  Version    1.2611.0 (00000000)
  User       admin
  UUID       46aa8b77-3f41-4268-952e-37c07f938b86
  CPI        openstack
  dns        enabled (domain_name: microbosh)
  compiled_package_cache disabled
  snapshots  disabled

  not set

Next up, Will try steps #5, #6, #7 and report back here.

March 17, 2014

Generating a Bitcoin Private Key and Address

Filed under: bitcoin — Davanum Srinivas @ 11:30 am

Ken Shirriff’s blog post here has an excellent introduction to Bitcoin. One of his code snippets shows a sample python code to generate a private key in WIF format and an address. I tweaked it just a bit to replace usage of python’s random module with os.urandom and stripped it down to just what’s needed to show the exponent, private key and address. Here’s the effort in a gist:

October 17, 2012

Scripts to start/stop OpenStack environment built using DevStack

Filed under: Uncategorized — Davanum Srinivas @ 4:14 pm

Work in progress…Once i bootstrapped an OpenStack install using DevStack i wanted keep/use the environment that was just built. I could not find scripts to start/stop all the services, here’s my effort. If someone has a better way, please let me know!.


rm -rf /var/log/nova/*.log

service mysql start
service rabbitmq-server start

cd /opt/stack/glance/bin
/opt/stack/glance/bin/glance-registry --config-file=/etc/glance/glance-registry.conf > /var/log/nova/glance-registry.log 2>&1 &

cd /opt/stack/glance/bin
/opt/stack/glance/bin/glance-api --config-file=/etc/glance/glance-api.conf > /var/log/nova/glance-api.log 2>&1 &
echo "Waiting for g-api to start..."
if ! timeout 60 sh -c "while ! wget --no-proxy -q -O-;
do sleep 1; done"; then
        echo "g-api did not start"
        exit 1
echo "Done."

cd /opt/stack/keystone/bin
/opt/stack/keystone/bin/keystone-all --config-file /etc/keystone/keystone.conf --log-config /etc/keystone/logging.conf -d --debug > /var/log/nova/keystone-all.log 2>&1 &
echo "Waiting for keystone to start..."
if ! timeout 60 sh -c "while ! wget --no-proxy -q -O-;
do sleep 1; done"; then
        echo "keystone did not start"
        exit 1
echo "Done."

cd /opt/stack/cinder/bin/
/opt/stack/cinder/bin/cinder-api --config-file /etc/cinder/cinder.conf > /var/log/nova/cinder-api.log 2>&1 &

cd /opt/stack/cinder/bin/
/opt/stack/cinder/bin/cinder-volume --config-file /etc/cinder/cinder.conf > /var/log/nova/cinder-volume.log 2>&1 &

cd /opt/stack/cinder/bin/
/opt/stack/cinder/bin/cinder-scheduler --config-file /etc/cinder/cinder.conf > /var/log/nova/cinder-scheduler.log 2>&1 &

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-api > /var/log/nova/nova-api.log 2>&1 &
echo "Waiting for nova-api to start..."
if ! timeout 60 sh -c "while ! wget --no-proxy -q -O-;
do sleep 1; done"; then
        echo "nova-api did not start"
        exit 1
echo "Done."

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-scheduler > /var/log/nova/nova-scheduler.log 2>&1 &

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-cert > /var/log/nova/nova-cert.log 2>&1 &

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-objectstore > /var/log/nova/nova-objectstore.log 2>&1 &

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-network > /var/log/nova/nova-network.log 2>&1 &

cd /opt/stack/nova/bin 
/opt/stack/nova/bin/nova-compute > /var/log/nova/nova-compute.log 2>&1 &

cd /opt/stack/noVNC
/opt/stack/noVNC/utils/nova-novncproxy --config-file /etc/nova/nova.conf  --web . > /var/log/nova/nova-novncproxy.log 2>&1 &

cd /opt/stack/nova/bin/
/opt/stack/nova/bin/nova-xvpvncproxy --config-file /etc/nova/nova.conf > /var/log/nova/nova-xvpvncproxy.log 2>&1 &

cd /opt/stack/nova/bin/
/opt/stack/nova/bin/nova-consoleauth > /var/log/nova/nova-consoleauth.log 2>&1 &

service apache2 start

kill -9 `ps aux | grep -v grep | grep /opt/stack | awk '{print $2}'`

service apache2 stop
service rabbitmq-server stop
service mysql stop

May 6, 2012

Deploying a Scala based Servlet in Liberty

Filed under: websphere — Tags: , , — Davanum Srinivas @ 8:16 pm

Fetch the scala servlet sample using git:

dims@dims-laptop-520:~$ git clone git://

Take a peek at the servlet code:

package org.peat.scalaservlet

import javax.servlet.http._
import javax.servlet._

class Example extends HttpServlet {

  override def doGet(request: HttpServletRequest, response: HttpServletResponse) = response.getWriter().println( exampleContent )

  def exampleContent(): String = { "Hello world, from a Scala servlet!" }

Build the sample using maven:

dims@dims-laptop-520:~/scala-servlet$ mvn war:war

Follow instructions from the following url to deploy the ScalaServlet.war:

Don’t forget to enable the servlet feature in the defaultServer/server.xml. Once the server is started peek at the logs

dims@dims-laptop-520:~/wlp$ cat ./usr/servers/defaultServer/logs/trace_12.05.04_22.50.54.0.log
[04/May/2012 22:50:54:141 -0700]  00000001 * id= [AUDIT   ] CWWKE0001I: The kernel was launched.
[04/May/2012 22:50:55:645 -0700]  0000001d * id=            [AUDIT   ] CWWKF0007I: A feature update is starting after 1.735 seconds.
[04/May/2012 22:50:55:682 -0700]  00000001 * id= [AUDIT   ] CWWKE0002I: The kernel started after 1.774
[04/May/2012 22:50:56:263 -0700]  00000016 * id=       [AUDIT   ] CWWKZ0058I: Application monitor started after 2.353 seconds.
[04/May/2012 22:50:56:265 -0700]  0000001d * id=            [AUDIT   ] CWWKF0008I: The feature update is complete after 2.356 seconds.
[04/May/2012 22:50:56:314 -0700]  0000001f * id=                    [AUDIT   ] CWWKO0219I: TCP Channel defaultHttpEndpoint has been started and is now listening for requests on host localhost  (IPv4: port 9080.
[04/May/2012 22:50:56:805 -0700]  00000020 * id=                     [AUDIT   ] CWWKT0016I: Web application available (default_host): http://localhost:9080/ScalaServlet/*
[04/May/2012 22:50:56:811 -0700]  00000020 * id=                    [AUDIT   ] SRVE9998A: Application ScalaServlet added to web container.
[04/May/2012 22:50:56:814 -0700]  00000020 * id=              [AUDIT   ] CWWKZ0001I: The application ScalaServlet has started successfully.

All’s well, now just point your browser to http://localhost:9080/ScalaServlet/

May 3, 2012

Developers Kit for Pure Application System

Filed under: Uncategorized — Davanum Srinivas @ 11:54 am

The deployment engine / kit for Pure Application System is available to try as a vmdk –  There is also a developerworks guide with a 5 part series that dives deep as well. See to get started.

January 4, 2012

Follow a user in Lotus Connections 3.0

Filed under: Uncategorized — Tags: , — Davanum Srinivas @ 3:35 pm

One main feature in Lotus Connections 3.0 is the asymmetric follow of someone (like twitter). The API documentation is here. Since sample code is always better here’s a HTTP GET python snippet to lookup the userid, given an email and then a quick HTTP POST to follow that user.

import sys,urllib,urllib2,traceback,base64
from xml.dom import minidom

xml_data_header = """
<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="">
   <category term="resource-follow" scheme=""></category>
   <category term="profiles" scheme=""></category>
   <category term="profile" scheme=""></category>
   <category term="
xml_data_footer = """
" scheme=""></category>

if len(sys.argv) != 4: 
        print 'Usage: follow <userid> <password> <email-of-user-to-follow>' 

base64string = base64.encodestring('%s:%s' % (sys.argv[1], sys.argv[2]))[:-1]

def getUuidForUser(email):
	uri = "" % email
	req = urllib2.Request(uri)
	req.add_header('Authorization', "Basic %s" % base64string)
	dom = minidom.parse(urllib2.urlopen(req))
	element = dom.getElementsByTagNameNS('', 'userid')[0]

def followUser(uuid):
	uri = ''
	query_string_values = {'source': 'profiles', 'type'  : 'profile'}
	payload = '%s%s%s' % (xml_data_header, uuid, xml_data_footer)

	try :
		if query_string_values:
		    uri = ''.join([uri, '?', urllib.urlencode(query_string_values)])
		req = urllib2.Request(uri, data=payload)
		req.add_header('Authorization', "Basic %s" % base64string)
		response = urllib2.urlopen(req)
	except urllib2.HTTPError, error:

print followUser(getUuidForUser(sys.argv[3]))
Older Posts »

The Shocking Blue Green Theme. Blog at


Get every new post delivered to your Inbox.

Join 135 other followers