More Accessible Neutron Development

As a quark/neutron developer I have to stand up neutron servers all the time. The way that openstack prefers this be done is with devstack. This is really great and cool if you want to deal with the whole stack and watch how the services interact. If you just want to develop your plugin for Neutron this is quite a bit more than overkill.

The OpenStack community is, rightfully so, pot-committed with their auth solution keystone. This can be seen by clients having --help outputs like:

$ neutron --help
  ...
  --os-auth-strategy <auth-strategy>
  DEPRECATED! Only keystone is supported.
  ...

This means that if you wish to be a simple, no-nonsense neutron developer and make use of the clients you’ll need to stand up keystone. Despite having done such a thing multiple times I still think it’s a pain in the ass. It’s simply a fact of life that OpenStack is somewhat complicated, maybe needlessly so, and abandoning support for ease-of-use solutions like noauth is a big reason for that.

We forked the client awhile ago to add auth extensions, and because of that noauth is now supported in the client. Now developing for neutron is much more accessible for everyone.

It’s these little things that make open-source development — open.

If you’d like the OpenStack community to embrace the mentality of ease-of-entry, and more openness please show your support in getting these auth extensions upstream.

How to Use noauth

Using noauth is quite simple.

neutron --os-auth-strategy noauth --os-url <service endpoint> COMMAND

Some examples:

List networks
neutron --os-auth-strategy noauth --os-url http://localhost:9696/v2.0 net-list
Create a network
neutron --os-auth-strategy noauth --os-url http://localhost:9696/v2.0 net-create my_nets --tenant-id me

Because you’re doing noauth you’ll need to pass in the tenant-id. This isn’t a big deal though because you can make up whatever you want!

If more examples are necessary just comment.

Installing rackspace-python-neutronclient

Installation of this client is very similar to the existing steps to install OpenStack clients, but I will put my methods of doing it here for quick reference.

Quick Install Steps (the TL;DR)

This will create a new directory, install the client and extensions, as well as give you a little file to start working on for the profile.

If you want to be safe use this to install into a virtualenv:

mkdir neutronclient && cd neutronclient
virtualenv --prompt='(neutronclient)' .venv
source .venv/bin/activate
pip install rackspace-neutronclient
wget http://tinyurl.com/neutron-profile

If you don’t want to be safe and fill your system with random things:

sudo pip install rackspace-neutronclient
wget http://tinyurl.com/neutron-profile

Make edits to the neutron-profile file you just downloaded and you should be set to go.

Installing the Client with Extensions

Assuming you’re in a virtualenv, which I highly recommend, you can just run these commands without using sudo:

pip install rackspace-neutronclient

Using the client

Because we didn’t rename the binary installed by the package the client will still be named neutron. This may cause some issues (see Troubleshooting).

For these examples os-auth-url needs to be given.

Example: Creating a network

neutron --os-auth-url https://identity.api.rackspacecloud.com/v2.0 --os-auth-strategy rackspace --os-username USERNAME --os-password API_KEY net-create test_net_name

Example: Listing Networks

neutron --os-auth-url https://identity.api.rackspacecloud.com/v2.0 --os-auth-strategy rackspace --os-username USERNAME --os-password API_KEY net-list

Using the client with Environment Args

If you don’t want to pass the arguments to the client as above you may also put all that information into your environment. This is very similar to the way it works with the nova client.

Create a new file or put the following into your ~/.bash_profile:

export OS_AUTH_URL=https://identity.api.rackspacecloud.com/v2.0/
export OS_AUTH_STRATEGY=rackspace
export OS_REGION_NAME=DFW
export OS_USERNAME=<username>
export OS_TENANT_NAME=<tenant_id>
export OS_PROJECT_ID=<tenant_id>
export OS_PASSWORD=<api_key>
export OS_NO_CACHE=1

GOTCHA: OS_AUTH_STRATEGY entry is different than OS_AUTH_SYSTEM.

Then source the file (or ~/.bash_profile). You should now be able to do the above commands like so:

Example: Creating a network

neutron net-create test_net_name

Example: Listing Networks

neutron net-list

Using the Client with supernova

The instructions on how to do this were taken from here. If you’ve never used supernova before:

  1. You really should
  2. You should read how to use it before continuing

So you’re a supernova expert now? Ok. You should have a .supernova file. Insert into that file a new section like so (note: this is the most of the above script without the exports):

[neutron-dfw]
OS_EXECUTABLE=neutron
OS_AUTH_URL=https://identity.api.rackspacecloud.com/v2.0/
OS_AUTH_STRATEGY=rackspace
OS_REGION_NAME=DFW
OS_USERNAME=<username>
OS_TENANT_NAME=<tenant_id>
OS_PROJECT_ID=<tenant_id>
OS_PASSWORD=<api_key>
OS_NO_CACHE=1

The relevant part is the OS_EXECUTABLE entry.

GOTCHA: OS_AUTH_STRATEGY entry is different than OS_AUTH_SYSTEM.

You should now be able to use supernova.

Example: Creating a network

supernova neutron-dfw net-create test_net_name

Example: Listing Networks

supernova neutron-dfw net-list

Troubleshooting

The client is upgraded often, so if you’re running into any troubles the first thing you want to do is (prefix sudo if necessary):

pip install --upgrade rackspace-neutronclient

Otherwise, a vast majority of the problems encountered are due to having python-neutronclient (upstream client) and rackspace-python-neutronclient (forked client) installed at the same time.

If you are having any problems first try the following and reinstall using the above instructions:

pip uninstall -y python-neutronclient
pip uninstall -y rackspace-neutronclient
pip uninstall -y rackspace-auth-neutronclientext

When both are installed, what is happening?

Because the forked client uses the exact code as the upstream code they are placed into the site-packages (libs) in the same place. The one installed last will be what is used. Even more troublesome is that if you uninstall one of the clients you are removing code that both depend on. Always uninstall both if having problems.

Getting OpenStack neutron+quark Working Locally

Our neutron plugin quark accepts many drivers and will use however many that are configured at the same time. Quark currently has no default working configuration in which it can be tested.

This setup will create a neutron server that really doesn’t do anything, but it will not require any other services to run! It also assumes that you have a mysql server configured. If you don’t want to use mysql you can change the sqlalchemy connection string in the neutron.conf provided at the end of the post.

Installing the Packages

First thing we need to do is download the code somewhere and install it. The things we really need to install are:

Create a directory somewhere. This will be where we install all the packages. We will be using virtualenv because we’re good python developers, aren’t we?

cd my_directory_of_choice
virtualenv --prompt='(quark)' .venv
source .venv/bin/activate

Your shell prompt should, at least if you’re using bash, have (quark) in front now.

The rest of this is pretty easy. Since we’re looking to develop quark/neutron we will need source and not packages.

git clone git@github.com:openstack/neutron && cd neutron && python setup.py develop
git clone git@github.com:rackerlabs/quark && cd quark && python setup.py develop

If you run pip freeze you should see some references to neutron and quark in there. There will most likely be a bunch of other packages installed that are dependencies for neutron or quark.

Configuration Details

Make a configuration directory. I call mine etc for giggles. See the full file at the end of this to copy-pasta.

My default testing configuration uses our unmanaged driver. This driver is essentially a noop driver and doesn’t require any extra software to be installed to use quark. Caveat: because it is noop you will have no connectivity if placed into an environment.

Put the following configuration into your neutron.conf (I usually put it at the bottom so the config grouping doesn’t get confused).

[QUARK]
net_driver=quark.drivers.unmanaged.UnmanagedDriver
default_network_type=UNMANAGED

  • set the core_plugin value in the neutron.conf to quark.plugin.Plugin.
  • setting the auth_strategy to noauth is very helpful especially if you are using wafflehaus.try_context to fake out DB connections

Creating the Database

You’ll need to setup a database (probably mysql). Then modify your neutron.conf with the appropriate connection string (see this). To create the initial quark database you need to run:

quark-db-manage --config-file <path-to>neutron.conf upgrade head

Creating the Default Network Artifacts

This resty call will create the mac address ranges for this simple setup:

POST /v2.0/mac_address_ranges.json -d '{"mac_address_range": {"cidr" : "AA:BB:CC", "tenant_id": "provider"}}'

And thisresty call will create the public network:

POST /v2.0/networks -d '{"network": {"id": "00000000-0000-0000-0000-000000000000", "tenant_id": "provider", "name": "public"}}'

Finally you will need to make this resty call to make a subnet.

POST /v2.0/subnets -d '{"subnet": {"network_id": "00000000-0000-0000-0000-000000000000", "segment_id": "blah", "cidr": "10.0.0.0/16", "tenant_id": "derp", "ip_version": "4"}}'

Isolated (tenant owned) networks are currently not supported in this configuration. A more operative driver is necessary. There is an NVP/NSX driver included with quark but it requires standing up a NVP/NSX Controller. Doing such a thing is well beyond the scope of this post.

Running Neutron

Then to run the neutron server locally you run the command (preferably in a virtualenv):

neutron-server --config-dir /home/jhammond/quark_server/etc

Usage and Initial Smoke Test

Some useful tidbits of information about how we’ve changed how things work in quark:

  • to support cells we had to add segment_id to subnets on provider networks
  • ports created on provider networks require a segment_id as well
  • during testing this value can be anything as long as they match

This script uses resty because I dislike curl. Note the segment id in the POST subnets and POST ports.

#!/bin/bash
endpoint="http://localhost:9696/v2.0"
tenant='random'
net_name='test_network'
source ~/bin/resty/resty -W $endpoint -H "Content-type: application/json"
net_id='00000000-0000-0000-0000-000000000000'

subnet=`POST /subnets -d '{"subnet": {"network_id": "'$net_id'", "tenant_id": "'$tenant'", "cidr": "192.168.7.1/24", "ip_version": 4, "segment_id":
"test"}}'`

subnet_id=`echo $subnet | python -c 'import sys, json; print json.load(sys.stdin)["subnet"]["id"]'`
echo "Made subnet $subnet_id"

port=`POST /ports -d '{"port": {"tenant_id": "'$tenant'", "name": "weeble", "network_id": "'$net_id'", "segment_id": "test", "fixed_ips": [{"ip_address":
"192.168.7.50", "subnet_id": "'$subnet_id'"}]}}'`

port_id=`echo $port | python -c 'import sys, json; print json.load(sys.stdin)["port"]["id"]'`
echo "Made port $port_id"

# CLEAN-UP
DELETE /ports/$port_id
DELETE /subnets/$subnet_id
GET /networks

My etc Contents

api-paste.ini
dhcp_agent.ini
fwaas_driver.ini
init.d/
l3_agent.ini
lbaas_agent.ini
metadata_agent.ini
metering_agent.ini
neutron/
neutron.conf
policy.json
rootwrap.conf
services.conf
vpn_agent.ini

Full neutron.conf

Fill out the database connection info. If you remove the database info completely sqlalchemy will attempt to connect to a mysql server with the following connection string:

mysql://root:password@127.0.0.1/neutron?charset=utf8

That may work for you, but you should probably change it.

[DEFAULT]
lock_path = $state_path/lock
core_plugin = quark.plugin.Plugin
auth_strategy = noauth
fake_rabbit = true
rpc_backend = fake

[DATABASE]
connection = mysql://db_user:db_password@127.0.0.1/neutron?charset=utf8
reconnect_interval = 5
auto_create_schema = False
min_pool_size = 100
max_pool_size = 500
max_overflow = 500

[QUARK]
net_driver=quark.drivers.unmanaged.UnmanagedDriver
default_net_strategy='{"00000000-0000-0000-0000-000000000000": {"bridge": "publicnet"}, "11111111-1111-1111-1111-111111111111": {"bridge": "servicenet"}}'
default_network_type=UNMANAGED
# IPAM recycling interval, in seconds
ipam_reuse_after= 300
ipam_retry=1
show_subnet_ip_policy_id = False
show_allocation_pools = True
pessimistic_connection_pooling = True
environment_capabilities = ""

Troubleshooting

TypeError: %d format: a number is required, not NoneType

This is caused by oslo_messaging and rpc functions I do not use. Ensure you have the config variable rpc_backend = fake. This will disable that feature.

Running the Quark Unit Tests

We use tox to run the unit tests. A gotcha with quark is that it has assumed configurations for local mysql DB testing (tox -e mysql). If your local DB doesn’t match these assumptions they will fail.

The following will allow those to work:

QUARK_MYSQL_TESTS_URL="mysql://user:password@localhost/quark_functional_tests" tox -e mysql

This will produce a lot of sqlalchemy output. I use this little snippet to keep the output to something sane:

QUARK_MYSQL_TESTS_URL=&quot;mysql://user:password@localhost/quark_functional_tests&quot; tox -e mysql 2&gt;&amp;1 &gt;/dev/null | grep -B 25 'end captured stdout'Getting OpenStack neutron+quark Working Locally