• map instance’s hostnames to ip addresses in DNS
• provide REST API to manage DNS

To solve first task service monitor message bus (RabbitMQ). For every started instance service add DNS record, for terminated – remove.  DNS name is choose in form hostname.tenant_name.root_zone. If zone for tenant_name doesn’t exists yet, it is created and populated automatically.

To solve second task service start REST API server on specified ip/port. REST API supports authentication against keystone and utilize keystone’s RBAC.

As a DNS backend PowerDNS is used, but other backends (for example for Bind) can be added later.

Next we are going to add support for PTR zones and  add ability to create personal zone for every started instance.

Links:

• source code
• issue tracker
• documentation
• yum repository with RHEL package
• yum repository with CentOS package

horizon-billing is packaged to a homonymous RPM (path for RHEL: http://yum.griddynamics.net/yum/diablo/, for CentOS: http://yum.griddynamics.net/yum/diablo-centos/).

To enable it in dashboard, install the horizon_billing package and turn it
on in /etc/openstack-dashboard/local/local_settings.py:

• add ‘horizon_billing’ to INSTALLED_APPS tuple;
• add ‘billing’ to ‘dashboards’ key in HORIZON_CONFIG.

After installation, a new “Billing” panel is added after “Project” and “Admin”.

So, these repos are renamed:

• osc-robot-keystone;
• osc-robot-nova;
• osc-robot-openstackx;
• osc-robot-glance;
• osc-robot-swift;
• osc-robot-openstack-compute;
• osc-robot-noVNC;
• osc-robot-python-novaclient.

Also, we add python-keystoneclient repo (forked from openstack/python-keystoneclient) that contains patches for RedHat packaging.

We have added ability to use LVM volumes for instances disks in OpenStack Nova according to the blueprint.

You can specify which storage type you want to use in nova.conf:

–local_images_type={raw, qcow, lvm, legacy}

legacy value (used by default) was added for backward compatibility. In case of legacy storage type will be controlled by old value:

–use_cow_images={True, False}

If you use other values, use_cow_images flag will be ignored. With lvm type you should specify LVM group to use for VM storage with lvm_volume_group flag:

–lvm_volume_group=<existing volume group name>

From point of user there is no difference at all between LVM and other storage types. He can do all usual operations on instance: reboot, migrate, resize, snapshot. One difference is that we disable snapshotting of running instance and add force_snapshot option to snapshot_instance method of OpenStack API to override this behaviour.

• Code: https://github.com/griddynamics/osc-robot-nova/commit/a1cd719481ff76b871d58bc3d5bf683850dc4a04
• RHEL repository: http://yum.griddynamics.net/yum/diablo
• CentOS repository: http://yum.griddynamics.net/yum/diablo-centos

Currently it collects information about instances and images and snapshost in glance.  Local disk usage (Gb*h), memory (Mb*h) and virtual CPU count (vcpus*h) are accounted for VM instances and disk space (Gb*h) is accounted for images in glance .

Next we are going to include other resources like network storages (volumes).

Links:

• source code
• issue tracker
• documentation
• yum repository with RHEL package
• yum repository with CentOS package

Information can be received from REST interface or you can use CLI command `nova2ools-billing` from `nova2ools` set (available from Yum repositories).

• Avoid hardcoded values in test scenarios. Lettuce offers the only way for data-driven scenarios. This is done via scenario outlines. What means that all data is stored in script itself. External data sources may be only supported by your own test code. We may fill that gap introducing Jinja2 templates and YAML configuration files. Feature templates stay readable but gain flexibility. The regular Lettuce scripts are generated on every test execution thus saving BDD style. Stories remain comprehensible for an end-user.
• Write test fixtures explicitly. It should be possible to write setup and teardown BDD stories. It is also important to have behavior specifications for installation procedures.
• Share and re-use test fixtures, specify that test depends on specific fixtures Tests should be self-suffucient and should not rely on the state created by other tests. However it is often a huge overhead when each test performs its own setup which may always be the same. It is required to provide the concept of “dependencies” between tests and test fixtures.
• Tests are environment agnostic but test fixtures are environment dependent. We may enable writing environment agnostic tests by moving all environment specific action into test fixtures. Thus test fixtures may have multiple versions aimed for different environments.

So we gathered these requirements and planned the following features for implementation:

1. Support for scenario templates. Scenarios become parameterizable and do not loose BDD spirit. Foreign Lettuce scenarios can be adopted by replacing hardcoded values with template variables. (YAML and Jinja2 were used for that)
2. Explicit separation of test fixtures from tests (setup, teardown and test scripts). Test scenario should not contain any actions that are specific to platform/configuration and should only perform actions on the product under test.
3. Dependency specification for setup/teardown fixtures and sharing fixtures among tests.  Most of tests rely on the same state while other may have specific prerequisites.
4. Parallel test execution. This is often required for long running end-to-end scenarios. That feature introduces a requirement that which are planned to be executed in parallel should be independent from each other and does not use the same resources.

Points 1-3 are implemented now. That should be enough to start using it.  However there is a plenty of stuff planned on the road-map.

We also shared our OpenStack test suite which runs within our CI workflow. So you may have a look how Bunch works. Links to tool, doc and tests are below:

• Bunch source code: https://github.com/griddynamics/bunch
• Bunch documentation: http://www.griddynamics.com/openstack/docs/bunch/
• OpenStack test suite: https://github.com/griddynamics/openstack-core-test

Just install Bunch, check out tests, adjust config.yaml and execute:
bunch -e clean openstack-core-test/smoketests/basic/ <resultdir>


Both Bunch and tests are under active development, so there are not much tests in the repo. This is just the beginning. Stay tuned.

We want to share our roadmap with a community. In the nearest future Grid Dynamics are going to focus on new component and services development. All services will be open sourced.

What are we going to present soon:

• Now we are developing resource accounting system for OpenStack (nova-billing).
• We are developing DNS service, so every VM in a cloud will be resolved by its name and administrators of tenants will own one subdomain and will be able to add custom DNS records or subdomains.
• We have created separate set of utilities called nova2ools and going to present them soon. They will offer utilities to work with nova and utilities to work with our services like billing or DNS.
• We are going to start working on hardware provisioning for OpenStack. Sometimes it is very important to be able to work on real hardware. Ability to get hardware in a OpenStack way through REST API should be very useful.

To reflect this activities we have created a site with documentation and additional information: http://www.griddynamics.com/openstack.

$ nova boot <server_name> --image <image_id> --flavor <flavor_id> [--key_path [<path_to_the_public_key>], --key_name <keypair_name>]

As you can see you are free to use both methods (old and new) and it is up to you what the way you go.

Here is a sample sequence of commands to work with the new feature:

$ nova keypair-add key1 > private_key_file — save returned private key to the file.

$ chmod 600 private_key_file — only owner shoud have access to the private key (-rw-------).

$ nova boot <server_name> --image 3 --flavor 1 --key_name key1 — boot new server with specified image id, flavor id and also inject previously created private key by its name.

$ nova list – show available servers. Check the status of the newly created server and if it is BUILD (creating in progress) perform this command until the status becomes ACTIVE. After that you can connect to the server by ssh. Let’s say servers’s ip is 172.30.254.4 (You can find out ip from the same table as the status).

$ ssh -i private_key_file root@172.30.254.4 — it is done — you are in the created server’s shell.

You can see sources on

https://github.com/griddynamics/osc-robot-python-novaclient

or binaries for RHEL

http://yum.griddynamics.net/yum/diablo/

and for CentOS

http://yum.griddynamics.net/yum/diablo-centos/

Some details about our build system. All packages are built in a chroot, populated by Mock tool. It allows us to build packages in an isolated environment with minimal packages installed and only right repositories set for dependency resolution. For RHEL it is only repo from distribution ISO and Grid Dynamics repo itself. So to bootstrap we had to build many dependencies from sources.  That is not all the story. Having ready RPM specs and sources was not enough for good automated builds. It is also necessary to have a uniform mechanism for gathering everything into tarball: sources, patches, specs, scripts. We chose Gear as a tool for this purpose. It automates source code checkouts from Git and packs everything according to “gear-rules” which should be added to Git repo. We also improved Gear by adding integration with Mock, so we could produce a chroot-built SRPMs directly from Git. So basically commands to build Gear repo are the following:

1. In the Git repo dir execute: gear-mock --root=<mock-chroot-config>
   (gear-mock is command introduced in our fork of Gear)
2. Take SRPM and build it via Mock:  mock --rebuild <mock-chroot-config> <SRPM>

Summarizing benefits of using Mock and Gear:

1. It is possible to build for different platforms (e.g. RedHat, CentOS, SL, Fedora)
2. Built packages are independent of current host configuration
3. Convenient source code updates (It is stored in Git, not tarballs)
4. No more manual patch creation (Gear is able to generate .patch files from commits and put it into SRPM automatically)
5. Uniform way to manage projects’ source code

RPM packages for Mock and Gear are also available at our YUM repository. You may try to do it yourself.

Starting from now this issue is over. Now you can perform following commands:

nova keypair-add [--pub_key <pub_key>] – creates public (if the pass to it is not passed) and private keys, saves public key into database and prints the private key into console.

nova keypair-delete – deletes keypair if it exists.

nova keypair-list – shows user’s keypairs like the eucatools do.

You can see sources on

https://github.com/griddynamics/osc-robot-python-novaclient

or binaries for RHEL

http://yum.griddynamics.net/yum/diablo/

and for CentOS

http://yum.griddynamics.net/yum/diablo-centos/