Monthly Archives: January 2012

Billing plugin for Horizon

Now nova-billing has a Django-based web interface – horizon-billing (https://github.com/griddynamics/horizon-billing).

horizon-billing is packaged to a homonymous RPM (path for RHEL: http://yum.griddynamics.net/yum/diablo/, for CentOS: http://yum.griddynamics.net/yum/diablo-centos/).

To enable it in dashboard, install the horizon_billing package and turn it
on in /etc/openstack-dashboard/local/local_settings.py:

  • add ‘horizon_billing’ to INSTALLED_APPS tuple;
  • add ‘billing’ to ‘dashboards’ key in HORIZON_CONFIG.

After installation, a new “Billing” panel is added after “Project” and “Admin”.

Advertisements

Grid Dynamics’ GitHub repos are renamed

In order to unify our repo names with openstack GitHub origanization, “osc-robot-” prefix is removed.

So, these repos are renamed:

  • osc-robot-keystone;
  • osc-robot-nova;
  • osc-robot-openstackx;
  • osc-robot-glance;
  • osc-robot-swift;
  • osc-robot-openstack-compute;
  • osc-robot-noVNC;
  • osc-robot-python-novaclient.

Also, we add python-keystoneclient repo (forked from openstack/python-keystoneclient) that contains patches for RedHat packaging.

Using LVM as disk storage for OpenStack

We have added ability to use LVM volumes for instances disks in OpenStack Nova according to the blueprint.

You can specify which storage type you want to use in nova.conf:

–local_images_type={raw, qcow, lvm, legacy}

legacy value (used by default) was added for backward compatibility. In case of legacy storage type will be controlled by old value:

–use_cow_images={True, False}

If you use other values, use_cow_images flag will be ignored. With lvm type you should specify LVM group to use for VM storage with lvm_volume_group flag:

–lvm_volume_group=<existing volume group name>

From point of user there is no difference at all between LVM and other storage types. He can do all usual operations on instance: reboot, migrate, resize, snapshot. One difference is that we disable snapshotting of running instance and add force_snapshot option to snapshot_instance method of OpenStack API to override this behaviour.

Resource usage accounting system for OpenStack

`nova-billing` service is ready for community review. It is a service which accounts resource usage per project (tenant). This information can be used for charging client later.

Currently it collects information about instances and images and snapshost in glance.  Local disk usage (Gb*h), memory (Mb*h) and virtual CPU count (vcpus*h) are accounted for VM instances and disk space (Gb*h) is accounted for images in glance .

Next we are going to include other resources like network storages (volumes).

Links:

Information can be received from REST interface or you can use CLI command `nova2ools-billing` from `nova2ools` set (available from Yum repositories).

Testing framework

Today we would like to share with you one of our recent subprojects. It is our testing framework. We have used to automate our tests for OpenStack with Lettuce. However we found that we need more features to build robust framework with it. So we created a Bunch tool. The main reason for Bunch was the motivation to write more flexible and powerful tests with Lettuce. Key points to improve were:

  • Avoid hardcoded values in test scenarios. Lettuce offers the only way for data-driven scenarios. This is done via scenario outlines. What means that all data is stored in script itself. External data sources may be only supported by your own test code. We may fill that gap introducing Jinja2 templates and YAML configuration files. Feature templates stay readable but gain flexibility. The regular Lettuce scripts are generated on every test execution thus saving BDD style. Stories remain comprehensible for an end-user.
  • Write test fixtures explicitly. It should be possible to write setup and teardown BDD stories. It is also important to have behavior specifications for installation procedures.
  • Share and re-use test fixtures, specify that test depends on specific fixtures Tests should be self-suffucient and should not rely on the state created by other tests. However it is often a huge overhead when each test performs its own setup which may always be the same. It is required to provide the concept of “dependencies” between tests and test fixtures.
  • Tests are environment agnostic but test fixtures are environment dependent. We may enable writing environment agnostic tests by moving all environment specific action into test fixtures. Thus test fixtures may have multiple versions aimed for different environments.

So we gathered these requirements and planned the following features for implementation:

  1. Support for scenario templates. Scenarios become parameterizable and do not loose BDD spirit. Foreign Lettuce scenarios can be adopted by replacing hardcoded values with template variables. (YAML and Jinja2 were used for that)
  2. Explicit separation of test fixtures from tests (setup, teardown and test scripts). Test scenario should not contain any actions that are specific to platform/configuration and should only perform actions on the product under test.
  3. Dependency specification for setup/teardown fixtures and sharing fixtures among tests.  Most of tests rely on the same state while other may have specific prerequisites.
  4. Parallel test execution. This is often required for long running end-to-end scenarios. That feature introduces a requirement that which are planned to be executed in parallel should be independent from each other and does not use the same resources.

Points 1-3 are implemented now. That should be enough to start using it.  However there is a plenty of stuff planned on the road-map.

We also shared our OpenStack test suite which runs within our CI workflow. So you may have a look how Bunch works. Links to tool, doc and tests are below:

Just install Bunch, check out tests, adjust config.yaml and execute:
bunch -e clean openstack-core-test/smoketests/basic/ <resultdir>

Both Bunch and tests are under active development, so there are not much tests in the repo. This is just the beginning. Stay tuned.

Grid Dynamics Open Stack development activities

Hello everybody.

We want to share our roadmap with a community. In the nearest future Grid Dynamics are going to focus on new component and services development. All services will be open sourced.

What are we going to present soon:

  • Now we are developing resource accounting system for OpenStack (nova-billing).
  • We are developing DNS service, so every VM in a cloud will be resolved by its name and administrators of tenants will own one subdomain and will be able to add custom DNS records or subdomains.
  • We have created separate set of utilities called nova2ools and going to present them soon. They will offer utilities to work with nova and utilities to work with our services like billing or DNS.
  • We are going to start working on hardware provisioning for OpenStack. Sometimes it is very important to be able to work on real hardware. Ability to get hardware in a OpenStack way through REST API should be very useful.

To reflect this activities we have created a site with documentation and additional information: http://www.griddynamics.com/openstack.