QUELLE

Introduction

This page provides supplementary material for the paper:

Daniel Moldovan, Georgiana Copil, Hong-Linh Truong, Schahram Dustdar, QUELLE - a Framework for Accelerating the Development of Elastic Systems, Third European Conference on Service-Oriented and Cloud Computing - ESOCC 2014, 2-4 September, Manchester, United Kingdom, http://link.springer.com/chapter/10.1007/978-3-662-44879-3_7, ppt

Abstract

A large number of cloud providers offer diverse types of cloud services for constructing complex "cloud-native" software. However, there is a lack of supporting tools and mechanisms for accelerating the development of cloud-native software-defined elastic systems (SES) based on elasticity capabilities of cloud services. In this paper we introduce QUELLE -- a framework for evaluating and recommending SES deployment configurations. QUELLE presents models for describing the elasticity capabilities of cloud services and capturing elasticity requirements of SESs. Based on that QUELLE introduces novel functions and algorithms for quantifying the elasticity capabilities of cloud services. QUELLE's algorithms can recommend SES deployment configurations from cloud services that both provide the required elasticity, and fulfill cost, quality, and resource requirements, and thus can be incorporated into different phases of the development of SESs. We present several experiments based on real-world cloud services for the development of an elastic machine-to-machine data-as-a-service system.

Overview

To develop cloud native elastic systems, in current approaches, the developer has to manually investigate all cloud services offered by various cloud providers, and evaluate if their elasticity capabilities provide the required elasticity control options. Then, s/he can use existing design and modeling tools such as Winery to design and deploy the DaaS on cloud infrastructures. Manually selecting each cloud service needed for constructing the DaaS is laborious, complex, and error prone. These problems can be reduced and the development can be accelerated if we could provide features, shown in the above figure for:

capturing and modeling elasticity capabilities of cloud services from different ecosystems and multi-level SES requirements (indicated by 1)
providing service elasticity quantification functions for software development tools (indicated by 2)
recommending SES configurations, which can later on be mapped to software-interpretable deployment descriptor (indicated by 3)

QUELLE Prototype

In the following we briefly describe the current QUELLE prototype. For managing the Cloud Services Model, we implemented a graph-based Neo4j Cloud Services Persistence Adapter. The population of the cloud services repository should ideally be an automatic process, with the increase in cloud providers' description APIs. However, currently we rely on available JSON description services and HTML parsing to populate our model. For SES configurations visualization, we have implemented a TOSCA-based output formatter for Winery

Capturing Elasticity Capabilities of Cloud Services

For explaining the concept of Elasticity Dependency , let's focus on Amazon EC2 m3.xlarge service unit, and model its elasticity it capabilities. The above figure shows the elasticity capabilities offered by this service: one mandatory association to another service unit which decreases the elasticity (left part of the figure), one optional association to a StorageQuality and StorageQuality Cost, and 8 Cost possibilities. To further understand the elasticity of the service unit, we must focus on each individual Elasticity Capabilities. For example, focusing on the Storage Elasticity Capabilities we discover that it provides another two Elasticity Capabilities targeting Storage Quality.

Neo4J Cloud Services Persistence Adapter

For managing the cloud services description model introduced in the paper, we define 7 Neo4J graph node types: Service, Resource, Quality, Cost, CostFunction, and ElasticityCapability. We also define several types of entity relationships for expressing express dependencies between entities: hasResource, hasQuality, hasCost, hasCostFunction, hasElasticityCapability, and elasticityCapabilityFor. Graph nodes capture information for categorizing the type of stored entity, and not particular property values; the actual values are captured as relationships properties. For example, two Service entities could target the same CPU Resource node, and have on their respective hasResource relationships the different CPU properties. This enables selecting services with specific elasticity capabilities, and then analyzing their capabilities properties from their respective relationships. For managing the cloud services description model introduced in the paper, we define 7 Neo4J graph node types: Service, Resource, Quality, Cost, CostFunction, and ElasticityCapability. We also define several types of entity relationships for expressing express dependencies between entities: hasResource, hasQuality, hasCost, hasCostFunction, hasElasticityCapability, and elasticityCapabilityFor. Graph nodes capture information for categorizing the type of stored entity, and not particular property values; the actual values are captured as relationships properties. For example, two Service entities could target the same CPU Resource node, and have on their respective hasResource relationships the different CPU properties. This enables selecting services with specific elasticity capabilities, and then analyzing their capabilities properties from their respective relationships.

Experiments

Quantifying Elasticity of Amazon EC2 IaaS Services

As the meaning of elasticity might vary depending on the stakeholder, QUELLE provides a customizable elasticity quantification function relying on user-defined VolatilityQ, ElDepQ, and ElPhaseQ coefficients. As the user is interested in building an elastic system, s/he expects services to be allocated/deallocated often. As Amazon bills its services minimum on a hourly basis, the supplied volatility quantification coefficient is VolatilityQ = 1/minLifetime (Hours), generating a volatility of 1 for hourly reserved services, and 1 / (365 * 24) for yearly reserved services. As the user wants to use services which have as few dependencies on other services, s/he supplies an elasticity dependency quantification coefficient ElDepQ = 1 if Optional-Association, and -1 if Mandatory-Association. Finally, the supplied elasticity phase quantification coefficient is ElPhaseQ = 0.33 if Instantiation-Time, 0.67 if Run-Time, 1 if Both, and all elasticity dimensions have same weight coefficient W_d=1.

The result of quantifying the elasticity of Amazon EC2 services over cost, quality, resources, and services associations is depicted in the below figure. As the defined VolatilityQ function quantifies close to zero all options of reserving a service for 1 or 3 years, the cost elasticity of most services, such as m3.large is quantified close to 2. Amazon EC2 services which have optional dependencies have additional cost and quality control options. Thus, Amazon EC2 IaaS services with can be associated with an EBS service have their service association elasticity quantified to >= 1, and cost elasticity quantified to <= 3.

Recommending SES Configurations

We aim to accelerate the development of SESs by recommending deployment configurations using cloud services providing the required elasticity for its units. Thus, we define a four phase recommendation process: (i) processing SES requirements, (ii) quantifying elasticity of cloud services, (iii) recommending elasticity-driven SES configurations, and (iv) exporting SES configurations as cloud deployment descriptor.

Mixed SES requirements w.r.t. cost, resource and quality, are described by the user in a top-down fashion, from the entire SES to individual units. At the SES level, a requirement for a Management as a Service (MaaS) cloud services with a monitoring frequency of 5 minutes is specified, which will be applied to all SES's units. As the units belonging to the Event Processing Topology level perform sensitive computation, a MaaS requirement for a monitoring cloud services providing a monitoring frequency of 1 minute is specified, overriding the SES level requirement. The Event Processing Unit requires an IaaS cloud services with over 2 CPU cores and 5 GB of RAM, and a Moderate network performance. In turn, the Messaging Unit requires a PaaS service of type messaging. Similarly, the Data End Unit requires an IaaS cloud services providing at least 10 GB of RAM, I/O Performance of at least 1000 IOps together with at least a Moderate network performance.
The requirements XML can be downloaded from here

Processing all IaaS, PaaS, and MaaS requirements refined above, our prototype generates a TOSCA descriptor containing the recommended SES configuration. For the Event Processing Topology , the recommendation is visualized in the figure below using Winery, a TOSCA modeling and visualization tool. The recommendation contains an m1.large IaaS service fulfilling the resource and network performance quality requirements with associated SpotCost , due the Minimum Cost strategy. A PaaS Monitoring Service with a High Monitoring Frequency is recommended for the monitoring frequency requirement, and a MaaS SimpleQueue service for the message oriented middleware requirements.

In a similar fashion, recommendations are provided for the Data End Topology , containing an m1.xlarge IaaS cloud service, an Elastic Block Store (EBS) IaaS cloud service, with IOps 4000 fulfilling the I/O quality requirement, and a PaaS Monitoring Service at 5 minutes frequency, fulfilling the SES level monitoring requirement.

Contact

For further information please contact Daniel Moldovan.