EMC customers can choose any cloud

25 Wednesday Jun 2014

Posted by in OpenStack

≈ Leave a comment

Tags

, , , ,

This is a reprint of my blog that was published in EMC executive ‘Reflections‘ blog site.

EMC leaders have time and time again emphasized a core aspect of the EMC Federation strategy and functioning model: “(we) offer best-of-breed, integrated, technology while preserving customers’ ability to choose…” In this blog, I reflect upon how this strategy applies to allowing customers the ability to choose Open Source Cloud OS – OpenStack.

Our customers are free to choose any Open Source Cloud OS

OpenStack is making rapid strides in both private and public cloud markets. The earlier skepticism on hype is waning as we see the momentum translate to tangible deployments. Let’s go over some numbers if you need convincing. A recent OpenStack user survey showed 512 total deployments; 209 of these being production deployments. This momentum seems even more impactful within the EMC customer base. A recent EMC pre-sales/field survey has revealed that 50% of customers are running OpenStack today; 53% of those are in production. So it is not surprising that EMC’s strategy of ‘providing of our customers with choice’ resulted in OpenStack being a viable option across our federated family. Let me explain this in more detail.

Pivotal Customers Can Choose OpenStack

Pivotal offers a comprehensive set of application and data services that run on top of PaaS (platform-as-a-service) called Cloud Foundry®. Cloud Foundry is an open source platform that can run on any cloud infrastructure like VMware, OpenStack, or Amazon Web Services. Even though the Pivotal distribution of Cloud Foundry runs on VMware vSphere, there are many prominent members of the Cloud Foundry community who offer Cloud Foundry PaaS on OpenStack. Piston Cloud, which had originally contributed Cloud Foundry interface for OpenStack, now offers PaaS on Piston Cloud, which is based on OpenStack. IBM, with BlueMix PaaS cloud, and HP, with Cloud Application PaaS, have recently announced Cloud Foundry running on OpenStack infrastructure. Not only is it being offered by many prominent vendors, you can also choose to run Pivotal applications and big data services on any of these multi-cloud platforms. Watch out for the inevitable open source progression: OpenStack to Cloud Foundry!

VMware Customers Can Choose OpenStack

Many are perplexed by the complex relationship between VMware and OpenStack. I think the relationship is largely complementary; OpenStack is an open and flexible framework to pull all the cloud IaaS components together, whereas VMware provides the best-of-breed cloud components. As we see a rising number of OpenStack production deployments, there is also a broad demand for the integration of the two for Enterprise class solutions and use cases. It is for this reason that VMware is heavily investing in the integration, giving its customers the choice of having an open framework with best-of-breed cloud components. VMware is now the fifth largest contributor of OpenStack code in the current IceHouse release.

Many immediately get network virtualization as an integration point, as VMware NSX is one of the founders of OpenStack networking, but the integration points are far more pervasive than just networking. You can now trigger complex vCenter functions like vMotion off the OpenStack compute module, Nova. vSphere storage policies and advanced vSAN capabilities can be facilitated through the OpenStack storage module, Cinder. vCOPs integration allows monitoring and troubleshooting through OpenStack. There are many more, and you can be rest assured that the roadmap will continue to get richer with time.

EMC Customers Can Choose OpenStack

EMC’s strategy for OpenStack is twofold:

  1. OpenStack projects allow vendors to add capabilities through “plugin” architecture. Every storage business unit at EMC is committed to providing direct plugins for OpenStack. We currently offer plugins for VNX, VMAX, and ViPR; plugins from Isilon, ScaleIO, and XtremIO are already available to customers for beta evaluation. OpenStack’s roots were with object storage over commodity hardware. However, as deployments mature, we now see it running a broad mix of production workloads. Plugins give customers the choice of a variety of EMC storage with appropriate service levels to run their workloads.
  2. ViPR is EMC’s software-defined storage platform and hence complements OpenStack by providing rich automation features, as well as integrated management of objects, blocks, and later files. In other words, you will get integrated access to EMC (what you get from direct plugins) and non-EMC storage, and on top of that you should be able to get enhanced automation features like provisioning, masking/zoning, FAST and VPLEX volumes, as well as rich data services. You can check-out my blog on ‘ViPR in the stack’ for more details.

Returning to my original point, EMC customers can choose any cloud. I hope to have shown you that OpenStack represents as a viable choice across each of the EMC federation’s horizontal stacks.

Introduction of OpenStack with EMC

19 Thursday Jun 2014

Posted by in Industry solutions, OpenStack

≈ Leave a comment

Tags

,

Here is a recording of my EMC World 2014 session on OpenStack.

Short abstract of the session:

OpenStack is fast evolving as a framework for building and deploying both private and public clouds. To EMC, OpenStack represents a viable choice for cloud computing that we offer to our customers. This session covers the storage solutions available from EMC and the value proposition they offer. This session also provides a preview to enterprise class use cases EMC is working on for the future.

http://t.co/C8plg0hz4q

 

 

State of the solution from OpenStack Atlanta Summit 2014

18 Sunday May 2014

Posted by in Industry solutions, OpenStack

≈ Leave a comment

Tags

, ,

Hello, OpenStack fans! Did you miss the summit by any chance? Don’t worry, here is my spin on it. If you don’t care for my commentary, you can always check out OpenStack.org for their coverage. As always, they cover it well with recordings of keynotes and sessions.

Adoption momentum continues

Let’s start with the highlights from the user survey. The momentum for adoption continues, although it has understandably cooled off from what we said in the October 2013 survey. Total deployments jumped from 387 in October to 506, while Summary of OS surveyproduction deployments increased to 209. For the first time, ‘Avoid vendor lock-in’ has replaced ‘cost’ as the top business driver for deployments. Nova service was the most popular (196 deployments), but Cinder (151) and Neutron (135) were close. This showed that most deployments are not only Nova, but in conjunction with Cinder and Neutron. I was surprised to see a relatively low number of Swift deployments (101). Perhaps this number does not account for the Swift compatible deployments, just the open source module.

Keynotes were bold on the adoption momentum as well, with many notable end users on stage. Organization’s chair, Jonathan Bryce, announced the launch of two major programs geared towards aiding deployments. An OpenStack Marketplace is established with the goal of aiding in solution display and selection. Super users were introduced for knowledge sharing and collaborative problem solving. The theme of IceHouse is also largely aligned to adoption theme, with features focused on ease of deployment, tighter integration, and reliability.

Takeaways from the session

The sessions on networking track seemed ultra-popular, and were overflowing with attendees. (Don’t have any statistics to support it, just a random observation.) The user survey numbers on Neutron adoption seemed to support this hypothesis on popularity. Dev/QA had 169 Neutron deployments; 156 PoCs; 156 production deployments. Watch out for some excellent solutions from VMW NSX in this area.

Move towards Enterprise class solutions, like High Availability, was a clear theme that no one could miss. Just glance through the session agenda and you will see what I am saying. VMWare sessions were very popular for this reason, emphasizing the message of running OpenStack on top of VMware vSphere® + NSX™ to build rich OpenStack clouds on top of their existing IT infrastructure.

Another segment, promoting solutions with higher service levels were the Telcos. They came in number (ATT, Ericsson, TSystem…), and showed off their presence describing use cases requiring Telco grade service levels and scale. Network Function Virtualization (NFV) use case is gaining traction, as many Telco users and OEMs described the use case.

EMC sessions

EMC participated in three sessions:

Laying Cinder blocks: Jim Ruddy defined the use of ViPR, Software defined storage, with Cinder. This brought about the new role of how cloud architects would define policies, through a Software defined tool like ViPR, in a heterogeneous environment, to be consumed through the service model of OpenStack. Jim made the session hilarious through “Seinfield” analogies. Check out Jim’s blog for the message, demo, and humor.

Storage consideration for OpenStack: A panel discussion where Brian Gracely defined the importance of integrated data types for the execution of complex use cases.

Introducing Manila: An incubation proposal for a File Share service; Shamail Tahir demonstrated a prototype of VNX driver delivering file share service. Isilon driver was also mentioned.

Takeaways from design sessions

My attention on the design session was limited to storage, where the focus was – you guessed it, High Availability. Xing Yang led a design proposal of adding ‘Consistency Groups’ in Cinder. The session was very well received, with broad support for the feature. The discussion was centered on the use cases. Proposal was to implement it for Snapshot in the first phase, followed by volume replication and backup in the next.

A separate session proposed incremental backup, which struggled with scope and feasibility. There was also a proposal on Mirroring, which has a good chance of making it into Juno.

Finally (I saved the best for last), here’s a mention of my personal highlight of the summit. I was pleasantly surprised to see a broad emphasis on policy based automation. Neutron had design sessions on group policies and policy abstractions. Swift has proposed policies based implementations where availability can be configured, i.e. you can reduce the number of copies or implement an alternate method for availability. Then, there was a proposal for an incubation project called Congress, a cross module policy engine. Hallelujah!

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

Preview of EMC World OpenStack demo#2: block volume backed live migration

02 Friday May 2014

Posted by in Industry solutions, OpenStack

≈ 3 Comments

Tags

,

As I stated in the last blog, I am hosting a session at the upcoming EMC World on May 5th and 6th. The session gives an overview of OpenStack with EMC. Here is a sneak peak of my demo#2 that I plan to showcase in the session. This one is an Enterprise class use case – block backed live migration using KVM and OpenStack.

Use Case

The use case for Live Migration that I hear from OpenStack customers is tied to host maintenance. Users want to migrate the VM or instance over to a temporary host, for the sake of maintenance / upgrade of the source host; after the maintenance is complete they would migrate the VM back. Enterprises are very much used to such use cases in VMWare world, through vMotion. Underlying VMFS file system in vSPhere makes vMotion easier and transparent to various storage systems and protocols. KVM being an ephemeral hypervisor needs external file system support for persistent storage, hence Live Migration is dependent on Libvirt implementation. This is doable and stable using shared file system, like NFS. Good news is that in Havana release, OpenStack Nova supports ‘block volume backed Live Migration’. With this feature one can theoretically migrate VM without moving or copying volume. But … This involves careful handshake between Nova libvirt and Cinder to attach / detach volume from one host to another, and that implementation is buggy for iSCSI and Fibre Channel. We are actively working on fixing these bugs and should have a firm time frame for bug fix in another month or so.

The Demo

In spite of the bugs we were able to pull together a concept demo in our lab through some Nova workaround. Before going into the demo, would like to give a big shout out for Xing Yang, Consultant Technologist in Advanced Development team in Office of CTO. Thanks to Xing we have made big strides in driving clarity of the use case and code, and were able to pull this demo for you all. Thanks for your hard work Xing!

Here is how the demo goes. Block LiveM

 

We first go through the motions in Horizon to create a volume from standard Cirros image, then launch an instance from newly create volume. The instance is running on Source host, …124 in the demo. We then SSH into the source host, …124, do a “virsh list” to see the running VM and then start the live migration.

We then come back to Horizon UI to see the migration in progress. Just to make it interesting we go to the VNX Unisphere tool and check out the volume being migrated. Interestingly Unisphere sees our LUN associated with two storage groups – …124 source as expected, but also by a …125 target. So while migration is in progress VNX tool can see both source and target association to LUN. Migration should end quickly, as it is only Ephemeral transfer, the volume does not move. After migration Horizon UI shows the instance now running on …125 target host.

We now SSH in to target host …125 and login to the VM without any issues. By this time supposedly the maintenance of our source host is complete. We give the live migration command again to move the VM back to the source host …124. Use case complete!

Conclusion

Host maintenance is a valuable use case asked by many of our customers. Many stackers had to leverage shared storage, like NFS, for this use case in the past BUT this will change soon once these Nova bugs get fixed. It is just another choice of use case that EMC is helping enable. Enjoy the demo and watch out for updates in the future on bug fixes for block backed live migration.

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

Preview of ScaleIO and OpenStack demo at EMC World

25 Friday Apr 2014

Posted by in Industry solutions, OpenStack

≈ 2 Comments

Tags

,

I am hosting a session at the upcoming EMC World on May 5th and 6th. The session gives an overview of OpenStack with EMC; it presents EMC wide strategy and solutions for OpenStack. I plan to show couple of demos in the session to showcase EMC providing the choice of solutions for both next generation scale out workloads and Enterprise class workloads.

Let me give you a glimpse of the first demo in this blog. It is about ScaleIO being configured and executing a scale out workload. But before I start, I want to give due credit to the creator of this use case – Steve Thellen, Principle Software Engineer, EMSD (VNX/VMAX) Advanced Development team. Steve put this platform together for a project to show the integration of EMC storage platform with OpenStack, for scale out workloads. I was able to pull the demo off Steve’s setup, and put a spin on it. Let’s check it out.

The demo starts with the creation of ScaleIO volume on commodity SSD based local storage. I then attach the volume to a compute instance. And then wallah, you are ready to roll. Of course this assumes that OpenStack, ScaleIO, ScaleIO drivers have already been installed, send me a note if you are looking for installation instructions. Below is the technical implementation of such ScaleIO setup with OpenStack Nova through the ScaleIO Cinder function. ScaleIO has a driver for both client (SDC) and remote storage (SDS), but for this setup I only use client (host) side driver.

SDC

I then use VDBench tool to mimic the configuration of a high IO workload. Such workloads would be sensitive to cost ($/GB), Performance (latency and Aggregate IOPS), and scale out management. Hence making it ideal for ScaleIO. Check out my previous blog on Application Characterization for detailed concepts and rationale for such analysis.

SSD vol

I run two scripts one against the ScaleIO volumes configured to utilize locally attached SSDs; and second one with remote iSCSI capacity storage to show the contrast. The script has a read percentage of 90%, xfer size of 4k, and random seek percentage of 100%. The test is a simple one that only uses a couple of OpenStack instances spread across multiple hosts designed to exercise basic functionality. Here are the net results, as the terminal output in the Camtasia demo is not clear.

vdbench output

As you can see, ScaleIO delivers excellent results on IOPS, throughput, and response times. Not shown in the demo, but the IOPS and throughput increased drastically by adding more nodes. However the key takeaway from the demo is more than that. Messages are that you should get good results if you map your workload service levels to the storage platform; secondly you now have the choice of running ScaleIO, software based storage, on OpenStack.

Enjoy the demo! I will review the second demo in my next blog.

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

Application characterization

04 Friday Apr 2014

Posted by in Industry solutions

≈ 1 Comment

Tags

, ,

Much of today’s chatter in the cloud has been about Infrastructure as a Service and, more recently, Platform as a Service. We love the two cloud models because they are dynamic and efficient solutions for the need of application workloads. However, we can get so enamored by the response (of IaaS and PaaS) that we sometimes lose sight of the original question: What are the Service Levels required by the Application workloads? Steve Todd, an EMC fellow, writes about this in his recent blog, describing it as the biggest problem VMWare is addressing. I will go farther than Steve to say that this is the largest issue EMC Federation is addressing, at all the different levels of the platform. In this blog, I will elaborate on the need for Application characterization on service levels and how EMC is addressing this need.

Problem statement PPG     Enterprise and providers have many workloads in their data center, and these workloads have different requirements from the infrastructure they consume. However, there is a lack of semantics to define and characterize an application workload. There are times when end users are guessing what infrastructure is to be provisioned. Expensive benchmarks are needed to optimize the infrastructure, but most need to determine the infrastructure a priori. There are times when costly re-architecture needs to occur to align with the required service levels. We see this specifically with OpenStack, where users start off with commodity hardware only to revert back to reliable storage with costly reimplementation. Another facet of this problem occurs when users move to the cloud with no clear way of defining the application workload to the provider. This problem has become more severe today than ever before. The new kinds of application workloads are emerging with mobile computing (MBaaS), scale out and big data applications, etc… The platform or infrastructure itself is going through unprecedented evolution with the advent of what IDC describes as the third platform. Storage, for example, can be cached in a flash attached to PCIe, or ephemeral at the compute, or at the hot edge of shared storage with all flash arrays, or hybrid arrays or scale out arrays or cold edge of the array or glacier edge in the cloud… N-N problem

We see this as an NxN problem. If there are N number of application workloads in a data center and N number of the types of infrastructure to provision them on, an IT administrator may have NxN possible combinations to evaluate for provisioning.  The variable N is increasing every day, leading to unsolvable NxN combinations. There is no common semantic to describe the problem, let alone solve it.

Service Level Objectives

The path EMC has chosen to resolve the above NxN issue is to characterize and manage the applications with Service Level objectives. Application Characteristics service levels Each workload can be assessed on the dials of the service level objectives, like the ones shown in the picture. Now, rather than determining and optimizing exact infrastructure, the end user focuses only on the rating of the service level dials, bringing the NxN problem down to a manageable number of service level dials. Solution implementation will also become easier, as there are now discreet design targets to shoot for. Let us take the spider chart visualization of a few workloads to illustrate thERPe point. Examples are derived from EMC Product Positioning guide and are meant to be representative and not exact.

The ERP OLTP workload tends to have critical requirements for Performance and Availability. For this reason, Service Levels for IOPS, Latency, QoS, and RAS are rated as Platinum+ (4-5 on the spider chart). Increasingly integrating the application and database layers into the overall IT stack is gaining momentum and deemed critical for this case. Cost is not a concern, hence the service levels for $/GB and $/IOPS are rated as Silver (2 in the spider chart).

I will take Big Data Hadoop as the next example to contrast a workload representinhadoopg the newer 3rd platform. Typically, Hadoop workloads place high value on Cost ($/GB) and Elasticity (Scale out/Scale down) and associate lower priority on performance (IOPS) and availability (QoS, RAS). Of course, this is just an approximate depiction; I have seen some Hadoop implementations requiring higher performance and availability. We have two distinct spider charts, obviously leading to two different storage infrastructures with the closest match. This was a simple example to prove a point; in reality, you may have thousands of workloads, making such manual selection virtually impossible.

How will the solution work?

Management by Service Level objectives is elegant, but unless it could be automated, it is not a solution. We need an abstraction layer and open interface for automation. Software defined storage, with ViPR, is a perfect fit to be the arbiter between the service levels required by the workloads and the service levels provisioned by the storage. ViPR already provides the capability of policy based provisioning. In the future, it will incorporate the interface for service level objectives, and will provision based on those objectives from a virtual pool of heterogeneous arrays. SDS to workload If you are wondering how you can ease your infrastructure decision making before ViPR automations comes through, you may be able to organize your plans based on recommendations by our EMC Product positioning guide at http://www.emc.com/ppg/index.htm. EMC solutions aside, coming up with an industry accepted definition of service levels is also critical for end users to fairly assess various cloud services offered by the industry. To that end, Open Data Center Alliance – a global association of Enterprise and providers- has made recommendations for the standard definitions of service attributes for Infrastructure as a Service. The alliance definitely has the broad representation and muscle to make such an adoption successful, but only time will tell.

Conclusion

Much has been said about EMC federation’s cloud offerings, from storage (EMC II) to infrastructure (VMW) to platform (Pivotal). However, the key to its success lies in the fundamentals of understanding the workload and provisioning accordingly. You will hear more announcements along these lines in the months and years to come.

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

The inevitable open source progression: OpenStack to Cloud Foundry

10 Monday Mar 2014

Posted by in Industry solutions, OpenStack

≈ Leave a comment

Tags

, , ,

In recent posts, I have described the EMC federation’s strategy of providing our customers with a choice of solutions through horizontal stacks. EMC storage platforms work equally well with VMWare, OpenStack, and other clouds. VMWare works well on EMC storage and other platforms.

Pivotal has taken this mantra of horizontal stack further  by  offering a comprehensive, multi-cloud Enterprise PaaS (Platform as a Service) comprised of a set of application and data services that run on top of Pivotal CF™, which is the leading enterprise distribution of the Cloud Foundry® platform. Cloud Foundry is an open source platform and has been in the news lately with the announcement from Pivotal to move to an independent governance model, similar to what OpenStack did back in 2012. In this blog I will introduce Platform as a Service (PaaS) horizontal stack – Cloud Foundry and how it offers the choice of platform on OpenStack cloud.

PaaS is hot

In case you have missed it, PaaS has been on a tear lately in the press. Let me highlight couple of salient data points from a recent cloud industry survey conducted by GigaOM Research and VC North Bridge.

OSS user survey

In my mind, there is a correlation between the above two trends. IT efficiency and scale are the main drivers behind the current popularity of Infrastructure as a Service, or IaaS. However, as cloud infrastructure starts to mature, the focus shifts to businesses, demanding agility and innovation through cloud. Hence, PaaS is emerging to make a major impact on both private and public cloud infrastructures, helping companies to decrease time-to-market significantly for product development and create the applications that people use each day.

The progression from IaaS to PaaS seems only natural, as the success of Platform requires a flexible and scalable infrastructure. Thereby, it’s not a surprise that lately the OpenStack community has been humming with the discussions on PaaS. I am not going to get into the debates in this post, if you are interested I would suggest Jesse Proudman’s post for a good summary. There is one thing clear from the intensity of these debates – the industry is gearing to make the progression from IaaS to PaaS.

Hottest PaaS in town – Cloud Foundry

Ready to seize and shape the trend is the darling of open source PaaS – Cloud Foundry. Cloud Foundry is showing many of the same patterns we saw with the early momentum of OpenStack. Take a look.

CF growth

I can argue that on couple of points above, Cloud Foundry is seeing better success than OpenStack did early-on. After the initial OpenStack deployments at Rackspace and NASA, there was a long time lag in the next wave of noteworthy deployments. Cloud Foundry can already claim some very large scale production deployments at both private and public clouds. It can also claim proven integration with both VMW and OpenStack infrastructure. This is a big achievement, knowing that the future is multi-cloud. In fact I am ready to declare the progression of Open Source Software. If OpenStack is the new Linux of Open Source, Cloud Foundry will be the next OpenStack of Open Source.

OSS progression

Cloud Foundry architecture and OpenStack integration

CF Arch

The picture shows the architecture, modeled on large scale distributed services. I will avoid the details of the architecture, which can be found on the Pivotal site, but will instead focus on the key integration points with OpenStack. The Cloud Foundry deployments are done through a tool called Bosh. Stackers can think of Bosh as the equivalent of Puppet or Chef or Juju, with added features for runtime management. The core Bosh engine (and Cloud Foundry as a whole) is abstracted from any particular IaaS through a Cloud Provider Interface or CPI. IaaS interfaces are implemented as plugins to BOSH. There are plugins available for OpenStack, VMW vSphere, and Amazon Web Services. The plugin compatibility is maintained by following and enforcing 14 imperatives defined in the CPI interface.

Services beyond the infrastructure can be plugged into Cloud Foundry, and thereby to your application, through a Service Broker. Service Brokers provides an interface to all native and external third party services. Service Processes run on service nodes or with external as-a-service providers (example: database, email etc.), so advanced OpenStack modules, like Trove database service, can be integrated into Cloud Foundry through a Service Broker. These services are presented and offered to the applications via a Service Catalog.

I will save more detailed explanations for later blogs, but hopefully it gives the stackers an idea of the interface points for OpenStack.

Cloud Foundry deployment solutions

Here are three paths to Cloud Foundry solutions; the options here are very similar to that of OpenStack solutions.

  1. Open Source Software: You can download Cloud Foundry, along with its relevant interfaces, plugins, and services, directly from Github. In doing so, you will have to ‘do it yourself’ (DIY). You can also download the plugin for OpenStack from Github. By the way, an interesting fact for stackers, parts of the CF community infrastructure are run by Piston Cloud, through an integrated VMW + OpenStack solution.
  2. Cloud Foundry Distributions: Pivotal offers supported Cloud Foundry software such as Pivotal CF, which is the leading Enterprise distribution. Canonical announced that the next major release of Ubuntu will provide an integrated OpenStack hosted PaaS from Cloud Foundry.
  3. Turnkey products or platforms powered with Cloud Foundry + OpenStack: These are abundant now. Examples of systems offered are Piston Cloud, IBM BlueMix PaaS cloud, HP Cloud Application PaaS… And a long list managed providers like IBM SoftLayer, BlueBox managed hosting, along with Anynines in Europe…

Wrap

And that’s a wrap! Look for deep dives on the intersections and usage of Cloud Foundry and OpenStack in the future. I will leave you with the parting thought on the progression of open source software – Cloud Foundry, the next OpenStack of the cloud platform! Noodle over it and send me your opinions.

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

Open Storms in OpenStack!

26 Wednesday Feb 2014

Posted by in Industry solutions, OpenStack

≈ Leave a comment

Tags

,

We are pleasantly surprised by the growing popularity of our beloved open source cloud OS – OpenStack. In such a honeymoon period one would expect that the community, the players, and the projects would be working in perfect harmony. Hardly, is this the case. In fact, there are some open storms – pardon my pun – brewing in the community, and the storms are leading to many a heated debate in the projects, blogs,and tweets… I don’t view this as a negative thing at all; such debates should be expected of any healthy open source community with such vast and varied interests. An organization’s future depends on community decisions, so tension and posturing in such debates should be expected. The good thing is that so far the debates are healthy and the community continues to make steady progress even with the arguments. After all, every member fully realizes that they cannot survive by single handedly forking the stack in a different direction.

I found it interesting to follow some of the high level debates and understand different perspectives. The debates are very insightful and illustrate the tensions and tradeoffs the community is trying to make on key decision points. An understanding of the debates could help one shape the cloud strategies in one’s company, both as an IT provider and – in my case – as a cloud solution provider. So let me give you my perspective, and how I and many of my colleagues are shaping EMC’s OpenStack strategy through these debates.

The following picture summarizes a few key debates.

Open storms

Debate#1: Native or Public APIs?

The API debate is well characterized by an open letter written by Randy Bias, the CEO of CloudScaling and a prominent figure in the community. The letter goes into some of the background of the current native APIs, which  is irrelevant to the strategic discussion we are having here. However, the key point here is that one camp is pitching that OpenStack should develop its own APIs, and the contrarian camp is pitching that we should embrace public APIs, like AWS. Why would you develop your own API? To allow unconstrained innovation. Why would you embrace public APIs? To allow ease of integration between private – hybrid – public clouds. My vote is to embrace the popular pubic cloud API or drive complete compatibility with it. This is what is best for the consumers; otherwise the consumers will eventually have to carry the burden of integrating diverse cloud. For this reason, EMC ViPR offers objects that are compatible to both OpenStack Swift and S3. Swift objects, created on ViPR, will also work as S3 objects.

Debate#2: Interoperability or Choice?

The integration debate is also elaborated by Randy Bias: yes, he has mastered the act of stirring the pot. OpenStack is a highly flexible framework because it is designed on the principles of modularity so that consumers have choices at many levels through Restful APIs. However this flexibility, which is so good for choice, is unfortunately very damaging for interoperability. I have struggled working with heterogeneous stack modules, and I am sure all of you have too. So now one camp is rightfully screaming for more interoperability between solutions, while the other wants to stay put with ‘choice’, as the defining attribute of OpenStack. My vote is to continue on the premise of ‘choice’, with some understandable pains for interoperability. It’s the practice of ‘choice’ that my company, EMC, follows; I believe it’s the choice that produces the healthiest environment for the consumers, even if it is at the expense of some integration pains.

An extension of the ‘choice’ versus ‘interoperability’ debate is that of the ‘Horizontal’ versus ‘vertical’ stacks. Vertical stacks get past the interoperability and consistency issues by providing synergistic, validated, and converged stack. It certainly makes an IT consumers life easier, but in doing so you lose the basic premise of OpenStack: choice. Sacrificing choice is never good for the consumers in the long run. It is for this reason that EMC, across its federated family of VMWare and Pivotal, is committed to providing choice to its customers by delivering horizontal stacks. My vote is, you guessed it, for horizontal stack.

Debate#3: Monolithic or Pluggable?

Mono plugginThis will need some explaining if you are not familiar with OpenStack architecture. OpenStack modules typically fall in two categories. One category is highly pluggable with a small core, typically to allow API compatibility on some core functions. Cinder block service is a pluggable module, with the richness of features and opinions largely coming through the contributed plugins. The second category is monolithic, with a large core that contains most of the functionality and a fixed opinion of things are done. A good example of monolithic architecture is Swift. The line between the two categories is not cast in stone, because while pluggable projects may try to add more features in the core and on the other hand many monolithic projects may try to add more plugins over time. There is always this tension when a feature is being considered – should we be letting it come through the plugin or core? Having a rich core gets you a nicely deployable module like Swift, which could work over commodity. Having a lean core provides you a nice abstraction layer with a vast ‘choice’ of plugins. There are gigantic discussions on how we define core, but I will spare you from those details. My vote in this debate is for pluggable modules, as that not only offers better choice for the consumers, but it also allows higher ecosystem innovation, allowing companies like EMC and VMW to contribute and add value by offering a rich, but pluggable, feature set.

Summary

Here is a summary of my point of view on ‘Open Storms’.

My guidance

You can play an influential role in shaping the outcome of the debates. All you have to do is keep these debates in mind when selecting, deploying, or recommending cloud solutions, and make deliberate choices along these debated lines. You will be surprised how strong and influential the collective voice of the end user could be!

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

Vote now for the Spring OpenStack Summit

24 Monday Feb 2014

Posted by in OpenStack

≈ Leave a comment

Tags

, ,

Voting is now open for the Spring 2014 OpenStack Summit in Atlanta! All sessions are voted on, after being submitted, and the final decision regarding sessions is made by the OpenStack Technical Committee BUT votes play a huge role in the decision making process.

To this end, EMC has the opportunity to present in multiple sessions at the upcoming summit. Below I have summarized each talk and provided a link to its page on the voting site.  As always, you must be a member of the OpenStack Foundation to vote. If you’re not already a member, just click here to join as an individual member for free. Your vote will help shape the OpenStack Summit agenda and promote these interesting storage topics.

I look forward to seeing you in Atlanta in May!

  • [Vote here] Enterprise class storage for OpenStack: In this session we will analyze the enterprise class storage use cases being explored, tested and deployed today, assess current gaps, and recommend future enhancements. Some examples of these use cases would be backup and recovery, high availability, encryption, tiering … I may include members of EMC federation like VMW and Pivotal in the session.
  • [Vote here] Ask the experts: Storage considerations for OpenStack (Panelists from EMC, NetApp, Rackspace, and others): Panel will discuss storage options and formats within an OpenStack deployment in enterprises.  Panel members will discuss market awareness, adoption and best practices for storage within OpenStack environments.
  • [Vote here] Manila: The OpenStack File Share Service – Technology deep dive: In this session, NetApp, Red Hat, EMC, Mirantis, and IBM representatives will introduce the Manila File Share Service, discussing its design principles and detailing use case examples.

ViPR in the stack!

08 Saturday Feb 2014

Posted by in Industry solutions, OpenStack

≈ 1 Comment

Tags

,

In my previous blog I described the need for storage plugins for OpenStack.  One should treat plugins as a necessity to connect your choice of storage to the OpenStack framework.

why vipr in stackHowever, plugins alone would not fit in with my definition of ‘solution’; that would be when ViPR complements the stack. ViPR helps ‘solutionize’ the storage in OpenStack by providing automation and efficiencies based on service levels. Let me explain this in depth.

Efficiency: one plugin fits all!

The first and foremost advantage of using ViPR is that it provides one universal plugin that fits all storage. That means that you don’t have to deploy a storage plugin for every array / protocol used. With ViPR and the one plugin interface you can manage heterogeneous EMC and non EMC arrays, including commodity in ViPR 1.1 (scheduled to be released in Q2 ’14). Unless you were planning on using just one array for OpenStack, which is highly unlikely, this is a big win.

efficiency1

If not for ViPR, most IT shops currently would need at least 3-4 plugins, for a list of arrays say VMAX, VNX, Isilon, perhaps NetApp…. In future releases, you will also need plugins for File and Data Protection. The integration, testing, and validation for each of these plugins, every six month could be expensive. Here at EMC E-Lab, where we integrate and validate OpenStack use cases, we have 7 plugins installed and we have year round testing going on, on the cadence of OpenStack releases, with several resources deployed. Of course, we won’t be able to avoid this complexity in our lab but most IT shops can with ViPR.

Efficiency: Reusable and reliable objects

ViPR implements objects differently, it actually encapsulates OpenStack Swift implementation in its data service. In other words it replaces Swift, while maintaining API compatibility, for a good reason.

efficiency2

Firstly, when you create objects in ViPR, they can work both as Swift objects and AMZN S3 objects. So you are creating reusable objects across many platforms. This is because ViPR, being a software defined storage platform, enables compatibility across different object implementations. The concept extends the notion of ‘choice’ to the next level. Why create objects that are silod to OpenStack or S3 for that matter. Most companies are using S3 object store, which is by far the world’s largest and widely known object storage system. Therefore, it would be invaluable to be able to exchange objects between the two for business, technical, and cost reasons. 2-0 for ViPR.

Secondly, the Swift objects create a 3-way copy to disks for availability. This provides a reasonable assurance against data loss. However I have heard from so many of my customers that 3-way redundancy could be expensive. The cost is not just of the larger disk capacities needed, but also increased TCO of discovering and maintaining disk failures. ViPR gives you the choice of storage over premier arrays, like VNX or Isilon with raid5, and in doing so you remove the need for 3 ways redundancy required in Swift. Yet another choice offered by ViPR. .

Automation: Policy based provisioning

Let’s face it – OpenStack is known for its flexibility, but not for its ease of use. For example, OpenStack Cinder has APIs to create / delete block volumes, but requirements could be much more complex and granular. What if you needed volumes with replication or migration capabilities? You will need to know beforehand which arrays support what capabilities. You could still issue the create volume command from OpenStack Cinder but only to a fixed known storage array, which makes the flexibility of OpenStack only partially effective. The provisioning of storage as a service use case is less interesting if the consumer has to piece various parts together before selecting the service.

auto1

ViPR provides the concept of virtual pools, pulled together based on policies and service levels. You can have logical grouping of storage with certain characteristics, for example IOPS and availability features. This logical grouping is put together in virtual pools, which is then exposed to OpenStack as storage type. You can create, say, virtual pools of bronze, silver, gold, platinum and expose these to OpenStack, which can be then appropriately consumed by a workload or end user.  Check out Chad Sakac’s demo, at the end, for a visual.

auto2

The virtual pools in ViPR are not just limited to storage volumes, but they discover, select, and group the entire storage and network paraphernalia needed for implementing the policy. This includes switches, SAN zones, masking, etc. That’s what I mean by a storage solution.

It’s free!

Now you know what I meant earlier by ViPR ‘solutionizing’ storage for OpenStack. But wait; there’s more. You can now download EMC ViPR at no charge for non-commercial use for as long as you want. How about I end this blog on that sweet offer!

Nikhil Sharma

http://SolutionizeIT.com/

Twitter: @NikhilS2000

 

Follow

Get every new post delivered to your Inbox.

Join 28 other followers