Tag Archives: virtualization

Cloud infrastructure Economics: Calculating IaaS cost

In our previous post, we referenced a number of financial factors and operational parameters that we need to take into account in order to calculate some meaningful costs for IaaS services. Let’s see now how these are combined to produce IaaS cost items.

Your mileage may vary on IaaS; you may be renting datacenter space, leasing equipment, operating your own facilities or simply using public clouds to run your business. However, at the end of the day, you need to utilize an apples-to-apples metric to find out which strategy is the most cost-effective. For IaaS, the metric is the footprint of your infrastructure, expressed in terms of virtual computing units: The monthly cost of a single virtual server, broken down to virtual CPU/Memory and virtual storage resources.

You can check online the formulas in this sheet here. If you want to take a peek on how the formulas work, continue reading.

For simplicity, we will not dive into virtua machine OS licensing costs here – they are easy to find out, anyway (those familiar with Microsoft SPLA should have an idea). We will calculate only monthly running costs of IaaS, expressed in the following cost items:

  • SRVCOST: Individual virtual server monthly cost: Regardless of virtual machine configuration, the monthly cost of spinning up a virtual machine.
  • COMPUTECOST: Virtual computing unit cost: The cost of operating one virtual memory GB and assorted CPU resources per month.
  • DISKCOST: Virtual disk unit cost: The cost of operating one virtual storage GB per month.

Almost all IaaS public cloud providers format their pricelists according to these three cost items, or bundle them in prepackaged virtual server sizes. Calculating these can help immensely in finding good answers to the “build or buy” question if you plan to adopt IaaS for your organization, or determine the sell price and margins if you are a public cloud provider.

Let’s see now each cost item one by one.

Individual server cost (SRVCOST)

How much does it cost to spin up a single virtual machine per month? What do we need to take into account? Well, a virtual machine, regardless of its footprint, needs some grooming and the infrastructure it will run on. The assorted marginal costs (cost to add one more virtual machine to our IaaS infrastructure) are the following:

  • C_SRV: Cost of maintaining datacenter network infrastructure (LAN switching, routing, firewall, uplinks) and computing infrastructure software costs (support & maintenance). We do not include here hardware costs since these are related to the footprint of the virtual machine.
  • C_DCOPS: Cost of manhours required to keep the virtual machine and related infrastructure up and running (keep the lights on)
  • C_NWHW: Cost of network related hardware infrastructure required to sustain one virtual machine. These are pure hardware costs and reflect the investment in network infrastructure needed to keep adding virtual machines.

An essential unit used in most calculation is the cost of rack unit. Referring to our older post for the EPC variable, this is expressed as


This gives us an approximation of the cost of one rack unit per month in terms of monthly electiricy and hosting cost (EPC).

C_SRV is expressed as a function of NETSUPP (monthly network operating & support costs), RU_NET (total network infrastructure footprint), CALCLIC (virtualization/computing infrastructure maintenance & software costs) and SRV (total virtual servers running). The formula is:


C_RU*RU_NET is the hosting cost of the entire networking infrastructure (switches, patch panels, load balancers, firewalls etc).

C_DCOPS is straightforward to calculate:


And finally C_NWHW is the hardware cost needed to add one more virtual server. To calculate C_NWHW we take into account the current network infrastructure cost and then we calculate how much money we have to borrow to expand it in order to provision one more virtual server. The way we do this is to divide the total network infrastructure cost with the number of provisioned virtual machines and spread this cost over the lifecycle of the hardware (AMORT), augmented with a monthly interest rate (INTRST):


Computing cost (COMPUTECOST)

As a computing unit, for simplicity we define one GB of virtual RAM coupled with an amount of processing power (CPU). Finding the perfect analogy between memory and CPU power is tricky and there is no golden rule here, so we define the metric as the amount of virtual RAM. The exact CPU power assigned to each virtual RAM GB depends on the amount of physical RAM configured in each physical server (SRVRAM) and the number of physical CPU cores of each server. COMPUTECOST is broken down to two cost items:

  • C_MEM: It is the cost associated with operating the hardware infrastructure that provisions each virtual RAM GB.
  • C_SRVHW: It is the cost associated with purchasing the hardware infrastructure required to provide each virtual RAM GB.

C_MEM depends on running costs and is the cost of compute rack units divided by the total virtual RAM deployed in our cloud:


Note that in some cases (like VMware’s VSPP program) you may need to add up to the above cost software subscription/license costs, if your virtualization platform is licensed per virtual GB.

C_SRVHW is calculated in a more complex way. First, we need to find out the cost of hardware associated with each virtual GB of RAM. This is the cost of one physical server equipped with RAM, divided with the amount of physical RAM adjusted with the memory overprovisioning factor:


In a similar way with C_NWHW, we calculate the acquisition cost spread over the period of infrastructure lifecycle, with the monthly interest rate:


Virtual storage cost (DISKCOST)

Calculating DISKCOST is simpler. The two cost items, in a similar way to COMPUTECOST are:

  • C_STOR: It is the cost associated with operating the hardware infrastructure that provisions each virtual RAM GB.
  • C_STORHW: It is the cost associated with purchasing the hardware infrastructure required to provide each virtual disk GB.

C_STOR is based on the existing operating costs for running the storage infrastructure and is calculated proportionally to the provisioned disk capacity:


C_STORHW is the cost of investment for each storage GB over the infrastructure lifecycle period:



One can elaborate on this model and add all sorts of costs and parameters, however, from our experience, this model is quite accurate for solving an IaaS financial exercise. What you need simple datacenter metrics and easily obtained costs.

Cloud infrastructure Economics: Cogs and operating costs

Perhaps the most important benefit of adopting cloud services (either from a public provider or internally from your organization) is that their cost can be quantified and attributed to organizational entities. If a cloud service cannot be metered and measured, then it should not be called a cloud service right?

So, whenever you need to purchase a cloud service or when you are called to develop one, you are presented with a service catalog and assorted pricelists, from where you can budget, plan and compare services. Understanding how the pricing has been formulated is not part of your business since you are on the consumer side. However, you should care: You need to get what you pay for. There must be a very good reason for a very expensive or a very cheap cloud service.

In the past, we have developed a few cloud services utilizing own resources and third party services. Each and every time, determining whether launching the service commercially would be a sound practice depended on two factors:

  • Would customers pay for the service? If yes, at what price?
  • If a similar service already was on the market, where would our competitors stand?
  • What is the operating cost of the service?

Answering the first two questions is straightforward: Visit a good number of trusted and loyal customers, talk to them, investigate competition. That’s a marketing and sales mini project. But answering the last question can be a hard thing to do.

Let us share some insight on the operating costs and cost-of-goods for a cloud service and in particular, infrastructure as a service (IaaS). Whether you already run IaaS for your organization or your customers, you are in one of the following states:

  1. Planning to launch IaaS
  2. Already running your datacenter

State (1) is where you have not yet invested anything. You need to work on implementation and operational scenarios (build or buy? Hire or rent?) and do a good part of marketing plans. State (2) is where you have already invested, you have people, processes and technology in place and are delivering services to your internal or external customers. In state (1) you need to develop a cost model, in state (2) you need to calculate your costs and discover your real operating cost.

In both cases, the first thing you need to do before you move on with cost calculation is to guesstimate (state 1) or calculate (state 2) the footprint of your investment and delivered services. From our experience, the following parameters are what you should absolutely take into account in order to properly find out how much your IaaS really costs.

Financial parameters (real money)

  • EPC: Electrical power and hosting cost. How much do (or would) you pay for electricity and hosting. This can be found from your electricity bill, your datacenter provider monthly invoice or from your financial controller (just make sure you ask them the right questions, unless you want to get billed with the entire company overhead costs). EPC is proportional to your infrastructure footprint (ie number of cabinets and hardware).
  • DCOPS: Payroll for the operations team. You need to calculate the total human resource costs here for the team that will operate IaaS services. You may include here also marketing & sales overhead costs.
  • CALCLIC: Software licensing and support costs for IaaS entire computing infrastructure layer. These are software costs associated with the infrastructure (eg, hypervisor licenses), not license costs for delivered services, eg Microsoft SPLA costs.
  • STORLIC: Software licensing and support costs for your entire storage infrastructure. Include here in their entirety also data backup software costs.
  • SERVER: Cost of a single computing server. It’s good to standardize on a particular server model (eg 2-way or 4-way, rackmount or blade). Here you should include the cost of a computing server, complete with processors but without RAM. RAM to CPU ratio is a resource that is adjusted according to your expected workloads and plays a substantial role in cost calculation. If you plan to use blade servers, you should factor here the blade chassis as well.
  • MEMORY: Average cost of 1 GB or RAM.
  • STORINFRA: Cost of your storage infrastructure, as is, or the storage infrastructure you plan to purchase. Storage costs are not that easy to calculate as a factor of 1 disk GB units, since you have to take into account SAN, backup infrastructure, array controllers, disk enclosures and single disks. Of course we assume you utilize a centralized storage infrastructure, pooled to your entire computing farm.
  • NETINFRA: Cost of data network. As above, include here datacenter LAN, load balancers, routers, even cabling.
  • NETSUPP: Cost of network support (monthly). Include here software licensing, antivirus subscriptions and datacenter network costs.

Operational parameters (Facts and figures)

  • RUAmount of available rack units in your datacenter. This is the RU number you can use to install equipment (protected with UPS, with dual power feeds etc).
  • RU_STOR: Rack units occupied by storage systems
  • RU_CALC: Rack units occupied by computing infrastructure (hypervisors)
  • RU_NET: Rack units occupied by network infrastructure
  • SRV: Virtual machines (already running or how many you plan to have within the next quarter)
  • INTRST: Interest rate (cost of money): Monthly interest rate of credit lines/business loans
  • TOTALMEM: Total amount of virtual memory your SRV occupy
  • TOTALSTOR: Total amount of virtual storage your SRV occupy
  • SRVRAM: Amount of physical memory for each physical server. This is the amount of RAM you install in each computing server. It is one of the most important factors, since it depends on your average workload. A rule of thumb is that for generic workloads, a hardware CPU thread can sustain up to 6 virtual computing cores (vcpu). For each vcpu, you need 4 GB of virtual RAM. So, for a 2-socket, 6-core server you need 2 (sockets) x 6 (cores) x 6 (vcpu) x 4 (GB RAM) = 288 GB RAM. For a 4-way, 8-core server beast with memory intensive workloads (say 8 GB per vcpu) you need 4 x 8 x 6 x 8 = 1536 GB RAM (1.5 TB).
  • MEMOVERPROV: Memory overprovisioning for virtual workloads. A factor that needs tuning from experience. If you plan conservatively, use a 1:1 overprovisioning factor (1 GB of physical RAM to 1 GB of virtual RAM). If you are more confident and plan to save costs, you can calculate an overprovisioning factor of up to 1.3. Do this if you trust your hypervisor technology and have homogenous workloads on your servers (for example, all-Windows ecosystem) so that your hypervisor can take advantage of copy-on-write algorithms and save physical memory.
  • AMORT: Amortization of your infrastructure. This is a logistics & accounting term, but here we mainly use this to calculate the lifespan of our infrastructure. It is expressed in months. A good value is 36 to 60 months (3 to 5 years), depending on your hardware warranty and support terms from your vendor.

If you can figure out the above factors, you can proceed with calculating your operating IaaS costs. Keep reading here!

Squeezing VDI in a box

Virtual desktop infrastructure (VDI) is nothing new. Decoupling a Windows desktop from a physical PC and converting it into a virtual image, accessible from more than one terminal has been around for many years, pioneered mostly by Citrix. XenDesktop is the platform of choice for large enterprises, rolling out hundreds and thousands of virtual desktops for internal users and mobile workers.

However, implementing a VDI solution is a complex project with lots of moving parts, and XenDesktop is no exception: For an end to end solution that can be used from all sides of the enterprise (intranet, Internet and extranet users) one needs all of these: Virtualization platform (hypervisors and shared storage), connection brokers, catalog repositories and asset database, provisioning services, desktop image preparation tools, connection proxies, firewalls and load balancers.

Well… Citrix has managed to squeeze all of the above in a single box, with VDI-in-a-box (ViaB). The acquisition of Kaviza in 2011 led to the release of ViaB with tight integration of HDX (the network protocol used by XenDesktop to move pixels, keystrokes and data from the virtual desktop to the user endpoint) and NetScaler (Citrix’s load balancer and application proxy). ViaB comes in the form of a virtual appliance, ready to boot in your favorite hypervisor (ESXi, XenServer and HyperV). The ViaB appliance talks directly to the hypervisor to provision virtual desktops, itself is a connection broker, provisioning server and image preparation platform and works in a grid with other ViaB instances, forming a VDI cluster by just setting up more hypervisor servers, each with a ViaB appliance and joining them in a single cluster. ViaB works with local storage in each hypervisor – no requirement here for DRS or shared SAN storage.

I recently had the chance to setup ViaB as a proof of concept. Literally, the solution is enclosed in a single box. Using a 16GB RAM dual-socket server with ESXi 5.1 (free edition), BiaB was setup and configured in less than two days, given that everything was configured from scratch. The recipe is:

  • A Windows 7 Pro DVD iso image (and corresponding valid key)
  • A Windows 2008R2 server iso image
  • A physical server. Anything with 16GB RAM and 60GB local storage is sufficient for a PoC with five concurrent desktops.
  • Citrix Netscaler 10 virtual appliance (I used version 10, build 71)
  • VDI in a box version 5.1.1 ESXi virtual appliance
  • To test with Internet desktops, two public IP addresses and a FQDN valid DNS entry pointing to the Internet IP address of ViaB. The other IP address is used for outbound connections from the desktops to the Internet via NAT, through NetScaler.

There are detailed guides from Citrix to setup your environment here; the process is quite straightforward, just pay attention to small details like setting up your ViaB to talk correctly to active directory services and your DNS server. In a nutshell, that’s that you do:

  1. Setup your hypervisor. A single Ethernet will do for Internet access. All the other subnets and port groups will be contained inside your hypervisor virtual switch. You need one virtual switch and three port groups in ESXi: an Internet port group, attached to your Internet public network, a private numbered port group to run your virtual desktops, the ViaB appliance, your Windows domain controller and the internal Netscaler proxy port, and finally another VMkernel port group, in the same IP private subnet as your VDI subnet, so that the hypervisor can be accessed from your ViaB appliance. Make sure you have configured an ESXi management address there. The setup I used is shown below:
  2. Install NetScaler with an access gateway license. NetScaler is a fully featured application delivery controller (ADC) which, in our context, will be used as an HDX proxy for desktop connections to the end users via Internet through SSL (TCP port 443) and also as a NAT gateway/firewall, so that all virtual desktops can send traffic to Internet hosts. Installation is easy, download the virtual appliance from Citrix and deploy on ESXi. Setup one NetScaler interface on the public (Internet) network and another interface on the internal private network.
  3. Install the ViaB appliance. The whole process takes minutes. Just deploy the OVF template, directly downloaded from Citrix. Add a license and configure ViaB to talk to your ESXi through the management port setup in the VMkernel port group.
  4. Install a Windows 7 image, enter a valid key, apply latest Windows updates, install VMware tools and leave it running. This image must have a single Ethernet interface attached on the internal VDI network.
  5. Install a Windows 2008R2 server, add active directory services, configure DHCP and DNS, apply Windows updates, install VMTools. Again, attach a single interface on the VDI network. Promote to domain controller and setup a new forest, which will be used to authenticate your desktop users, attach virtual desktops to the domain and apply group policies. This domain controller will be used also to host your users’ roaming profiles, since the desktops that I will deploy will be stateless, erased and recreated every time a user logs out. Here, you can of course use an existing domain controller, just make sure you configure your virtual networks and routing correctly. Best practice is to use separate OUs for desktops and users. Find a snapshot of the AD structure:

    AD structure

  6. Configure a public FQDN pointing to your external NetScaler IP address. Create also a NAT rule in NetScaler, permitting traffic from the internal VDI network towards the Internet.
  7. Now, go to Citrix and follow the instructions in this article. Configuration occurs in two places: NetScaler, to setup the access gateway and the ViaB appliance. The most tedious part is the configuration of NetScaler. I preferred the methid described above instead of using the access gateway wizard, since it’s easier to go back and correct mistakes.
  8. After you have configured NetScaler and access gateway, you are ready to start building desktop images. VDI in a box here is a great tool to use, since it hides all the mechanics of using sysprep and other tools: It prepares your Windows 7 image, installs Citrix HDX agents, configures Windows firewall and lots of other settings.
  9. After you test your image, create templates, add users or groups from your AD and you are set to go. To access virtual desktops, your users have to install Citrix Receiver and point any browser to your NetScaler external HTTP port. There, they enter valid credentials from your AD and connect to desktops.

My guinea pig was my 9-yr old daughter, which by herself logged in, installed Chrome (and flash) on the virtual desktop and accessed her favorite web site, all from the iPad:

Windows 7, iPad view

According to my trusted reviewer, the GUI was snappy, without latency and the whole thing felt much faster. Reasonable, since the desktop was running on a Xeon server.

This is the same view from a conventional PC:

Same view from Windows 7 desktop


A quick tour of Cloudstack

Cloud.com, now a part of Citrix, has developed a neat, compact, yet powerful platform for cloud management: Enter cloudstack, a provisioning, management and automation platform for KVM, Xen and VMware, already trusted for private and public cloud management frmo companies like Zynga (got Farmville?), Tata communications (public IaaS) and KT (major Korean Service provider).

Recently I had the chance to give cloudstack a spin in a small lab installation with one NFS repository and two Xenservers. Interested in how it breathes and hums? Read on, then.

Cloudstack was installed in a little VM in our production vSphere environment. Although it does support vSphere 4.1, we decided to try it with Xen and keep it off the production ESX servers. Installation was completed in 5 minutes (including the provisioning of the Ubuntu 10.04 server from a ready VMware tremplate) and cloudstack came to life, waiting for us to login:

The entire interface is AJAX – no local client. In fact, cloudstack can be deployed in a really small scale (a standalone server) or in a full-blown fashion, with redundant application and database servers to fulfill scalability and availability policies.

Configuring cloudstack is a somewhat more lengthy process and requires reading the admin guide. We decided to follow the simple networking paradigm, without VLANs and use NFS storage for simplicity. Then, it was time to define zones, pods and clusters, primary and secondary storage. In a nutshell:

  • A zone is a datacenter. A zone has a distinct secondary storage, used to store boot ISO images and preconfigured virtual machine templates.
  • A pod is servers and storage inside a zone, sharing the same network segments
  • A cluster is a group of servers with identical CPUs (to allow VM migration) inside a pod. Clusters share the same primary storage.
We created a single zone (test zone) with one pod and two clusters, each cluster consisting of a single PC (one CPU, 8 GB RAM) running Xenserver 5.6. Configuring two clusters was mandatory, since the two Xenservers were of different architectures (Core 2 and Xeon). After the configuration was finished, logging in to Cloudstack as administrator brings us to the dashboard.

In a neat window, the datacenter status is shown in clear, with events and status in the same frame. From here an administrator has full power over the entire deployment. This is a host (processing node in Openstack terms) view:

You can see the zone hierarchy in the left pane and the virtual machines (instances) running on the host shown in the pane on the right.

Pretty much, what an administrator can do is more or less what Xencenter and vCenter do: Create networks, virtual machine templates, configure hosts and so on. Let’s see how the cloudstack templates look like:

Cloudstack comes with some sample templates and internal system virtual machine templates. These are used internally, but more on them later. The administrator is free to upload templates for all three hypervisor clans (KVM, Xen and Vcenter). For KVM, qemu images, for VMware, .ova and for Xenserver VHD. We created one Windows 2008 server template quite easily, by creating a new VM in Xencenter, installing Xentools and then uploading the VHD file in Cloudstack:

As soon as the VHD upload is finished, it is stored internally in the Zone secondary storage area and is ready to be used by users (or customers).

How does cloudstack look like from the user/customer side? We created a customer account (Innova) and delegated access to our test zone:

Customers (depending on their wallet…) have access to one or more pods and can create virtual machines freely, either from templates of from ISO boot images they have access to, without bringing into the loop cloudstack administrators. Creating a new virtual machine (instance) is done through a wizard. First, select your favorite template:

Then, select a service offering from preconfigured sizes (looks similar to EC2?)

Then, select a virtual disk. A template comes with its own disk (in our case the VHD we uploaded earlier), but you can add more disks to your instances. This can also be done after the instance is deployed.

…and after configuring the network (step 4), you are good to go:

The template will be cloned to your new instance, boot up, and form this point on, you can log in through the web browser – no RDP or VNC client needed!

It’s kind of magic — doing this via an app server seems impossible, right? Correct. Cloudstack deploys silently and automagically its own system VMs that take care of template deployment to computing nodes and storage. Three special kinds of VMs are used:

  • Console proxies that relay to a web browser VNC, KVM console or RDP sessions of instances. One console proxy runs in every zone.
  • Secondary storage VM, that takes care of template provisioning
  • Virtual router, one for every domain (that is, customers), which supplies instances with DNS services, DHCP addressing and firewalling.
Through the virtual router users can add custom firewall rules, like this:
All these system virtual machines are managed directly from cloudstack. Login is not permitted and they are restarted upon failure. This was demonstrated during an unexpected Xenserver crash, which brought down the zone secondary storage VM. After the Xenserver was booted up, the secondary storage VM was restarted automatically by cloudstack and relevant messages showed up in the dashboard. Cool, huh?

Customers have full power over their instances, for example, they can directly interact with virtual disks (volumes), including creating snapshots:

In all, from our little cloudstack deployment we were really impressed. The platform is very solid, all advertised features do work (VM provisioning, management, user creation and delegation, templates, ISO booting, VM consoles, networking) and the required resources are literally peanuts: It is open source and all you need are L2 switches (if you go with basic networking), servers and some NFS storage. Service providers investigating options for their production IaaS platform definitely should look into cloud.com offerings, which has been a part of Citrix since July 2011.

A quick spin of Amazon EC2

This is my version of how-to-create-an-EC2-instance-with-pictures. Users of Amazon AWS stuff will find these trivial, others are welcome to see how to create your own little virtual servers in Amazon’s (non-free) cloud infrastructure.

The first thing you need to do, is of course to sign up to Amazon AWS. Point your browser to aws.amazon.com and click on the “Sign in to the AWS Management Console” link (top right). Creating an account is trivial, except that you have to enter your credit card number and a valid telephone number. The credit card number is mandatory (you have to be billed somehow to use AWS); the phone number will be used for Amazon’s automated billing service to literally give you a call and ask you to enter the four digit random challenge number that will show up on your browser. So, enter a valid phone number in a nearby phone, wait for it to ring, type the number and your account is created.

After that, you are a customer of Amazon Web Services. You will now be transferred to the AWS console, which looks like this:

The tabs at the top are all Amazon services available to customers. From here, you can create virtual machines, use elastic storage services, change networking rules, use platform tools and virtually run your own (virtual) datacenter from your browser. The limit is your wallet.

Before creating any resources, it is vital to do some geeky stuff, like downloading Amazon’s command line tools. Ubuntu people can do that like this:

# apt-get install ec2-api-tools

EC2 tools allow creation and management of AWS on the fly. They are front-end utilities to Amazon’s web services API, which is well documented and open, allowing Amazon customers to develop own applications and frameworks that directly interact with the AWS cloud. To make the EC2 API tools work, you need to take a few extra steps. Accessing the AWS API is not done via a password, but by using two authentication methods: A symmetric key to access  REST & query APIs and your personal X.509 certificate and private key signed by Amazon. These are used to make use of their SOAP web services API. Download them (.pem files) and store them in at least two safe locations. Note: The private key is generated only once. Amazon will not keep a copy; if you lose it, it is impossible to use the web services API again and you have to generate a new one.

In addition to the above, you will also need a keypair to log in to your instance. And now that you have your certificate and keys, you can fire up the console and start creating your own virtual servers. The easiest way is to select any of the offered, preconfigured Amazon Machine Images (AMI) that are available:

Amazon offers two free, no-cost (really really small…) Linux images and many more, with all sorts of operating systems (including Microsoft Windows) and middleware preinstalled. The AMI marketplace is growing, with images submitted from all major software vendors.

The customization options of your virtual server will look familiar to Xen and vCenter users (selection of memory, disk, CPUs etc), with the extra option of network parameters, like configuring access ports:

Firewall configuration

By default for a Linux AMI only SSH (port 22) is available. The next step is to start your virtual server. Select your instance and from the “Instance Actions” menu select “Start”. Wait a couple of minutes for EC2 and EBS to provision your virtual server, add another minute for booting and your machine comes to life:

Something that Xen and vCenter users would expect to be there and is not: The console. AWS does not provide (at this time) a console window where you can see your server booting up and running; rather, you have to wait until SSH (or RDP for Windows VMs) starts up. Then you can login like this:

SSH into your AMI instance

Remember what we said about keys? There is no password to log in via SSH, you have to use they keypair you have downloaded earlier. As soon as you log in, you can sudo to root (no password required) and configure your virtual server the way you like.

Apart from starting and stopping your virtual server, the AWS console allows you to create and restore disk snapshots, like this:

…and retrieve detailed report usage reports in CSV and XML format:

…and have a 10,000ft view of AWS status:

Many more features and services are available: S3 storage services, purpose-built AMIs, load balancers, CloudFront services, network latency and bandwidth options, all are available for a price, summarized in a single page:

That’s what true IaaS looks like. Signing up, creating a VM and bringing it up and live on the Internet does not take more than 15 minutes. The underlying infrastructure is massive and in constant development for close to 5 years now, yet, mature enough to be used from all kinds of customers, from freelancers up to large enterprises.

Building a cloud

Question: How many people do you need to build and run a cloud?

Answer: As many as you can fit in a meeting room.

A cloud offering IaaS and SaaS to customers is nothing more than a compact and complex technology stack. Starting from the bottom to the top, you have servers, storage (NFS/iSCSI/FC), networking (LIR, upstream connections, VLANs, load balancers) , data protection (snapshots, replication, backup/restore), virtualization (pick your flavor), cloud management (Applogic/Openstack/Cloudstack/OpenNebula/Abiquo/vCommander/you-name-it), metering & billing (eg WHCMS), helpdesk (like Kayako), user identity management, database platform (Hadoop), application servers, hosted applications and web services. All this stuff has to work. And work efficiently, if you want to attract, retain and expand your customer base, simply because your customers simultaneously use all these resources: From their browsers, customer actions ripple through firewalls, load balancers, switches, web and application servers, databases, hypervisors and disks, crossing the entire cloud stack up, down and sideways.

The only way to run this stack is… to use humans. Of what skills? System engineering, storage management, networking, security, application architecture, coding, coding, coding, web marketing, technical management and more coding. And all of them must be able to sit around the same table, talk and understand each other, if you want your cloud stack to simply work. This calls for a small headcount of gifted people (and well compensated – slide 8) that can not only deliver on the technical side but understand the cloud business and the Internet business as well.

The trick question: What kind of company can host this ecosystem? Service providers? Datacenter hosting? Web hosters? Software vendors? Well… this would depend on the company DNA. Take for example Amazon and Google. Neither was a datacenter/network provider or software vendor; Amazon is the largest online retailer, Google is the king of online advertising. Yet, both of them fostered the right kind of people that spun off what we have and use today.

Of datacenter transformation and clouds

Looking back a few years, when virtualization was still “under evaluation” and Facebook a fancy new thing, the term “cloud” did not exist. It’s not a coincidence that we started talking about cloud computing no sooner than Internet industries (Salesforce, Google, Amazon et al) reached the critical mass to offer services attractive to enterprise IT, and at the same time, the enterprise IT emerged as a business enabler.

What happened? Software and infrastructure as a service, a game well understood by cloud providers, came within the reach of enterprise datacenters. Private clouds now are a reality, a way to provide services to internal customers of an organization on demand, swiftly, on a consolidated multitenant architecture. For an IT worker, the change of the landscape is dramatic. The datacenter transformation from the established server-OS-application stack to the mesh of the cloud (server-hypervisor-shared storage-virtual networking-virtual OS-dynamic load balancing-automatic scaling-resource metering-automation-application server-AJAX stacks) is so immense that it’s very hard to keep up and sometimes, not understood at all.

Let’s spend 168 seconds and see what a private cloud stack would look like:

  • Uniform servers, which in some cases have dual or triple role (computing, storage, networking) with lots of RAM, CPU cores and network ports
  • The network is a massive switching core, tuned to offer troublesome automation and constant reconfiguration, using the same fabric for both data and storage traffic (check out OpenFlow)
  • Massive storage with lots of spindles to cope with mostly write traffic, integrated snapshots and data replication
  • Two or more hypervisor clans (XEN, VMware, HyperV, KVM), each with its own management and provisioning stack
  • A variety of virtual machine templates, all flavors of Linux and Windows, from desktop (VDI) to server
  • Metering of resource consumption and billing
  • Service catalog to end consumers, provisioning workflows
  • Licensing made quite complex (see this and this)
  • Lots of the usual enterprise stacks (eg Citrix, Oracle, Websphere) virtualized and highly available from the hypervisor layer
  • Virtual backup solutions (like Veeam)
  • And an automation stack to rule them all (see this)
Most of these require skills beyond those of an average IT engineer. Storage, networking and virtualization skills are mandatory in order to understand and handle this stack. But, that’s what a cloud is: more than the sum of its parts. 
Ironically, integrators that claim to offer “datacenter transformation” services in essence mean cloud computing, but have failed to realize this fact. They do not grasp the full picture and tend to offer point solutions (a hypervisor here, a smarter backup solution there and shiny powerful servers). The sooner system integrators realize this, the better for them and their customers.