Category: Cloud Speak

Bellingham Cloud Talk, Coming Right Up

Here’s the basic outline of what I intend to speak on at the upcoming presentation I have for the Bellingham, Washington .NET Users Group.  If you happen to be in the area you should swing by and give it a listen (or heckle, whatever you feel like doing).

On April 5th I have a talk lined up with the Bellingham .NET Users Group. So far here’s a quick one over of the talk:

What Cloud Computing REALLY is to us techies

  • Geographically dispersed data centers.
  • Node based – AKA grid computing configurations that are…
  • Highly Virtualized – thus distributed.
  • Primarily compute and storage functionality.
  • Auto-scalable based on demand.

What kind of offerings exist out in the wild?

  • Amazon Web Services
  • Rackspace
  • Orcs Web
  • GoGrid
  • Joyent
  • Heroku
  • EngineYard

…many others and then the arrival in the last year”ish” of…

  • Windows Azure
  • AppHarbor

Developing for the cloud, what are the fundamentals in the .NET world?

Well, let’s talk about who has been doing the work so far, pushing ahead this technology.

  • Linux is the OS of choice… free, *nix, most widely used on the Internet by a large margin, and extremely capable…
  • Java
  • Ruby on Rails
  • Javascript & jQuery, budding into Node.js via Google’s V8 Engine
  • The Heroku + EngineYard + Git + AWESOMESAUCE capabilities of pushing… LIVE to vastly scalable and distributable cloud provisions!

So where does that leave us .NETters?

AWS .NET SDK released a few years ago.
Windows Azure & SDK released about a year ago.

These two have however been lacking compared to Heroku and EngineYard for those that want something FAST,

something transformative, easy to use, without extra APIs or odd tightly coupled SDKs.

Enter…
AppHarbor

In Summary the .NET Platform has primarily:

AWS for the top IaaS and most widely available zones & capabilities at the absolutely lowest prices,

Windows Azure for the general build to PaaS Solution, and for the people lucky enough to be going the Git +

MVC + real Agile route, AppHarbor is the peeminent solution.

Demo Time…

Windows Azure Demo

AWS Demo

AppHarbor Demo

Git Rid of Windows Azure and Amazon Web Services (AWS) SDKs with .NET + Git + AppHarbor Deployment Revolution

I’ve been wanting to do a quick write up on the state of cloud apps from my perspective.  What’s my perspective?  Well I’m keeping up with  the SDKs from the big players; AWS and Windows Azure.  I’m also working on several cloud applications and providing consulting for some people and companies when approached related to which stack to go with, how to apply their current stacks (such as Ruby on Rails or .NET) in migrating to a cloud service provider.  Cloud services, or really more accurately utility computing has my personal and professional interest.  Above all, I keep trying to stay informed and know what the best path is for anyone that seeks my advice for moving into hosting & working in the SaaS, PaaS, or IaaS Space.  Feel free to contact me in regards to cloud questions:  adronhall at the famous gmail dot com.  🙂

Now on to the good tidbits that have been released lately.

The latest Microsoft goodies area available.  For the Windows Azure SDK go check out the Microsoft MSDN Site.

For the latest awesome from AWS (Amazon Web Services) SDK check out the AWS .NET Site.

These two SDKs are great for customers who want to build on the bare bones X platform.  Now whatever language & stack one builds in they are tied to that.  Ruby on Rails, .NET, Java, PHP, or whatever.  But getting tied to the stack is kind of like breathing air, one has to live with what air they have.  You can’t exactly get a refund very easily on that.

The Cloud SDKs though for Azure & AWS provide a certain amount of lock in, in addition to the stack lock in you’re using.  One of the easiest ways to prevent this lock in is to use a general deployment method backed by source control on something like Git or Mercurial.  So far though, .NET has been left out the cold.  There has been almost zero support for pushing .NET via Git or Mercurial into a cloud.

Heroku
Heroku

Ruby on Rails however has had support for this since…  well since the idea popped into the minds of the people at Heroku, EngineYard, and the other companies that are pushing this absolutely amazing and powerful technology pairing.

Engine Yard
Engine Yard

Again, for .NET, the problem is it has been left in the dust.  Smoked.  It has left a lot of .NET Developers moving to Ruby on Rails (which isn’t new, this is just one more thing that has pulled more developers away from the .NET stack).

 

Well, that’s changed a bit.  FINALLY someone has gotten the Git + .NET Pairing in the Cloud put together!  FINALLY you can get a cloud application running in a minute or two, instead of the absolutely inane amount of time it takes on Windows Azure (15+ minutes most of the time).  So who has done something about this?

AppHarbor is the first fully deployable solution for the cloud that allows Git + .NET to get going FAST!  I don’t work for these guys at all, so don’t think I’m shilling for them.  I’m just THAT happy that .NET has been pulled out of the dust bins and the community has this option.  I am flippin’ stoked matter of fact.

Currently, because of pricing and ease of deployment, I’ve been solely using AWS.  I can have a .NET MVC app running in AWS in about 5-10 minutes.  Between that speed of setup and the pricing, I pay 2/3 as much as Azure would be and can deploy much fast with a completely traditional .NET deployment.  No special project type needed, no extra configs, just a straight deployment with full control over the server (i.e. I can RDP in with no problem).  Anyway, the list of reasons I went with AWS over Azure really deserve an entire blog entry unto themselves.

AppHarbor
AppHarbor

With AppHarbor though I can step into the realm of doing exactly the same thing a Ruby on Rails Developer would do with Heroku or EngineYard.  Fully PaaS Capable with the scalability and features without needing to port or migrate to an entirely new stack!  I’ll probably keep a number of things running on AWS (such as the pending WordPress Websites I am about to push up to AWS), but will absolutely be starting up some applications to run in AppHarbor.

If you’re a .NET Developer and you’ve been wanting, looking for, and frustrated that the .NET Community didn’t have a Git + Cloud Deployment option for .NET, wait no longer.  Give AppHarbor a look ASAP!

Anyway… off to do a little work on my infrastructure project.  Cheers!

Overloaded to Death, “Cloud” Computing is Dead

I’ve gotten to a point where I won’t argue it anymore.  The word Cloud as defined on Wikipedia

“location-independent computing, whereby shared servers provide resources, software, and data tocomputers and other devices on demand, as with the electricity grid. Cloud computing is a natural evolution of the widespread adoption of virtualization, service-oriented architecture and utility computing. Details are abstracted from consumers, who no longer have need for expertise in, or control over, the technology infrastructure “in the cloud” that supports them.“

has been overloaded to such a large degree that it doesn’t actually mean anything anymore.  Cloud computing has become a casualty of media hype.  Utility computing, service oriented architecture, virtualization, and other things still have meaning, but cloud computing really is lost.  We have cloud services in Office 365, which isn’t particularly stored or operated from the cloud.  We have vertically distributed, geographically dependent databases & other services with are called cloud, but also don’t meet the basic definition above.  We have private cloud computing, which also is a perversion of the definition above.  Basically the term cloud has been stuck onto anything technology related.  Your phone, your website, your car, your everything is now all of a sudden supposed to be provided by the cloud.

In laymen’s terms, “The cloud is bullshit.”

I feared this would happen at some point, because the underlying technology of the original ideal around cloud computing is still very intact, very valuable, is drastically disruptive, and marks a massive change in the technology industry.  Virualization, horizontal architectures (hardware & software, PaaS, IaaS, and SaaS are already and will continue to change the enterprise, startup, and worldwide business landscape.

Why am I rambling on about this?  What is my frustration?

When things are redefined that means we, the practitioners, programmers, and users of actual utility, virtualization, and other computing that we had defined as cloud computing will have to find new language to use in defining this revolutionary shift in capabilities.  The specifics that are now needed in each conversation becomes required, the understanding now desperately needs to increase.  Let me provide an example of why.

Imagine a CTO or a CIO that is working with his or her team to identify a way to utilize cloud computing to enhance their services, increase their uptime, and generally utilize this new technology within their enterprise in which they work.  Say they’ve been turned onto the idea of a private cloud.  Not a virtual private cloud that’s hosted in the public cloud (like AWS’s offering), but just a big huge box that sits in their data center.

  • Does it provide location independent computing?  Nope.
  • Does it provide auto-scaling capabilities like AWS, Azure, Rackspace, or others?  Nope.
  • Does it offer the assumes virtualization, service orientation, or other features?  Questionable.
  • Does it offer the resiliency of connectivity like a geographically dispersed utility computing service?  Nope.

But that CTO or CIO will be told over and over that it is a cloud, a private cloud, one of their very own that they can take and hold and hug and love.  But it isn’t cloud computing by the definition on Wikipedia.  It isn’t cloud computing by definition of those hard core developers, web application experts, and others that have been using cloud computing for years now.  It isn’t for those marketers that have been bringing the message of utility, virtualization, and other features.  It’s a misrepresentation by the sales people and media who just want to put stickers with little clouds on everything and shove it out the door to clients who will shell out big money for things that say cloud.

I don’t have much of a problem with that, I’m just perturbed that now I have to explain even more about what cloud computing really is versus what it isn’t.  Now every time I talk to someone new in the field – in addition to the fear & misinformation about security in the cloud (It’s secure already, so drop it – it’s more secure than your data center you can be sure), I now have to explain to people that Office 365 is a SaaS Service, that isn’t particularly in the cloud yet (last I checked it isn’t running on Windows Azure, but it will or should be in the coming future).  I have to explain that Compute & storage that is limited to one geographic area stuck in your data center is called a “computer”, or maybe at best referred to as “High Performance Computer”, but it doesn’t meet the cloud computing definition or the generally accepted understanding of what entail cloud computing.  The number of explanations to keep conversations honest have just drastically increased.

Thanks media, you’ve just made my job harder.  For those still learning in the industry, beware of the misinformation out there, it’s become rampant!

…got that out of my system.  Now on to actual development with utility computing, on virtualized images, within a highly distributed, node based, geographically dispersed computing network system.  <-  How do ya like those apples?  Specific enough?  🙂

Windows Azure Web, Worker, and CGI Roles – How They Work

This is a write up I’ve put together of how the roles in Windows Azure work.  As far as I know, this is all correct – but if there are any Windows Azure Team Members out there that wouldn’t mind providing some feedback about specifics or adding to the details I have here – please do add comments!  🙂

Windows 2008 and Hyper-V

Windows Azure is built on top of Windows 2008 & Hyper-V. Hyper-V provides virtualization to the various instance types and allocation of resources to those instances. Windows 2008 provides the core operating system functionality for those systems and the Windows Azure Platform Roles and Storage.

The hypervisor that a Hyper-V installation implements does a few unique things compared to many of the other virtualization offerings in the industry. Xen (The Open Source Virtualization Software that Amazon Web Services use) & VMWare both use a shared resource model for utilization of physical resources within a system. This allows for more virtualized instances to be started per physical machine, but can sometimes allow hardware contention. On the other hand Hyper-V pins a particular amount of resources to a virtualized instance, which decreases the number of instances allowed on a physical machine. This enables Hyper-V to prevent hardware contention though. Both designs have their plusses and minuses and in cloud computing these design choices are rarely evident. The context however is important to know when working with high end computing within the cloud.

Windows Azure Fabric Controller

The Windows Azure Fabric Controller is kind of the magic glue that holds all the pieces of Windows Azure together. The Azure Fabric Controller automates all of the load balancing, switches, networking, and other networking configuration. Usually within an IaaS environment you’d have to setup the load balancer, static IP address, internal DNS that would allow for connection and routing by the external DNS, the switch configurations, configuring the DMZ, and a host of other configuration & ongoing maintenance is needed. With the Windows Azure Platform and the Fabric Controller, all of that is taken care of entirely. Maintenance for these things goes to zero.

The Windows Azure Fabric Controller has several primary tasks: networking, hardware, and operating system management, service modeling, and life cycle management of systems.

The low level hardware that the Windows Azure Fabric Controller manages includes switches, load balancers, nodes, load balancers, and other network elements. In addition it manipulates the appropriate internal DNS and other routing needed for communication within the cloud so that each URI is accessed seamlessly from the outside.

The service modeling that the fabric controller provides is a to map the topology of services, port usage, and as mentioned before the internal communication within the cloud. All of this is done by the Fabric Controller without any interaction other than creating an instance or storage service within Windows Azure.

The operating system management from the Fabric Controller involves patching the operating system to assure that security, memory and storage, and other integral operating system features are maintained and optimized. This allows the operating system to maintain uptime and application performance characteristics that are optimal.

Finally the Fabric Controller has the responsibility for service life cycle. This includes updates and configuration changes for domains and fault domains. The Fabric Controller does so in a way to maintain uptime for the services.

Each role has at least one instance running. A role however can have multiple instances, with a theoretically limitless number. In this way, the Fabric Controller, if an instance stops responding is recycled and a new instance takes over. This can sometimes take several minutes, and is a core reason behind the 99.99% uptime SLA requiring two instances within a role to be running. In addition to this the instance that is recycled is rebuilt from scratch, thus destroying any data that would be stored on the role instance itself. This is when Windows Azure Storage plays a pivotal role in maintaining Windows Azure Cloud Applications.

Web Role

The Windows Azure Web Role is designed as a simply to deploy IIS web site or services hosting platform feature. The Windows Azure Web Role can provide hosting for any .NET related web site such as; ASP.NET, ASP.NET MVC, MonoRails, and more.

The Windows Azure Web Role is provides this service hosting with a minimal amount of maintenance required. No routing or load balancing setup is needed; everything is handled by the Windows Azure Fabric Controller.

Uses: Hosting ASP.NET, ASP.NET MVC, MonoRails, or other .NET related web site in a managed, high uptime, highly resilient, controlled environment.

Worker Role

A worker role can be used to host any number of things that need to pull, push, or run continuously without any particular input. A service role can be used to setup a schedule or other type of service. This provides a role dedicated to what could closely be compared to a Windows Service. The options and capabilities of a Worker Role however vastly exceed a simple Windows Service.

CGI Role

This service role is designed to allow execution of technology stacks such as Ruby on Rails, PHP, Java, and other non-Microsoft options.

Windows Azure Storage

Windows Azure Storage is broken into three distinct features within the service. Windows Azure provides tables, blob, and queue for storage needs. Any of the Windows Azure Roles can also connect to the storage to maintain data across service lifecycle reboots, refreshes, and any temporary loss of a Windows Azure Role.

A note about Windows Azure Storage compared to most Cloud Storage Providers: None of the Azure Storage Services are “eventually consistent”. When a write is done, it is instantly visible to all subsequent readers. This simplifies coding but slows down the data storage mechanisms more than eventually consistent data architectures.

Shout it

Windows Azure and the PaaS Context

PaaS stands for Platform as a Service.  The new concept around Devops* (Developer + Operations) has allowed cloud computing to reach an apex of agility for business.  For developers PaaS provides an ultimately clean and agile experience around staging and deployment.  PaaS is also the highest level of cost savings for most prospective enterprise and mid-size business users of the cloud computing services.  Windows Azure has positioned itself with the vast majority of its services as a platform.

Working with a platform, instead of an infrastructure based cloud computing service allows Devops to focus almost solely on the business problems.  In addition this prevents an unnecessary staffing level for IT in most organizations.  With staff re-focused on business problems and eliminating the majority of hardware issues in an organization costs go down while return on investment dramatically increases.

The Ideal PaaS Scenario, Athenaeum Corporation

Imagine a company, I’ll call it Athenaeum Corporation that has around 250 people and provides a web based on demand service.  Right now they have 4 geographically dispersed data centers that incur real estate, staffing, energy, and other costs.  In each of those geographically dispersed data centers there are network switches and dedicated web servers connected to clustered with failover databases.  Each set of clustered databases is setup to replicate among all the geographically disperse locations everyday on a near real-time basis.  The website that these locations host is then balanced by load balancers, which also require maintenance and administration.

The headquarters of this company is located away from the data centers, but has a smaller duplicate data center of its own that also receives replicated data and hosts the website.  This is for internal and development purposes.  The development team consists of approximately 45 people out of the 250 staff.  The network operations staff is about 25 people, with internal IT making up another 15 people.  Altogether the direct support of development and operations is 85 people out of a 250 person staff.

At the headquarters are approximately 280 machines ranging from desktop PCs to Laptops.  These machines are used to support operations, sales, accounting, support, and every other part of the company.  These 280 machines are connected to approximately 60 internal servers that provide things like Exchange Services, file sharing directories, communications on instant messengers, Sharepoint services, and other IT related tools.  In addition there are other switches, cabling, and other items related to the routing, load balancing, and usage of these internal services.

The Athenaeum Corporation that I’ve described is a perfect scenario that could literally save hundreds of thousands of dollars with cloud computing services.  While saving that money they could possibly increase their physical service, better their uptime & system processing performance, and more just by migrating to the Windows Azure Platform.

Before jumping into how a company like the Athenaeum Corporation might jump into PaaS with the Windows Azure Platform, let’s take a quick review of the services that the Windows Azure Platform provides.

The Platform of Windows Azure

The core Windows Azure Platform is made up of compute and storage.  The compute is broken up into Web, Worker, and CGI Roles.  The storage is broken up into Table, Blob, and Queue services.  All of these features have a platform SDK that can be used or RESTful Web Service APIs.  From the basis of an operating system, it is abstracted away and only the platform is of concern to development.

Beyond the core compute and storage elements the Windows Azure Platform cloud has the Windows Azure AppFabric and the SQL Azure Relational Database for service bus, security access control, and storage of highly structured data.  The AppFabric is made up of two core features; the access control and the service bus.  The SQL Azure is really just a clustered, high end instance of SQL Server running with a hot swappable backup that is managed by Microsoft in their data centers.

The Windows Azure AppFabric is one of the features of Windows Azure Platform that makes working with on-premises, internal, disparate, and Windows Azure Platform or other cloud services easy.  With the AppFabric access control security, claims based identification, and other authentication mechanisms may be used for seemless single sign-on experiences.  With the systems secured with the access control, the AppFabric service bus can then be used as a way to manage and keep communication between those disparate systems flowing and active.  The AppFabric Access Control & Services Bus provides a way to incorporate any request to incorporate systems that a business enterprise, government, or other entity may have.

With SQL Azure, a hosted, high end solution to relational data storage needs is provided.  One big concern is that the data sizes are to 50GB in storage.  Although the there is this 50GB limit, once this size has been attained the data most likely should not be contained solely in a relational data store.  This is when the other Windows Azure Storage mediums come into play.  But for data under 50 GB, a relational data store setup to work seamlessly in Windows Azure like this provides additional platform capabilities for developers to port traditionally hosted applications into the cloud with minimal changes.

Now that the platform is covered, how would the Athenaeum Corporation move their system & website operations into the Windows Azure Platform for increased capabilities and decreased costs?  The first thing needed is a breakdown of the individual systems and interoperations.

  1. Relational databases in each of the geographically dispersed data centers with failover databases.
  2. Headquarters has 280 PCs and Laptops.
  3. Headquarters has 60 internal IT maintained servers with custom applications, file-sharing, and other tools running on Windows Server.
  4. Load balancing is done for the web based on demand services in house.
  5. 4 Data Centers geographically dispersed with respective real estate, staffing, energy, and other costs.
  6. Network operations requires approximately 25 staff for 24-hour a day operational uptime.
  7. Web Based On Demand Services.

I’ll start breaking down these 7 key functionalities and state how the move to Windows Azure would change costing by using the platform.  Relational databases in each of these data centers can be moved in a couple different ways.

  1. One is to move the databases into one single primary SQL Azure instance.  Since the databases are most likely located at each of the datacenters for location CDN reasons, it made sense before, but with the move to the cloud the Windows Azure CDN could be used and the database would likely have better access to the geographically dispersed web presence points.
  2. The second is to move the databases to affinity points within the cloud that already match the current locations, porting the replication functionality for the specific data that each site needs.
  3. The 280 PCs and Laptops would still need connectivity and access to all of the existing applications they have now.  The cloud changes little in regard to this situation.  However redundant machines could be removed and with the implementation of SaaS based solutions, which I’ll discuss further in the next section, would dramatically decrease the cost of machines that each employee would need along with a decrease in support, administration, and maintenance of the software they currently use.
  4. The 60 internal servers at headquarters that IT maintains could be migrated completely, especially if they’re all running a Windows Operating System.  For anything that isn’t, one may want to look to AWS, Rackspace, or other virtualization solution at these cloud providers.  In Windows Azure internal servers hosting IIS applications could likely have them moved to Web or Worker Roles.  Anything such as Ruby on Rails, PHP, or Java that is hosted via IIS can be moved to a CGI Role in Windows Azure.  For anything that has other complexities and such can be installed on a Windows Azure VM Role.
  5. Current in house load balancing can be eliminated entirely.  There is no need for in house management of this with a PaaS like the Windows Azure Platform.  So mark this off the cost list, it is included in the cost of the service and requires no configuration, management, or other interaction.
  6. Each data center that previously provided geographic locations for the web presence can be brought into the Windows Azure Cloud.  There are two primary locations in North America at this time, and several more in other countries throughout the world.  With this ability the need to have 4 different data centers is removed.  In most cases, the centers that Windows Azure is located in also have significant security and penetration tests done at a physical level.  This effectively increases the security of each of the geographic access points.  Removing one more cost, while providing more for the money.
  7. Network operations, effectively simplified by the removal of routing, load balancing, and other concerns that needed to be done in house.  The cloud offers 24x7x365 operational uptime.  This eliminates the need for the in house staffing, with only a 4-6 staff needed for this particular scenario.  The roles and requirements for the 4-6 staffing positions would primarily be there to maintain data, assure that systems that are custom are maintained and operational within the Windows Azure Cloud.
  8. The last item is easily moved into the Windows Azure Platform using a Windows Azure Web Role.  This provides everything needed to operate a SaaS Web Application with the Windows Azure Portal PaaS.

On that last point of moving the Athenaeum Software into the Windows Azure Cloud, is SaaS on the Windows Azure Platform.