Free Windows Azure Images…

I created these, prospectively for use in presentations or whatever may call for something of this sort.  Since others may be talking about Windows Azure and the innards of the system also, I thought I’d contribute these for free use on the web.  I don’t even really care if you attribute me (even though it would be cool if you did).  I’ll be adding a few more over the next couple of weeks as I prepare for some pending presentations.  (click image to see full size)

Windows Azure
Windows Azure
Spline Architecture
Spline Architecture

Windows Azure Web, Worker, and CGI Roles – How They Work

This is a write up I’ve put together of how the roles in Windows Azure work.  As far as I know, this is all correct – but if there are any Windows Azure Team Members out there that wouldn’t mind providing some feedback about specifics or adding to the details I have here – please do add comments!  🙂

Windows 2008 and Hyper-V

Windows Azure is built on top of Windows 2008 & Hyper-V. Hyper-V provides virtualization to the various instance types and allocation of resources to those instances. Windows 2008 provides the core operating system functionality for those systems and the Windows Azure Platform Roles and Storage.

The hypervisor that a Hyper-V installation implements does a few unique things compared to many of the other virtualization offerings in the industry. Xen (The Open Source Virtualization Software that Amazon Web Services use) & VMWare both use a shared resource model for utilization of physical resources within a system. This allows for more virtualized instances to be started per physical machine, but can sometimes allow hardware contention. On the other hand Hyper-V pins a particular amount of resources to a virtualized instance, which decreases the number of instances allowed on a physical machine. This enables Hyper-V to prevent hardware contention though. Both designs have their plusses and minuses and in cloud computing these design choices are rarely evident. The context however is important to know when working with high end computing within the cloud.

Windows Azure Fabric Controller

The Windows Azure Fabric Controller is kind of the magic glue that holds all the pieces of Windows Azure together. The Azure Fabric Controller automates all of the load balancing, switches, networking, and other networking configuration. Usually within an IaaS environment you’d have to setup the load balancer, static IP address, internal DNS that would allow for connection and routing by the external DNS, the switch configurations, configuring the DMZ, and a host of other configuration & ongoing maintenance is needed. With the Windows Azure Platform and the Fabric Controller, all of that is taken care of entirely. Maintenance for these things goes to zero.

The Windows Azure Fabric Controller has several primary tasks: networking, hardware, and operating system management, service modeling, and life cycle management of systems.

The low level hardware that the Windows Azure Fabric Controller manages includes switches, load balancers, nodes, load balancers, and other network elements. In addition it manipulates the appropriate internal DNS and other routing needed for communication within the cloud so that each URI is accessed seamlessly from the outside.

The service modeling that the fabric controller provides is a to map the topology of services, port usage, and as mentioned before the internal communication within the cloud. All of this is done by the Fabric Controller without any interaction other than creating an instance or storage service within Windows Azure.

The operating system management from the Fabric Controller involves patching the operating system to assure that security, memory and storage, and other integral operating system features are maintained and optimized. This allows the operating system to maintain uptime and application performance characteristics that are optimal.

Finally the Fabric Controller has the responsibility for service life cycle. This includes updates and configuration changes for domains and fault domains. The Fabric Controller does so in a way to maintain uptime for the services.

Each role has at least one instance running. A role however can have multiple instances, with a theoretically limitless number. In this way, the Fabric Controller, if an instance stops responding is recycled and a new instance takes over. This can sometimes take several minutes, and is a core reason behind the 99.99% uptime SLA requiring two instances within a role to be running. In addition to this the instance that is recycled is rebuilt from scratch, thus destroying any data that would be stored on the role instance itself. This is when Windows Azure Storage plays a pivotal role in maintaining Windows Azure Cloud Applications.

Web Role

The Windows Azure Web Role is designed as a simply to deploy IIS web site or services hosting platform feature. The Windows Azure Web Role can provide hosting for any .NET related web site such as; ASP.NET, ASP.NET MVC, MonoRails, and more.

The Windows Azure Web Role is provides this service hosting with a minimal amount of maintenance required. No routing or load balancing setup is needed; everything is handled by the Windows Azure Fabric Controller.

Uses: Hosting ASP.NET, ASP.NET MVC, MonoRails, or other .NET related web site in a managed, high uptime, highly resilient, controlled environment.

Worker Role

A worker role can be used to host any number of things that need to pull, push, or run continuously without any particular input. A service role can be used to setup a schedule or other type of service. This provides a role dedicated to what could closely be compared to a Windows Service. The options and capabilities of a Worker Role however vastly exceed a simple Windows Service.

CGI Role

This service role is designed to allow execution of technology stacks such as Ruby on Rails, PHP, Java, and other non-Microsoft options.

Windows Azure Storage

Windows Azure Storage is broken into three distinct features within the service. Windows Azure provides tables, blob, and queue for storage needs. Any of the Windows Azure Roles can also connect to the storage to maintain data across service lifecycle reboots, refreshes, and any temporary loss of a Windows Azure Role.

A note about Windows Azure Storage compared to most Cloud Storage Providers: None of the Azure Storage Services are “eventually consistent”. When a write is done, it is instantly visible to all subsequent readers. This simplifies coding but slows down the data storage mechanisms more than eventually consistent data architectures.

Shout it

Cloud Software Architect Necessities

Cloud Architecture is becoming more and more relevant in the software industry today.  A lot of efforts are becoming less about software and more about cloud software.  The whole gamut of cloud technology; platform, service, infrastructure, or platforms on platforms are growing rapidly in number.  The days you didn’t need to know what the cloud was are rapidly coming to an end.

As I move further along in my efforts with cloud development, both at application level and services levels, I’ve come to a few conclusions about what I do and do not need.  The following points are what I have recently drawn conclusions about, that cause frustration or are drawn from the beauty of cloud architecture.

REST is King, Period

When I use REST in this context, I don’t just mean a nice clean URI or “not SOAP”.  I’m talking about the whole entire enchilada of REST Architecture Principles.  A good place to start learning about REST is the Wikipedia Article.  Other key resources to check out include;  Roy T. Fielding’s Dissertation, specifically chapter 5, Principled Design of the Modern Web Architecture, and the O’Reilly Book RESTful Web Services.

REST Architecture is fundamental to the web and the fact that the web is continuous in uptime.  The web, or Internet, doesn’t go down, doesn’t crash, and is always available in some way.  REST Architecture and the principles around it are what enables the web to be this way.  The cloud seeks to have the same abilities, functionality, and basically be always on, thus REST Architecture is key to that underpinning.

Core Development

Currently I do most of my development with C# using the .NET Framework.  This is great for developing in cloud environments in Windows Azure and Amazon Web Services.  The .NET Framework has a lot of libraries that work around, with, and attempt to provide good RESTful Architecture.  However, there are also a lot of issues, such as the fact that WCF and ASP.NET at their core aren’t built with good intent against RESTful Architecture.  ASP.NET MVC and some of the latest WCF Releases (the out of band stuff) are extensively cleaning this up and I hear that this may be sooner than later.  I look forward to have cleaner, bare bones, fast implementations to use for RESTful development in Azure or AWS.

The current standing ASP.NET (Web Forms) Architecture is built well above the level it should be to utilize REST Architecture well.  Thus it creates a lot of overhead and unnecessary hardware utilization, especially for big sites that want to follow good RESTful Practice.

WCF on the other hand had some great ideas behind it, but as REST increased in popularity and demanded better use of the core principles the web is founded on, the WCF model has had to bend and give way some of its functional context in order to meet basic REST Architecture.

Anyway, that’s enough for my ramblings right now.  Just wanted to get some clear initial thoughts written down around Cloud Architecture needs for the software developer.

Aggregated Web Services Pt II – Tying it Together

I’ve been working through a project recently that I ended up creating an interesting abstraction of an assembly/classes between multiple web services projects.  I wrote about it initially in Aggregated Web Services Pt I.  In this blog entry is going to cover a few things, based on a full end-to-end implementation of a project from the WCF RESTful Web Services, to the ASP.NET MVC Site, and finally the jQuery calling those same services.

Simple Architecture
Simple Architecture

Before I got started, there is one thing I need to point out.  The communication with Javascript/jQuery/AJAX has a lot of tricky bits one needs to be aware of.  One of those is the same site origin and of course the famous cross domain solution issues.  That is why in this walk through I will place the web services and the site pages in the same project, yes, WCF and MVC living happily in a single project.  🙂  You can of course, if your architecture requires it, break these into separate projects, but for this example I’ll have both in the same project.

First create a new solution.  I always like to start with a new solution because it keeps the naming structured right, just from the practice.

(Click on any of the images to see a larger full size copy)

New Solution
New Solution

Once you create all of that then add an ASP.NET MVC 2 Project.

Adding the ASP.NET MVC 2 Project
Adding the ASP.NET MVC 2 Project

You can create an ASP.NET MVC 2 Unit Test Project if you want to, but I’m skipping it for now.  (yes, I’m still a big TDD advocate, but just setting up a prototype for this example)

Next I wiped out some files I don’t use, and suggest you zap em’ too.

Get Rid of the Nasty MS AJAX
Get Rid of the Nasty MS AJAX

Setting up the WCF Parts

Now that we’ve cleaned up those nasty bits, we’ll add our basic model we’ll be using.

[sourcecode language=”csharp”]
using System;

namespace EndToEnd.Mvc.Models
{
public class Investor
{
public string Id { get; set; }
public string Text { get; set; }
public decimal Money { get; set; }
public DateTime Stamp { get; set; }
}
}
[/sourcecode]

Now add a interface for the RESTful services to the root of the MVC Project as shown below.

[sourcecode language=”csharp”]
using System.Collections.Generic;
using System.ServiceModel;
using System.ServiceModel.Web;
using EndToEnd.Mvc.Models;

namespace EndToEnd.Mvc
{
[ServiceContract]
public interface IEndToEndService
{
[OperationContract]
[WebGet(UriTemplate = "Investors/{pageStart}/{pageEnd}", ResponseFormat = WebMessageFormat.Json)]
List<Investor> GetIncidents( string pageStart, string pageEnd);

[OperationContract]
[WebGet(UriTemplate = "Investor/", ResponseFormat = WebMessageFormat.Json)]
Investor GetInvestor();
}
}
[/sourcecode]

Now add the following abstract base class at the root level also.

[sourcecode language=”csharp”]
using System;
using System.Collections.Generic;
using EndToEnd.Mvc.Models;

namespace EndToEnd.Mvc
{
public abstract class InvestorBase : IEndToEndService
{
#region IEndToEndService Members

public List<Investor> GetIncidents(string pageStart, string pageEnd)
{
return new List<Investor>
{
new Investor
{
Id = Guid.NewGuid().ToString(),
Money = (decimal) (DateTime.Now.Second*2.27),
Stamp = DateTime.Now,
Text = "Lorum ipsum 1"
},
new Investor
{
Id = Guid.NewGuid().ToString(),
Money = (decimal) (DateTime.Now.Second*1.32),
Stamp = DateTime.Now,
Text = "Lorum ipsum 2"
}
};
}

public Investor GetInvestor()
{
return new Investor
{
Id = Guid.NewGuid().ToString(),
Money = (decimal) (DateTime.Now.Second*1.27),
Stamp = DateTime.Now,
Text = "Lorum ipsum"
};
}

#endregion
}
}
[/sourcecode]

Now add a WCF Service file and remove the interface file.  Then change the WCF class itself as shown below.  The reasons for the abstract class, inheriting from the interface, is that it removes any manipulation being needed with the actual *.svc file.  It just seems, at least to me, a little bit cleaner this way.

[sourcecode language=”csharp”]
namespace EndToEnd.Mvc
{
public class EndToEndService : InvestorBase
{}
}
[/sourcecode]

For the last touches for the WCF RESTful Service we need to setup the Web.Config file.  I’ve added the section toward the bottom of the config file in the <System.ServiceModel> section.  I’ve included the full config file below, so you can easily just copy and paste it if you’re working through step by step with me.

[sourcecode language=”html”]
<?xml version="1.0"?>
<configuration>
<system.web>
<compilation debug="true" targetFramework="4.0">
<assemblies>
<add assembly="System.Web.Abstractions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Web.Routing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Web.Mvc, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
</assemblies>
</compilation>
<authentication mode="Forms">
<forms loginUrl="~/Account/LogOn" timeout="2880" />
</authentication>
<pages>
<namespaces>
<add namespace="System.Web.Mvc" />
<add namespace="System.Web.Mvc.Ajax" />
<add namespace="System.Web.Mvc.Html" />
<add namespace="System.Web.Routing" />
</namespaces>
</pages>
</system.web>
<system.webServer>
<validation validateIntegratedModeConfiguration="false"/>
<modules runAllManagedModulesForAllRequests="true"/>
</system.webServer>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="System.Web.Mvc" publicKeyToken="31bf3856ad364e35" />
<bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0" />
</dependentAssembly>
</assemblyBinding>
</runtime>

<system.serviceModel>
<behaviors>
<endpointBehaviors>
<behavior name="httpBehavior">
<webHttp />
</behavior>
</endpointBehaviors>
<serviceBehaviors>
<behavior name="ServicesBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
</behavior>
<behavior name="">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="false" />
</behavior>
</serviceBehaviors>
</behaviors>
<serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
<services>
<service behaviorConfiguration="ServicesBehavior"
name="EndToEnd.Mvc.EndToEndService">
<endpoint address="" behaviorConfiguration="httpBehavior" binding="webHttpBinding"
contract="EndToEnd.Mvc.IEndToEndService" />
</service>
</services>
</system.serviceModel>

</configuration>
[/sourcecode]

One of the things I always do at this point is to setup the project properties.  I do this for a number of reasons, primarily to assure that the port number doesn’t go and change itself on me.  The other thing I set is the default startup page.  With ASP.NET MVC things get out of sync with Visual Studio, and Visual Studio tries to startup actual *.aspx files.  So what I do is just set the startup to an empty root startup.  These settings are shown below.

Project Properties
Project Properties

Setting up the MVC Parts

First add a home directory, a HomeController.cs, and then add a Core.Master master page to the project.

MVC Project Parts
MVC Project Parts

Next setup the Core.Master file with the following content sections.

[sourcecode language=”html”]
<%@ Master Language="C#" Inherits="System.Web.Mvc.ViewMasterPage" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt;

<html xmlns="http://www.w3.org/1999/xhtml&quot; >
<head runat="server">
<title><asp:ContentPlaceHolder ID="TitleContent" runat="server" /></title>

<script type="text/javascript" src="../../Scripts/jquery-1.4.1.js"></script>

<asp:ContentPlaceHolder ID=HeaderContent runat=server>
</asp:ContentPlaceHolder>

</head>
<body>
<div>
<asp:ContentPlaceHolder ID="MainContent" runat="server">
</asp:ContentPlaceHolder>
</div>
</body>
</html>
[/sourcecode]

One of the things I’ll point out is that with Visual Studio 2010 you get Intellisense with jQuery.  The reason I don’t use the x.x.min.js version of the jQuery is that it doesn’t have the appropriate setup to provide the Intellisense.  So be sure for development to use the fully expanded version and you can go to the zipped min version when you go live in production.  Another thing I do, which may vary on how you want to develop, is use the hosted jQuery on Google or Microsoft.  I did a write up previously for using the hosted jQuery so check it out for reference locations.

In the controller add the following code.

[sourcecode language=”csharp”]
using System.Web.Mvc;

namespace EndToEnd.Mvc.Controllers
{
public class HomeController : Controller
{
public ActionResult Index()
{
return View();
}
}
}
[/sourcecode]

Now that we have the Site.Master and the home controller, create an Index.aspx View in the Home folder of the project.

Adding the Index.aspx View
Adding the Index.aspx View

In the view add the following code for the jQuery calls to the services layer.

[sourcecode language=”html”]
<%@ Page Title="" Language="C#" MasterPageFile="~/Views/Shared/Core.Master" Inherits="System.Web.Mvc.ViewPage<dynamic>" %>

<asp:Content ID="Content1" ContentPlaceHolderID="TitleContent" runat="server">
jQuery AJAX Calls to RESTful WCF Web Services
</asp:Content>
<asp:Content ID="Content3" ContentPlaceHolderID="HeaderContent" runat="server">
<script type="text/javascript">
var domainRoot = "http://localhost:1000/EndToEndService.svc/&quot;;
var investorUri = domainRoot + "investor/";
var investorsUri = domainRoot + "investors/10/15";
</script>
</asp:Content>
<asp:Content ID="Content2" ContentPlaceHolderID="MainContent" runat="server">
<h2>
jQuery AJAX Calls to RESTful WCF Web Services
</h2>

<div id="InvestorUri">
</div>
<div id="InvestorResult">
</div>

<hr />

<div id="InvestorsUri">
</div>
<div id="InvestorsResult">
</div>

<script type="text/javascript">

$(‘#InvestorUri’).html(investorUri);
$(‘#InvestorsUri’).html(investorsUri);

$.getJSON(investorUri, function investor_complete(json) {
$(‘#InvestorResult’).html(‘<li>’ + json.Id + ‘</li>’ + ‘<li>’ + json.Text + ‘</li>’);
});

$.getJSON(investorsUri, function investors_complete(json) {
var builtHtml = ”;

$.each(json, function (x, y) {
builtHtml += ‘<li>’ + json[x].Id + ‘</li>’ + ‘<li>’ + json[x].Text + ‘</li>’;
});

$(‘#InvestorsResult’).html(builtHtml);
});

</script>
</asp:Content>

[/sourcecode]

Run it and you should get the following displayed on screen.

Browser Results
Browser Results

Let me know if you run into any issues trying this out.  Thanks!

Shout it

Aggregated Web Services Pt I

I’ve been working through an architecture scenario recently.  This is what I have so far.  Multiple external web services, some SOAP and some REST, and some data sources in a SQL Server Database, Azure Table Storage, and flat files of some sort.  All of these sources need to be accessed by a web site for read-only display.  In the diagram below I’ve drawn out the primary three points of reference.

  1. The services that are external; Contract, Table Store, Document, Search, and Help Desk Services.
  2. The Website Web Services Facade, which would be an aggregated layer that then provides the various services via an internally controlled services layer.
  3. On top of that will be the web site, accessing the services from the aggregated layer with jQuery.
base three tiers
Basic Three Tiers

After creating this to get some basic idea of how these things should fit together, I moved on to elaborate on the web services aggregation layer.  What I’ve sketched in this diagram is the correlation to architectural elements and the physical environments they would prospectively be deployed to.  Again, broken out by the three tiers as shown above.

  1. Website and the respective jQuery, AJAX, and Market/CSS for display.
  2. Web Services, which include the actual architecture breakout;  Facade Interface, Facade Aggregation Component, Cache & Non-cached DTOs (Data Transfer Objects), Cache Database/Storage, Caching Process, Lower Layer Aggregation Component, and the Poller Process for polling the external services.
  3. The cache is intended to use SQL Server, thus the red call out to the physical SQL Server cluster.
  4. The last tier, which isn’t being developed, but just providing data is the External Services, primarily shown to provide a full picture of all the layers.
aggregate web services
Aggregate Web Services

I primarily drew up these diagrams for discussion of the architecture, poke holes in it, or otherwise. Which speaking of, if any readers have input, question, or are curious please type up a comment and I’ll answer it ASAP.

As the effort continues there are some other great how-to write ups I will be putting together.  Everything from unit testing, mocking (with moq), how to setup test services, test services, and other elements of the project.  I’ll have all this coming, so keep reading & let me know what you think of the design so far, subscribe via e-mail (look to the metadata section below), or grab the RSS for the blog (see below also).

kick it on DotNetKicks.com

Shout it