Put Stuff in Your Windows Azure Junk Trunk – Repository Base

Alright, so the title is rather stupid, but hey, it’s fun! 🙂

This project I setup to provide some basic functionality with Windows Azure Storage. I wanted to use each of the three mediums; Table, Blob, and Queue, and this example will cover each of these things. The application will upload and store images, provide a listing, some worker processing, and deletion of the images & associated metadata. This entry is part 1 of this series, with the following schedule for subsequent entries:

Title aside, schedule laid out, description of the project completed, I’ll dive right in!

Putting Stuff in Your Junk Trunk

Create a new Windows Azure Project called PutJunkInIt. (Click any screenshot for the full size, and also note some of the text may be off – I had to recreate a number of these images)

Next select the ASP.NET MVC 2 Web Application and also a Worker Role and name the projects JunkTrunk and JunkTrunk.WorkerRole.

In the next dialog choose to create the unit test project and click OK.

After the project is created the following projects are setup within the PutJunkInIt Solution. There should be a JunkTrunk, JunkTrunk.Worker, JunkTrunk Windows Azure Deployment Project, and a JunkTrunk.Tests Project.

Next add a Windows Class Library Project and title it JunkTrunk.Storage.

Add a reference to the Microsoft.WindowsAzure.ServiceRuntime and Microsoft.WindowsAzure.StorageClient assemblies to the JunkTrunk.Storage Project. Rename the Class1.cs file and class to JunkTrunkBase. Now open up the Class1.cs file in the JunkTrunk.Storage Project. First add the following fields and constructor to the class.

[sourcecode language=”csharp”]
public const string QueueName = "metadataqueue";
public const string BlobContainerName = "photos";
public const string TableName = "MetaData";
static JunkTrunkBase()
{
CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
{
configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
RoleEnvironment.Changed
+= (sender, arg) =>
{
if (!arg.Changes.OfType()
.Any(change => (change.ConfigurationSettingName == configName)))
return;
if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
{
RoleEnvironment.RequestRecycle();
}
};
});
}
[/sourcecode]

After that add the following blob container and reference methods.

[sourcecode language=”csharp”]
protected static CloudBlobContainer Blob
{
get { return BlobClient.GetContainerReference(BlobContainerName); }
}
private static CloudBlobClient BlobClient
{
get
{
return Account.CreateCloudBlobClient();
}
}
[/sourcecode]

Now add code for the table & queue client and reference methods.

[sourcecode language=”csharp”]
protected static CloudQueue Queue
{
get { return QueueClient.GetQueueReference(QueueName); }
}
private static CloudQueueClient QueueClient
{
get { return Account.CreateCloudQueueClient(); }
}
protected static CloudTableClient Table
{
get { return Account.CreateCloudTableClient(); }
}
protected static CloudStorageAccount Account
{
get
{
return
CloudStorageAccount
.FromConfigurationSetting("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
}
}
[/sourcecode]

This class now provides the basic underpinnings needed to retrieve the appropriate information from the configuration. This base class can then provide that connection information to connect to the table, queue, or blob mediums.

Next step is to create some initialization code to get the containers created if they don’t exist in Windows Azure. Add a new class file to the PutJunkInIt Project.

[sourcecode language=”csharp”]
public class JunkTrunkSetup : JunkTrunkBase
{
public static void CreateContainersQueuesTables()
{
Blob.CreateIfNotExist();
Queue.CreateIfNotExist();
Table.CreateTableIfNotExist(TableName);
}
}
[/sourcecode]

Next add the System.Data.Services.Client Assembly to the project. After adding the assembly add two new classes and name them BlobMeta.cs and Table.cs. Add the following code to the Table.cs Class.

[sourcecode language=”csharp”]
public class Table
{
public static string PartitionKey;
}
[/sourcecode]

Next add another class file and name it BlobMetaContext.cs and add the following code.

[sourcecode language=”csharp”]
public class BlobMetaContext : TableServiceContext
{
public BlobMetaContext(string baseAddress, StorageCredentials credentials)
: base(baseAddress, credentials)
{
IgnoreResourceNotFoundException = true;
}
public IQueryable Data
{
get { return CreateQuery(RepositoryBase.TableName); }
}
public void Add(BlobMeta data)
{
data.RowKey = data.RowKey.Replace("/", "_");
BlobMeta original = (from e in Data
where e.RowKey == data.RowKey
&& e.PartitionKey == Table.PartitionKey
select e).FirstOrDefault();
if (original != null)
{
Update(original, data);
}
else
{
AddObject(RepositoryBase.TableName, data);
}
SaveChanges();
}
public void Update(BlobMeta original, BlobMeta data)
{
original.Date = data.Date;
original.ResourceUri = data.ResourceUri;
UpdateObject(original);
SaveChanges();
}
}
[/sourcecode]

Now add the following code to the BlobMeta Class.

[sourcecode language=”csharp”]
public class BlobMeta : TableServiceEntity
{
public BlobMeta()
{
PartitionKey = Table.PartitionKey;
}
public DateTime Date { get; set; }
public string ResourceUri { get; set; }
}
[/sourcecode]

At this point, everything should build. Give it a go to be sure nothing got keyed in wrong (or copied in wrong). Once assured the build is still solid, add the Blob.cs Class to the project.

[sourcecode language=”csharp”]
public class Blob : JunkTrunkBase
{
public static string PutBlob(Stream stream, string fileName)
{
var blobRef = Blob.GetBlobReference(fileName);
blobRef.UploadFromStream(stream);
return blobRef.Uri.ToString();
}
public static Stream GetBlob(string blobAddress)
{
var stream = new MemoryStream();
Blob.GetBlobReference(blobAddress)
.DownloadToStream(stream);
return stream;
}
public static Dictionary<string, string> GetBlobList()
{
var blobs = Blob.ListBlobs();
var blobDictionary =
blobs.ToDictionary(
listBlobItem => listBlobItem.Uri.ToString(),
listBlobItem => listBlobItem.Uri.ToString());
return blobDictionary;
}
public static void DeleteBlob(string blobAddress)
{
Blob.GetBlobReference(blobAddress).DeleteIfExists();
}
}
[/sourcecode]

After that finalize the Table Class with the following changes and additions.

[sourcecode language=”csharp”]
public class Table : RepositoryBase
{
public const string PartitionKey = "BlobMeta";
public static void Add(BlobMeta data)
{
Context.Add(data);
}
public static BlobMeta GetMetaData(Guid key)
{
return (from e in Context.Data
where e.RowKey == key.ToString() &&
e.PartitionKey == PartitionKey
select e).SingleOrDefault();
}
public static void DeleteMetaDataAndBlob(Guid key)
{
var ctxt = new BlobMetaContext(
Account.TableEndpoint.AbsoluteUri,
Account.Credentials);
var entity = (from e in ctxt.Data
where e.RowKey == key.ToString() &&
e.PartitionKey == PartitionKey
select e).SingleOrDefault();
ctxt.DeleteObject(entity);
Repository.Blob.DeleteBlob(entity.ResourceUri);
ctxt.SaveChanges();
}
public static List<BlobMeta> GetAll()
{
return (from e in Context.Data
select e).ToList();
}
public static BlobMetaContext Context
{
get
{
return new BlobMetaContext(
Account.TableEndpoint.AbsoluteUri,
Account.Credentials);
}
}
}
[/sourcecode]

The final file to add is the Queue.cs Class File. Add that and then add the following code to the class.

[sourcecode language=”csharp”]
public class Queue : JunkTrunkBase
{
public static void Add(CloudQueueMessage msg)
{
Queue.AddMessage(msg);
}
public static CloudQueueMessage GetNextMessage()
{
return Queue.PeekMessage() != null ? Queue.GetMessage() : null;
}
public static List<CloudQueueMessage> GetAllMessages()
{
var count = Queue.RetrieveApproximateMessageCount();
return Queue.GetMessages(count).ToList();
}
public static void DeleteMessage(CloudQueueMessage msg)
{
Queue.DeleteMessage(msg);
}
}
[/sourcecode]

The now gives us a fully functional class that utilizes the Windows Azure SDK. In Part 2 I’ll start building on top of that using the ASP.NET MVC 2 Web Project. Part 2 will be published tomorrow, so stay tuned.

Cuttin’ Teeth With NuGet

If you’re in the .NET Dev space you might have heard about the release of nuget. This is an absolutely great tool for pulling in dependencies within a Visual Studio 2010 Project. (It can also help a lot in Visual Studio 2008, and maybe even earlier versions)

Instead of writing up another tutorial I decided I’d put together specific links for putting together packages, installing packages, and other activities around using this tool. In order of importance in getting started, I’ve itemized the list below.

The first site to check out is the Nuget Codeplex Site.
Check out the Nuget Gallery.
Then check out the getting start page.

Once you’ve checked out all those sites & got rolling with nuget, be sure to check out some of the blogs from the guys that have put time in on the development and ongoing awesomeness of it!

David Ebbo – Blog – @davidebbo
Phil Haack – Haacked – @haack

That’s all I’ve got for now, check out the nuget, if you code against the .NET stack you owe it to yourself!

Also some of my cohorts have put together a few pieces of information related to nuget and getting it tied together a bit like Ruby on Rails Gems:

Gritty Technical Info on Windows Azure Worker Roles

In the last blog entry, “Gritty Technical Info on Windows Azure Web Roles“, I covered the creation and startup of a web role within the Windows Azure Development Fabric and observing the web role with the Windows Azure Compute Emulator. In this blog entry I’ll cover the worker role.

Open the Windows Azure Web Role Sample Solution. Right click on the Windows Azure and select New Worker Role Project.

Once the worker role project SampleWorkerRole is added the solution explorer will display the project just like the web role, albeit fewer files.

Next right click on the SampleWorkerRole instance in the Windows Azure Web Role Sample and select properties. Now set the instance count to 2 and the VM size to extra large.

Click on F5 to run the application. Now when the application executes the 6 web role instances will start and the 2 worker role instances will start.

Examine the first worker role instance.

The worker role instance displays a number of new diagnostic messages in a similar way to the web role. The first half of the trace diagnostics are configuration and instance messages. The second half of the trace diagnostics are status messages that are printed from the worker role running.

Open up the code in the WorkerRole.cs file in the SampleWorkerRole Project. As a comparison open the WebRole.cs file in the SampleWebRole Project.

[sourcecode language=”csharp”]
using System.Diagnostics;
using System.Net;
using System.Threading;
using Microsoft.WindowsAzure.ServiceRuntime;

namespace SampleWorkerRole
{
public class WorkerRole : RoleEntryPoint
{
public override void Run()
{
Trace.WriteLine("SampleWorkerRole entry point called", "Information");

while (true)
{
Thread.Sleep(10000);
Trace.WriteLine("Working", "Information");
}
}

public override bool OnStart()
{
ServicePointManager.DefaultConnectionLimit = 12;
return base.OnStart();
}
}
}
[/sourcecode]

In the WorkerRole.cs file the code inherites from the RoleEntryPoint for the WorkerRole. In the WorkerRole Class the Run and OnStart Methods are overridden to provide some basic trace information and set the default connection limit.

The Run method has a basic while loop that updates every 10000 milliseconds, which displays on the Windows Azure Compute Emulator as “Information: Working”.

[sourcecode language=”csharp”]
using Microsoft.WindowsAzure.ServiceRuntime;

namespace SampleWebRole
{
public class WebRole : RoleEntryPoint
{
public override bool OnStart()
{
return base.OnStart();
}
}
}
[/sourcecode]

In the code for the WebRole.cs file there is very little actually going on. Take a closer look at the OnStart method override. Technically this code doesn’t even need to be in the generated file and can be deleted, but provides a good starting point to add any other code needed in the start of the web role.

Next I’ll add some code in the worker role to provide a telnet prompt that responds with worker role information. To work through this exercise completely download a telnet client like Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/).

If Visual Studio 2010 is no longer open, launch it and open the Windows Azure Web Role Sample Solution. Right click on the SampleWorkRole Role in the Windows Azure Web Role Sample Project. Click on the Endpoints tab of the properties window and click on Add Endpoint and call it TelnetServiceEndpoint.

Add a private member and create a run method with the following code.

[sourcecode language=”csharp”]
private readonly AutoResetEvent _connectionWait = new AutoResetEvent(false);

public override void Run()
{
Trace.WriteLine("Starting Telnet Service…", "Information");

TcpListener listener;
try
{
listener = new TcpListener(
RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["TelnetServiceEndpoint"].IPEndpoint) { ExclusiveAddressUse = false };
listener.Start();

Trace.WriteLine("Started Telnet Service.", "Information");
}
catch (SocketException se)
{
Trace.Write("Telnet Service could not start.", "Error");
return;
}

while (true)
{
listener.BeginAcceptTcpClient(HandleAsyncConnection, listener);
_connectionWait.WaitOne();
}
}
[/sourcecode]

After adding this code, add the following code for the role information to write to a stream.

[sourcecode language=”csharp”]
private static void WriteRoleInformation(Guid clientId, StreamWriter writer)
{
writer.WriteLine("— Current Client ID, Date & Time —-");
writer.WriteLine("Current date: " + DateTime.Now.ToLongDateString() + " " + DateTime.Now.ToLongTimeString());
writer.WriteLine("Connection ID: " + clientId);
writer.WriteLine();

writer.WriteLine("— Current Role Instance Information —-");
writer.WriteLine("Role ID: " + RoleEnvironment.CurrentRoleInstance.Id);
writer.WriteLine("Role Count: " + RoleEnvironment.Roles.Count);
writer.WriteLine("Deployment ID: " + RoleEnvironment.DeploymentId);
writer.WriteLine();

writer.WriteLine("— Instance Endpoints —-");

foreach (KeyValuePair<string, RoleInstanceEndpoint> instanceEndpoint in RoleEnvironment.CurrentRoleInstance.InstanceEndpoints)
{
writer.WriteLine("Instance Endpoint Key: " + instanceEndpoint.Key);

RoleInstanceEndpoint roleInstanceEndpoint = instanceEndpoint.Value;

writer.WriteLine("Instance Endpoint IP: " + roleInstanceEndpoint.IPEndpoint);
writer.WriteLine("Instance Endpoint Protocol: " + roleInstanceEndpoint.Protocol);
writer.WriteLine("Instance Endpoint Type: " + roleInstanceEndpoint);
writer.WriteLine();
}
}
[/sourcecode]

Now add a handle method for the asynchronous call.

[sourcecode language=”csharp”]
private void HandleAsyncConnection(IAsyncResult result)
{
var listener = (TcpListener)result.AsyncState;
var client = listener.EndAcceptTcpClient(result);
_connectionWait.Set();

var clientId = Guid.NewGuid();
Trace.WriteLine("Connection ID: " + clientId, "Information");

var netStream = client.GetStream();
var reader = new StreamReader(netStream);
var writer = new StreamWriter(netStream);
writer.AutoFlush = true;

var input = string.Empty;
while (input != "3")
{
writer.WriteLine(" 1) Display Worker Role Information");
writer.WriteLine(" 2) Recycle");
writer.WriteLine(" 3) Quit");
writer.Write("Enter your choice: ");

input = reader.ReadLine();
writer.WriteLine();

switch (input)
{
case "1":
WriteRoleInformation(clientId, writer);
break;
case "2":
RoleEnvironment.RequestRecycle();
break;
}

writer.WriteLine();
}

client.Close();
}
[/sourcecode]

Finally override the OnStart() method and setup the RoleEnvironmentChanging Event.

[sourcecode language=”csharp”]
public override bool OnStart()
{
ServicePointManager.DefaultConnectionLimit = 12;

DiagnosticMonitor.Start("DiagnosticsConnectionString");

RoleEnvironment.Changing += RoleEnvironmentChanging;

return base.OnStart();
}

private static void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
{
if (e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange))
{
e.Cancel = true;
}
}
[/sourcecode]

Now run the role by hitting F5. When the application runs open up the Windows Azure Compute Emulator to check for the end point and verify the instances of the role are running.

Endpoint displayed in the Windows Azure Compute Emulator

The Service Name should be Windows_Azure_Web_Role, with an Interface Type of SampleWorkerRole running on the tcp://*:1234 URL and the IP is 127.0.0.1P:1234.

Click on one of the instances, which should be green, and assure that each has started up appropriately.

Startup a telnet application, such as Putty and enter the information in as shown in screenshot above.

Start the telnet prompt connecting to the Windows Azure Worker Role. The prompt with the three choices will display. Click to recycle and then display worker role information a few times, just to make sure all the information is available and the worker role telnet application service is working. Select 3 to exit out and the prompt should close while the role continues to run on the development fabric.

Gritty Technical Info on Windows Azure Web Roles

This is a follow up to the previous blog entry I wrote pertaining to Windows Azure Roles. I wanted to cover the bases on the various technical aspects of creating a Windows Azure Web Role & Worker Role in Visual Studio 2010. Without interruption let’s just dive right in. Start Visual Studio 2010 and initiate a new project. File, new, and then project will open the new project dialog.

Select a cloud template type and name your project. Click OK and the New Windows Azure Project Dialog will appear to select the role types you can choose from.

Select an ASP.NET MVC Web Application, name it appropriately, and then click OK. When prompted for a test project select yes and click OK. When the solution is finished generating from the chosen templates there will be a SampleWebRole ASP.NET MVC Web Application, the test project titled SampleWebRole.Tests, and a Windows Azure Project titled Windows Azure Web Role Sample.

After that run the application to assure that the Development Fabric & other parts of the web application startup appropriately.

With the web application still running, click on the Development Fabric Icon in the status bar of Windows 7 and select the Show Computer Emulator UI.

The Windows Azure Compute Emulator will display. Click on the Service Deployments tree until you can see each individual instance (the little green lights should be showing). Figure 4.6 shows this tree opened with one of the instances selected to view the status trace.

Select Shift + F5 to stop the web application from running. In the Solution Explorer right click on the SampleWebRole under the Windows Azure Web Role Sample Project and select Properties.

Under the configuration tab of the SampleWebRole Properties set the Instance Count to 6 and the VM Size to Extra Large.

Now select F5 to run the web application again in the Windows Azure Development Fabric. The Windows Azure Compute Emulator (if it is closed right click back on the status icon to launch it again) will now display each of the 6 instances launching under the SampleWebRole.

Click on one of the green lights to show that specific instance status in the primary window area.

When you select the specific instance the status of that instance is displayed. The instance that is displayed in figure 4.10 has a number of events being recorded with the diagnostics, MonAgentHost, and the runtime. This particular instance had gone through a rough start. During the lifecycle of a Windows Azure Web, Worker, or CGI Role there are a number of events similar to these that can occur.

Read through the first few lines. These lines show that another agent was running, which could be a number of things that conflicted with this web role starting up cleanly. Eventually the web role was able to startup appropriately as shown in the runtime lines stating that the OnStart() is called and then complete, with the Run() executing next.

Reading further through the diagnostics the web role eventually requests a shutdown and then prepares for that shutdown pending the exit of the parent process 6924.

These types of events are common place when reviewing the actions a web role will go through; generally, don’t get too alarmed by any particular set of messages. As long as the role has green lights on the instances, things are going swimmingly. When the lights change to purple or red then it is important to really start paying attention to the diagnostics.

Windows Azure Worker Roles

In the next blog entry (Part II) I want to show is how to add a worker role and how to analyze the activities within the role. The worker role is somewhat different than a web role. The primary difference between a web role and a worker role is that one is built around providing compute work, while one is built around providing web compute. Think of the worker role as something similar to a Windows Service, which runs ongoing to execute jobs & other processes, often backend type processes. A web role is what is built to host Silverlight and web applications such as ASP.NET or ASP.NET MVC.

Part II published on Monday the 17th.

Git Updates

Here’s my latest Git Ignore file for .NET Development.

#Begin .ignore file
#OS junk files
[Tt]humbs.db
*.DS_Store

#Visual Studio files
*.[Oo]bj
*.dat
*.crc
*.exe
*.pdb
*.user
*.aps
*.pch
*.vspscc
*.vssscc
*_i.c
*_p.c
*.ncb
*.suo
*.tlb
*.tlh
*.bak
*.[Cc]ache
*.ilk
*.log
*.lib
*.sbr
*.sdf
ipch/
obj/
[Bb]in
[Dd]ebug*/
[Rr]elease*/
Ankh.NoLoad

#Tooling
_ReSharper*/
*.resharper
[Tt]est[Rr]esult*

#Project files
[Bb]uild/

#Subversion files
.svn

# Office Temp Files
~$*

#End .ignore file

I added a starting Git workflow on my domain here: http://www.adronbhall.com/Home/GitWorkflow

Category: How-to

Cuttin’ Teeth With NuGet

Like this:

Git Updates

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: