Category: Iron Foundry

Architectural PaaS Cracks or Crack PaaS

Over the last couple years there have been two prominent open source PaaS Solutions come onto the market. Cloud Foundry & OpenShift. There’s been a lot of talk about these plays and the talk has slowly but steadily turned into traction. Large enterprises are picking these up and giving their developers and operations staff a real chance to make changes. Sometimes disruptive in a very good way.

However, with all the grandeur I’m going to hit on the negatives. These are the missing parts, the serious pain points beyond just some little deployment nuisance. Then a last note on why, even amidst the pain points, you still need to make real movement with PaaS tooling and technologies.

Negative: The Data Story is Lacking

Both Cloud Foundry and OpenShift have a way to plug into databases easily.

Cloud Foundry provides ways to build a Cloud Foundry Service that becomes the bound and hooked in SQL Server, MySQL, Postgresql, Redis or whatever data storage service you need. For more details on building a service, check out the echo example on the vcap sample github project.

OpenShift has what are called Cartridges which provide the ability to add databases and other services into the system. For more information about the cartridges check out Red Hat’s OpenShift Documentation and also the forums.

Cloud Foundry and OpenShift however have distinctive weak spots when it comes to services that go beyond a mere single instance database. In the case of a true distributed database such as Cassandra, HBase or Riak, it is inordinately difficult to integrate a system that any PaaS inter-operates with well. In some cases it’s irrelevant to even try.

The key problem being that both of the PaaS systems assume the mantle of master while subjugating the distributed database a lower tier of coordination. The way to resolve this at the moment is to do an autonomous installation of Riak, Cassandra, Neo4j or other database that may be distributed, stored hot swappable, or otherwise spread across multiple machine or instance points. Then create a bound connection between it and the PaaS Application that is hosted. This is the big negative in PaaS systems and tooling right now, the data story just doesn’t expand well to the latest in data and database technologies. I’ll elaborate more about this below.

Negative: Deployment is Sometimes Easy, Maintenance is Sometimes Hard

Cloud Foundry is extremely rough to deploy, unless you use Bosh to deploy to either VMware Virtualized instances or AWS. Now, you could if resources were available get Bosh to deploy your Cloud Foundry environment anywhere you wanted. However, that’s not easy to do. Bosh is still a bit of a black box. I myself along with others in the community are working to document Bosh, but it is slow going.

OpenShift is dramatically easier to deploy, but is missing a few key pieces once deployed that draw some additional operational overhead. One of those is that OpenShift requires more networking management to handle routing between various parts of the PaaS Ecosystem.

Overall, this boils down to what you need between the two PaaS tool chains. If you want Cloud Foundry’s automatic routing and management between nodes. This is a viable route, but if your team wants to manage the networking tier more autonomous from the PaaS environment then maybe OpenShift is the way to go. In the end, it’s negative bumpy territory to determine which you may or may not want based on that.

Negative: Full Spectrum Polyglot, Missing Some

Cloud Foundry has a wider selection of languages and frameworks with community involvement around those with groups like Iron Foundry. OpenShift I’m sure will be getting to parity in the coming months. I have no doubt between both of these PaaS Ecosystems that they’ll expand to new languages and frameworks over time. Being polyglot after all is a no brainer these days!

Why PaaS Is, IMHO, Still Vitally Important

First toss out the idea that huge, web scale, Facebooks and Googles need to be built. Think about what the majority of developers out there in the world work on. Tons and tons and tons of legacy or greenfield enterprise applications. Sometimes the developer is lucky enough to work on a full vertical mix of things for a small business, but generally, the standard developer in the world is working on an enterprise app.

PaaS tooling takes the vast majority of that enterprise app maintenance from an operational side and tosses it out. Instead of managing a bunch of servers with a bunch of different apps the operations team manages an ecosystem that has a bunch of apps. This, for the enterprises that have enough foresight and have managed their IT assets well enough to be able to implement and use PaaS tooling, is HUGE!

For companies working to stay relevant in the enterprise, for companies looking to make inroads into the enterprise and especially for enterprises that are looking to maintain, grow or struggling to keep ahead of the curve – PaaS tooling is something that is a must have.

Just ask a dev, do they want to spend a few hours configuring and testing a server?  Do they want to deploy their application and focus on building more value into that application?

…being I’ve spent a few years being the developer, I’ll hedge on the side of adding value.

What’s Next?

So what’s next? Two major things in my opinion.

1. Fill the data gap. Most of the PaaS tooling needs to bridge the gap with the data story. I’m working my part with testing, development and efforts to get real options built into these environments, but this often leads back to the data story of PaaS being weak. What’s the solution here? I’m in talks, ongoing, planning sessions ongoing, and we’ll eventually get a solid solution around the data side.

2. Fix deployments & deployment management. Bosh isn’t straight forward or obvious in what it does, Cloud Foundry is easily the hardest thing to deploy with many dependencies. OpenShift is easier to deploy and neither of them actually have a solid management story over time. Bosh does some impressive updates of Cloud Foundry, and OpenShift has some upgrade methods, but still over time and during day to day operations there hasn’t been any clear cut wins with viewing, monitoring and managing nodes and data within these environments.

Thor HAMMA! OS-X Cocoa UI for Cloud Foundry

So today we’re super excited to release Thor release candidate from the furnaces of the Iron Foundry. We’ve had number of people working not he project and core Objective-C Coder Benjamin van der Veen @bvanderveen (Twitter), @bvanderveen (Github) and site tearing through tests, implementation, refactoring and UI hacking non-stop these last few weeks. I’ll admit, I think he’s slept some, but nobody knows.

With this new release, the features around…  well…  check out the video.

For a more complete list of the features check out Github, Github Issues & the Iron Foundry Blog.

Thor Project Opens Up, Building the Cloud Foundry Ecosystem with the Community

The Iron Foundry Team are big advocates of open source software. We write code across all sorts of languages, just like many of the development shops out there do. Sometimes we’re heavy on the .NET, other times we’re all up in some Java, Ruby on Rails, spooling up a Node.js Application or something else. So keeping with our love of open source and our polyglot nature we’ve created the Thor Project with three distinct apps.

Before jumping into the applications though, a little context for what and where Thor is in the grand scheme of things. We need to roll back to the Cloud Foundry Project to get into that. The Cloud Foundry Project is an open source project built around software for PaaS (Platform as a Service) which can be used to build your own PaaS internally or externally, in a cloud provider or directly on hardware. It’s your choice how, when and where you want to use it. For more context on PaaS check out my previous entry “The Confusions of IaaS, PaaS and SaaS“.

Thor Project

Cocoa for OS-X

Thor Odinson
Thor Odinson, God of Thunder

You know who Thor is right? He’s this mythic Norse God, also known as the God of Thunder. Since we’re all about bringing the hamma we welcomed Thor into our team’s stable of applications. So starting immediately we’ve released Thor into the realms for contributions and fighting the good open source software battle! If you’d like to join the effort, check out the github project and feel free to join us!

Technically, what is the Thor Application? This is a Cocoa Application built for OS-X that is used for managing, deploying and publishing applications to Cloud Foundry enabled and or Iron Foundry extended PaaS Environments.

.NET for Windows 7

The .NET Metro version of the Thor Application is also released via github with a provided installer. We’ve almost taken the same path, except of course for the very different UX and UI queues with Windows 7 and the Metro UX design guidelines.

WinRT for Windows 8

I wasn’t really sure what to call this version. Is it Metro or WinRT or Windows 8 or something else? Anyway, there is a project, it is albeit empty at this point, but it is the project where the Windows 8 version of Thor will go! For now get the Windows 7 version and install it on Windows 8, it won’t have touch interface support and things, but should work just like a regular application on Windows 8.

The Code

To get started with these, generally you’d just clone the repo and do a build, then get started checking out the code. There is one catch, for the OS-X version you’ll want to pull down the sub-modules with the following command.

[sourcecode language=”bash”]
git clone git@github.com:YourForkHere/Thor.git
git submodule update –init –recursive
[/sourcecode]

Once you do that in XCode just make sure to then select the right project as the starting build project.

…then when the application is launched…

Thor Running in OS-X
Thor Running in OS-X

I’ll have more in the coming days and weeks about Thor & Iron Foundry. For now, check out the blog entry on the Iron Foundry Blog and subscribe there for more information.

Ways to Interact Asynchronously with C#

NOTE: All of this code is available at my Github Project “Remembering” (https://github.com/Adron/Remembering). Feel free to fork it, share it, or send me corrections or pull requests.

While working on the Thor Project there have been numerous situations where I need to fire off an asynchronous callback in C# while maintaining good responsiveness in the actual user interface. Benjamin (@bvanderveen) has been using Reactive Extensions with subscriptions to do this in the Objective-C code for the Cocoa based OS-X Thor user interface. See my previous blog entry for an example of what he’s doing.

For a good example of asynchronous calls against Cloud Foundry I setup the following project using the Iron Foundry Project VCAP Client Library. The first thing I setup was a static class with a few constants to use across the examples for the URI, username and password for the Cloud Foundry Account.

[sourcecode language=”csharp”]
public static class YourSecrets
{
public const string Username = "youremail@someplace.com";
public const string Password = "AnAwesom3HardPassw0rd!";
public const string Uri = "http://api.yourpaas.com";
}
[/sourcecode]

Next step was to setup the delegate and method I’d use for calling out to the Cloud Foundry environment and retrieving data in parallel to my active console or user interface. That code snippet looked like this. I also added a private variable _finished for use in tracking when the request was completed in the begin and end invoke example below.

[sourcecode language=”csharp”]
private bool _finished;

IEnumerable TheMethodToConnectThatWillTakeLongTime(string uri)
{
var client = new VcapClient(uri);
client.Login(TheSecretBits.YourSecrets.Username, TheSecretBits.YourSecrets.Password);

_finished = false;

return client.GetApplications();
}

delegate IEnumerable MethodDelegate(string uri);
[/sourcecode]

Once I had that setup I was ready to create my baseline method that would make a synchronous call. A synchronous call is one that makes the call as if it just called the method directly. There’s no real reason to create one like I’ve done here, but I was just using it to provide a basic example of calling the delegate.

[sourcecode language=”csharp”]
public void SynchronousCall()
{
var starting = DateTime.Now.ToLongTimeString();

var delegateMethod = new MethodDelegate(TheMethodToConnectThatWillTakeLongTime);
var returnedBits = delegateMethod(TheSecretBits.YourSecrets.Uri);

var ending = DateTime.Now.ToLongTimeString();

Console.WriteLine(string.Format("The delegate call returned \n\n{0}\n\nstarting at {1} and

ending at {2} which takes a while of waiting.",
returnedBits, starting, ending));

_finished = false;
}
[/sourcecode]

That gets us a baseline. If you run a synchronous call against anything with a console application or a windows app, WPF or whatever it will lock up the calling thread while it is waiting for a response. In any type of user interface that is unacceptable. One of the best options is to fire of an asynchronous callback. The way I did this, which is an ideal way to make calls with the Iron Foundry Client Library against a Cloud Foundry Environment, is shown below.

This is my asynchronous call.

[sourcecode language=”csharp”]
public void DemoCall()
{
Console.WriteLine("Callback:");
var delegateMethod = new MethodDelegate(TheMethodToConnectThatWillTakeLongTime);

var callbackDelegate = new AsyncCallback(MyAsyncCallback);

Console.WriteLine(" starting…{0}", DateTime.Now.ToLongTimeString());
delegateMethod.BeginInvoke(TheSecretBits.YourSecrets.Uri, callbackDelegate, delegateMethod);
Console.WriteLine(" ending…{0}", DateTime.Now.ToLongTimeString());
}
[/sourcecode]

Now the simple callback.

[sourcecode language=”csharp”]
public void MyAsyncCallback(IAsyncResult ar)
{
Console.WriteLine("Things happening, async state calling.");

var delegateMethod = (MethodDelegate)ar.AsyncState;

Console.WriteLine(" called…{0}", DateTime.Now.ToLongTimeString());

var returnedBits = delegateMethod.EndInvoke(ar);

Console.WriteLine(" end invoked…{0}", DateTime.Now.ToLongTimeString());

foreach (Application application in returnedBits)
{
Console.WriteLine("Application {0} is in {1} state…",
application.Name, application.State);
Console.WriteLine(" with {0} running instances, {1} memory per instance, {2} disk allocated…",
application.RunningInstances, application.Resources.Memory, application.Resources.Disk);
Console.Write(" hosted at ");
foreach (var uri in application.Uris)
{
Console.Write(uri + " ");
}
Console.WriteLine(".");
}
}
[/sourcecode]

That’ll get the call running on a parallel thread and when it is wrapped up it returns the data.

The User Interface Interaction Issue

This is all fine and dandy for the command console. But if you want to give control back to the UI thread in a UI application and make sure that the background thread can actually update a control when fired off, do the same thing as I’ve discussed here except set the control up to invoke the dispatcher, so that the “threads don’t cross” when trying to return information to a control that needs updated. In order to do this take the control that needs updated and set the Dispatcher Invoke method as shown below.

[sourcecode language=”csharp”]
private void Write(string updateText)
{
UpdatingTextBlock.Dispatcher.Invoke(
System.Windows.Threading.DispatcherPriority.Normal,
new Action(delegate
{
UpdatingTextBlock.Text += updateText;
}
));
}
[/sourcecode]

For more on the Iron Foundry Project and the library I’ve used here, check out the Iron Foundry Blog & Site. For more information on Thor and to follow or get involved check out the Thor Project Site (Hosted with Cloud Foundry at Tier 3!).

All of this code is available in my Github Project “Remembering” (https://github.com/Adron/Remembering). Feel free to fork it, share it, or send me corrections or pull requests.

Where Am I?

That title sounds like a Dream Theater song or something. But alas, I’m going to try and answer the question for the next few weeks per my calendar of logistics.

  • September 10th (Monday) – September 14th (Friday) I’ll be in Seattle for networking, work and a few rounds. Maybe a geek lunch or two too, who’s up for it?
  • September 20th (Thursday) I’ll be in Seattle for the Software Craftsman’s Meetup at Getty Images.
  • September 22nd (Saturday) I’ll be at the Portland Streetcar eastside Loop Party. Yeah yeah, it doesn’t really have anything to do with tech, but I’m a transit nerd, so gotta go see the new streetcar line.
  • October 8th (Monday) – October 17th (Wednesday) I’ll be in San Francisco for the RICON (A Distributed System Conference for Developers) and the HTML 5 Dev Conf (js, html5 and all that developer conference).

Thor & Cloud Foundry Hackathon & Installfest

Curious if anyone is up for meeting and doing an installfest or hacking on Cloud Foundry, Iron Foundry or checking out Thor this coming week in Seattle? Any takers? Leave a comment and I’ll also ping the people of the Twitterverse and App.net. I’m up for meeting at a coffee shop or other space and would be happy to come to an office or other environment if anyone is interested.