OSCON: The Web, It’s HUGE! Cloud Computing More Realistically…

It is day 3 of OSCON data & java, and the kick off to the main keynotes and core conference. There are a repeating topics throughout the conference:

The Web, It’s Still HUGE!  Imagine that!

HTML 5, CSS3, JavaScript/jQuery/Node.js – This is starting to look like it will be the development stack of the web. If you use ASP.NET MVC, Ruby on Rails, PHP, Java, or some other web stack these core technologies are here to augment or even in some cases completely replace traditional web stacks.

Node.js can replace web servers in some situations when core APIs or other fundamental simple services are needed. In addition to that the Node server will eventually, I have no doubt, be able to completely replace traditional web servers like Apache, Tomcat, or IIS for almost any web site. In addition to web sites though, Node provides a very valuable engine to develop and test hard core JavaScript, building reusable libraries, and other server oriented needs. The other huge boost for Node.js is the ability for a dev shop to be able to centralize development around a single language. Something that Java and .NET have tried in the past, yet failed to ever achieve. The big irony is JavaScript never started out with this intent, but here it is!

In addition to Node.js making inroads to the server environments worldwide, JavaScript in general is starting to be used for all sorts of tools, stacks, and frameworks outside of just the browser. It can be used to submit a request against Hadoop, it can create a way to access and manipulate CouchDb, MongoDb, and other databases. Javascript is becoming the one language to rule them all (please excuse my Tolkenism 😉 )

Cloud Computing or More Realistically, “Distributed, Geographically Dispersed, Highly Available, Highly Available, Resilient, Compute and Storage Segmented Functionality, and not to forget, Business Agility Oriented Utility Computing“.

Long enough title? There are numerous open source cloud platforms and infrastructure offerings available. At OSCON there was discussion and multiple session about OpenStack, the Open Cloud Initiative, Stratos, and other open software solutions for cloud computing. This is great news for developer working with cloud computing technologies, especially for ongoing efforts and pushes to gain adoption of cloud computing within Enterprise.

Companies will continue to push their own proprietary capabilities and features, but it would behoove the industry to standardize on an open platform such as OpenStack. Currently most major cloud/utility computing providers such as Amazon Web Services and Windows Azure lock a company into their specific APIs, SDKs, and custom way of doing things. A development team that is savvy can prevent that, but if the core feature sets around comput, storage, and otherwise were standardized this lock in issue could be resolved.

Half Way Mark, Check

So far the conference has provided lots of insight into the open source community. Announcements have been made that keep the open source community moving forward into the future. With that, some of the things to look forward to:

  • I’ll have some in depth coverage of products, product releases, and services for some of the top open source companies.
  • I will hopefully win a Github T-shirt, to go along with my score of t-shirts for Heroku and others that I’ve received!
  • I’ll dig into some of the bleeding edge technologies around cloud computing including the likes of DotCloud!

So stay tuned, I’ll be back with the action packed details shortly.  Cheers!

At OSCON Day #1 and “Learning Node.js”

Playful Exploration of Public & Personal Data

I arrived, just after the keynote, because I was being lame on this Monday. Sounds like it was a great keynote with some great people speaking.

The first thing I did was get my directions straight for where all the rooms and expo area is. Once I did that I headed to Playful Explorations of Public and Personal Data that Andrew Turner was giving. His presentation showed some great information and data for or from GeoIQ, Geocommons, and others. The areas in which data can be used to overlay crime, friendly or unfriendly areas based on that, where there are dogs, coffee shops, and what can be derived.

I did notice though that a lot of the data is at a very high level. Leaders can make decisions on this kind of data, but it is dangerous making decision based on one or two of these data points. Leaders need to truly understand events and triggers at a macro level also. At the end of the session Andrew actually hit on the note that users of this data must be careful in what they correlate.

Architectural Anti-patterns for Handling Data

I’ll let the tweets I tweeted tell the tweet of the presentation.  🙂

Adron Hall#oscon #oscond RDBMS antipattern – Email data stored in databases. <- Can get VERY nasty. Couch? Mongo? NoSQL anybody? BigTable? 😐

Adron Hall#oscon #oscond RDBMS antipattern: “Stoned Procedures” i.e. Stored Procedures, Triggers, etc… Bad! Lock in and horrible maintainability.

Adron HallOooh! #oscon #oscond Another antipattern, are you ready for this one? ORMs. Impedance mis-match w/ RDBMS. The cure as bad as the disease! :/

Adron Hall#oscon #oscond Another antipattern. Tables as a log file. 😮 Hmmm. Yup, I’ve done that. 🙁 Probably wasn’t the greatest idea…but meh. 😀

Adron Hall#oscon #oscond antipattern The Alignment creating columns “a1, a2, a3, a4…?!?!?!?!” WTF!? If you do this – STOP NOW! k thx.

Adron Hall (originally posted on yfrog) Click for larger image

Click for larger image

antipatterns w/ RDBMSs… #oscon #oscond

Adron Hall#oscon #oscond antipattern RDBMS “Dynamic Table Creation”… yeah, not a good idea. 🙁 makes sad pandas.

Adron HallAnti-pattern for RDBMS: “Not everything fits on/in a RDBMS…”#oscon #oscond

Adron HallOH: “What is the main mySQL tuning technique used in panic situations?” A: “Install PostgreSQL” :O Doh! #oscon #oscond

Adron Hall@LusciousPear En route now… 🙂 to @gleicon session.

Bradford S.LusciousPear

by adronbhIf you’re at #oscon / #oscond head NOW to C123 to see @gleicon‘s talk on “Architectural Anti-Patterns for Data Handling”. There’s lulz + win

As described on the OSCON Site, Learning Node.js

Presented by Tom Hughes-Croucher (Joyent). Learn how to build scalable Internet applications with Node.js, the event-driven server-side JavaScript framework. You’ll see how Node.js solves many scaling and speed problems that weigh down other web application frameworks.

This is the first session I lined up for myself. I’ve been on a Node.js kick lately and have still found myself not having much time to work with the technology. However, having a clearcut session dedicated to the topic, and tomorrow is Node.js day, I’m pretty stoked to really get to using the technology hard core!

Tom did a good job with this session, with a bit of lively retort thrown in here and there. The session covered installation, the basic apps that are displayed on the Node.js Site, and then into additional exercises that got us all running the bits like pros (ok, like total newbs I’m sure). The session was a decent pace, it gave me time to work through the exercises and also wrap up coverage of the 1st day of the conference. However, I’m still going to have to sit down and go through the exercises again and actually determine what Node is actually doing. There are some interesting going ons behind the scenes that I know I’ve missed, but sure will pick them up ASAP with a quick review.  🙂

Day Number One Summary

This is my first full size OSS Conference I’ve attended. I did attend OS Bridge, which was similar, but this one has a large price tag to it. So expectations are different. Overall I’ve been very happy so far. The Swag is awesome, the sessions have been good, and the Node.js Intro was very informative and taught me a few things I didn’t know. The one thing I regrest, is that I can’t attend more of the sessions. This however tends to be the problem with any conference that is worth the time! I’m looking forward to day two, for now, I’m off to socialize and try to do a few non-computer related things.

Keeping Up With the Conference

To keep up with today’s events, and ongoing events during the Conference follow/search the Twitter feed with the #oscon for the main conference, #osconj for the Java Track, and #oscond for the Big Data Track!

Watch the live streaming feed.

OSCON Impact!

OSCON 2011OSCON, one of the most exciting and largest Open Source Software Conferences, is kicking off this Monday. If you’re into closed source, open source, or software creation, design, coding, or otherwise you should be paying attention! I’ll have coverage, interviews, and more here so stay tuned!

Also check out live streaming!

For more information about the conference, check out the tracks…

OSCON 2011 Tracks

…or check out the Node.js Day!  Anyway you look at it, the conference is going to be jam packed with great info, work, learning, and more. Check back soon, I’m sure I’ll have more tasty tech bits each and every day!

JavaScript Development Environment

JavaScript is pretty easy to develop with, usually only a browser and a text editor is the only thing needed. But seriously, that’s a bit slow going. It helps a lot of have certain tools, editors, debuggers, or other pieces to make the development move along smoothly. So far, over the last few months of doing a little JavaScript here and there I’ve accrued the following tools to help with my JavaScript Development. First a few descriptions of what I’ve been using and and why, then toward the end I’ll provide a full list of tools.

Web Browsers: Primarily there is Firefox, Internet Explorer, Safari, Opera, and Chrome. I’d almost say it doesn’t matter which one you use to view the web in. With Internet Explorer 9 they’re all pretty fast with JavaScript and all render the latest HTML fairly well. Albeit none are 100% complete some are much more complete than others. But since this is about JavaScript I’m focusing on that aspect.

Debugging: Of the web browsers the top two for the last few years have been Firefox with Chrome coming in second. One of the main reasons is because of Firebug in both browsers. It originally came out for Firefox if I recall correctly and then became available for some others. Even though Firebug is mainly a player on Firefox each of the browsers have pretty extensive debugging capabilities now for Javascript. I’ve not tested Opera too much for debugging, but each of the others I know have the ability to set break points, actually pinpoint where real errors in code are, and more.

IDEs: WebStorm by Jetbrains is awesome for JavaScript. Matter of fact many of their IDEs are great for JavaScript. Visual Studio is also pretty decent once you add the various plug ins/add ons for JavaScript to gain more Intellisense support and other features.  Overall these are the primary IDEs that I’ve been using for development of JavaScript and they’ve all served me pretty well. Do I long for more capabilities and options in the IDEs? Sure, but overall the IDEs today are much improved over just the last couple of years.

Integrated Development Environments, Browsers, and Addons

  • Visual Studio 2010 w/ ReSharper 6
  • WebStorm by Jetbrains
  • Rubymine by Jetbrains
  • Firefox
  • Chrome
  • Internet Explorer
  • Opera
  • Safari
  • Firebug

Over the next few days I’ll be attending OSCON, picking up a lot more tools, tricks, and other JavaScript goodies, so will post a follow up to this blog entry. Please feel free to add any suggestions or comments and I’ll add those to the “unabridged” list that I put together!

A SQL Server .NET ASP.NET MVC RESTful Web Services Facade – Part I

Did I get enough of the acronyms and key words in the header?  It looks like soup!  :O

This is a somewhat messy project to build a prototype layer around SQL Server. The reason for this, shockingly, is to allow for a SQL Server to be used by frameworks and systems that normally don’t or can’t access the database directly. In my particular scenario we’re working on getting Ruby on Rails running with JRuby in a Windows Environment. Because we will need to utilize a lot of SQL Server Databases, it seemed like a great idea to build out a layer over the SQL Server (or Servers) so that a Ruby on Rails Web App, ASP.NET MVC, or even a PHP or pure Javascript Application could access the data in the database. What better way to do that then to create a RESTful Web Services Facade over the database.

Some of you might be thinking “Why not use RIA Services?!?!?! Are you mad!!” Well, there is a big problem, RIA Services doesn’t work against SQL 2000 or SQL 2005, which is the database technology that this particular requirement dictated. Well, now that you have context, I’ll dig straight in to what I did building this prototype out.

Kick Out a SQL Server Database Project

I need some data, and a database, with just some of the standard junk you’d expect in a production database. One of the best ways to throw together a database in a really short amount of time, with data, is to use a SQL Server Database Project.

New Database Project (Click for larger image)

New Database Project (Click for larger image)

You might see this and think, “But you said that the facade is against a SQL Server 2000 or 2005 database!” Well, it is, but to get a database running locally and have this project type work, I’m using my local SQL Server 2008 Express installation. However, I’m limiting myself to data types primarily available to SQL Server 2000 and 2005. So no worries, this works just fine against those archaic databases.  😛

First I ran the following script to create the database and some sample tables with various data types.

DROP DATABASE SomeExistingOrMigratedDatabase
GO
CREATE DATABASE SomeExistingOrMigratedDatabase
GO
USE SomeExistingOrMigratedDatabase
GO
IF  EXISTS (SELECT * FROM sys.foreign_keys WHERE object_id = OBJECT_ID(N'[dbo].[FK_Person_Village]') AND parent_object_id = OBJECT_ID(N'[dbo].[Person]'))
ALTER TABLE [dbo].[Person] DROP CONSTRAINT [FK_Person_Village]
GO
IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[Person]') AND type in (N'U'))
DROP TABLE [dbo].[Person]
GO
IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SomeFlatDenormalizedDataTable]') AND type in (N'U'))
DROP TABLE [dbo].[SomeFlatDenormalizedDataTable]
GO
IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[Village]') AND type in (N'U'))
DROP TABLE [dbo].[Village]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[Village]') AND type in (N'U'))
BEGIN
CREATE TABLE [dbo].[Village](
	[Id] [uniqueidentifier] NOT NULL,
	[Village] [nvarchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
 CONSTRAINT [PK_Village] PRIMARY KEY CLUSTERED
(
	[Id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON)
)
END
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SomeFlatDenormalizedDataTable]') AND type in (N'U'))
BEGIN
CREATE TABLE [dbo].[SomeFlatDenormalizedDataTable](
	[Id] [uniqueidentifier] NOT NULL,
	[StarzDate] [datetime] NOT NULL,
	[Numerals] [int] NULL,
	[Numberals] [int] NULL,
	[Monies] [decimal](14, 4) NOT NULL,
	[Day] [int] NOT NULL,
	[Month] [int] NOT NULL,
	[Year] [int] NOT NULL,
	[BigNonsense] [ntext] COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[Flotsam] [float] NULL,
	[Jetsam] [float] NULL,
	[SmallishText] [nvarchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
	[BiggishText] [nvarchar](2999) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
 CONSTRAINT [PK_SomeFlatDenormalizedDataTable] PRIMARY KEY CLUSTERED
(
	[Id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON)
)
END
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[Person]') AND type in (N'U'))
BEGIN
CREATE TABLE [dbo].[Person](
	[Id] [uniqueidentifier] NOT NULL,
	[Name] [nvarchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
	[DateOfBirth] [datetime] NOT NULL,
	[VillageId] [uniqueidentifier] NULL,
 CONSTRAINT [PK_Person] PRIMARY KEY CLUSTERED
(
	[Id] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON)
)
END
GO
IF NOT EXISTS (SELECT * FROM sys.foreign_keys WHERE object_id = OBJECT_ID(N'[dbo].[FK_Person_Village]') AND parent_object_id = OBJECT_ID(N'[dbo].[Person]'))
ALTER TABLE [dbo].[Person]  WITH CHECK ADD  CONSTRAINT [FK_Person_Village] FOREIGN KEY([VillageId])
REFERENCES [dbo].[Village] ([Id])
GO
IF  EXISTS (SELECT * FROM sys.foreign_keys WHERE object_id = OBJECT_ID(N'[dbo].[FK_Person_Village]') AND parent_object_id = OBJECT_ID(N'[dbo].[Person]'))
ALTER TABLE [dbo].[Person] CHECK CONSTRAINT [FK_Person_Village]

Once the database and tables are created, import the database into the database project. To do this select the “Import Database Objects and Settings…” by right clicking the context menu on the Database Project.

Import Database Objects and Settings...

Import Database Objects and Settings…

Select the database just created and click on start. Once the script generation is done, navigate into the project directories and you will see the following scripts have been created.

Generated Scripts (click for larger image)

Generated Scripts (click for larger image)

Next create a new data generation plan in the Data Generation Plans folder (notice I already cheated and have one in the above image).

Creating a Data Generation Plan

Creating a Data Generation Plan

Open up the file this creates (I called mine BuildSomeData.dgen). In the file, note I selected the relationship between the Village and People Tables, and set the ratio to 60:1. When you change the data in the Village table it then automatically updates how much data will be generated for the People Table.

Data Generation Plan

Data Generation Plan

When all that is done, hit F5, select the database and the data will be generated. That gets us a database with data to use as an existing source. From here I’ll jump into creating the actual Facade Layer.

NOTES: Once you generate data, depending on how much you decided to generate, you may want to see how big your database is by using the sp_dbhelp stored procedure. I am however, unsure which versions of SQL Server this stored procedure is available in.

Code for this project is available here: https://github.com/Adron/ExistingSqlServerProject