More Cloud News

Recently Amazon jumped into the relational database cloud competition with Microsoft.  Up until the 6th of this month, Microsoft had the only cloud with a real dedicated relational database offering in SQL Azure.  Now Amazon has their Relational Database Service heating up the competition.

In other news, Google finally joined the storage party with their recent launch announcement at the I/O Conference.  So now we have Amazon, Microsoft, and Google as the big companies on the block throwing down on the storage offerings.  Stay tuned for more!

In other news I have been working through the katas setup for TDD practice.  They’re actually a lot of fun and would suggest anyone out there interested in TDD or just unit testing to just go out and give one a test drive.  : )

Currently I am working on a code kata putting together ideas from Roy Osherove‘s The Art of Unit Testing and what one needs to know for testing in enterprise environments, abstracting the appropriate code to take into account web services, files, I/O, architectural issues, and other elements of coding.

Test Driven Development Built Name Generator Part 3

Part 1, Part 2.

Now it is time finally for the random name generation.  I’ve gotten everything into place, so it’s just a matter of grabbing the names and randomizing among them.  Just as a check, after Part 2 was finished I had a total of 98,572 rows of data, or simply 98,572 names.

Many times I would not write actual unit tests against the database as it breaks isolation.  This time though, I’m making an exception.  First I added a reference to Generator.Core, System.Data, and System.Data.Entity.  I then created a new unit test class called DatabaseTests and added the following test to get things rolling.  Keep in mind, since these tests break isolation, I don’t write them in the red light green light idea of TDD.  My assumption is that they only prove the database is setup, running, the connection string is correct, and other minor configuration based and server based components.  In addition, yes, these are somewhat silly tests, such as the RetrieveFirstName() test has a bit of redundancy to it, but the point is to have an automated test to assure that the data has been moved into the database and is returning appropriately.

[sourcecode language=”csharp”]
[TestClass]
public class DatabaseTests
{
[TestMethod]
public void RetrieveFirstFemaleName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Female First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveLastName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Last Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveFirstMaleName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Male First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveFirstName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty && returnName.Type == "Female First Names" ||
returnName.Type == "Male First Names");
}
}
[/sourcecode]

The next step now was to refactor the existing tests to prove the assembly of the name data transfer objects that would hold the first name and last name derived from the returned entity objects.

The first thing I did was break out the actual name objects into their respective files.  I did this using ReSharper, because it’s super insanely simple with ReSharper.  After that I put the class files in the project that has the model.  Next I fixed the namespaces to Generator.Core.Model.  The class files after these changes looked like the code below.

IFullName.c

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public interface IFullName
{
string FirstName { get; set; }
string LastName { get; set; }
}
}
[/sourcecode]

FullName.cs

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public class FullName : IFullName
{
public FullName()
{
FirstName = string.Empty;
LastName = string.Empty;
}

#region IFullName Members

public string FirstName { get; set; }
public string LastName { get; set; }

#endregion
}
}
[/sourcecode]

NameFactory.cs

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public class NameFactory
{
public static IFullName Build()
{
return new FullName {FirstName = "TestFirst", LastName = "TestLast"};
}
}
}
[/sourcecode]

Now that we have these files moved out to the project where all the munging will happen, I’m going to start the actual refactor of the tests.  What I’ve come up with for the test refactor is as follows.

[sourcecode language=”csharp”]
using Generator.Core.Model;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Factory.Tests
{
[TestClass]
public class FactoryNameBuilder
{
[TestMethod]
public void VerifyFullNameObjectInstantiatesWithNames()
{
var name = new FullName();
Assert.IsTrue(name.FirstName.Length == 0);
Assert.IsTrue(name.LastName.Length == 0);
}

[TestMethod]
public void VerifyFullNameObjectReturnsFromFactory()
{
var factory = new NameFactory();
IFullName name = factory.Build();
Assert.IsTrue(name.FirstName.Length > 0);
Assert.IsTrue(name.LastName.Length > 0);
}

[TestMethod]
public void VerifyFullNameIsRandom()
{
var factory = new NameFactory();
IFullName nameOne = factory.Build();
IFullName nameTwo = factory.Build();

Assert.AreNotEqual(nameOne.FirstName, nameTwo.FirstName);
Assert.AreNotEqual(nameOne.LastName, nameTwo.LastName);
}
}
}
[/sourcecode]

In the factory I removed the fake return and fleshed out the appropriate build method.

[sourcecode language=”csharp”]
using System;
using System.Collections.Generic;
using System.Linq;

namespace Generator.Core.Model
{
public class NameFactory
{
public IFullName Build()
{
Random rand = new Random(DateTime.Now.Millisecond);

GeneratorEntities entities = new GeneratorEntities();

List FirstNames = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).ToList();
List LastNames = (from name in entities.Names
where name.Type == "Last Names"
select name).ToList();

FullName fullName = new FullName();
fullName.FirstName = FirstNames[rand.Next(0, FirstNames.Count)].Name;
fullName.LastName = LastNames[rand.Next(0, LastNames.Count)].Name;

return fullName;
}
}
}
[/sourcecode]

Run the tests now and you have a random name gold mine.  However, I wanted a bit more than that.  With another test added…

[sourcecode language=”csharp”]
[TestMethod]
public void VerifyMultipleNameReturn()
{
var factory = new NameFactory();
List<IFullName> list = factory.Build(1000);

for (int i = 0; i < list.Count; i++)
{
int compareUp = i + 1;
if (compareUp == list.Count)
{
compareUp = 0;
}
Assert.AreNotEqual(list[compareUp].FirstName + list[compareUp].LastName, list[i].FirstName + list[i].LastName);
}
}
[/sourcecode]

…then the extra Build method.

[sourcecode language=”csharp”]
public List<IFullName> Build(int numberOfNames)
{
var rand = new Random(DateTime.Now.Millisecond);
var entities = new GeneratorEntities();

List<Names> FirstNames = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).ToList();
List<Names> LastNames = (from name in entities.Names
where name.Type == "Last Names"
select name).ToList();

var names = new List<IFullName>();

for (int i = 0; i < numberOfNames; i++)
{
var fullName = new FullName();
fullName.FirstName = FirstNames[rand.Next(0, FirstNames.Count)].Name;
fullName.LastName = LastNames[rand.Next(0, LastNames.Count)].Name;
names.Add(fullName);
}

return names;
}
[/sourcecode]

and now I have a strong verification that I get solid randoms for 1000 name pairs.  I bumped it up to 2000 too just for fun, and didn’t hit duplicates until I put it up to 200,000.

Test Driven Development Built Name Generator Part 2

Part 1Part 3.

Alright, time to tear into getting that name data imported into the database.  To review the previous entry check out part 1.

Next I set the Windows App Project I added in part 1 to be the startup project.  After that I created a screen that looks like the image below (click to see larger image).

Also add an open file dialog control.  Next double click on the Process button and add the following code.  Make sure not to miss the instantiation of the names list property in the initialization.

[sourcecode language=”csharp”]
using System;
using System.Collections.Generic;
using System.IO;
using System.Text.RegularExpressions;
using System.Windows.Forms;

namespace TextFileImporter
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
Names = new List();
}

private List Names { get; set; }

private void buttonProcessFile_Click(object sender, EventArgs e)
{
if (openFile.ShowDialog() == DialogResult.OK)
{
if (File.Exists(openFile.FileName))
{
TextReader tr = new StreamReader(openFile.FileName);
textProcessedFile.Text = tr.ReadToEnd();
tr.Close();
// Cleanup text.
textProcessedFile.Text = ProcessName(textProcessedFile.Text);
}
else
{
MessageBox.Show("Select a file that exists.", "File doesn’t exist.", MessageBoxButtons.OK,
MessageBoxIcon.Information, MessageBoxDefaultButton.Button1);
}
}
}

private string ProcessName(string textBody)
{
var regex = new Regex("[^A-Z’]+");
textBody = regex.Replace(textBody, " ");

foreach (var s in textBody.Split(Convert.ToChar(" ")))
{
Names.Add(s);
}

return textBody;
}
}
}
[/sourcecode]

With the beauty of those regular expressions the files will parse appropriately in about 1-5 seconds.  Next we need a way to write this data into the database.  I did this by adding a Class Project named Generator.Core and added a folder called Model.

In the Model Directory I added a new ADO.NET Entity Framework Model.  The images below show my steps through the wizard.

  • Wizard # 1 - Adding Data Model.Wizard # 1 – Adding Data Model.
  • Wizard # 2 - Adding Data Model.Wizard # 2 – Adding Data Model.
  • Wizard # 3 - Adding Data Model.Wizard # 3 – Adding Data Model.
  • Wizard # 4 - Adding Data Model.Wizard # 4 – Adding Data Model.
  • Wizard # 5 - Adding Data Model.Wizard # 5 – Adding Data Model.

.smugcontainer div {overflow: hidden;line-height: 1.1;margin-top: 10px;font-family: verdana, arial, lucida, sans-serif;font-size: 85%;background-color1: rgb(20, 20, 20);}
.smugimg li {float: left;display: block;width: 160px;height: 200px;background-position: 50% 50%;margin-right: 10px;margin-bottom: 10px;}
.smugimg li img {width: auto; height: auto; border: solid 1px #aaa; background: #555; padding: 2px; vertical-align: middle;}
.smugimg a {display: block;text-align: center;text-decoration: none; color1: rgb(240,240,240);}
.smugimg a:hover img {border: 3px solid #6da6d1; padding: 0px;}

.smugcontainer div {overflow: hidden;line-height: 1.1;margin-top: 10px;font-family: verdana, arial, lucida, sans-serif;font-size: 85%;background-color1: rgb(20, 20, 20);}
.smugimg li {float: left;display: block;width: 160px;height: 200px;background-position: 50% 50%;margin-right: 10px;margin-bottom: 10px;}
.smugimg li img {width: auto; height: auto; border: solid 1px #aaa; background: #555; padding: 2px; vertical-align: middle;}
.smugimg a {display: block;text-align: center;text-decoration: none; color1: rgb(240,240,240);}
.smugimg a:hover img {border: 3px solid #6da6d1; padding: 0px;}

At the end I also deleted the Class1.cs file from the project.

I then added a reference to the Generator.Core Project in the TextFileImporter Application Project.  Next add an application configuration file to the TextFileImporter Project with the following configuration section.

[sourcecode language=”xml”]
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<connectionStrings>
<add name="GeneratorEntities" connectionString="metadata=res://*/Model.Data.csdl|res://*/Model.Data.ssdl|res://*/Model.Data.msl;provider=System.Data.SqlClient;provider connection string=&quot;Data Source=.\carriage;Initial Catalog=Generator;Integrated Security=True;Pooling=False;MultipleActiveResultSets=True&quot;" providerName="System.Data.EntityClient" />
</connectionStrings>
</configuration>
[/sourcecode]

At the last minute, and to make sure anyone following along with these instructions is really paying attention, I made a minor change to the database table.  I changed the column Type to a data type of nvarchar(50).  I then updated the Data.edmx file in the Generator.Core Project.  To do this open up the file and right click anywhere in the white space around the model.  Then click on Update Model from Database….  Now, as things go, the stupid model didn’t update for me.  This is something that seems to occur very frequetly with the ADO.NET Entity Framework Model Files.  Since it didn’t update I had to manually right click on the Type Property and then set the Type of the Type (yes I know, redundant naming) to String.

So if anyone from Microsoft is reading this, the entity model files are STILL broken for some reason.  It’s been about 6 months since the entity framework and such where in beta and they still behave in many ways like they’re in beta, please fix this.  k thx bye.

Next I double clicked the Save (buttonSaveToDb) button and entered the following code to write the file data into the database.

[sourcecode language=”csharp”]
private void buttonSaveToDb_Click(object sender, EventArgs e)
{
if (comboNameType.Text == string.Empty)
{
MessageBox.Show("Select a type of name.", "Select Name", MessageBoxButtons.OK,
MessageBoxIcon.Information, MessageBoxDefaultButton.Button1);
}
else
{
// TODO Write to DB.
var generatorEntities = new GeneratorEntities();
foreach (var name in Names)
{
var newName =
new Names
{
NameId = Guid.NewGuid(),
Name = name,
Type = comboNameType.Text
};
generatorEntities.AddToNames(newName);
}
generatorEntities.SaveChanges(true);
}
}
[/sourcecode]

Then a quick reference and that finishes up the TextFileImporter Project.

[sourcecode language=”csharp”]
using Generator.Core.Model;
[/sourcecode]

Now mind you, I broke TDD principles because this app was merely a utility, would be used a few times, and above all breaks isolation and crosses boundaries.  A good rule is to not write tests for a project like this.  Unless you feel it absolutely necessary then I wouldn’t even think of it.

Now run the application and open and import each of the three files into the database.  This will give us the data we need for the next step of our random name generator.

Test Driven Development Built Name Generator Part 1

Part 2Part 3.

This is going to be a multi-part series on building a straight forward database driven name generator.  I’ve tried the random name generator thing and it generally isn’t so great.  I’d tried in the past this idea with the database table of census names and it works great.  So this is part 1.  I’ll post these entries consecutively over the next few days so stay tuned.

First I started a new clean solution and added a test project.  I figured I wasn’t even going to add the actual assembly project yet, just jump right in and start writing a test, get red, and go to the next step.

I added a test file and wrote the following test.

[sourcecode language=”csharp”]
[TestMethod]
public void VerifyFullNameObjectInstantiatesWithNames()
{
FullName fullName = new FullName();
Assert.IsTrue(fullName.FirstName.Length == 0);
Assert.IsTrue(fullName.LastName.Length == 0);
}
[/sourcecode]

After that I used ReSharper Alt+Enter Shortcut plus a little additional keying in myself to flesh out the skeletal class and get a green light.  I ended up with the class below.

[sourcecode language=”csharp”]
public class FullName
{
public string FirstName { get; set; }
public string LastName { get; set; }
}
[/sourcecode]

Next I wanted to enforce a contract so I could create a factory to build my FullName objects with.  With that change the interface and class I had now looked like this.

[sourcecode language=”csharp”]
public interface IFullName
{
string FirstName { get; set; }
string LastName { get; set; }
}

public class FullName : IFullName
{
public FullName()
{
FirstName = string.Empty;
LastName = string.Empty;
}

public string FirstName { get; set; }
public string LastName { get; set; }
}
[/sourcecode]

That gave me a green light on my first test.  After that I built a test for the factory that could build the names.

[sourcecode language=”csharp”]
[TestMethod]
public void VerifyFullNameObjectReturnsFromFactory()
{
IFullName name = NameFactory.Build();
Assert.IsTrue(name.FirstName.Length > 0);
Assert.IsTrue(name.LastName.Length > 0);
}
[/sourcecode]

I then took the NameFactory object and fleshed it out so I could build, run the test, and get green lighted.  Below is the NameFactory Class.

[sourcecode language=”csharp”]
public class NameFactory
{
public static IFullName Build()
{
return new FullName { FirstName = "TestFirst", LastName = "TestLast" };
}
}
[/sourcecode]

So now I have a green light on the name factory.  But even though I have a green light, it doesn’t exactly do what it needs to do, which is get some good unique and random names.  Next step, write a test for getting back some random names.

[sourcecode language=”csharp”]
[TestMethod]
public void VerifyFullNameIsRandom()
{
IFullName nameOne = NameFactory.Build();
IFullName nameTwo = NameFactory.Build();

Assert.AreNotEqual(nameOne.FirstName, nameTwo.FirstName);
Assert.AreNotEqual(nameOne.LastName, nameTwo.LastName);
}
[/sourcecode]

After creating this test, I have to dive a little deeper.  First I grabbed the census names for first and last names off of the Internet.  After that I added two projects to my overall Visual Studio Solution.  One is a database project and one is Windows App to use to manipulate the text file data and get it into our database.

Next I created the Generator Database a table to store the names that are stored in the files.

The SQL create script is shown below.

[sourcecode language=”sql”]
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[Names]’) AND type in (N’U’))
BEGIN
CREATE TABLE [dbo].[Names](
[NameId] [uniqueidentifier] NOT NULL,
[Name] [nvarchar](50) NOT NULL,
[Type] [smallint] NOT NULL,
CONSTRAINT [PK_Names] PRIMARY KEY CLUSTERED
(
[NameId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
END
[/sourcecode]

I also added an extended property to outline how I intended to use the “Type” column.

[sourcecode language=”sql”]
IF NOT EXISTS (SELECT * FROM ::fn_listextendedproperty(N’MS_Description’ , N’SCHEMA’,N’dbo’, N’TABLE’,N’Names’, N’COLUMN’,N’Type’))
EXEC sys.sp_addextendedproperty @name=N’MS_Description’, @value=N’1 = Male, 2 = Female, 3 = Last Name’ , @level0type=N’SCHEMA’,@level0name=N’dbo’, @level1type=N’TABLE’,@level1name=N’Names’, @level2type=N’COLUMN’,@level2name=N’Type’
[/sourcecode]

My intention is for the “Type” column to have a 1 for the male first name, a 2 for a female first name, and a 3 for the last name.

I’ve covered creating the initial tests and objects to use.  Also the database table that is needed and the create scripts have been provided.  Next steps are to build a quick app to get the names imported into the database table.  Stay tuned and that will be posted tomorrow.