DSE6 + .NET v?

Project Repo: Interoperability Black Box

First steps. Let’s get .NET installed and setup. I’m running Ubuntu 18.04 for this setup and start of project. To install .NET on Ubuntu one needs to go through a multi-command process of keys and some other stuff, fortunately Microsoft’s teams have made this almost easy by providing the commands for the various Linux distributions here. The commands I ran are as follows to get all this initial setup done.

[sourcecode language=”bash”]
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg –dearmor > microsoft.asc.gpg
sudo mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
wget -q https://packages.microsoft.com/config/ubuntu/18.04/prod.list
sudo mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
sudo chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
sudo chown root:root /etc/apt/sources.list.d/microsoft-prod.list
[/sourcecode]

After all this I could then install the .NET SDK. It’s been so long since I actually installed .NET on anything that I wasn’t sure if I just needed the runtime, the SDK, or what I’d actually need. I just assumed it would be safe to install the SDK and then install the runtime too.

[sourcecode language=”bash”]
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install dotnet-sdk-2.1
[/sourcecode]

Then the runtime.

[sourcecode language=”bash”]
sudo apt-get install aspnetcore-runtime-2.1
[/sourcecode]

logoAlright. Now with this installed, I wanted to also see if Jetbrains Rider would detect – or at least what would I have to do – to have the IDE detect that .NET is now installed. So I opened up the IDE to see what the results would be. Over the left hand side of the new solution dialog, if anything isn’t installed Rider usually will display a message that X whatever needs installed. But it looked like everything is showing up as installed, “yay for things working (at this point)!

rider-01

Next up is to get a solution started with the pertinent projects for what I want to build.

dse2

Kazam_screenshot_00001

For the next stage I created three projects.

  1. InteroperationalBlackBox – A basic class library that will be used by a console application or whatever other application or service that may need access to the specific business logic or what not.
  2. InteroperationalBlackBox.Tests – An xunit testing project for testing anything that might need some good ole’ testing.
  3. InteroperationalBlackBox.Cli – A console application (CLI) that I’ll use to interact with the class library and add capabilities going forward.

Alright, now that all the basic projects are setup in the solution, I’ll go out and see about the .NET DataStax Enterprise driver. Inside Jetbrains Rider I can right click on a particular project that I want to add or manage dependencies for. I did that and then put “dse” in the search box. The dialog pops up from the bottom of the IDE and you can add it by clicking on the bottom right plus sign in the description box to the right. Once you click the plus sign, once installed, it becomes a little red x.

dse-adding-package

Alright. Now it’s almost time to get some code working. We need ourselves a database first however. I’m going to setup a cluster in Google Cloud Platform (GCP), but feel free to use whatever cluster you’ve got. These instructions will basically be reusable across wherever you’ve got your cluster setup. I wrote up a walk through and instructions for the GCP Marketplace a few weeks ago. I used the same offering to get this example cluster up and running to use. So, now back to getting the first snippets of code working.

Let’s write a test first.

[sourcecode language=”csharp”]
[Fact]
public void ConfirmDatabase_Connects_False()
{
var box = new BlackBox();
Assert.Equal(false, box.ConfirmConnection());
}
[/sourcecode]

In this test, I named the class called BlackBox and am planning to have a parameterless constructor. But as things go tests are very fluid, or ought to be, and I may change it in the next iteration. I’m thinking, at least to get started, that I’ll have a method to test and confirm a connection for the CLI. I’ve named it ConfirmConnection for that purpose. Initially I’m going to test for false, but that’s primarily just to get started. Now, time to implement.

[sourcecode language=”csharp”]
namespace InteroperabilityBlackBox
using System;
using Dse;
using Dse.Auth;

namespace InteroperabilityBlackBox
{
public class BlackBox
{
public BlackBox()
{}

public bool ConfirmConnection()
{
return false;
}
}
}
[/sourcecode]

That gives a passing test and I move forward. For more of the run through of moving from this first step to the finished code session check out this

By the end of the coding session I had a few tests.

[sourcecode language=”csharp”]
using Xunit;

namespace InteroperabilityBlackBox.Tests
{
public class MakingSureItWorksIntegrationTests
{
[Fact]
public void ConfirmDatabase_Connects_False()
{
var box = new BlackBox();
Assert.Equal(false, box.ConfirmConnection());
}

[Fact]
public void ConfirmDatabase_PassedValuesConnects_True()
{
var box = new BlackBox(“cassandra”, “”, “”);
Assert.Equal(false, box.ConfirmConnection());
}

[Fact]
public void ConfirmDatabase_PassedValuesConnects_False()
{
var box = new BlackBox(“cassandra”, “notThePassword”, “”);
Assert.Equal(false, box.ConfirmConnection());
}
}
}
[/sourcecode]

The respective code for connecting to the database cluster, per the walk through I wrote about here, at session end looked like this.

[sourcecode language=”csharp”]
using System;
using Dse;
using Dse.Auth;

namespace InteroperabilityBlackBox
{
public class BlackBox : IBoxConnection
{
public BlackBox(string username, string password, string contactPoint)
{
UserName = username;
Password = password;
ContactPoint = contactPoint;
}

public BlackBox()
{
UserName = “ConfigValueFromSecretsVault”;
Password = “ConfigValueFromSecretsVault”;
ContactPoint = “ConfigValue”;
}

public string ContactPoint { get; set; }
public string UserName { get; set; }
public string Password { get; set; }

public bool ConfirmConnection()
{
IDseCluster cluster = DseCluster.Builder()
.AddContactPoint(ContactPoint)
.WithAuthProvider(new DsePlainTextAuthProvider(UserName, Password))
.Build();

try
{
cluster.Connect();
return true;
}
catch (Exception e)
{
Console.WriteLine(e);
return false;
}

}
}
}
[/sourcecode]

With my interface providing the contract to meet.

[sourcecode language=”csharp”]
namespace InteroperabilityBlackBox
{
public interface IBoxConnection
{
string ContactPoint { get; set; }
string UserName { get; set; }
string Password { get; set; }
bool ConfirmConnection();
}
}
[/sourcecode]

Conclusions & Next Steps

After I wrapped up the session two things stood out that needed fixed for the next session. I’ll be sure to add these as objectives for the next coding session at 3pm PST on Thursday.

  1. The tests really needed to more resiliently confirm the integrations that I was working to prove out. My plan at this point is to add some Docker images that would provide the development integration tests a point to work against. This would alleviate the need for something outside of the actual project in the repository to exist. Removing that fragility.
  2. The application, in its “Black Box”, should do something. For the next session we’ll write up some feature requests we’d want, or maybe someone has some suggestions of functionality they’d like to see implemented in a CLI using .NET Core working against a DataStax Enterprise Cassandra Database Cluster? Feel free to leave a comment or three about a feature, I’ll work on adding it during the next session.

Distributed Coding Prefunc: Installing QuickCheck for Great Testing

Erlang LogoA few weeks ago I kicked off this series of “Distributed Coding Prefunc: Up and Running with Erlang” and had wanted to keep up the momentum, but as life goes I had to tackle a few other things first. But now, it’s time to get back on track with some distributed computing. I intend to write tests with my samples, as I often do, I decided to take a stab at .

Before going forward, note that there is QuickCheck for Haskell and there is a QuickCheck for Erlang. Since the point of this “Distributed Coding Prefunc” is to get started coding with Erlang from zero, I’ll be talking about the Erlang version here. This version is created by John Hughes and Koen Claessen, starting the Quviq Company in 2006.

To download QuickCheck choose the version you intend to use, I’ve chosen the commercial license version from the download page.

At the command prompt, install QuickCheck by running Erlang and then run the install with these commands.

Launch Erlang:

[sourcecode language=”bash”]
$ erl
Erlang R15B01 (erts-5.9.1) [source] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.1 (abort with ^G)
1>
[/sourcecode]

Then execute the install:

[sourcecode language=”Erlang”]
1> eqc_install:install().
[/sourcecode]

If the execution of the install displays this error, you’ll need to use sudo.

[sourcecode language=”bash”]
Installing ["pulse-1.27.7","eqc-1.27.7","eqc_mcerlang-1.27.7"].
Failed to copy pulse-1.27.7–copy returned {error,eacces}??
** exception exit: {{error,eacces},"pulse-1.27.7"}
in function eqc_install:’-copy_quickcheck/3-lc$^0/1-0-‘/3 (../src/eqc_install.erl, line 63)
in call from eqc_install:install2/4 (../src/eqc_install.erl, line 44)
[/sourcecode]

Kill Erlang with a ctrl+c and restart Erlang with the sudo command.

[sourcecode language=”bash”]
$ sudo erl
[/sourcecode]

Now when you install you should see the following result or something similar. You’ll be asked to continue, select lowercase ‘y’ to continue. It may be different for some, but when I hit uppercase ‘Y’ (I suppose I got overzealous to install QuickCheck) it finished as if I’d hit no or something else.

[sourcecode language=”bash”]
1> eqc_install:install().
Installation program for "Quviq QuickCheck" version 1.27.7.
Installing in directory /usr/local/lib/erlang/lib.
This will delete conflicting versions of QuickCheck, namely
[]
Proceed? y
Installing ["pulse-1.27.7","eqc-1.27.7","eqc_mcerlang-1.27.7"].
Quviq QuickCheck is installed successfully.
Looking in "/Users/adronhall"… .emacs not found
Could not find your .emacs file!
Try install("path-to-emacs-file") or install(new_emacs).
Bookmark the documentation at /usr/local/lib/erlang/lib/eqc-1.27.7/doc/index.html.
ok
[/sourcecode]

You’ll note above, I don’t currently have emacs installed. The reason it looks for emacs is because QuickCheck has templates/ops mode for emacs. So if you use emacs you’re in luck. I on the other hand, don’t, so I’ll just be using this from wherever I’m using it.

In addition to the lack of emacs, another important thing to note from the message is the link to documentation. Once you get this link open it up and check out the docs. They’re broken out into easily readily topic spaces and are a good place to do initial reference checking while you’re writing up your specs.

If you have a license, it is important to note, that if you’ve used sudo with your installation you’ll need to kill your running Erlang session and start it anew without sudo. Otherwise you’ll run into issue down the road trying to use the libs (unless of course you want to go hack on your permissions manually). Once you’re ready to register the software it’s simply one command, where xxxxx is your license key.

[sourcecode language=”bash”]
eqc:registration("xxxxxxxxxxxx").
[/sourcecode]

Alright, next time we’re on to next steps…

Urban Lean Agile Tech Breakfast Meetup, Be There This Wednesday!

Are you hard core into technology and software development like Node.js, JavaScript, Ruby, Rails, .NET, Java, Clojure and more?

Do you like the ideas behind the agile manifesto, lean startup, kanban, and thinking outside of the box?

Are you digging that ASP.NET MVC Framework or waiting for the next ALT.NET meetup?

Loving that ease of Ruby on Rails to wow your user base with ease, to implement with Sinatra those clean JavaScript & jQuery enabled UX for your clients?

Want to talk shop, eat some grub, have a beverage, and get a nerd kick start in the morning?

In that case meet us for Urban Lean Agile Tech Breakfast Meetup at Mod Pizza @ 1302 6th Avenue @ 8 am on Wednesday, August 3rd.

Windows Azure SDK Unit Testing Dilemma — F5DD Plz K Thx Bye

I’m a huge advocate for high quality code. I will admit I don’t always get to write, or am always able to write high quality code. But day in and out I make my best effort at figuring out the best way to write solid, high quality, easy to maintain, easy to read code.

Over the last year or so I’ve been working with Windows Azure (Amazon Web Services and other Cloud/Utility Platforms & Infrastructure also). One of the largest gaps that I’ve experienced when working with Windows Azure is the gross disregard for unit testing and especially unit testing in a Test Driven Development style way. The design of the SDK doesn’t make unit testing a high priority, and instead focuses mostly on what one might call F5 & Run Development.

I’ll be the first to stand up and point out why F5 Driven Development (for more on this, check out Jeff Schumacher‘s Blog Entry) is the slowest & distracting ways to build high quality code. I’d also be one to admit that F5 Development encourages poor design and development. A developer has to juggle far too many things to waste time hitting F5 every few seconds to assure that the build is running and code changes, additions, or deletions have been made correctly. If a developer disregards running the application when forced to do F5 Development the tendancy is to produce a lot of code, most likely not refactored or tested, during each run of the application. The list of reasons to not develop this way can get long pretty quick. A developer needs to be able to write a test, implement the code, and run the test without a framework launching the development fabric, or worse being forced to not write a test and running code that launches a whole development fabric framework.

Now don’t get me wrong, the development fabric is freaking AWESOME!! It is one of the things that really sets Windows Azure apart from other platforms and infrastructure models that one can develop to. But the level of work and effort makes effectively, cleanly, and intelligently unit testing code against Windows Azure with the development fabric almost impossible.

But with that context, I’m on a search to find some effective ways, with the current SDK limitations and frustrations, to write unit tests and encourage test driven design (TDD) or behaviour driven design (BDD) against Windows Azure, preferably using the SDK.

So far I’ve found the following methods of doing TDD against Windows Azure.

  • Don’t use the SDK. The easiest way to go TDD or BDD against Windows Azure and not being tightly bound to the SDK & Development Fabric is to ignore the SDK altogether and use regular service calls against the Windows Azure service end points. The problem with this however, is that it basically requires one rewrite all the things that the SDK wraps (albeit with better design principles). This is very time consuming but truly gives one absolute control over what they’re writing and also releases one from the issues/nuances that the Windows Azure SDK (1.3 comes to mind) has had.
  • Abstract, abstract, and abstract with a lock of stubbing, mocking, more stubbing, and some more abstractions underneath all of that to make sure the development fabric doesn’t kick off every time the tests are run.  I don’t want to abstract something just to fake, stub, or mock it.  The level of indirection needed gets a bit absurd because of the design issues with the SDK.  The big problem with this design process to move forward with TDD and BDD is that it requires the SDK to basically be rewritten as a whole virtual stubbed, faked, and mocked layer. Reminds me of many of the reasons the Entity Framework is so difficult to work with for testing (has the EF been cleaned up, opened up, and those nasty sealed classes removed yet??)

Now I’ll admit, sometimes I miss the obvious things and maybe there is a magic “build tests real easy right here” button for Windows Azure, but I haven’t found it.  I’d love to hear what else people are doing to enable good design principles around Windows Azure’s SDK. Any thoughts, ideas, or things I ought to try would be absolutely great – I’d love to read them. Please do comment!

Test Driven Development Built Name Generator Part 3

Part 1, Part 2.

Now it is time finally for the random name generation.  I’ve gotten everything into place, so it’s just a matter of grabbing the names and randomizing among them.  Just as a check, after Part 2 was finished I had a total of 98,572 rows of data, or simply 98,572 names.

Many times I would not write actual unit tests against the database as it breaks isolation.  This time though, I’m making an exception.  First I added a reference to Generator.Core, System.Data, and System.Data.Entity.  I then created a new unit test class called DatabaseTests and added the following test to get things rolling.  Keep in mind, since these tests break isolation, I don’t write them in the red light green light idea of TDD.  My assumption is that they only prove the database is setup, running, the connection string is correct, and other minor configuration based and server based components.  In addition, yes, these are somewhat silly tests, such as the RetrieveFirstName() test has a bit of redundancy to it, but the point is to have an automated test to assure that the data has been moved into the database and is returning appropriately.

[sourcecode language=”csharp”]
[TestClass]
public class DatabaseTests
{
[TestMethod]
public void RetrieveFirstFemaleName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Female First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveLastName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Last Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveFirstMaleName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Male First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty);
}

[TestMethod]
public void RetrieveFirstName()
{
var entities = new GeneratorEntities();
var returnName = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).FirstOrDefault();
Assert.IsNotNull(returnName);
Assert.IsTrue(returnName.Name != string.Empty && returnName.Type == "Female First Names" ||
returnName.Type == "Male First Names");
}
}
[/sourcecode]

The next step now was to refactor the existing tests to prove the assembly of the name data transfer objects that would hold the first name and last name derived from the returned entity objects.

The first thing I did was break out the actual name objects into their respective files.  I did this using ReSharper, because it’s super insanely simple with ReSharper.  After that I put the class files in the project that has the model.  Next I fixed the namespaces to Generator.Core.Model.  The class files after these changes looked like the code below.

IFullName.c

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public interface IFullName
{
string FirstName { get; set; }
string LastName { get; set; }
}
}
[/sourcecode]

FullName.cs

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public class FullName : IFullName
{
public FullName()
{
FirstName = string.Empty;
LastName = string.Empty;
}

#region IFullName Members

public string FirstName { get; set; }
public string LastName { get; set; }

#endregion
}
}
[/sourcecode]

NameFactory.cs

[sourcecode language=”csharp”]
namespace Generator.Core.Model
{
public class NameFactory
{
public static IFullName Build()
{
return new FullName {FirstName = "TestFirst", LastName = "TestLast"};
}
}
}
[/sourcecode]

Now that we have these files moved out to the project where all the munging will happen, I’m going to start the actual refactor of the tests.  What I’ve come up with for the test refactor is as follows.

[sourcecode language=”csharp”]
using Generator.Core.Model;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Factory.Tests
{
[TestClass]
public class FactoryNameBuilder
{
[TestMethod]
public void VerifyFullNameObjectInstantiatesWithNames()
{
var name = new FullName();
Assert.IsTrue(name.FirstName.Length == 0);
Assert.IsTrue(name.LastName.Length == 0);
}

[TestMethod]
public void VerifyFullNameObjectReturnsFromFactory()
{
var factory = new NameFactory();
IFullName name = factory.Build();
Assert.IsTrue(name.FirstName.Length > 0);
Assert.IsTrue(name.LastName.Length > 0);
}

[TestMethod]
public void VerifyFullNameIsRandom()
{
var factory = new NameFactory();
IFullName nameOne = factory.Build();
IFullName nameTwo = factory.Build();

Assert.AreNotEqual(nameOne.FirstName, nameTwo.FirstName);
Assert.AreNotEqual(nameOne.LastName, nameTwo.LastName);
}
}
}
[/sourcecode]

In the factory I removed the fake return and fleshed out the appropriate build method.

[sourcecode language=”csharp”]
using System;
using System.Collections.Generic;
using System.Linq;

namespace Generator.Core.Model
{
public class NameFactory
{
public IFullName Build()
{
Random rand = new Random(DateTime.Now.Millisecond);

GeneratorEntities entities = new GeneratorEntities();

List FirstNames = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).ToList();
List LastNames = (from name in entities.Names
where name.Type == "Last Names"
select name).ToList();

FullName fullName = new FullName();
fullName.FirstName = FirstNames[rand.Next(0, FirstNames.Count)].Name;
fullName.LastName = LastNames[rand.Next(0, LastNames.Count)].Name;

return fullName;
}
}
}
[/sourcecode]

Run the tests now and you have a random name gold mine.  However, I wanted a bit more than that.  With another test added…

[sourcecode language=”csharp”]
[TestMethod]
public void VerifyMultipleNameReturn()
{
var factory = new NameFactory();
List<IFullName> list = factory.Build(1000);

for (int i = 0; i < list.Count; i++)
{
int compareUp = i + 1;
if (compareUp == list.Count)
{
compareUp = 0;
}
Assert.AreNotEqual(list[compareUp].FirstName + list[compareUp].LastName, list[i].FirstName + list[i].LastName);
}
}
[/sourcecode]

…then the extra Build method.

[sourcecode language=”csharp”]
public List<IFullName> Build(int numberOfNames)
{
var rand = new Random(DateTime.Now.Millisecond);
var entities = new GeneratorEntities();

List<Names> FirstNames = (from name in entities.Names
where name.Type == "Female First Names" || name.Type == "Male First Names"
select name).ToList();
List<Names> LastNames = (from name in entities.Names
where name.Type == "Last Names"
select name).ToList();

var names = new List<IFullName>();

for (int i = 0; i < numberOfNames; i++)
{
var fullName = new FullName();
fullName.FirstName = FirstNames[rand.Next(0, FirstNames.Count)].Name;
fullName.LastName = LastNames[rand.Next(0, LastNames.Count)].Name;
names.Add(fullName);
}

return names;
}
[/sourcecode]

and now I have a strong verification that I get solid randoms for 1000 name pairs.  I bumped it up to 2000 too just for fun, and didn’t hit duplicates until I put it up to 200,000.