DSE6 + .NET v?

Project Repo: Interoperability Black Box

First steps. Let’s get .NET installed and setup. I’m running Ubuntu 18.04 for this setup and start of project. To install .NET on Ubuntu one needs to go through a multi-command process of keys and some other stuff, fortunately Microsoft’s teams have made this almost easy by providing the commands for the various Linux distributions here. The commands I ran are as follows to get all this initial setup done.

[sourcecode language=”bash”]
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg –dearmor > microsoft.asc.gpg
sudo mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
wget -q https://packages.microsoft.com/config/ubuntu/18.04/prod.list
sudo mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
sudo chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
sudo chown root:root /etc/apt/sources.list.d/microsoft-prod.list
[/sourcecode]

After all this I could then install the .NET SDK. It’s been so long since I actually installed .NET on anything that I wasn’t sure if I just needed the runtime, the SDK, or what I’d actually need. I just assumed it would be safe to install the SDK and then install the runtime too.

[sourcecode language=”bash”]
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install dotnet-sdk-2.1
[/sourcecode]

Then the runtime.

[sourcecode language=”bash”]
sudo apt-get install aspnetcore-runtime-2.1
[/sourcecode]

logoAlright. Now with this installed, I wanted to also see if Jetbrains Rider would detect – or at least what would I have to do – to have the IDE detect that .NET is now installed. So I opened up the IDE to see what the results would be. Over the left hand side of the new solution dialog, if anything isn’t installed Rider usually will display a message that X whatever needs installed. But it looked like everything is showing up as installed, “yay for things working (at this point)!

rider-01

Next up is to get a solution started with the pertinent projects for what I want to build.

dse2

Kazam_screenshot_00001

For the next stage I created three projects.

  1. InteroperationalBlackBox – A basic class library that will be used by a console application or whatever other application or service that may need access to the specific business logic or what not.
  2. InteroperationalBlackBox.Tests – An xunit testing project for testing anything that might need some good ole’ testing.
  3. InteroperationalBlackBox.Cli – A console application (CLI) that I’ll use to interact with the class library and add capabilities going forward.

Alright, now that all the basic projects are setup in the solution, I’ll go out and see about the .NET DataStax Enterprise driver. Inside Jetbrains Rider I can right click on a particular project that I want to add or manage dependencies for. I did that and then put “dse” in the search box. The dialog pops up from the bottom of the IDE and you can add it by clicking on the bottom right plus sign in the description box to the right. Once you click the plus sign, once installed, it becomes a little red x.

dse-adding-package

Alright. Now it’s almost time to get some code working. We need ourselves a database first however. I’m going to setup a cluster in Google Cloud Platform (GCP), but feel free to use whatever cluster you’ve got. These instructions will basically be reusable across wherever you’ve got your cluster setup. I wrote up a walk through and instructions for the GCP Marketplace a few weeks ago. I used the same offering to get this example cluster up and running to use. So, now back to getting the first snippets of code working.

Let’s write a test first.

[sourcecode language=”csharp”]
[Fact]
public void ConfirmDatabase_Connects_False()
{
var box = new BlackBox();
Assert.Equal(false, box.ConfirmConnection());
}
[/sourcecode]

In this test, I named the class called BlackBox and am planning to have a parameterless constructor. But as things go tests are very fluid, or ought to be, and I may change it in the next iteration. I’m thinking, at least to get started, that I’ll have a method to test and confirm a connection for the CLI. I’ve named it ConfirmConnection for that purpose. Initially I’m going to test for false, but that’s primarily just to get started. Now, time to implement.

[sourcecode language=”csharp”]
namespace InteroperabilityBlackBox
using System;
using Dse;
using Dse.Auth;

namespace InteroperabilityBlackBox
{
public class BlackBox
{
public BlackBox()
{}

public bool ConfirmConnection()
{
return false;
}
}
}
[/sourcecode]

That gives a passing test and I move forward. For more of the run through of moving from this first step to the finished code session check out this

By the end of the coding session I had a few tests.

[sourcecode language=”csharp”]
using Xunit;

namespace InteroperabilityBlackBox.Tests
{
public class MakingSureItWorksIntegrationTests
{
[Fact]
public void ConfirmDatabase_Connects_False()
{
var box = new BlackBox();
Assert.Equal(false, box.ConfirmConnection());
}

[Fact]
public void ConfirmDatabase_PassedValuesConnects_True()
{
var box = new BlackBox(“cassandra”, “”, “”);
Assert.Equal(false, box.ConfirmConnection());
}

[Fact]
public void ConfirmDatabase_PassedValuesConnects_False()
{
var box = new BlackBox(“cassandra”, “notThePassword”, “”);
Assert.Equal(false, box.ConfirmConnection());
}
}
}
[/sourcecode]

The respective code for connecting to the database cluster, per the walk through I wrote about here, at session end looked like this.

[sourcecode language=”csharp”]
using System;
using Dse;
using Dse.Auth;

namespace InteroperabilityBlackBox
{
public class BlackBox : IBoxConnection
{
public BlackBox(string username, string password, string contactPoint)
{
UserName = username;
Password = password;
ContactPoint = contactPoint;
}

public BlackBox()
{
UserName = “ConfigValueFromSecretsVault”;
Password = “ConfigValueFromSecretsVault”;
ContactPoint = “ConfigValue”;
}

public string ContactPoint { get; set; }
public string UserName { get; set; }
public string Password { get; set; }

public bool ConfirmConnection()
{
IDseCluster cluster = DseCluster.Builder()
.AddContactPoint(ContactPoint)
.WithAuthProvider(new DsePlainTextAuthProvider(UserName, Password))
.Build();

try
{
cluster.Connect();
return true;
}
catch (Exception e)
{
Console.WriteLine(e);
return false;
}

}
}
}
[/sourcecode]

With my interface providing the contract to meet.

[sourcecode language=”csharp”]
namespace InteroperabilityBlackBox
{
public interface IBoxConnection
{
string ContactPoint { get; set; }
string UserName { get; set; }
string Password { get; set; }
bool ConfirmConnection();
}
}
[/sourcecode]

Conclusions & Next Steps

After I wrapped up the session two things stood out that needed fixed for the next session. I’ll be sure to add these as objectives for the next coding session at 3pm PST on Thursday.

  1. The tests really needed to more resiliently confirm the integrations that I was working to prove out. My plan at this point is to add some Docker images that would provide the development integration tests a point to work against. This would alleviate the need for something outside of the actual project in the repository to exist. Removing that fragility.
  2. The application, in its “Black Box”, should do something. For the next session we’ll write up some feature requests we’d want, or maybe someone has some suggestions of functionality they’d like to see implemented in a CLI using .NET Core working against a DataStax Enterprise Cassandra Database Cluster? Feel free to leave a comment or three about a feature, I’ll work on adding it during the next session.

Chapter 2 in My Twitch Streaming

A while back I started down the path of getting a Twitch Channel started. At this point I’ve gotten a channel setup which I’ve dubbed Thrashing Code albeit it still just has “adronhall” all over it. I’ll get those details further refined as I work on it more.

Today I recorded a new Twitch stream about doing a twitch stream and created an edited video of all the pieces and cameras and angles. I could prospectively help people get started, it’s just my experiences so far and efforts to get everything connected right. The actual video stream recording is available, and I’ll leave it on the channel. However the video I edited will be available and I’ll post a link here.

Tomorrow will be my first official Twitch stream at 3pm PST. If you’re interested in watching check out my Twitch profile here follow and it’ll ping you when I go live. This first streaming session, or episode, or whatever you want to call it, will include a couple topics. I’ll be approaching these topics from that of someone just starting, so if you join help hold me to that! Don’t let me skip ahead or if you notice I left out something key please join and chat at me during the process. I want to make sure I’m covering all the bases as I step through toward achieving the key objectives. Which speaking of…

Tomorrow’s Mission Objectives

  1. Create a DataStax Enterprise Cassandra Cluster in Google Cloud Platform.
  2. Create a .NET project using the latest cross-platform magical bits that will have a library for abstracting the data source(s), a console interface for using the application, and of course a test project.
  3. Configure & connect to the distributed database cluster.

Mission Stretch Objectives

  1. Start a github repo to share the project with others.
  2. Setup some .github templates for feature request issues or related issues.
  3. Write up some Github Issue Feature requests and maybe even sdd some extra features to the CLI for…??? no idea ??? determine 2-3 during the Twitch stream.

If you’d like to follow along, here’s what I have installed. You’re welcome to a range of tooling to follow along with that is the same as what I’ve got here or a variance of other things. Feel free to bring up tooling if you’re curious about it via chat and I’ll answer questions where and when I can.

  • Ubuntu v18.04
  • .NET core v2.1
  • DataStax Enterprise v6

_____101 |> F# Coding Ecosystem: Paket && Atom w/ Paket

One extremely useful tool to use with F# is Paket. Paket is a package manager that provides a super clean way to manage your dependencies. Paket can handle everything from Nuget dependencies to git or file dependencies. It really opens up your project capabilities to easily pull in and handle dependencies, whereever they are located.

I cloned the Paket Project first, since I would like to have the very latest and help out if anything came up. For more information on Paket check out the about page.

[sourcecode language=”bash”]
git clone git@github.com:fsprojects/Paket.git
[/sourcecode]

I built that project with the respective ./build.sh script and all went well.

[sourcecode language=”bash”]./build.sh[/sourcecode]

NOTE – Get That Command Line Action

One thing I didn’t notice immediately in the docs (I’m putting in a PR right after this blog entry) was anyway to actually get Paket setup for the command line. On bash, Windows, or whatever, it seemed a pretty fundamental missing piece so I’m going to doc that right here but also submit a PR based on the issue I added here). It could be I just missed it, but either way, here’s the next step that will get you setup the rest of the way.

[sourcecode language=”bash”]./install.sh[/sourcecode]

Yeah, that’s all it was. Kind of silly eh? Maybe that’s why it isn’t documented that I could see? After the installation script is run, just execute paket and you’ll get the list of the various commands, as shown below.

[sourcecode language=”bash”]
$ paket
Paket version 1.31.1.0
Command was:
/usr/local/lib/paket/paket.exe
available commands:

add: Adds a new package to your paket.dependencies file.
config: Allows to store global configuration values like NuGet credentials.
convert-from-nuget: Converts from using NuGet to Paket.
find-refs: Finds all project files that have the given NuGet packages installed.
init: Creates an empty paket.dependencies file in the working directory.
auto-restore: Enables or disables automatic Package Restore in Visual Studio during the build process.
install: Download the dependencies specified by the paket.dependencies or paket.lock file into the `packages/` directory and update projects.
outdated: Lists all dependencies that have newer versions available.
remove: Removes a package from your paket.dependencies file and all paket.references files.
restore: Download the dependencies specified by the paket.lock file into the `packages/` directory.
simplify: Simplifies your paket.dependencies file by removing transitive dependencies.
update: Update one or all dependencies to their latest version and update projects.
find-packages: EXPERIMENTAL: Allows to search for packages.
find-package-versions: EXPERIMENTAL: Allows to search for package versions.
show-installed-packages: EXPERIMENTAL: Shows all installed top-level packages.
pack: Packs all paket.template files within this repository
push: Pushes all `.nupkg` files from the given directory.

–help [-h|/h|/help|/?]: display this list of options.
[/sourcecode]

Paket Elsewhere && Atom

If you’re interested in Paket with Visual Studio I’ll let you dig into that on your own. Some resources are Paket Visual Studio on Github and Paket for Visual Studio. What I was curious though was Paket integration with either Atom or Visual Studio Code.

Krzysztof Cieślak (@k_cieslak) and Stephen Forkmann (@sforkmann) maintain the Paket.Atom Project and Krzysztof Cieślak also handles the atom-fsharp project for Atom. Watch this gif for some of the awesome goodies that Atom gets with the Paket.Atom Plugin.

Click for fullsize image of the gif.
Click for fullsize image of the gif.

Getting Started and Adding Dependencies

I’m hacking along and want to add some libraries, how do I do that with Paket? Let’s take a look. This is actually super easy, and doesn’t make the project dependentant on peripheral tooling like Visual Studio when using Paket.

The first thing to do, is inside the directory or project where I need the dependency I’ll intialize the it for paket.

[sourcecode language=”bash”]paket init[/sourcecode]

The next step is to add the dependency or dependencies that I’ll need. I’ll add a Nuget package that I’ll need shortly. The first package I want to grab for this project is FsUnit, a testing framework project managed and maintained by Dan Mohl @dmohl and Sergey Tihon @sergey_tihon.

[sourcecode language=”bash”]paket add nuget FsUnit[/sourcecode]

When executing this dependency addition the results displayed show what other dependencies were installed and which versions were pegged for this particular dependency.

[sourcecode language=”bash”]
✔ ~/Codez/sharpPaketsExample
15:33 $ paket add nuget FsUnit
Paket version 1.33.0.0
Adding FsUnit to /Users/halla/Codez/sharpPaketsExample/paket.dependencies
Resolving packages:
– FsUnit 1.3.1
– NUnit 2.6.4
Locked version resolution written to /Users/halla/Codez/sharpPaketsExample/paket.lock
Dependencies files saved to /Users/halla/Codez/sharpPaketsExample/paket.dependencies
Downloading FsUnit 1.3.1 to /Users/halla/.local/share/NuGet/Cache/FsUnit.1.3.1.nupkg
NUnit 2.6.4 unzipped to /Users/halla/Codez/sharpPaketsExample/packages/NUnit
FsUnit 1.3.1 unzipped to /Users/halla/Codez/sharpPaketsExample/packages/FsUnit
3 seconds – ready.
[/sourcecode]

I took a look in the packet.dependencies and packet.lock file to see what were added for me with the paket add nuget command. The packet.dependencies file looked like this now.

[sourcecode language=”bash”]source https://nuget.org/api/v2

nuget FsUnit[/sourcecode]

The packet.lock file looked like this.

[sourcecode language=”bash”]NUGET
remote: https://nuget.org/api/v2
specs:
FsUnit (1.3.1)
NUnit (2.6.4)
NUnit (2.6.4)
[/sourcecode]

There are a few more dependencies that I want, so I went to work adding those. First of this batch that I added was FAKE (more on this in a subsequent blog entry), which is a build tool based off of RAKE.

[sourcecode language=”bash”]paket add nuget FAKE[/sourcecode]

Next up was FsCheck.

[sourcecode language=”bash”]paket add nuget FsCheck[/sourcecode]

The paket.dependencies file now had the following content.

[sourcecode language=”bash”]
source https://nuget.org/api/v2

nuget FAKE
nuget FsCheck
nuget FsUnit
[/sourcecode]

The paket.lock file had the following items added.

[sourcecode language=”bash”]
NUGET
remote: https://nuget.org/api/v2
specs:
FAKE (4.1.4)
FsCheck (2.0.7)
FSharp.Core (>= 3.1.2.5)
FSharp.Core (4.0.0.1)
FsUnit (1.3.1)
NUnit (2.6.4)
NUnit (2.6.4)
[/sourcecode]

Well, that got me started. The code repository at this state is located on this branch here of the sharpSystemExamples repository. So on to some coding and the next topic. Keep reading, subsribe, or hit me up on twitter @adron.

References

_____100 |> F# Some Troubleshooting Linux

In the last article I wrote on writing a code kata with F# on OS-X or Windows, I had wanted to use Linux but things just weren’t cooperating with me. Well, since that article I have resolved some of the issues I ran into, and this is the log of those issues.

Issue 1: “How can I resolve the “Could not fix timestamps in …” “…Error: The requested feature is not implemented.””

The first issue I ran into with running the ProjectScaffold build on Linux I wrote up and posted to Stack Overflow titled “How can I resolve the “Could not fix timestamps in …” “…Error: The requested feature is not implemented.”“. You can read more about the errors I receiving on the StackOverflow Article, but below is the immediate fix. This fix should probably be added to any F# Installation instructions for Linux as part of the default.

First ensure that you have the latest version of mono. If you use the instructions to do a make and make install off of the fsharp.org site you may not actually have the latest version of mono. Instead, here’s a good way to get the latest version of mono using apt-get. More information can be found about this on the mono page here.

[sourcecode language=”bash”]
apt-get install mono-devel
apt-get install mono-complete
[/sourcecode]

Issue 2: “ProjectScaffold Error on Linux Generating Documentation”

The second issue I ran into I also posted to Stack Overflow titled “ProjectScaffold Error on Linux Generating Documentation“. This one took a lot more effort. It also spilled over from Stack Overflow to become an actual Github Issue (323) on the project. So check out those issues in case you run into any issues there.

In the next issue, to be published tomorrow, I’ll have some script tricks to use mono more efficiently to run *.exe commands and get things done with paket and fake in F# running on any operating system.

______10 |> F# – Moar Thinking Functionally (Notes)

More notes on the “Thinking Functionally” series. Previous notes are @ “_______1 |> F# – Getting Started, Thinking Functionally“.

#6 Partial Application

Breaking down functions into single parameter functions is the mathematically correct way of doing it, but that is not the only reason it is done — it also leads to a very powerful technique called partial function application.

For example:

[sourcecode language=”fsharp”]
let add42 = (+) 42 // partial application
add42 1
add42 3

[1;2;3] |> List.map add42

let twoIsLessThan = (<) 2 // partial application twoIsLessThan 1 twoIsLessThan 3 // filter each element with the twoIsLessThan function [1;2;3] |> List.filter twoIsLessThan

let printer = printfn "printing param=%i"

[1;2;3] |> List.iter printer
[/sourcecode]

Each case a partially applied function above it can then be reused in multiple contexts. It can also fix function parameters.

[sourcecode language=”fsharp”]
let add1 = (+) 1
let add1ToEach = List.map add1 // fix the "add1" function

add1ToEach [1;2;3;4]

let filterEvens =
List.filter (fun i -> i%2 = 0) // fix the filter function

filterEvens [1;2;3;4]
[/sourcecode]

Then the following shows plug in behavior that is transparent.

[sourcecode language=”fsharp”]
let adderWithPluggableLogger logger x y =
logger "x" x
logger "y" y
let result = x + y
logger "x+y" result
result

let consoleLogger argName argValue =
printfn "%s=%A" argName argValue

let addWithConsoleLogger = adderWithPluggableLogger consoleLogger
addWithConsoleLogger 1 2
addWithConsoleLogger 42 99

let popupLogger argName argValue =
let message = sprintf "%s=%A" argName argValue
System.Windows.Forms.MessageBox.Show(
text=message,caption="Logger")
|> ignore

let addWithPopupLogger = adderWithPluggableLogger popupLogger
addWithPopupLogger 1 2
addWithPopupLogger 42 99
[/sourcecode]

Designing Functions for Partial Application

Sample calls to the list library:

[sourcecode language=”fsharp”]
List.map (fun i -> i+1) [0;1;2;3]
List.filter (fun i -> i>1) [0;1;2;3]
List.sortBy (fun i -> -i ) [0;1;2;3]
[/sourcecode]

Here are the same examples using partial application:

[sourcecode language=”fsharp”]
let eachAdd1 = List.map (fun i -> i+1)
eachAdd1 [0;1;2;3]

let excludeOneOrLess = List.filter (fun i -> i>1)
excludeOneOrLess [0;1;2;3]

let sortDesc = List.sortBy (fun i -> -i)
sortDesc [0;1;2;3]
[/sourcecode]

Commonly accepted guidelines to multi-parameter function design.

  1. Put earlier: parameters ore likely to be static. The parameters that are most likely to be “fixed” with partial application should be first.
  2. Put last: the data structure or collection (or most varying argument). Makes it easier to pipe a structure or collection from function to function. Like:

    [sourcecode language=”fsharp”]
    let result =
    [1..10]
    |> List.map (fun i -> i+1)
    |> List.filter (fun i -> i>5)
    [/sourcecode]

  3. For well-known operations such as “subtract”, put in the expected order.

Wrapping BCL Function for Partial Application

Since the data parameter is generally last versus most BCL calls that have the data parameter first, it’s good to wrap the BCL.

[sourcecode language=”fsharp”]
let replace oldStr newStr (s:string) =
s.Replace(oldValue=oldStr, newValue=newStr)

let startsWith lookFor (s:string) =
s.StartsWith(lookFor)
[/sourcecode]

Then pipes can be used with the BCL call in the expected way.

[sourcecode language=”fsharp”]
let result =
"hello"
|> replace "h" "j"
|> startsWith "j"

["the"; "quick"; "brown"; "fox"]
|> List.filter (startsWith "f")
[/sourcecode]

…or we can use function composition.

[sourcecode language=”fsharp”]
let compositeOp = replace "h" "j" >> startsWith "j"
let result = compositeOp "hello"
[/sourcecode]

Understanding the Pipe Function

The pipe function is defined as:

[sourcecode language=”fsharp”]
let (|>) x f = f x
[/sourcecode]

It allows us to put the function argument in front of the function instead of after.

[sourcecode language=”fsharp”]
let doSomething x y z = x+y+z
doSomething 1 2 3
[/sourcecode]

If the function has multiple parameters, then it appears that the input is the final parameter. Actually what is happening is that the function is partially applied, returning a function that has a single parameter: the input.

[sourcecode language=”fsharp”]
let doSomething x y =
let intermediateFn z = x+y+z
intermediateFn // return intermediateFn

let doSomethingPartial = doSomething 1 2
doSomethingPartial 3
3 |> doSomethingPartial
[/sourcecode]

#7 Function Associativity and Composition

Function Associativity

This…

[sourcecode language=”fsharp”]
let F x y z = x y z
[/sourcecode]

…means this…

[sourcecode language=”fsharp”]
let F x y z = (x y) z
[/sourcecode]

Also three equivalent forms.

[sourcecode language=”fsharp”]
let F x y z = x (y z)
let F x y z = y z |> x
let F x y z = x <| y z
[/sourcecode]

Function Composition

Here’s an example

[sourcecode language=”fsharp”]
let f (x:int) = float x * 3.0 // f is int->float
let g (x:float) = x > 4.0 // g is float->bool
[/sourcecode]

We can create a new function h that takes the output of “f” and uses it as the input for “g”.

[sourcecode language=”fsharp”]
let h (x:int) =
let y = f(x)
g(y) // return output of g
[/sourcecode]

A much more compact way is this:

[sourcecode language=”fsharp”]
let h (x:int) = g ( f(x) ) // h is int->bool

//test
h 1
h 2
[/sourcecode]

These are notes, to read more check out the Function Composition.