Building a Data Thrashing CLI Tool in Go

I need a tool just to do some testing against an API end point. I figured I’d throw one together real quick in Go. With a few libraries it’s just a few steps to get the job done. The following is that project. Eventually I’ll create the services that will run in some containers I’ll throw into a Kubernetes cluster, but for now, it’s all CLI. Onward.The first thing I’ll need is Cobra. Continue reading “Building a Data Thrashing CLI Tool in Go” →

Golang Execution on OS-X vs. Linux

Recently I was dorking about with some Go code on OS-X. Just working with it via iTerm 2 and using some basic go run whateverthefile.go commands and displaying results out the the shell. All was well, and then I dived into some code that used so Go Routines. The code looked something like this.

package main

import (
    "time"
    "fmt"
)

func say(s string) {
    for i := 0; i<5; i++ {
        time.Sleep(2 * time.Second)
        fmt.Println(s)
    }
}

func main() {
    go say("1. First thread is running.")
    go say("  2. Second thread is running.")
    say("    3. Thread is running in main.")
    fmt.Println("Ended.")
}

That was fine. I ran the code and the results were as expected.

OSX$   2. Second thread is running.
    3. Thread is running in main.
1. First thread is running.
1. First thread is running.
  2. Second thread is running.
    3. Thread is running in main.
    3. Thread is running in main.
1. First thread is running.
  2. Second thread is running.
  2. Second thread is running.
    3. Thread is running in main.
1. First thread is running.
1. First thread is running.
  2. Second thread is running.
    3. Thread is running in main.
Ended.

Then I ran it again. Also these results were expected.

OSX$     3. Thread is running in main.
  2. Second thread is running.
1. First thread is running.
    3. Thread is running in main.
1. First thread is running.
  2. Second thread is running.
  2. Second thread is running.
    3. Thread is running in main.
1. First thread is running.
1. First thread is running.
    3. Thread is running in main.
  2. Second thread is running.
    3. Thread is running in main.
Ended.
1. First thread is running.

You’ll notice the slightly out of order response as each thread completes. With the first set of results the last response is “Ended.” and with the second set of results “Ended.” is second to last. This of course is expected as the threads are likely to be executing while the last line of code is executed. But, there’s a catch to this depending where you run this code.

On the MacOS/OS-X platforms, when you run the shell it runs certain things with an assumed “&” after the command. Now, I’m not sure the specifics of what is happening (I’m looking it up and will happily post a super detailed reference link if you’ve got one, lemme know @Adron.) The “&” is however used on Linux/Unix systems to designate to the shell, run this command on the background so that the user remains in control of the shell. Specifically, from the man pages.

If a command is terminated by the control operator &, the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0.

The reason I bring this up, is if I run the same thing and those same results come back on Linux or Unix, you would not see anything after the “Ended.” runs, because the application terminates at that point and none of the background threads are actively connected to the shell. In other words the result where “Ended.” occurs second to last would look like this.

LINUX-SHELL$     3. Thread is running in main.
  2. Second thread is running.
1. First thread is running.
    3. Thread is running in main.
1. First thread is running.
  2. Second thread is running.
  2. Second thread is running.
    3. Thread is running in main.
1. First thread is running.
1. First thread is running.
    3. Thread is running in main.
  2. Second thread is running.
    3. Thread is running in main.
Ended.

At first glance, it might seem like the application has run successfully, but look closely and count the number of responses from each thread. In this example, the shell has lost priority when the “Ended.” message appears and there is only four messages that read “1. First thread is running.”. However, after the “Ended.” message there should have been another one, like the second example above that I ran on OS-X. On Linux, the solution to executing this example is actually to add the ampersand at the end of the command.

go run whateverthefile.go & and then you’ll see the execution of the program with the correct results.

Well that’s it for this often overlooked and confusing behavior on Linux versus OS-X. Happy hacking, cheers!

Addition/Correction April 3rd, 2017: It was brought to my attention that this is very specifically the shell, not the operating system. This is an important thing to note, and sometimes is easily confused or mistaken outright. The effect can even be different behavior between shell use on the same operating system, regardless of OS-X, Linux, UNIX, or whatever.

Back From Scandinavia, Back to Project Coding, Writing and Organizing

vikingI just got back from Scandinavia (and Amsterdam). I went for a million reasons, mostly for the adventure of it. Visiting Stockholm, Copenhagen and Reykjavik I saw about a zillion bikes, great architecture, Tivoli, amazing and beautiful waterways, Viking boat building museums, design to die for and so much more. It’s truly one of the amazing areas of the world. But now I’m back in ‘Merica and ready to get back to working on projects, design efforts and all the things I love to do. This blog post is a summary of my immediate return to projects, here’s the list broken into coding, writing and organizing:

Coding

  • Deconstructed – [site] This is the startup I’ve cofounded with Aaron Gray @agray. Check out our main site at Deconstructed. Check out some of the open source projects we’ve started here and listed below.
  • Deconstructed Docs – [site] [JavaScript] [Node.js] I’m using Wintersmith to build docs with static site generation. The docs are located at docs.deconstructed.io. Previous blog entries I did on building a static site with Wintersmith are available at Wintersmith Creating Documentation and Working in -34c, Wintersmith Customization & Github Hosting.
  • Symphonize.js – [site] [JavaScript] [Node.js] [issues] This is a project I started to use configuration as a basis for creating data for any database, but specifically Orchestrate (see blog entries under the writing section I did for Orchestrate). The idea behind this started since I needed something to generate test data for Deconstructed. This one is incomplete, but I’ll be pushing it forward to a deployable NPM Module soon that will be easy to download and just use. There’s also a possibility that this becomes a service that I make available in the near future.
  • Orchestrate.NET – [site] [c#] [issues] I’ve been helping out Robert Smith and a crew to manage the effort around the .NET client driver for Orchestrate. This is currently functional and we’d love anybody and everybody using it to really test it out. Currently I’m using this for the OrchestrateExecute Project listed below too.
  • orchestrate-rapping – [repo] [go] [issues] [group] Yo yo yo, hit the beat. This is an effort that I and others have kicked off to put together a wrapper for Orchestrate’s API. The reason is simple, we want to be able to develop sitting far away from wifi and connectivity, in a park or a cabin in the woods, with a beer in hand and a fire crackling. All while knowing we’re building something that will work when we reconnect to the land of the internet!
  • OrchestrateExecutive – [repo] [c#] [issues] For a very serious enterprise application, I’ve started hacking together a C# Application using Xamarin that will provide a library tier (that could be used as a sample) to use in building to Android, iOS or Windows Phone and all of the native Windows, Linux or OS-X apps that might be needed. In the application I’ll be using Orchestrate and Deconstructed to build out the application. Stay tuned at blog.deconstructed.io for more on this.
  • …and also inspired by Rick Turoczy @turoczy eternally another fucking side project will be going live soon. 😮

Writing

Organizing

  • Bike n’ Hack – Follow @bikenhack for information and more coming soon.
  • Node PDX – More to come on this soon.

…subscribe to the RSS link, hit the e-mail subscription or just ping me or follow me @adron on Twitter and I’ll keep you posted on the goings on of all my efforts and others. Cheers!

Orchestrate.io JavaScript Client Library

Today I’m starting a project working with Orchestrate.io’s API & open source software collaborations. More about the project in a moment, let’s get up to speed on what I’ll be including in this project. My main focus is to build a client library to access Orchestrate.io. During building this I’ll dive into the key value, graph and other storage mechanisms that the client library will provide. Beyond that, I’ll take a stroll through building an NPM library and the pertinent JavaScript the library. So buckle up, we’re going on a code slinging hash writing hacking session.

Over the course of putting together this material, I’ll be posting most of the core material on Orchestrate.io’s blog, so subscribe for updates as they come out. Feedly is a good option, connect via searching for “orchestrate.io” or navigate over to the Orchestrate.io blog itself. 😉

Project Effort Context

During building the client I’ll take a dive into who, what, where, when, why and how to interact with the various data structures. I’ll aim for the client to follow the model of the existing Go Client Library that is available at Orchestrate Go Client on Github. It follows a basic model as shown below in Go language.

[sourcecode language=”cpp”]
c := client.NewClient("Your API Key")
// Get a value
value, _ := c.Get("collection", "key")
// Put a value
c.Put("collection", "key", strings.NewReader("Some JSON"))
// Search
results, _ := c.Search("collection", "A Lucene Query")
// Get Events
events, _ := c.GetEvents("collection", "key", "kind")
// Put Event
c.PutEvent("collection", "key", "kind", strings.NewReader("Some JSON"))
// Get Relations
relations, _ := c.GetRelations("collection", "key", []string{"kind", "kind"})
// Put Relation
c.PutRelation("sourceCollection", "sourceKey", "kind", "sinkCollection", "sinkKey")
[/sourcecode]

I’ll be working on this client, but don’t hold back on me, feel free to jump in with some of your own code or telling me I wrote some code wrong or whatever. I’d gladly accept any committers jumping in to help out. The more we all work together the more useful information I can provide during this project.

Once this project has produced a workable client pending interest from the community I’ll put together some material about where, how and some best uses around using the client in your Node.js Application. Even prospectively build a JavaScript client side library prospectively for use with Angular or other popular client side libraries.

References