Category Archives: ThrashingCode

Learning Go Episode 5 – Functions (and Methods and lots of other things)

Episode Post & Video Links:  1, 2, 3, 4, 5 (this post), 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

In this session we covered a host of topics around Go Functions. Along with some troubleshooting, debugging, and other features in Jetbrains Goland IDE.

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan as a starting point. I’m using it as a simple guideline, but also doing a lot more in each stream that includes ecosystem, dependency management with godep, IDE use of Goland from Jetbrains, and more. In this session I get specifically into: functions, signatures, declarations, recursion, return values, and more.

Video Time Points & Topics

2:50 – Introduction to the snowy wonderland of Seattle and episode 5 of the Learning Go series. Introduction to the various screen transitions and such.
6:40 – Getting started, opening up JetBrains Goland and creating a new project. The project exists on Github as https://github.com/adron/learning-go-….
12:18 – Starting with functions in Go. See the blog entry I wrote on the topic for more additional information around this first code session within the episode 5 session.

Code – This first example I setup a basic function in Go that is called by the main function. The sample function below I’ve named exampleExecutor, and the signature is made up of an int parameter called this, a string parameter called that, and a return parameter of type int and one of type string. In summary for the function signature we have two input parameters going in and two return parameters coming out.

The function does very little besides print the parameters passed in and then return the parameters back out as the return parameters.

Recursion with Go & HTML Parsing

28:10 – Here I get into recursion and the application example, largely taken from the book but with some very distinctive modifications, that parses HTML and the various nodes within an HTML document.

For the recursion section I use an example from the book with an expanded sample set of HTML. The HTML is included in the repo under the function-recursion branch. For this example I setup a set of types and variables up that are needed throughout the code.

First a type setup called NodeType of type int32. A constant array of ErrorNode, TextNode, DocumentNode, ElementNode, CommentNode, and DoctypeNode for determining the different nodes within an HTML document. Then a general struct called Node with Type, Data, Attr for attribute, and FirstChild with NextSibling setup as a pointer to *Node, which gives a type of memory recursion to the underlying type. Then finally the Attribute struct with a Key a Value.

One of the first functions I then end up with is the visit function. It turns out as shown below. Here the function takes a links parameter that is of type string array, a parameter name n that is a pointer reference to an a node within html, and then the function returns a parameter of type string array.

After that function I worked through and created two additional functions, one for countsWordsAndImages and one called CountWordsAndImages. The casing being specific to scope and use of the functions, and they respectively look like this in completion.

Then all that is wrapped up, with recursive calls and more, in the main function for program execution.

Starting Error Handling && Anonymous Functions

1:32:40 – At this point in the episode 5 session I get into a simple Error handling function, and further into function signatures and how to set them up.
1:52:24 – Setting up some anonymous functions and reviewing what they are.
1:59:00 – Introduction to panics in Go. After this short introduction I also discuss some of the pedantic specifics of methods vs functions and related verbiage around the Go language. Additionally I provide more examples around these specifics for declaring functions, various scope, and other types for function calls and related usage.

With that done the wrap up of the session is then a short introduction to anonymous functions.

Learning Go Episode 2 – Further into packages, dependencies, application creation, and IDE’s

Episode Post & Video Links:  1, 2 (this post), 3, 4, 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

In episode two I went over a lot of the material that I covered in the first episode, but added more context, historical reasons certain things are the way they are in Go and the stack, and went over a number of new elements of information too. One thing I got further into this episode is the package and dependency management with Go Dep and also how to create a package, or dependency library for use in other Go libraries or applications. It’s just a small introduction in this episode, but pivotal to future episodes, as I’ll be jumping further into library creation and related details.

In this post I’ve got the time point breakdown like usual,  but also a few additional bits of information and code examples, plus links to the repository I’ve setup for this particular episode. The quick links to those references are below, and also I’ll link at particular call out points within the time points.

Quick Links:

Key Topics Covered

Data Types, Packages, and Dependency Management

2:52 – Fumbling through getting started. It is after all Saturday morning!
3:00 – Recap of what we covered in the first session. Includes a quick review of the previous session code too, such as the random data generation library we setup and used.
6:40 – Covering some specifics of the IDE’s, the stories of the benefits of Go having a specific and somewhat detailed convention to the way syntax, variables, and related features are used.
7:40 – Covering gofmt and what it gives us.
9:45 – Looking at the gofmt plugins and IDE features around the conventions.
14:06 – New example time! In this one, I work through an example showing how to find duplicate lines in passed in text.

Duplicate Line Finder

I went through the various steps of creating the code, but then took a little bit of a detour from the example in the book. Instead of lines by the CLI it takes in content from a text file. The code in main.go ended up like this.

Then if you’d like to check out the text file and remaining content in that project, check out the master branch of the episode 2 repo.

36:34 – Here I take a thorough step through committing this project to github, which is the included repo in this post. However I step through the interface of using Jetbrains Goland to do the commit, how it enables gofmt and other features to improve the condition of code and ensure it meets linter demands and related crtieria. I also cover the .gitignore file and other elements to create a usable repository.
44:30 – Setting up the repository for today’s code at https://github.com/Adron/learning-go-…
50:00 – Setup of the key again for using Github. How to setup your ssh keys using ssh-keygen.
56:00 – Going beyond just the language, and building out a Go build on Travis CI.
1:10:16 – Creating a new branch for the next code examples and topics. At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

The tempApp Branch is available in the repository here.

At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

During this point, we take a look at our first package. This package ended up looking like this.

In the main.go file, I showed how you can use this package by adding a respective import shown in this code.

1:17:54 – At this point, to increase readability of font sizes I get into the various Goland IDE options.
1:38:12 – Creating the final branch for this session to pull in a public package and use it in project. For this, I pull in a random data generation package to use in some application code.

1:44:50 – Further discussion and explanation of what to include in .gitignore files to manage projects, but also what is and isn’t included for dependencies and other details around all of this.
2:13:22 – The wicked awesome hacker outtro.

Learning Go Episode 1 – Environment, Go Workspace, GOPATH/GOROOT, Types, and more Introduction

Episode Post & Video Links:  1 (this post), 2, 3, 4, 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

This is episode one of a multi-part series on “The Go Programming Language“. Not necessary, but if you’d like to follow along you can also pick up the book “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan. At the bottom of the description I have a link to the book publisher’s website and the respective book. I’ll be using that as a guideline and using a number of examples from the book. However I’ll also be adding a lot of additional material around Goland IDE from Jetbrains and Visual Studio Code. The video link to this session is at the bottom of the post, scroll all the way down and it’s waiting for you there.

3:28 – Getting started, introducing that the session I’m starting with a completely new Ubuntu Linux load so that I ensure we cover all of the steps to get up and running. These steps, even though they’re on Linux are reproducible on Windows 10 and MacOS, so any operating system is usable to follow along with, with only minor discrepancies.

5:04 – Introducing the book that I’ll be using as a guideline reference so that viewers can also follow along with a physical book. I’m a big fan of multisensory learning, so between a book, the stream, being able to ask questions in the channel, it’ll give viewers a chance to learn and undertake their coding adventures in Go using all sorts of methods.

Book Reference: “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan

6:58 – Discussing where Go is located on the web related to Github and the golang.org site that is useful in that one can even try out little snippets of Go code itself, on the site!
Github: https://github.com/golang/go
Golang: https://golang.org
10:40 – Setting export in the .bashrc file (or .bash_profile on MacOS or environment variables on Windows 10). Speaking of Windows 10 specifically, Linda Gregier wrote up a great blog post on getting Go setup on Windows specifically.
14:50 – Setting up the Go workspace path for GOPATH using the standard Go convention. From here I get into the first “Hello World!” with Go.
15:34 – Mention of setting up Go on a Docker container and how it is easier, but we’re staying focused on setting it up completely from scratch.
18:20 – Starting first code, a standard “Hello World” program.
19:50 – First build of that “Hello World” program.
20:34 – Here I introduce go run and how to execute a singular file instead of building an entire project.
21:32 – Installing some IDE’s to use for developing Go applications. The first two up for installation is Visual Studio Code and JetBrains Goland.
29:00 – A first variable, what is it, and how to declare one in Go in one of the ways one can declare a variable in Go!
31:08 – Introducing the terminal in Visual Studio Code.
37:12 – A little example of OBS, how I’m using it, and how I interact back and forth with chat and related tooling plus the virtual machine itself.
42:36 – Changing themes and adding plugins for Goland. In the plugins I also find the most epic of status bars, the Nyan Cat!
59:00 – Here I start to get back into some more specific Go details. Starting off with a Go command line parsing application. At this point I also cover several additional ways to declare variables, speak more about short declarations, and other ways to declare, assign, and use variables in Go.

At this point I also go through a number of examples to exemplify how to go about declaring variables, build, run, and explore the details of the code. Further along I also get into string formatting, concatenating, and related string manipulation with Go.

Other details include taking a look at extra ways to figure out Go code using autocomplete inside Goland and other exploratory features. Eventually before wrapping up I broach pointers, tuple declaration techniques, and how to declare additional functions beyond func main().
1:58:40 – Adding dependencies and generating random data. At this point I bring in a dependency. But before pulling in the dependency, after introducing it, I show how to go about doing.
2:00:10 – New machine, and I run into git not being available. The go get command uses git to pull dependencies so I go about why this is needed and the steps to install on Ubuntu.
2:09:20 – Introduction to more concepts around dependencies, what go get does versus managing dependencies with Go Dep.
2:10:00 – Installing Go Dep; MacOS, using curl, Linux installation, and a question sort of remains to get it running on Windows. The easiest method is using chocolatey however, so check that out if you’re going the Windows route.
2:15:20 – Setting up Go Dep integration with Goland.
2:23:55 – Showing off Goland’s commit dialog and some of the respective options.

Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation

Part 2 of 3 – Coding Session in Go – Cobra + Viper CLI for Parsing Text Files, Retrieval of Twitter Data, Exports to various file formats, and export to Apache Cassandra.

UPDATED PARTS:

  1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
  2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
  3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

Updated links to each part will be posted at bottom of  this post when I publish them. For code, written walk through, and the like scroll down below the video and timestamps.

Hacking Together a CLI Installing Cassandra, Setting Up the Twitter API, ENV Vars, etc.

0:04 Kick ass intro. Just the standard rocking tune.

3:40 A quick recap. Check out the previous write “Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files” of this series.

4:30 Beginning of completion of twitz parse command for exporting out to XML, JSON, and CSV (already did the text export previous session). This segment also includes a number of refactorings to clean up the functions, break out the control structures and make the code more readable.

In the end of refactoring twitz parse came out like this. The completed list is put together by calling the buildTwitterList() function which is actually in the helpers.go file. Then prints that list out as is, and checks to see if a file export should be done. If there is a configuration setting set for file export then that process starts with a call to exportParsedTwitterList(exportFilename string, exportFormat string, ... etc ... ). Then a simple single level control if then else structure to determine which format to export the data to, and a call to the respective export function to do the actual export of data and writing of the file to the underlying system. There’s some more refactoring that could be done, but for now, this is cleaned up pretty nicely considering the splattering of code I started with at first.

50:00 I walk through a quick install of an Apache Cassandra single node that I’ll use for development use later. I also show quickly how to start and stop post-installation.

Reference: Apache Cassandra, Download Page, and Installation Instructions.

53:50 Choosing the go-twitter API library for Go. I look at a few real quickly just to insure that is the library I want to use.

Reference: go-twitter library

56:35 At this point I go through how I set a Twitter App within the API interface. This is a key part of the series where I take a look at the consumer keys and access token and access token secrets and where they’re at in the Twitter interface and how one needs to reset them if they just showed the keys on a stream (like I just did, shockers!)

57:55 Here I discuss and show where to setup the environment variables inside of Goland IDE to building and execution of the CLI. Once these are setup they’ll be the main mechanism I use in the IDE to test the CLI as I go through building out further features.

1:00:18 Updating the twitz config command to show the keys that we just added as environment variables. I set these up also with some string parsing and cutting off the end of the secrets so that the whole variable value isn’t shown but just enough to confirm that it is indeed a set configuration or environment variable.

1:16:53 At this point I work through some additional refactoring of functions to clean up some of the code mess that exists. Using Goland’s extract method feature and other tooling I work through several refactoring efforts that clean up the code.

1:23:17 Copying a build configuration in Goland. A handy little thing to know you can do when you have a bunch of build configuration options.

1:37:32 At this part of the video I look at the app-auth example in the code library, but I gotta add the caveat, I run into problems using the exact example. But I work through it and get to the first error messages that anybody would get to pending they’re using the same examples. I get them fixed however in the next session, this segment of the video however provides a basis for my pending PR’s and related work I’ll submit to the repo.

The remainder of the video is trying to figure out what is or isn’t exactly happening with the error.

I’ll include the working findem code in the next post on this series. Until then, watch the wrap up and enjoy!

1:59:20 Wrap up of video and upcoming stream schedule on Twitch.

UPDATED SERIES PARTS

    1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
    2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
    3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

 

Behind the Scenes @ DevRel Week in Santa Clara & San Francisco

The following are some lagniappe, a little extra, about the behind the scenes adventures I’m off on when I travel or am in between coding. Ya know, coding being life and all.  😉

The first episode in this series I posted a while back on my gear I use to record the Twitch sessions and pretty much everything else. These are the story of the first half of the trip to Santa Clara and San Francisco. The rest, are still in post-production, and will be out real soon. Along with videos on a host of other adventures that will offer you good information on where the good food is, the best coding places, best meetups, and all that stuff. So subscribe on my Youtube Channel and on Twitch – the shows are coming to Youtube, and now and again I’ll pre-watch one with my Twitch audience. Cheers!

Setting Up Nodes, Firewall, & Instances in Google Cloud Platform

Here’s the run down of what I covered in the latest Thrashing Code Session (go subscribe here to the channel for future sessions or on Twitch). The core focus of this session was getting some further progress on my Terraform Project around getting a basic Cassandra and DataStax Enterprise Apache Cassandra Cluster up and running in Google Cloud Platform.

The code and configuration from the work is available on Github at terraform-fields and a summary of code changes and other actions taken during the session are further along in this blog entry.

Streaming Session Video

In this session I worked toward completing a few key tasks for the Terraform project around creating a Cassandra Cluster. Here’s a run down of the time points where I tackle specific topics.

  • 3:03 – Welcome & objectives list: Working toward DataStax Enterprise Apache Cassandra Cluster and standard Apache Cassandra Cluster.
  • 3:40 – Review of what infrastructure exists from where we left off in the previous episode.
  • 5:00 – Found music to play that is copyright safe! \m/
  • 5:50 – Recap on where the project lives on Github in the terraformed-fields repo.
  • 8:52 – Adding a google_compute_address for use with the instances. Leads into determining static public and private google_compute_address resources. The idea being to know the IP for our cluster to make joining them together easier.
  • 11:44 – Working to get the access_config and related properties set on the instance to assign the google_compute_address resources that I’ve created. I run into a few issues but work through those.
  • 22:28 – Bastion server is set with the IP.
  • 37:05 – I setup some files, following a kind of “bad process” as I note. Which I’ll refactor and clean up in a subsequent episode. But the bad process also limits the amount of resources I have in one file, so it’s a little easier to follow along.
  • 54:27 – Starting to look at provisioners to execute script files and commands before or after the instance creation. Super helpful, with the aim to use this feature to download and install the DataStax Enterprise Apache Cassandra or standard Apache Cassandra software.
  • 1:16:18 – Ah, a need for firewall rule for ssh & port 22. I work through adding those and then end up with an issue that we’ll be resolving next episode!

Session Content

Starting Point: I started this episode from where I left off last session.

Work Done: In this session I added a number of resources to the project and worked through a number of troubleshooting scenarios, as one does.

Added firewall resources to open up port 22 and icmp (ping, etc).

resource "google_compute_firewall" "bastion-ssh" {
  name    = "gimme-bastion-ssh"
  network = "${google_compute_network.dev-network.name}"

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }
}

resource "google_compute_firewall" "bastion-icmp" {
  name    = "gimme-bastion-icmp"
  network = "${google_compute_network.dev-network.name}"

  allow {
    protocol = "icmp"
  }
}

I also broke out the files so that each instances has its own IP addresses with it in the file specific to that instance. Later I’ll add context for why I gave the project file bloat, by refactoring to use modules.

terraform-files.png

Added each node resource as follows. I just increased each specific node count by one for each subsequent node, such as making this node1_internal IP google_compute_address increment to node2_internal. Everything also statically defined, adding to my file and configuration bloat.

resource "google_compute_address" "node1_internal" {
  name         = "node-1-internal"
  subnetwork   = "${google_compute_subnetwork.dev-sub-west1.self_link}"
  address_type = "INTERNAL"
  address      = "10.1.0.5"
}

resource "google_compute_instance" "node_frank" {
  name         = "frank"
  machine_type = "n1-standard-1"
  zone         = "us-west1-a"

  boot_disk {
    initialize_params {
      image = "ubuntu-minimal-1804-bionic-v20180814"
    }
  }

  network_interface {
    subnetwork = "${google_compute_subnetwork.dev-sub-west1.self_link}"
    address    = "${google_compute_address.node1_internal.address}"
  }

  service_account {
    scopes = [
      "userinfo-email",
      "compute-ro",
      "storage-ro",
    ]
  }
}

I also setup the bastion server so it looks like this. Specifically designating a public IP so that I can connect via SSH.

resource "google_compute_address" "bastion_a" {
  name = "bastion-a"
}

resource "google_compute_instance" "a" {
  name         = "a"
  machine_type = "n1-standard-1"
  zone         = "us-west1-a"

  provisioner "file" {
  source      = "install-c.sh"
  destination = "install-c.sh"

  connection {
    type     = "ssh"
    user     = "root"
    password = "${var.root_password}"
    }
  }

  boot_disk {
    initialize_params {
      image = "ubuntu-minimal-1804-bionic-v20180814"
    }
  }

  network_interface {
    subnetwork = "${google_compute_subnetwork.dev-sub-west1.self_link}"
    access_config {
      nat_ip = "${google_compute_address.bastion_a.address}"
    }
  }

  service_account {
    scopes = [
      "userinfo-email",
      "compute-ro",
      "storage-ro",
    ]
  }
}

Plans for next session include getting the nodes setup so that the bastion server can work with and deploy or execute commands against them without the nodes being exposed publicly to the internet. We’ll talk more about that then. For now, happy thrashing code!

Thrashing Code Twitch Schedule September 19th-October 2nd

I’ve got everything queued back up with some extra Thrashing Code Sessions and will have some on the rails travel streams. Here’s what the schedule looks like so far.

Today at 3pm PST (UPDATED: Sep 19th 2018)

thrashing-code-terraformUPDATED: Video available https://youtu.be/NmlzGKUnln4

I’m going to get back into the roll of things this session after the travels last week. In this session I’m aiming to do several things:

  1. Complete next steps toward getting a DataStax Enterprise Apache Cassandra cluster up and running via Terraform in Google Cloud Platform. My estimate is I’ll get to the point that I’ll have three instances that launch and will automate the installation of Cassandra on the three instances. Later I’ll aim to expand this, but for now I’m just going to deploy 3 nodes and then take it from there. Another future option is to bake the installation into a Packer deployed image and use it for the Terraform execution. Tune in to find out the steps and what I decide to go with.
  2. I’m going to pull up the InteroperabilityBlackBox and start to flesh out some objects for our model. The idea, is based around something I stumbled into last week during travels, the thread on that is here.

Friday (Today) @ 3pm PST

thrashing-code-gopherThis Friday I’m aiming to cover some Go basics before moving further into the Colligere CLI  app. Here are the highlights of the plan.

  1.  I’m going to cover some of the topics around program structure including: type declarations, tuple assignment, variable lifetime, pointers, and other variables.
  2.  I’m going to cover some basics on packages, initialization of packages, imports, and scope. This is an important aspect of ongoing development with Colligere since we’ll be pulling in a number of packages for generation of the data.
  3. Setting up configuration and schema for the Colligere application using Viper and related tooling.

Tuesday, October 2nd @ 3pm PST

thrashing-code-terraformThis session I’m aiming to get some more Terraform work done around the spin up and shutdown of the cluster. I’ll dig into some more specific points depending on where I progress to in sessions previous to this one. But it’s on the schedule, so I’ll update this one in the coming days.