Tag Archives: programming

Learning Go Episode 3 – More Data Types, Casting, Rendering an SVG file, and writing to Files.

Episode Post & Video Links:  1, 2, 3 (this post), 4 (almost done)

Episode 3 of my recurring “Learning Go” Saturday stream really got more into the particulars of Go data types including integers, strings, more string formatting verbs, concatenation, type casting, and lots of other pedantic details. In the episode I also delve into some OBS details with the audience, and we get the Twitch interface I’ve setup a bit more streamlined for easier readability. Overall, I think it’s looking much better than just the last episode! Hats off to the conversational assist from the audience.

Here’s the play by play of what was covered in episode 3 with the code in gists plus the repo is available on Github. Video below the timeline.

Timeline

0:00 Intro
6:08 The point I fix the sound. Just skip that first bit!
6:24 Re-introducing the book I’m using as a kind of curriculum guide for these go learning sessions.
7:44 Quick fix of the VM, a few updates, discussion of Goland updates, and fixing the Material Theme to a more visually less caustic theme. Also showing where it is in the IDE.
9:52 Getting into the learning flow, starting a new project with Go 1.11.4 using the Goland IDE new project dialog.


10:50 Creating the Github repo for learning-go-episode-3.
12:14 Setting up the initial project and CODING! Finally getting into some coding. It takes a while when we do it from nothing like this, but it’s a fundamentally important part of starting a project!
13:04 From nothing, creating the core basic elements of a code file for go with main.go. In this part I start showing the various ways to declare types, such as int and int64 with options on style.
14:14 Taking a look at printing out the various values of the variables using formatter verbs via the fmt.Printf function.
17:00 Looking at converting values from one type to another type. There are a number of ways to do this in Go.

I also, just recently, posted a quick spot video and code (blog entry + code) on getting the minimum and maximum value in Go for a number of types. This isn’t the course video, just a quick spot. Keep reading for the main episode below.


18:16 Oh dear the mouse falls on the ground. The ongoing battle of streaming, falling objects! But yeah, I get into adding a function – one of the earlier functions being built in the series – and we add a signature with a return int64 value. I continue, with addition of another function and looking at specifics of the signature.
25:50 Build this code and take a look at the results. At this point, some of the formatting is goofed up so I take a look into the formatter verbs to figure out what should be used for the output formatting.
33:40 I change a few things and take a look at more output from the various calculations that I’ve made, showing how various int, int64, and related calculations can be seen.
37:10 Adding a constant, what it is, and when and where and why to declare something as a constant.
38:05 Writing out another for loop for output results of sets.
42:40 A little git work to create a branch, update the .gitignore, and push the content to github. Repo is here btw: https://github.com/Adron/learning-go-episode-3

At this point I had to take a short interruption to get my ssh keys setup for this particular VM so I could push the code! I snagged just a snippet of the video and made a quick spot video out of it too. Seems a useful thing to do.

47:44 Have to add a new ssh key for the virtual machine to github, so this is a good little snippet of a video showing how that is done.
56:38 Building out a rendering of an SVG file to build a graphic. The complete snippet is below, watch the video for more details, troubleshooting, and working through additions and refactoring of the code.

1:15:32 We begin the mission of bumping up the font size in Goland. It’s a little tricky but we get it figured out.
1:33:20 Upon realization, we need to modify for our work, that this outputs directly to a file instead of just the console. Things will work better that way so I work into the code a write out to file.
1:40:05 Through this process of changing it to output to file, I have to work through additional string conversions, refactoring, and more. There’s a lot of nuance and various things to learn during this section of the video, albeit a little slow. i.e. LOTS of strconv usage.
2:01:24 First view of the generated SVG file! Yay! Oh dear!
2:09:10 More troubleshooting to try and figure out where the math problem is!
2:22:50 Wrapping up with the math a little off kilter, but sort of fixed, I move on to getting a look into the build but also pushing each of the respective branches on github. Repo is here btw: https://github.com/Adron/learning-go-episode-3

Learning Go Episode 1 – Environment, Go Workspace, GOPATH/GOROOT, Types, and more Introduction

Episode Post & Video Links:  1 (this post), 2, 3, 4 (almost done)

This is episode one of a multi-part series on “The Go Programming Language“. Not necessary, but if you’d like to follow along you can also pick up the book “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan. At the bottom of the description I have a link to the book publisher’s website and the respective book. I’ll be using that as a guideline and using a number of examples from the book. However I’ll also be adding a lot of additional material around Goland IDE from Jetbrains and Visual Studio Code. The video link to this session is at the bottom of the post, scroll all the way down and it’s waiting for you there.

3:28 – Getting started, introducing that the session I’m starting with a completely new Ubuntu Linux load so that I ensure we cover all of the steps to get up and running. These steps, even though they’re on Linux are reproducible on Windows 10 and MacOS, so any operating system is usable to follow along with, with only minor discrepancies.

5:04 – Introducing the book that I’ll be using as a guideline reference so that viewers can also follow along with a physical book. I’m a big fan of multisensory learning, so between a book, the stream, being able to ask questions in the channel, it’ll give viewers a chance to learn and undertake their coding adventures in Go using all sorts of methods.

Book Reference: “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan

6:58 – Discussing where Go is located on the web related to Github and the golang.org site that is useful in that one can even try out little snippets of Go code itself, on the site!
Github: https://github.com/golang/go
Golang: https://golang.org
10:40 – Setting export in the .bashrc file (or .bash_profile on MacOS or environment variables on Windows 10). Speaking of Windows 10 specifically, Linda Gregier wrote up a great blog post on getting Go setup on Windows specifically.
14:50 – Setting up the Go workspace path for GOPATH using the standard Go convention. From here I get into the first “Hello World!” with Go.
15:34 – Mention of setting up Go on a Docker container and how it is easier, but we’re staying focused on setting it up completely from scratch.
18:20 – Starting first code, a standard “Hello World” program.
19:50 – First build of that “Hello World” program.
20:34 – Here I introduce go run and how to execute a singular file instead of building an entire project.
21:32 – Installing some IDE’s to use for developing Go applications. The first two up for installation is Visual Studio Code and JetBrains Goland.
29:00 – A first variable, what is it, and how to declare one in Go in one of the ways one can declare a variable in Go!
31:08 – Introducing the terminal in Visual Studio Code.
37:12 – A little example of OBS, how I’m using it, and how I interact back and forth with chat and related tooling plus the virtual machine itself.
42:36 – Changing themes and adding plugins for Goland. In the plugins I also find the most epic of status bars, the Nyan Cat!
59:00 – Here I start to get back into some more specific Go details. Starting off with a Go command line parsing application. At this point I also cover several additional ways to declare variables, speak more about short declarations, and other ways to declare, assign, and use variables in Go.

At this point I also go through a number of examples to exemplify how to go about declaring variables, build, run, and explore the details of the code. Further along I also get into string formatting, concatenating, and related string manipulation with Go.

Other details include taking a look at extra ways to figure out Go code using autocomplete inside Goland and other exploratory features. Eventually before wrapping up I broach pointers, tuple declaration techniques, and how to declare additional functions beyond func main().
1:58:40 – Adding dependencies and generating random data. At this point I bring in a dependency. But before pulling in the dependency, after introducing it, I show how to go about doing.
2:00:10 – New machine, and I run into git not being available. The go get command uses git to pull dependencies so I go about why this is needed and the steps to install on Ubuntu.
2:09:20 – Introduction to more concepts around dependencies, what go get does versus managing dependencies with Go Dep.
2:10:00 – Installing Go Dep; MacOS, using curl, Linux installation, and a question sort of remains to get it running on Windows. The easiest method is using chocolatey however, so check that out if you’re going the Windows route.
2:15:20 – Setting up Go Dep integration with Goland.
2:23:55 – Showing off Goland’s commit dialog and some of the respective options.

 

Creating Distributed Database Application Starter Kits

I’ve boarded a bus, and as always, when I board a bus I almost always code. Unless of course there are people I’m hanging out with then I chit chat, but right now this is the 212 and I don’t know anybody on this chariot anyway. So into the code I go.

I’ve been re-reviewing the Docker and related collateral we offer at DataStax. In that review it seems like it would be worth having some starter kit applications along with these “default” Docker options. This post I’ve created to provide the first language & tech stack of several starter kits I’m going to create.

Starter Kit – The Todo List Template

This first set of starter kits will be based upon a todo list application. It’s really simple, minimal in features, and offers a complete top to bottom implementation of a service, and an application on top of that service all built on Apache Cassandra. In some places, and I’ll clearly mark these places, I might add a few DataStax Enterprise features around search, analytics, or graph.

The Todo List

Features: The following detail the features, from the users perspective, that this application will provide. Each implementation will provide all of these features.

  • A user wants to create a user account to create todo lists with.
  • A user wants to be able to store a username, full name, email, and some simple notes with their account.
  • A user wants to be able to create a todo list that is identified by a user defined name. (i.e. “Grocery List”, “Guitar List”, or “Stuff to do List”)
  • A user want to be able to logout and return, then retrieve a list from a list of their lists.
  • A user wants to be able to delete a todo list.
  • A user wants to be able to update a todo list name.
  • A user wants to be able to add items to a todo list.
  • A user wants to be able to update items in the todo list.
  • A user wants to be able to delete items in a todo list.

Architecture: The following is the architecture of the todo list starter kit application.

  • Database: Apache Cassandra.
  • Service: A small service to manage the data tier of the application.
  • User Interface: A web interface using React/Vuejs ??

As you can see, some of the items are incomplete, but I’ll decide on them soon. My next review is to check out what I really want to use for the user interface, and also to get a user account system figured out. I don’t really want to create the entire user interface, but instead would like to use something like Auth0 or Okta.

May I Ask?

There are numerous things I’d love help with. Are there any user stories you think are missing? Should I add something? What would make these helpful to you? Leave a comment, or tweet at me @Adron. I’d be happy to get some feedback and other’s thoughts on the matter so that I can ensure that these are simple, to the point, usable, and helpful to people. Cheers!

Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation

Part 2 of 3 – Coding Session in Go – Cobra + Viper CLI for Parsing Text Files, Retrieval of Twitter Data, Exports to various file formats, and export to Apache Cassandra.

UPDATED PARTS:

  1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
  2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
  3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

Updated links to each part will be posted at bottom of  this post when I publish them. For code, written walk through, and the like scroll down below the video and timestamps.

Hacking Together a CLI Installing Cassandra, Setting Up the Twitter API, ENV Vars, etc.

0:04 Kick ass intro. Just the standard rocking tune.

3:40 A quick recap. Check out the previous write “Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files” of this series.

4:30 Beginning of completion of twitz parse command for exporting out to XML, JSON, and CSV (already did the text export previous session). This segment also includes a number of refactorings to clean up the functions, break out the control structures and make the code more readable.

In the end of refactoring twitz parse came out like this. The completed list is put together by calling the buildTwitterList() function which is actually in the helpers.go file. Then prints that list out as is, and checks to see if a file export should be done. If there is a configuration setting set for file export then that process starts with a call to exportParsedTwitterList(exportFilename string, exportFormat string, ... etc ... ). Then a simple single level control if then else structure to determine which format to export the data to, and a call to the respective export function to do the actual export of data and writing of the file to the underlying system. There’s some more refactoring that could be done, but for now, this is cleaned up pretty nicely considering the splattering of code I started with at first.

50:00 I walk through a quick install of an Apache Cassandra single node that I’ll use for development use later. I also show quickly how to start and stop post-installation.

Reference: Apache Cassandra, Download Page, and Installation Instructions.

53:50 Choosing the go-twitter API library for Go. I look at a few real quickly just to insure that is the library I want to use.

Reference: go-twitter library

56:35 At this point I go through how I set a Twitter App within the API interface. This is a key part of the series where I take a look at the consumer keys and access token and access token secrets and where they’re at in the Twitter interface and how one needs to reset them if they just showed the keys on a stream (like I just did, shockers!)

57:55 Here I discuss and show where to setup the environment variables inside of Goland IDE to building and execution of the CLI. Once these are setup they’ll be the main mechanism I use in the IDE to test the CLI as I go through building out further features.

1:00:18 Updating the twitz config command to show the keys that we just added as environment variables. I set these up also with some string parsing and cutting off the end of the secrets so that the whole variable value isn’t shown but just enough to confirm that it is indeed a set configuration or environment variable.

1:16:53 At this point I work through some additional refactoring of functions to clean up some of the code mess that exists. Using Goland’s extract method feature and other tooling I work through several refactoring efforts that clean up the code.

1:23:17 Copying a build configuration in Goland. A handy little thing to know you can do when you have a bunch of build configuration options.

1:37:32 At this part of the video I look at the app-auth example in the code library, but I gotta add the caveat, I run into problems using the exact example. But I work through it and get to the first error messages that anybody would get to pending they’re using the same examples. I get them fixed however in the next session, this segment of the video however provides a basis for my pending PR’s and related work I’ll submit to the repo.

The remainder of the video is trying to figure out what is or isn’t exactly happening with the error.

I’ll include the working findem code in the next post on this series. Until then, watch the wrap up and enjoy!

1:59:20 Wrap up of video and upcoming stream schedule on Twitch.

UPDATED SERIES PARTS

    1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
    2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
    3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

 

New Live Coding Streams and Episodes!

I’ve been working away in Valhalla on the next episodes of Thrashing Code TV and subsequent content for upcoming Thrashing Code Sessions on Twitch (follow) and Youtube (subscribe). The following I’ve broken into the main streams and shows that I’ll be putting together over the next days, weeks, and months and links to sessions and shows already recorded. If you’ve got any ideas, questions, thoughts, just send them my way.

Colligere (Next Session)

Coding has been going a little slow, in light of other priorities and all, but it’ll still be one of the featured projects I’ll be working on. Past episodes are available here, however join in on Friday and I’ll catch everybody up, so you can skip past episodes if you aren’t after specific details and just want to join in on future work and sessions.

In this next session, this Friday the 9th at 3:33pm PST, I’m going to be working on reading in JSON, determining what type of structure the JSON should be unmarshalled into, and how best to make that determination through logic and flow.

Since Go needs something to unmarshall JSON into, a specific structure, I’ll be working on determining a good way to pre-read information in the schema configuration files (detailed in the issue listed below) so that a logic flow can be implemented that will then begin the standard Go JSON unmarshalling of the object. So this will likely end up including some hackery around reading in JSON without the assistance of the Go JSON library. Join in and check out what solution(s) I come up with.

The specific issue I’ll be working on is located on Github here. These sessions I’m going to continue working on, but will be a little vague and will start working on the Colligere CLI primarily on Saturday’s at 10am. So you can put that on your schedule and join me then for hacks. If you’d like to contribute, as always, reach out via here, @Adron, or via the Github Colligere Repository and let’s discuss what you’d like to add.

Getting Started with Go

This set of sessions, which I’ve detailed in “Getting Started with Go“, I’ll be starting on January 12th at 4pm PST. You can get the full outline and further details of what I’ll be covering on my “Getting Started with Gopage and of course the first of these sessions I’ve posted details on the Twitch event page here.

  • Packages & the Go Tool – import paths, package declarations, blank imports, naming, and more.
  • Structure – names, declarations, variables, assignments, scope, etc., etc.
  • Basic Types – integers, floats, complex numbers, booleans, strings, and constants.

Infrastructure as Code with Terraform and Apache Cassandra

I’ll be continuing the Terraform, bash, and related configuration and coding of using infrastructure as code practices to build out, maintain, and operate Apache Cassandra distributed database clusters. At some point I’ll likely add Kubernetes, some additional on the metal cluster systems and start looking at Kubernetes Operators and how one can manage distributed systems on Kubernetes using this on the metal environment. But for now, these sessions will continue real soon as we’ve got some systems to build!

Existing episodes of this series you can check out here.

Getting Started with Multi-model Databases

This set of sessions I’ve detailed in “Getting Started with a Multi-model Database“, and this one I’ll be starting in the new years also. Here’s the short run down of the next several streams. So stay tuned, subscribe or follow my Twitch and Youtube and of course subscribe to the Composite Code blog (should be to the left, or if on mobile click the little vertical ellipses button)

  1. An introduction to a range of databases: Apache Cassandra, Postgresql and SQL Server, Neo4j, and … in memory database. Kind of like 7 Databases in 7 Weeks but a bunch of databases in just a short session!
  2. An Introduction – Apache Cassandra and what it is, how to get a minimal cluster started, options for deploying something quickly to try it out.
  3. Adding to Apache Cassandra with DataStax Enterprise, gaining analytics, graph, and search. In this session I’ll dive into what each of these capabilities within DataStax Enterprise give us and how the architecture all fits together.
  4. Deployment of Apache Cassandra and getting a cluster built. Options around ways to effectively deploy and maintain Apache Cassandra in production.
  5. Moving to DataStax Enterprise (DSE) from Apache Cassandra. Getting a DSE Cluster up and running with OpsCenter, Lifecycle Manager (LCM), and getting some queries tried out with Studio.

Top 10 West Coast Confs for 2019

I’ve been putting together a list of conferences that I want to aim to attend this coming year. I made it, then thought, “somebody else could use this list probably” so here it is. If you think of any other specific conferences I ought to add and attend please leave a comment. Enjoy!

March 7-10 is SCALE Southern California Linux Expo in Pasadena, California

March 25-28 is O’Reilly Strata in San Francisco, California

April 26-28 is LinuxFest Northwest in Bellingham, Washington

June 3-5 is Monitorama in Portland, Oregon

June 10-13 is O’Reilly Velocity in San Jose, California

June 10-13 is O’Reilly Software Architecture Conference SACON in San Jose, California

July 15-18 is O’Reilly OSCON in Portland, Oregon

August 21-23 is the Open Source Summit in San Diego, California

September 9-12 is the O’Reilly Artificial Intelligence in San Jose, California

November 18-21 KubeCon 2019 in San Diego, California

Without Dates – Conferences that are really great that don’t currently have a date just yet.

Polyglot Conf in Vancouver BC

Seattle Code Camp

Microsoft Build

GDG DevFest

What others should I add that are awesome Seattle or immediate surrounding area conferences?

 

A Go Repo & Some Go Code

Wrapped up another Twitch stream (follow here) and streamed live via YouTube where the video is now (subscribe here, video here). However the more interesting part IMHO was where I broke down a few key parts of the application building features around file writes, reads, JSON marshalling, and some other functionality. I also put together a repo of the code I put together here on Github (code shown and explained below). I dove into this topic in the later part of the stream which I’ve time tagged below. For the full timeline and the rest of the video just watch from the beginning of the video. I do make some progress working on Colligere, but I’ll cover that topic in a subsequent blog entry and Twitch stream.

  • 1:32:40 – Creating a new application with Jetbrains Goland to show off how to use the Go core libraries around the user, JSON, and some basic file creation and writing.
  • 1:36:40 – At this point I start adding some basic code to pull the current user and collect some information about that user.
  • 1:40:42 – I extract the error code using the Goland refactoring feature and setup a func check() for error checking. That cleans up the inline code a bit.
  • 1:43:10 – Here I add to the user data retrieved some environment variables to that list of collected data. I also cover again, as I have a number of times, how the environment variables are pulled in IDE versus user session versus out of IDE.
  • 1:47:46 – Now I add a file exists check and start working on that logic.
  • 1:48:56 – Props to Edd Turtle on a solid site on Go. Here’s the blog entry, and respectively the path to more of Edd’s material @ https://golangcode.com/. Also, Edd seems like a good guy to follow @eddturtle.
  • 2:01:33 – Starting the JSON Work here to marshal and unmarshall.
  • 2:26:33 – At this point I push the code up to Github (repo here) using the built in Goland VCS features. I realize I’ve named the repo “adron” by accident so I rename it, close Goland, and then clone the code back down locally with Goland’s VCS features. It’s kind of interesting to see Goland go through the 2-factor auth for this too.
  • 2:58:56 – The Seattle Thrashing Code outtro!

There were a few notes I took during the session with my collected links and references for things I looked up. Those included the following:

The code for the Github repo, ended in this state. Just a single main.go file, which shows how to use several features of Go and capabilities of the core libraries.

package main

import (
	"encoding/json"
	"io/ioutil"
	"log"
	"os"
	"os/user"
	"strconv"
)

type UserInformation struct {
	Name     string   `json:"name"`
	UserId   int64    `json:"userid"`
	GroupId  int64    `json:"groupid"`
	HomeDir  string   `json:"homedir"`
	UserName string   `json:"username"`
	GroupIds []string `json:"groupids"`
	GoPath   string   `json:"gopath"`
	EnvVar   string   `json:"environmentvariable"`
}

func main() {
	currentUser, err := user.Current()
	check(err)
	changeDataForMarshalling(currentUser)
}

func changeDataForMarshalling(currentUser *user.User) {
	groupsIds, err := currentUser.GroupIds()
	check(err)
	userId, err := strconv.ParseInt(currentUser.Uid, 6, 64)
	check(err)
	groupId, err := strconv.ParseInt(currentUser.Gid, 6, 64)
	check(err)

	workingUserInformation := &UserInformation{
		Name:     currentUser.Name,
		UserId:   userId,
		GroupId:  groupId,
		HomeDir:  currentUser.HomeDir,
		UserName: currentUser.Username,
		GroupIds: groupsIds,
		GoPath:   os.Getenv("GOPATH"),
		EnvVar:   os.Getenv("NEW_STRING_ENVIRONMENT_VARIABLE"),
	}

	resultingUserInformation, _ := json.Marshal(workingUserInformation)

	filename := "collected_values.json"
	if _, err := os.Stat(filename); os.IsNotExist(err) {
		writeFileContents(err, filename, string(resultingUserInformation))
	} else {
		newUserInformation, err := openFileMakeChanges(filename)
		writeFileContents(err, filename, newUserInformation)
	}
}

func openFileMakeChanges(filename string) (string, error) {
	jsonFile, err := os.Open(filename)
	check(err)
	defer jsonFile.Close()
var changingUserInformation UserInformation
	byteValue, _ := ioutil.ReadAll(jsonFile)
	json.Unmarshal(byteValue, &changingUserInformation)
changingUserInformation.Name = "Adron Hall"
	changingUserInformation.GoPath = "/where/is/the/goland"
	newUserInformation, _ := json.Marshal(changingUserInformation)
return string(newUserInformation), err
}

func writeFileContents(err error, filename string, text string) {
	f, err := os.OpenFile(filename, os.O_CREATE|os.O_WRONLY, 0600)
	check(err)
	defer f.Close()

	if _, err = f.WriteString(text); err != nil {
		panic(err)
	}
}

func check(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

The import section includes several core libraries for the application “encoding/json”, “io/ioutil”, “log”, “os”, “os/user”, and “strconv”. Then I’ve got a structure declared that I use throughout the code with various fields.

Then just for the heck of it I created a changeDataForMarshalling function and passed in userInfo and parsed the string results of gid and uid to int data types. From there the remaining values are passed in then marshalled to JSON and written to a file, depending on if it’s a new file or existing. The writing however I broke out to a function dubbed writeFileContents. Definitely some more refactoring and tweaking to really make it functional and usable, but shows easily what strconv, marshalling, and other features of these core libraries do. This code provides some examples on what functionality Colligere will use to read in schema configuration, edit, and save that schema. More about that in the next Twitch few streams.