Category Archives: golang

Learning Go Episode 4 – Composite Types, Slices, Arrays, Etc.

Episode Post & Video Links:  1, 2, 3, 4 (this post), & 5 (almost done)

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan. I use it throughout these sessions to provide a guideline. I however add a bunch of other material about IDE’s, development tips n’ tricks and other material.

8:00 Starting the core content with some notes. Composite types, arrays, slices, etc.
11:40 Announcement of my first reload – https://compositecode.blog/2019/02/11/… – second successful time blogged here) of the XPS 15 I have, which – https://youtu.be/f0z1chi4v1Q – actually ended in catastrophe the first time!
14:08 Starting the project for this session.
16:48 Setting up arrays, the things that could be confusing, and setup of our first code for the day. I work through assignment, creation, new vs. comparison, and various other characteristics of working with arrays during this time.

29:36 Creation of a type, called Currency, of type int, setting up constants, and using this kind of like an enumerator to work with code that reads cleaner. Plus of course, all the various things that you might want to, or need for a setup of types, ints, and related composite types like this.

43:48 Creating an example directly from the aforementioned book enumerating the months of the year. This is a great example I just had to work through it a bit for an example.

52:40 Here I start showing, and in the process, doing some learning of my own about runes. I wasn’t really familiar with them before digging in just now!

1:09:40 Here I break things down and start a new branch for some additional examples. I also derail off into some other things about meetups and such for a short bit. Skip to the next code bits at the next time point.
1:23:58 From here on to the remainder of the video I work through a few examples of how to setup maps, how make works, and related coding around how to retrieve, set, and otherwise manipulate the maps one you’ve got them.

That’s it for this synopsis. Until next episode, happy code thrashing and go coding!

Learning Go Episode 3 – More Data Types, Casting, Rendering an SVG file, and writing to Files.

Episode Post & Video Links:  1, 2, 3 (this post), 4, & 5 (almost done)

Episode 3 of my recurring “Learning Go” Saturday stream really got more into the particulars of Go data types including integers, strings, more string formatting verbs, concatenation, type casting, and lots of other pedantic details. In the episode I also delve into some OBS details with the audience, and we get the Twitch interface I’ve setup a bit more streamlined for easier readability. Overall, I think it’s looking much better than just the last episode! Hats off to the conversational assist from the audience.

Here’s the play by play of what was covered in episode 3 with the code in gists plus the repo is available on Github. Video below the timeline.

Timeline

0:00 Intro
6:08 The point I fix the sound. Just skip that first bit!
6:24 Re-introducing the book I’m using as a kind of curriculum guide for these go learning sessions.
7:44 Quick fix of the VM, a few updates, discussion of Goland updates, and fixing the Material Theme to a more visually less caustic theme. Also showing where it is in the IDE.
9:52 Getting into the learning flow, starting a new project with Go 1.11.4 using the Goland IDE new project dialog.

10:50 Creating the Github repo for learning-go-episode-3.
12:14 Setting up the initial project and CODING! Finally getting into some coding. It takes a while when we do it from nothing like this, but it’s a fundamentally important part of starting a project!
13:04 From nothing, creating the core basic elements of a code file for go with main.go. In this part I start showing the various ways to declare types, such as int and int64 with options on style.
14:14 Taking a look at printing out the various values of the variables using formatter verbs via the fmt.Printf function.
17:00 Looking at converting values from one type to another type. There are a number of ways to do this in Go.

I also, just recently, posted a quick spot video and code (blog entry + code) on getting the minimum and maximum value in Go for a number of types. This isn’t the course video, just a quick spot. Keep reading for the main episode below.


18:16 Oh dear the mouse falls on the ground. The ongoing battle of streaming, falling objects! But yeah, I get into adding a function – one of the earlier functions being built in the series – and we add a signature with a return int64 value. I continue, with addition of another function and looking at specifics of the signature.
25:50 Build this code and take a look at the results. At this point, some of the formatting is goofed up so I take a look into the formatter verbs to figure out what should be used for the output formatting.
33:40 I change a few things and take a look at more output from the various calculations that I’ve made, showing how various int, int64, and related calculations can be seen.
37:10 Adding a constant, what it is, and when and where and why to declare something as a constant.
38:05 Writing out another for loop for output results of sets.
42:40 A little git work to create a branch, update the .gitignore, and push the content to github. Repo is here btw: https://github.com/Adron/learning-go-episode-3

At this point I had to take a short interruption to get my ssh keys setup for this particular VM so I could push the code! I snagged just a snippet of the video and made a quick spot video out of it too. Seems a useful thing to do.

47:44 Have to add a new ssh key for the virtual machine to github, so this is a good little snippet of a video showing how that is done.
56:38 Building out a rendering of an SVG file to build a graphic. The complete snippet is below, watch the video for more details, troubleshooting, and working through additions and refactoring of the code.

1:15:32 We begin the mission of bumping up the font size in Goland. It’s a little tricky but we get it figured out.
1:33:20 Upon realization, we need to modify for our work, that this outputs directly to a file instead of just the console. Things will work better that way so I work into the code a write out to file.
1:40:05 Through this process of changing it to output to file, I have to work through additional string conversions, refactoring, and more. There’s a lot of nuance and various things to learn during this section of the video, albeit a little slow. i.e. LOTS of strconv usage.
2:01:24 First view of the generated SVG file! Yay! Oh dear!
2:09:10 More troubleshooting to try and figure out where the math problem is!
2:22:50 Wrapping up with the math a little off kilter, but sort of fixed, I move on to getting a look into the build but also pushing each of the respective branches on github. Repo is here btw: https://github.com/Adron/learning-go-episode-3

A Short Spot on Coding up Go Types Sizing Limits

In an effort, finally, to get around to putting more content together I’ve created this video. It’s the first of many that I’ll put together. This one is a quick coding example of getting the maximum size of particular types in Go. At just about a minute, it’s a quick way to pick up a way to do something coding. If you like this video, do leave a comment, if you thought it wasn’t useful leave a comment. Either way I’d like to read the over under on material like this and if it is or isn’t useful (or entertaining) for you. It’ll mostly be just little tips n’ tricks on how to get things done.

The code gist.

 

Learning Go Episode 2 – Further into packages, dependencies, application creation, and IDE’s

Episode Post & Video Links:  1, 2 (this post), 34, & 5 (almost done)

In episode two I went over a lot of the material that I covered in the first episode, but added more context, historical reasons certain things are the way they are in Go and the stack, and went over a number of new elements of information too. One thing I got further into this episode is the package and dependency management with Go Dep and also how to create a package, or dependency library for use in other Go libraries or applications. It’s just a small introduction in this episode, but pivotal to future episodes, as I’ll be jumping further into library creation and related details.

In this post I’ve got the time point breakdown like usual,  but also a few additional bits of information and code examples, plus links to the repository I’ve setup for this particular episode. The quick links to those references are below, and also I’ll link at particular call out points within the time points.

Quick Links:

Key Topics Covered

Data Types, Packages, and Dependency Management

2:52 – Fumbling through getting started. It is after all Saturday morning!
3:00 – Recap of what we covered in the first session. Includes a quick review of the previous session code too, such as the random data generation library we setup and used.
6:40 – Covering some specifics of the IDE’s, the stories of the benefits of Go having a specific and somewhat detailed convention to the way syntax, variables, and related features are used.
7:40 – Covering gofmt and what it gives us.
9:45 – Looking at the gofmt plugins and IDE features around the conventions.
14:06 – New example time! In this one, I work through an example showing how to find duplicate lines in passed in text.

Duplicate Line Finder

I went through the various steps of creating the code, but then took a little bit of a detour from the example in the book. Instead of lines by the CLI it takes in content from a text file. The code in main.go ended up like this.

Then if you’d like to check out the text file and remaining content in that project, check out the master branch of the episode 2 repo.

36:34 – Here I take a thorough step through committing this project to github, which is the included repo in this post. However I step through the interface of using Jetbrains Goland to do the commit, how it enables gofmt and other features to improve the condition of code and ensure it meets linter demands and related crtieria. I also cover the .gitignore file and other elements to create a usable repository.
44:30 – Setting up the repository for today’s code at https://github.com/Adron/learning-go-…
50:00 – Setup of the key again for using Github. How to setup your ssh keys using ssh-keygen.
56:00 – Going beyond just the language, and building out a Go build on Travis CI.
1:10:16 – Creating a new branch for the next code examples and topics. At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

The tempApp Branch is available in the repository here.

At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

During this point, we take a look at our first package. This package ended up looking like this.

In the main.go file, I showed how you can use this package by adding a respective import shown in this code.

1:17:54 – At this point, to increase readability of font sizes I get into the various Goland IDE options.
1:38:12 – Creating the final branch for this session to pull in a public package and use it in project. For this, I pull in a random data generation package to use in some application code.

1:44:50 – Further discussion and explanation of what to include in .gitignore files to manage projects, but also what is and isn’t included for dependencies and other details around all of this.
2:13:22 – The wicked awesome hacker outtro.

Learning Go Episode 1 – Environment, Go Workspace, GOPATH/GOROOT, Types, and more Introduction

Episode Post & Video Links:  1 (this post), 2, 3, 4, 5 (almost done)

This is episode one of a multi-part series on “The Go Programming Language“. Not necessary, but if you’d like to follow along you can also pick up the book “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan. At the bottom of the description I have a link to the book publisher’s website and the respective book. I’ll be using that as a guideline and using a number of examples from the book. However I’ll also be adding a lot of additional material around Goland IDE from Jetbrains and Visual Studio Code. The video link to this session is at the bottom of the post, scroll all the way down and it’s waiting for you there.

3:28 – Getting started, introducing that the session I’m starting with a completely new Ubuntu Linux load so that I ensure we cover all of the steps to get up and running. These steps, even though they’re on Linux are reproducible on Windows 10 and MacOS, so any operating system is usable to follow along with, with only minor discrepancies.

5:04 – Introducing the book that I’ll be using as a guideline reference so that viewers can also follow along with a physical book. I’m a big fan of multisensory learning, so between a book, the stream, being able to ask questions in the channel, it’ll give viewers a chance to learn and undertake their coding adventures in Go using all sorts of methods.

Book Reference: “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan

6:58 – Discussing where Go is located on the web related to Github and the golang.org site that is useful in that one can even try out little snippets of Go code itself, on the site!
Github: https://github.com/golang/go
Golang: https://golang.org
10:40 – Setting export in the .bashrc file (or .bash_profile on MacOS or environment variables on Windows 10). Speaking of Windows 10 specifically, Linda Gregier wrote up a great blog post on getting Go setup on Windows specifically.
14:50 – Setting up the Go workspace path for GOPATH using the standard Go convention. From here I get into the first “Hello World!” with Go.
15:34 – Mention of setting up Go on a Docker container and how it is easier, but we’re staying focused on setting it up completely from scratch.
18:20 – Starting first code, a standard “Hello World” program.
19:50 – First build of that “Hello World” program.
20:34 – Here I introduce go run and how to execute a singular file instead of building an entire project.
21:32 – Installing some IDE’s to use for developing Go applications. The first two up for installation is Visual Studio Code and JetBrains Goland.
29:00 – A first variable, what is it, and how to declare one in Go in one of the ways one can declare a variable in Go!
31:08 – Introducing the terminal in Visual Studio Code.
37:12 – A little example of OBS, how I’m using it, and how I interact back and forth with chat and related tooling plus the virtual machine itself.
42:36 – Changing themes and adding plugins for Goland. In the plugins I also find the most epic of status bars, the Nyan Cat!
59:00 – Here I start to get back into some more specific Go details. Starting off with a Go command line parsing application. At this point I also cover several additional ways to declare variables, speak more about short declarations, and other ways to declare, assign, and use variables in Go.

At this point I also go through a number of examples to exemplify how to go about declaring variables, build, run, and explore the details of the code. Further along I also get into string formatting, concatenating, and related string manipulation with Go.

Other details include taking a look at extra ways to figure out Go code using autocomplete inside Goland and other exploratory features. Eventually before wrapping up I broach pointers, tuple declaration techniques, and how to declare additional functions beyond func main().
1:58:40 – Adding dependencies and generating random data. At this point I bring in a dependency. But before pulling in the dependency, after introducing it, I show how to go about doing.
2:00:10 – New machine, and I run into git not being available. The go get command uses git to pull dependencies so I go about why this is needed and the steps to install on Ubuntu.
2:09:20 – Introduction to more concepts around dependencies, what go get does versus managing dependencies with Go Dep.
2:10:00 – Installing Go Dep; MacOS, using curl, Linux installation, and a question sort of remains to get it running on Windows. The easiest method is using chocolatey however, so check that out if you’re going the Windows route.
2:15:20 – Setting up Go Dep integration with Goland.
2:23:55 – Showing off Goland’s commit dialog and some of the respective options.

 

Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

Part 3 of 3 – Coding Session in Go – Cobra + Viper CLI for Parsing Text Files, Retrieval of Twitter Data, and Exports to various file formats.

UPDATED PARTS:

  1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
  2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation
  3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval (this post)

Updated links to each part will be posted at bottom of  this post when I publish them. For code, written walk through, and the like scroll down below the video and timestamps.

0:54 The thrashing introduction.
3:40 Getting started, with a recap of the previous sessions but I’ve not got the sound on so ignore this until 5:20.
5:20 I notice, and turn on the volume. Now I manage to get the recap, talking about some of the issues with the Twitter API. I step through setup of the app and getting the appropriate ID’s and such for the Twitter API Keys and Secrets.
9:12 I open up the code base, and review where the previous sessions got us to. Using Cobra w/ Go, parsing and refactoring that was previously done.
10:30 Here I talk about configuration again and the specifics of getting it setup for running the application.
12:50 Talking about Go’s fatal panic I was getting. The dependency reference to Github for the application was different than what is in application and don’t show the code that is actually executing. I show a quick fix and move on.
17:12 Back to the Twitter API use by using the go-twitter library. Here I review the issue and what the fix was for another issue I was having previous session with getting the active token! Thought the library handled it but that wasn’t the case!
19:26 Now I step through creating a function to get the active oath bearer token to use.
28:30 After deleting much of the code that doesn’t work from the last session, I go about writing the code around handling the retrieval of Twitter results for various passed in Twitter Accounts.

The bulk of the next section is where I work through a number of functions, a little refactoring, and answering some questions from the audience/Twitch Chat (working on a way to get it into the video!), fighting with some dependency tree issues, and a whole slew of silliness. Once that wraps up I get some things committed into the Github repo and wrap up the core functionality of the Twitz Application.

58:00 Reviewing some of the other examples in the go-twitter library repo. I also do a quick review of the other function calls form the library that take action against the Twitter API.
59:40 One of the PR’s I submitted to the project itself I review and merge into the repo that adds documentation and a build badge for the README.md.
1:02:48 Here I add some more information about the configuration settings to the README.md file.

1:05:48 The Twitz page is now updated: https://adron.github.io/twitz/
1:06:48 Setup of the continuous integration for the project on Travis CI itself: https://travis-ci.org/Adron/twitz
1:08:58 Setup fo the actual travis.yml file for Go. After this I go through a few stages of troubleshooting getitng the build going, with some white space in the ole’ yaml file and such. Including also, the famous casing issue! Ugh!
1:26:20 Here I start a wrap up of what is accomplished in this session.

NOTE: Yes, I realize I spaced and forgot the feature where I export it out to Apache Cassandra. Yes, I will indeed have a future stream where I build out the part that exports the responses to Apache Cassandra! So subcribe, stay tuned, and I’ll get that one done ASAP!!!

1:31:10 Further CI troubleshooting as one build is green and one build is yellow. More CI troubleshooting! Learn about the travis yaml here.
1:34:32 Finished, just the bad ass outtro now!

The Codez

In the previous posts I outlined two specific functions that were built out:

  • Part 1 – The config function for the twitz config command.
  • Part 2 – The parse function for the twitz parse command.

In this post I focused on updating both of these and adding additional functions for the bearer token retrieval for auth and ident against the Twitter API and other functionality. Let’s take a look at what the functions looked like and read like after this last session wrap up.

The config command basically ended up being 5 lines of fmt.Printf functions to print out pertinent configuration values and environment variables that are needed for the CLI to be used.

The parse command was a small bit changed. A fair amount of the functionality I refactored out to the buildTwitterList() and exportFile, and rebuildForExport functions. The buildTwitterList() I put in the helper.go file, which I’ll cover a littler later. But in this file, which could still use some refactoring which I’ll get to, I have several pieces of functionality; the export to formats functions, and the if else if logic of the exportParsedTwitterList function.

Next up after parse, it seems fitting to cover the helpers.go file code. First I have the check function, which simply wraps the routinely copied error handling code snippet. Check out the file directly for that. Then below that I have the buildTwitterList() function which gets the config setting for the file name to open to parse for Twitter accounts. Then the code reads the file, splits the results of the text file into fields, then steps through and parses out the Twitter accounts. This is done with a REGEX (I know I know now I have two problems, but hey, this is super simple!). It basically finds fields that start with an @ and then verifies the alphanumeric nature, combined with a possible underscore, that then remove unnecessary characters on those fields. Wrapping all that up by putting the fields into a string/slice array and returning that string array to the calling code.

The next function in the Helpers.go file is the getBearerToken function. This was a tricky bit of code. This function takes in the consumer key and secret from the Twitter app (check out the video at 5:20 for where to set it up). It returns a string and error, empty string if there’s an error, as shown below.

The code starts out with establishing a POST request against the Twitter API, asking for a token and passing the client credentials. Catches an error if that doesn’t work out, but if it can the code then sets up the b64Token variable with the standard encoding functionality when it receives the token string byte array ( lines 9 and 10). After that the request then has the header built based on the needed authoriztaion and content-type properties (properties, values? I don’t recall what spec calls these), then the request is made with http.DefaultClient.Do(req). The response is returned, or error and empty response (or nil? I didn’t check the exact function signature logic). Next up is the defer to ensure the response is closed when everything is done.

Next up the JSON result is parsed (unmarshalled) into the v struct which I now realize as I write this I probably ought to rename to something that isn’t a single letter. But it works for now, and v has the pertinent AccessToken variable which is then returned.

Wow, ok, that’s a fair bit of work. Up next, the findem.go file and related function for twitz. Here I start off with a few informative prints to the console just to know where the CLI has gotten to at certain points. The twitter list is put together, reusing that same function – yay code reuse right! Then the access token is retrieved. Next up the http client is built, the twitter client is passed that and initialized, and the user lookup request is sent. Finally the users are printed out and below that a count and print out of the count of users is printed.

I realized, just as I wrapped this up I completely spaced on the Apache Cassandra export. I’ll have those post coming soon and will likely do another refactor to get the output into a more usable state before I call this one done. But the core functionality, setup of the systemic environment needed for the tool, the pertinent data and API access, and other elements are done. For now, that’s a wrap, if you’re curious about the final refactor and the Apache Cassandra export then subscribe to my Twitch @adronhall and/or my YouTube channel ThrashingCode.

UPDATED SERIES PARTS

    1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
    2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation
    3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval (this post)

     

Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation

Part 2 of 3 – Coding Session in Go – Cobra + Viper CLI for Parsing Text Files, Retrieval of Twitter Data, Exports to various file formats, and export to Apache Cassandra.

UPDATED PARTS:

  1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
  2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
  3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

Updated links to each part will be posted at bottom of  this post when I publish them. For code, written walk through, and the like scroll down below the video and timestamps.

Hacking Together a CLI Installing Cassandra, Setting Up the Twitter API, ENV Vars, etc.

0:04 Kick ass intro. Just the standard rocking tune.

3:40 A quick recap. Check out the previous write “Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files” of this series.

4:30 Beginning of completion of twitz parse command for exporting out to XML, JSON, and CSV (already did the text export previous session). This segment also includes a number of refactorings to clean up the functions, break out the control structures and make the code more readable.

In the end of refactoring twitz parse came out like this. The completed list is put together by calling the buildTwitterList() function which is actually in the helpers.go file. Then prints that list out as is, and checks to see if a file export should be done. If there is a configuration setting set for file export then that process starts with a call to exportParsedTwitterList(exportFilename string, exportFormat string, ... etc ... ). Then a simple single level control if then else structure to determine which format to export the data to, and a call to the respective export function to do the actual export of data and writing of the file to the underlying system. There’s some more refactoring that could be done, but for now, this is cleaned up pretty nicely considering the splattering of code I started with at first.

50:00 I walk through a quick install of an Apache Cassandra single node that I’ll use for development use later. I also show quickly how to start and stop post-installation.

Reference: Apache Cassandra, Download Page, and Installation Instructions.

53:50 Choosing the go-twitter API library for Go. I look at a few real quickly just to insure that is the library I want to use.

Reference: go-twitter library

56:35 At this point I go through how I set a Twitter App within the API interface. This is a key part of the series where I take a look at the consumer keys and access token and access token secrets and where they’re at in the Twitter interface and how one needs to reset them if they just showed the keys on a stream (like I just did, shockers!)

57:55 Here I discuss and show where to setup the environment variables inside of Goland IDE to building and execution of the CLI. Once these are setup they’ll be the main mechanism I use in the IDE to test the CLI as I go through building out further features.

1:00:18 Updating the twitz config command to show the keys that we just added as environment variables. I set these up also with some string parsing and cutting off the end of the secrets so that the whole variable value isn’t shown but just enough to confirm that it is indeed a set configuration or environment variable.

1:16:53 At this point I work through some additional refactoring of functions to clean up some of the code mess that exists. Using Goland’s extract method feature and other tooling I work through several refactoring efforts that clean up the code.

1:23:17 Copying a build configuration in Goland. A handy little thing to know you can do when you have a bunch of build configuration options.

1:37:32 At this part of the video I look at the app-auth example in the code library, but I gotta add the caveat, I run into problems using the exact example. But I work through it and get to the first error messages that anybody would get to pending they’re using the same examples. I get them fixed however in the next session, this segment of the video however provides a basis for my pending PR’s and related work I’ll submit to the repo.

The remainder of the video is trying to figure out what is or isn’t exactly happening with the error.

I’ll include the working findem code in the next post on this series. Until then, watch the wrap up and enjoy!

1:59:20 Wrap up of video and upcoming stream schedule on Twitch.

UPDATED SERIES PARTS

    1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
    2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
    3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval