Category: golang

Creating a Go Module & Writing Tests in Less Than 3 Minutes

In this short (less than 4 minutes) video I put together a Go module library, then setup some first initial tests calling against functions in the module.

 

00:12 – Writing unit tests.

00:24 – Create the first go file.

00:40 – Creating the Go go.mod file. When creating the go.mode file, note the command is go mod init [repopath] where repopath is the actual URI to the repo. The go.mod file that is generated would look like this.

module github.com/adron/awesomeLib

go 1.13

00:56 – Enabling Go mod integration for Goland IDE. This dialog has an option to turn off the proxy or go direct to repo bypassing the proxy. For more details on the Go proxy, check out goproxy.io and the Go docs for more details.

01:09 – Instead of TDD, first I cover a simple function implementation. Note the casing is very important in setting up a test in Go. The test function needs to be public, thus capitalized, and the function needs accessible, thus also capitalized. The function in the awesomeLib.go file, just to have a function that would return some value, looks like this.

package awesomeLib

func GetKnownResult() string {
    return "known"
}

01:30 – Creating the test file, awesomeLib_test.go, then creating the test. In this I also show some of the features of the Goland IDE that extracts and offers the function names of a module prefaced with Test and other naming that provides they perform a test, performance, or related testing functionality.

package awesomeLib

import "testing"

func TestGetKnownResult(t *testing.T) {
    got := GetKnownResult()
    if got != "known" {
        t.Error("Known result not received, test failed.")
    }
}

02:26 – Running the standard go test to execute the test. To run tests for all files within this module, such as if we’ve added multiple directories and other files, would be go test ./....

02:36 – Using Goland to run the tests with singular or multiple tests being executed. With Goland there are other capabilities to show test coverage, what percentage of functionality is covered by tests, and other various code metrics around testing, performance, and other telemetry.

That’s it, now the project is ready for elaboration and is setup for a TDD, BDD, or implement and test style approach to development.

For JavaScript, Go, Python, Terraform, and more infrastructure, web dev, and coding in general I stream regularly on Twitch at https://twitch.tv/adronhall, post the VOD’s to YouTube along with entirely new tech and metal content at https://youtube.com/c/ThrashingCode.

Cobra + Viper Go CLI Sample

I just released a Go Cobra + Viper library CLI App sample app here. Here are the main up to date details as of release today.

The app example is using the Cobra and Viper libraries for Go. This application provides commands (per Cobra) to read, write, update, and delete configuration records from a configuration file using Viper.

Using the example CLI App.

<cli> config -h provides the following documentation for using this CLI App.

Available Commands:
  add         The 'add' subcommand will add a passed in key value pair to the application configuration file.
  delete      The 'delete' subcommand removes a key value pair from the configuration file. 
  update      The 'update' subcommand will update a passed in key value pair for an existing set of data to the application configuration file.
  view        The 'view' subcommand will provide a list of keys and a map of the values.

Flags:
  -h, --help           help for config
  -k, --key string     The key for the key value set to add to the configuration.
  -v, --value string   The value for the key value set to add to the configuration.

Currently I got this to work but it isn’t in the best shape. Check out the code here.

Get Involved, Add Samples, Make Requests

If you’d like to get involved and add samples or make request for additional samples please file an issue here!

If you’ve found any bugs or issues with the code please file a bug report here!

Examples, The CRUD!

./cli config add -k "blog" -v "https://compositecode.blog/" example writes a record to the configuration file with a key of “blog” and a value of “https://compositecode.blog/“.

./cli config view displays the contents of the configuration file and CLI specific environment variables. These are the configuration files located in the .cobrae-cli-samples.yml and environment variables prefaced with COBRACLISAMPLES. The list of keys is displaced first and then the keys and values are displayed below that.

./cli config update -k "blog" -v "not found" will update the blog entry in the configuration to read not found for the value.

./cli config delete ... will delete the key and value from the configuration file.

Building the Project

Following a fairly standard clone, one can build this project with a single step using the ./build.sh file. If you’d like to contribute the same for Windows, feel free I’d be happy to pull that PR in. Once the project is built use the CLI as defined above.

Installing the CLI App

This application can be installed as a CLI app by referencing it’s location in your bash (powershell? etc) startup script.

Learning Go Episode 5 – Functions (and Methods and lots of other things)

Episode Post & Video Links:  1, 2, 3, 4, 5 (this post), 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

In this session we covered a host of topics around Go Functions. Along with some troubleshooting, debugging, and other features in Jetbrains Goland IDE.

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan as a starting point. I’m using it as a simple guideline, but also doing a lot more in each stream that includes ecosystem, dependency management with godep, IDE use of Goland from Jetbrains, and more. In this session I get specifically into: functions, signatures, declarations, recursion, return values, and more.

Video Time Points & Topics

2:50 – Introduction to the snowy wonderland of Seattle and episode 5 of the Learning Go series. Introduction to the various screen transitions and such.
6:40 – Getting started, opening up JetBrains Goland and creating a new project. The project exists on Github as https://github.com/adron/learning-go-….
12:18 – Starting with functions in Go. See the blog entry I wrote on the topic for more additional information around this first code session within the episode 5 session.

Code – This first example I setup a basic function in Go that is called by the main function. The sample function below I’ve named exampleExecutor, and the signature is made up of an int parameter called this, a string parameter called that, and a return parameter of type int and one of type string. In summary for the function signature we have two input parameters going in and two return parameters coming out.

The function does very little besides print the parameters passed in and then return the parameters back out as the return parameters.


package main
import "fmt"
func main() {
var this, result int
var that, message string
this = 2
that = "42"
result, message = exampleExecutor(this, that)
fmt.Printf("%s\n%d", message, result)
}
func exampleExecutor(this int, that string) (int, string) {
fmt.Printf("Numbers: %d, %s\n", this, that)
return this, "This is the result: " + that
}

view raw

func_main_plus_example_executor.go

hosted with ❤ by GitHub

Recursion with Go & HTML Parsing

28:10 – Here I get into recursion and the application example, largely taken from the book but with some very distinctive modifications, that parses HTML and the various nodes within an HTML document.

For the recursion section I use an example from the book with an expanded sample set of HTML. The HTML is included in the repo under the function-recursion branch. For this example I setup a set of types and variables up that are needed throughout the code.

First a type setup called NodeType of type int32. A constant array of ErrorNode, TextNode, DocumentNode, ElementNode, CommentNode, and DoctypeNode for determining the different nodes within an HTML document. Then a general struct called Node with Type, Data, Attr for attribute, and FirstChild with NextSibling setup as a pointer to *Node, which gives a type of memory recursion to the underlying type. Then finally the Attribute struct with a Key a Value.


type NodeType int32
const (
ErrorNode NodeType = iota
TextNode
DocumentNode
ElementNode
Commentnode
DoctypeNode
)
type Node struct {
Type NodeType
Data string
Attr []Attribute
FirstChild, NextSibling *Node
}
type Attribute struct {
Key, Val string
}

view raw

types_nodes_etc.go

hosted with ❤ by GitHub

One of the first functions I then end up with is the visit function. It turns out as shown below. Here the function takes a links parameter that is of type string array, a parameter name n that is a pointer reference to an a node within html, and then the function returns a parameter of type string array.


func visit(links []string, n *html.Node) []string {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key == "href" {
links = append(links, a.Val)
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
links = visit(links, c)
}
return links
}

view raw

visit_func.go

hosted with ❤ by GitHub

After that function I worked through and created two additional functions, one for countsWordsAndImages and one called CountWordsAndImages. The casing being specific to scope and use of the functions, and they respectively look like this in completion.


func CountWordsAndImages(url string) (words, images int, err error) {
resp, err := http.Get(url)
if err != nil {
return
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
err = fmt.Errorf("parsing HTML: %s", err)
}
words, images = countWordsAndImages(doc)
return
}
func countWordsAndImages(n *html.Node) (words, images int) {
if n.Type == html.TextNode {
words += len(strings.Split(n.Data, " "))
return
}
if n.Data == "img" {
images++
} else {
if n.FirstChild != nil {
w, i := countWordsAndImages(n.FirstChild)
words += w
images += i
}
if n.NextSibling != nil {
w, i := countWordsAndImages(n.NextSibling)
words += w
images += i
}
}
return
}

view raw

counting.go

hosted with ❤ by GitHub

Then all that is wrapped up, with recursive calls and more, in the main function for program execution.


func main() {
htmlContent, err := ioutil.ReadFile("compositecode.html")
if err != nil {
fmt.Println(err)
}
htmlData := string(htmlContent)
r := strings.NewReader(htmlData)
doc, err := html.Parse(r)
if err != nil {
fmt.Fprintf(os.Stderr, "find links: %v\n", err)
os.Exit(1)
}
for _, link := range visit(nil, doc) {
fmt.Println(link)
}
w, i, _ := CountWordsAndImages("https://compositecode.blog&quot;)
fmt.Printf("Words: %d\nImages: %d\n", w, i)
}

view raw

TheCountingRecursionOfThings.go

hosted with ❤ by GitHub

Starting Error Handling && Anonymous Functions

1:32:40 – At this point in the episode 5 session I get into a simple Error handling function, and further into function signatures and how to set them up.
1:52:24 – Setting up some anonymous functions and reviewing what they are.
1:59:00 – Introduction to panics in Go. After this short introduction I also discuss some of the pedantic specifics of methods vs functions and related verbiage around the Go language. Additionally I provide more examples around these specifics for declaring functions, various scope, and other types for function calls and related usage.

With that done the wrap up of the session is then a short introduction to anonymous functions.

Learning Go Episode 4 – Composite Types, Slices, Arrays, Etc.

Episode Post & Video Links:  1, 2, 3, 4 (this post), 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan. I use it throughout these sessions to provide a guideline. I however add a bunch of other material about IDE’s, development tips n’ tricks and other material.

8:00 Starting the core content with some notes. Composite types, arrays, slices, etc.
11:40 Announcement of my first reload – http://compositecode.blog/2019/02/11/… – second successful time blogged here) of the XPS 15 I have, which – https://youtu.be/f0z1chi4v1Q – actually ended in catastrophe the first time!
14:08 Starting the project for this session.
16:48 Setting up arrays, the things that could be confusing, and setup of our first code for the day. I work through assignment, creation, new vs. comparison, and various other characteristics of working with arrays during this time.


fmt.Println("Hello, let's talk composite types.")
basketOfStuff := [3]string{"The first string","second","This string."}
var zeeValues [2]int
for i, v := range basketOfStuff {
fmt.Printf("Value %d: %s\n", i, v)
}
fmt.Println(zeeValues)
if zeeValues[0] == zeeValues[1] {
fmt.Println("The values are the same, this doesn't instantiate like the `new` keyword.")
} else {
fmt.Println("The way go appears to instantiate unset variable values, such as in this array is like the `new` keyword instantiation.")
}
zeeValues[0] = 1 + 52 * 3
zeeValues[1] = 9
fmt.Println(zeeValues[len(zeeValues) – 1])

view raw

testing-out-composite-types.go

hosted with ❤ by GitHub

29:36 Creation of a type, called Currency, of type int, setting up constants, and using this kind of like an enumerator to work with code that reads cleaner. Plus of course, all the various things that you might want to, or need for a setup of types, ints, and related composite types like this.


type Currency int
const (
USD Currency = iota
CAN
EUR
GBP
JPY
NOK
SEK
DKK
)
symbol := […]string{USD: "$", CAN: "$", EUR: "€", GBP: "£", JPY:"¥", NOK:"kr", SEK:"kr",DKK:"kr"}
fmt.Println(EUR, symbol[EUR])
fmt.Println(JPY, symbol[JPY])
r := […]int{99: -1}
r[36] = 425
r[42] = 42
fmt.Println(r[36] + r[42])
fmt.Println(strconv.Itoa(r[36]))

view raw

currency.go

hosted with ❤ by GitHub

43:48 Creating an example directly from the aforementioned book enumerating the months of the year. This is a great example I just had to work through it a bit for an example.


months := […]string{1: "January", 2:"February", 3: "March", 4:"April", 12:"December"}
for _, s := range months {
fmt.Printf("The month: %s\n", s)
}
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}

view raw

the_months.go

hosted with ❤ by GitHub

52:40 Here I start showing, and in the process, doing some learning of my own about runes. I wasn’t really familiar with them before digging in just now!


… the rest of the main.go file is here…
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}
}
func appendInt(x []int, i int) []int {
var z []int
zlen := len(x) + 1
if zlen <= cap(x) {
z = x[:zlen]
} else {
zcap := zlen
if zcap < 2* len(x) {
zcap = 2 * len(x)
}
z = make([]int, zlen, zcap)
copy(z, x)
}
return z
}

view raw

main.go

hosted with ❤ by GitHub

1:09:40 Here I break things down and start a new branch for some additional examples. I also derail off into some other things about meetups and such for a short bit. Skip to the next code bits at the next time point.
1:23:58 From here on to the remainder of the video I work through a few examples of how to setup maps, how make works, and related coding around how to retrieve, set, and otherwise manipulate the maps one you’ve got them.


package main
import "fmt"
func main() {
ages := map[string]int{
"Peterson": 52,
"Sally": 22,
"Javovia": 15,
"Ben": 42,
}
jobAssociation := make(map[string]string)
jobAssociation["Peterson"] = "Engineer"
jobAssociation["Sally"] = "CEO"
jobAssociation["Jovovia"] = "Gamer"
jobAssociation["Ben"] = "Programmer"
printAges(ages)
printJobAssociations(jobAssociation)
fmt.Println(ages)
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"])
fmt.Println(jobAssociation["Frank"]) // Blank! 😮
fmt.Println(ages["Sally"])
delete(ages, "Sally")
fmt.Println(ages)
fmt.Println(ages["Sally"])
delete(jobAssociation, "Jovovia")
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"]) // Blank.
ages2 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
ages3 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
if equal(ages, ages2) {
fmt.Println("Naw, not really equal.")
} else {
fmt.Println("This is correct, not equal.")
}
if equal(ages2, ages3) {
fmt.Println("True, these are effectively the same map values and keys.")
}
}
func printJobAssociations(associations map[string]string) {
for name, job := range associations {
fmt.Printf("%s\t%s\n", name, job)
}
}
func printAges(ages map[string]int) {
for name, age := range ages {
fmt.Printf("%s\t%d\n", name, age)
}
}
func equal(x, y map[string]int) bool {
if len(x) != len(y) {
return false
}
for k, xv := range x {
if yv, ok := y[k]; !ok || yv != xv {
return false
}
}
return true
}

view raw

map_types.go

hosted with ❤ by GitHub

That’s it for this synopsis. Until next episode, happy code thrashing and go coding!

Learning Go Episode 3 – More Data Types, Casting, Rendering an SVG file, and writing to Files.

Episode Post & Video Links:  1, 2, 3 (this post), 4, 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

Episode 3 of my recurring “Learning Go” Saturday stream really got more into the particulars of Go data types including integers, strings, more string formatting verbs, concatenation, type casting, and lots of other pedantic details. In the episode I also delve into some OBS details with the audience, and we get the Twitch interface I’ve setup a bit more streamlined for easier readability. Overall, I think it’s looking much better than just the last episode! Hats off to the conversational assist from the audience.

Here’s the play by play of what was covered in episode 3 with the code in gists plus the repo is available on Github. Video below the timeline.

Timeline

0:00 Intro
6:08 The point I fix the sound. Just skip that first bit!
6:24 Re-introducing the book I’m using as a kind of curriculum guide for these go learning sessions.
7:44 Quick fix of the VM, a few updates, discussion of Goland updates, and fixing the Material Theme to a more visually less caustic theme. Also showing where it is in the IDE.
9:52 Getting into the learning flow, starting a new project with Go 1.11.4 using the Goland IDE new project dialog.


package main
import (
"fmt"
"math"
"strconv"
)
func main() {
var someInt int
var anotherValue int64
thisValue := 45
thisVAlue, thatValue, someValue := 20, 23, 15
someInt = thisVAlue
anotherValue = 64
const knownSet = 7.12904
const anotherKnown = 9.10347
for x := 0; x < 20; x++ {
fmt.Printf("x = %d e = %8.3f\n", x, math.Exp(float64(x)))
}
fmt.Printf("Known Set and Another Known: %2.7f, %2.7f", knownSet, anotherKnown)
fmt.Printf("The values: %6d, %6d, %6d, and %s.\n", thisValue, thisVAlue, thatValue, strconv.Itoa(someValue))
fmt.Printf("Other values: %4d, %4d.\n", someInt, anotherValue)
fmt.Printf("Function doThings: %6d\n", doThings(thisVAlue, thisValue))
fmt.Printf("Function doMoreThings: %s", doMoreThings(someInt, int(anotherValue), "Adron"))
}
func doThings (valueOne int, valueTwo int) int64 {
result := valueOne * valueTwo
return int64(result)
}
func doMoreThings (valueOne int, valueTwo int, name string) string {
result := valueTwo + valueOne * 10
return "The result added and exponentially multiplied for you " + name + " this: " + strconv.Itoa(result)
}

view raw

main.go

hosted with ❤ by GitHub

10:50 Creating the Github repo for learning-go-episode-3.
12:14 Setting up the initial project and CODING! Finally getting into some coding. It takes a while when we do it from nothing like this, but it’s a fundamentally important part of starting a project!
13:04 From nothing, creating the core basic elements of a code file for go with main.go. In this part I start showing the various ways to declare types, such as int and int64 with options on style.
14:14 Taking a look at printing out the various values of the variables using formatter verbs via the fmt.Printf function.
17:00 Looking at converting values from one type to another type. There are a number of ways to do this in Go.

I also, just recently, posted a quick spot video and code (blog entry + code) on getting the minimum and maximum value in Go for a number of types. This isn’t the course video, just a quick spot. Keep reading for the main episode below.

18:16 Oh dear the mouse falls on the ground. The ongoing battle of streaming, falling objects! But yeah, I get into adding a function – one of the earlier functions being built in the series – and we add a signature with a return int64 value. I continue, with addition of another function and looking at specifics of the signature.
25:50 Build this code and take a look at the results. At this point, some of the formatting is goofed up so I take a look into the formatter verbs to figure out what should be used for the output formatting.
33:40 I change a few things and take a look at more output from the various calculations that I’ve made, showing how various int, int64, and related calculations can be seen.
37:10 Adding a constant, what it is, and when and where and why to declare something as a constant.
38:05 Writing out another for loop for output results of sets.
42:40 A little git work to create a branch, update the .gitignore, and push the content to github. Repo is here btw: https://github.com/Adron/learning-go-episode-3

At this point I had to take a short interruption to get my ssh keys setup for this particular VM so I could push the code! I snagged just a snippet of the video and made a quick spot video out of it too. Seems a useful thing to do.

47:44 Have to add a new ssh key for the virtual machine to github, so this is a good little snippet of a video showing how that is done.
56:38 Building out a rendering of an SVG file to build a graphic. The complete snippet is below, watch the video for more details, troubleshooting, and working through additions and refactoring of the code.


package main
import (
"fmt"
"io/ioutil"
"math"
"strconv"
)
const (
width, height = 600, 600
cells = 100
xyrange = 30.0
xyscale = width / 2 / xyrange
zscale = height * 0.4
angle = math.Pi / 6
)
var sin30, cos30 = math.Sin(angle), math.Cos(angle)
func main() {
var outputResult string
outputResult = "<svg xmlns='http://www.w3.org/2000/svg&#39; " +
"style='stroke: blue; fill: white; stroke-width: 0.7' " +
"width='" + strconv.Itoa(width) + "' height='" + strconv.Itoa(height) + "'>"
for i := 0; i < cells; i++ {
for j := 0; j < cells; j++ {
ax, ay := corner(i+1, j)
bx, by := corner(i, j)
cx, cy := corner(i, j+1)
dx, dy := corner(i+1, j+1)
outputResult = outputResult + "<polygon points='" +
ax + "," + ay +
bx + "," + by +
cx + "," + cy +
dx + "," + dy +
"'/>\n"
}
}
outputResult = outputResult + "</svg>"
fmt.Printf(outputResult)
err := ioutil.WriteFile("surface-plot.svg", []byte(outputResult), 0644)
if err != nil {
fmt.Printf("Error: %s", err)
}
}
func corner(i, j int) (string, string) {
x := xyrange * (float64(i)/cells – 0.5)
y := xyrange * (float64(j)/cells – 0.5)
z := f(x, y)
sx := width/2 + (x-y)*cos30*xyscale
sy := height/2 + (x+y)*sin30*xyscale – z*zscale
xResult := strconv.FormatFloat(sx, 'f', -1, 64)
yResult := strconv.FormatFloat(sy, 'f', -1, 64)
return xResult, yResult
}
func f(x, y float64) float64 {
r := math.Hypot(x, y)
return math.Sin(r)
}

view raw

main.go

hosted with ❤ by GitHub

1:15:32 We begin the mission of bumping up the font size in Goland. It’s a little tricky but we get it figured out.
1:33:20 Upon realization, we need to modify for our work, that this outputs directly to a file instead of just the console. Things will work better that way so I work into the code a write out to file.
1:40:05 Through this process of changing it to output to file, I have to work through additional string conversions, refactoring, and more. There’s a lot of nuance and various things to learn during this section of the video, albeit a little slow. i.e. LOTS of strconv usage.
2:01:24 First view of the generated SVG file! Yay! Oh dear!
2:09:10 More troubleshooting to try and figure out where the math problem is!
2:22:50 Wrapping up with the math a little off kilter, but sort of fixed, I move on to getting a look into the build but also pushing each of the respective branches on github. Repo is here btw: https://github.com/Adron/learning-go-episode-3