Language Stack Installation for Python & Go

Previously I’ve gone through the steps I take to get a solid development machine setup. From the base operating system load, to the browser and basic IDEs I install. Now I’ve got more videos and the respective notes and details about what two language stacks I setup next; Python and Go.

Python

In regards to the Python stack, this one can often be somewhat confusing. Depending on the operating system I setup the stack a little differently.

MacOS

For MacOS I’ve written two posts about this previously, one titled “Getting Started with Python Right!” and one “Unbreaking Python Through Virtual Environments“. Those two posts cover most of the nuance to getting a base Python stack installed on MacOS and then using virtual environments to manage project specific versions per repository.

Linux

For the Linux OS, usually a debian variant, the systems tend to have Python 3 installed by default. I then take the next step of installing pip3 and work from there. The IDE, PyCharm from Jetbrains uses virtualenv to setup virtual environments per repository from that point forward.

Python Setup && Reasons

For more details about the specific walk through, I’ve created this video to walk through setting up Python 3 on Ubuntu and verifying, and also by use of PyCharm to setup a small verification app it shows how the virtualenv sets up a specific environment for the new verification project.

The reasons for installing Python first are numerous. One of the first reasons is that Python is required for installing and using numerous Python related CLIs, such as AWS’s CLI, among many others. It pays off to just have a good install at the system level (i.e. not particular just in a virutal environment, but executable at the terminal on system) to ensure it is available for any and all CLIs that would need it. If you’re into data science work, that’s a huge second reason, because Python is used in about every aspect of data science work, machine learning, and related endeavors.

Go Setup && Reasons

The reason I go for Go as my second language stack install is driven by two primary reasons:

  1. I like writing Go and use it myself for a number of reasons. Such as, it is ridiculously quick and minimal work to build a CLI for use in systems that requires only a single binary executable for use.
  2. I use Go for a lot of other work-related efforts, around Kubernetes, Docker, Terraform, and others.

With that, here’s the quick install and initial project for verification setup.

If you’d like to take a few other quick tours of Go, here are some posts, with videos, putting together a Go module project and writing an initial test in under 3 minutes and setting up an HTTP server in about 15 minutes.

That’s it for now. However, if you’re interested in joining me for next steps, language stack setup, and more in addition to writing some JavaScript, Go, Python, Terraform, and more infrastructure, web dev, and all sorts of coding I stream regularly on Twitch at https://twitch.tv/adronhall, post the VOD’s to YouTube along with entirely new tech and metal content at https://youtube.com/c/ThrashingCode. Feel free to check out a coding session, ask questions, interject, or just come and enjoy the tunes!

For more on my open source efforts and related projects, sign up for the Thrashing Code Newsletter!

My Top 2 Reading List for Go & Data

Right now I’ve got a couple books in queue.

“Black Hat Go” Go Programming for Hackers and Pentesters by Tom Steele, Chris Patten, and Dan Kottmann.

Alt Text

I picked this book up after a good look at the index. There is a basic intro to Go at the beginning of the book to cover language fundamentals, but immediately after that dives into the meat of the topic. Immediately getting into understanding the TCP handshake, TCP itself, writing a scanner, and a proxy. There is then some basics about HTTP Servers, routers, and middleware in chapter 3 and 4, but returns immediately into topics of interest in chapter 5 around exploiting DNS, and writing DNS clients. I perused the index a bit further to note it covers SMB and NTLM, a chapter on “Abusing Databases and Filesystems”, then on to raw packet processing, writing and porting exploit code, and a host of other topics. This is going to be an interesting book to dig into and write about in the coming weeks. I’m pretty excited about it and hope to write a thorough review upon completion.

“Designing Data-Intensive Applications” by Martin Kleppmann

Alt Text

This book is familiar territory, as I’ve spent much of my career working on similar topics that this book covers. What I suspect is that I’ll enjoy reading the material presented in an orderly and concise way versus the chaos and disruptive way I’ve acquired the knowledge on these topics.

From the index, the book starts off with foundations of data systems and the ideas around building reliable, scalable, and maintainable applications. This provides a good basis in which to dive into the other topics. From there it looks like we’ll get a run into the birth of NoSQL, object-relational mismatches and the related insanity that this has bred in the industry! Then a solid dive into graph-like, traditional, and multi-model data modeling. With the beginning quarter of the book then covering everything from hash indexes, SSTables (another familiar topic), LSM-Trees, B-Trees, and related indexing structures before wrapping up this first 25% of the book with stars and snowflake topics for analytics and column-oriented storage, compression, sort orders in column storage, and related material on aggregation in data cubes and materialized views.

That’s just the first 25%! From there Martin covers a wide range of topics, that if you’re in the industry and plan to deal with large scale data-intensive applications these are topics you need to be intimately familiar with!

Reading

Over the next few months while I read through these books I hope to provide summaries and related notes on the material. Who knows, maybe you’ll want to dive into the material yourself! Until then happy thrashing code and may you have high retention and comprehension in reading!

Simple Go HTTP Server Starter in 15 Minutes

This is a quick starter, to show some of the features of Go’s http library. (sample repo here) The docs for the library are located here if you want to dig in deeper. In this post however I cover putting together a simple http server with a video on creating the http server, setting a status code, which provides the basic elements you need to further build out a fully featured server. This post is paired with a video I’ve put together, included below.

00:15 Reference to the Github Repo where I’ve put the code written in this sample.
https://github.com/Adron/coro-era-coding-go-sample
00:18 Using Goland IDE (Jetbrains) to clone the repository from Github.
00:40 Creating code files.
01:00 Pasted in some sample code, and review the code and what is being done.
02:06 First run of the web server.
02:24 First function handler to handle the request and response of an HTTP request and response cycle.
04:56 Checking out the response in the browser.
05:40 Checking out the same interaction with Postman. Also adding a header value and seeing it returned via the browser & related header information.
09:28 Starting the next function to provide further HTTP handler functionality.
10:08 Setting the status code to 200.
13:28 Changing the status code to 500 to display an error.

Getting a Server Running

I start off the project by pulling an empty repository that I had created before starting the video. In this I use the Jetbrains Goland IDE to pull this repository from Github.

1

Next I create two files; main.go and main_test.go. We won’t use the main_test.go file right now, but in a subsequent post I’ll put together some unit tests specifically to test out our HTTP handlers we’ll create. Once those are created I pasted in some code that just has a basic handler, using an anonymous function, provides some static file hosting, and then sets up the server and starts listening. I make a few tweaks, outlined in the video, and execute a first go with the following code.

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Welcome to my website!")
    })

    http.ListenAndServe(":8080", nil)
}

When that executes, opening a browser to localhost:8080 will bring up the website which then prints out “Welcome to my website!”.

2

Adding a Function as an HTTP Handler

The next thing I want to add is a function that can act as an HTTP handler for the server. To do this create a function just like we’d create any function in Go. For this example, the function I built included several print line calls to the ResponseWriter with Request properties and a string passed in.

func RootHandler(w http.ResponseWriter, r *http.Request){
    fmt.Fprintln(w, "This is my content.")
    fmt.Fprintln(w, r.Header)
    fmt.Fprintln(w, r.Body)
}

In the func main I changed out the root handler to use this newly created handler instead of the anonymous function that it currently has in place. So swap out this…

http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Welcome to my website!")
})

with this…

http.HandleFunc("/", RootHandler)

Now the full file reads as shown.

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", RootHandler)
    http.ListenAndServe(":8080", nil)
}

Now executing this and navigating to localhost:8080 will display the following.

3

The string is displayed first “This is my content.”, then the header, and body respectively. The body, we can see is empty. Just enclosed with two braces {}. The header is more interesting. It is returned as a map type, between the brackets []. Showing an accept, accept-encoding, accept-language, user-agent, and other header information that was passed.

This is a good thing to explore further, check out how to view or set the values associated with the header values in HTTP responses, requests, and their related metadata. To go a step further, and get into this metadata a tool like Postman comes in handy. I open this tool up, setup a GET request and add an extra header value just to test things out.

4

Printing Readable Body Contents

For the next change I wanted to get a better print out of body contents, as the previous display was actually just attempting to print out the body in an unreadable way. In this next section I used an ioutil function to get the body to print out in a readable format. The ioutil.ReadAll function takes the body, then I retrieve a body variable with the results, pending no error, the body variable is then cast as a string and print out to the ResponseWriter on the last line. The RootHandler function then reads like this with the changes.

func RootHandler(w http.ResponseWriter, r *http.Request){
    fmt.Fprintln(w, "This is my content.")
    fmt.Fprintln(w, r.Header)

    defer r.Body.Close()

    body, err := ioutil.ReadAll(r.Body)
    if err != nil {
        fmt.Fprintln(w, err)
    }
    fmt.Fprintln(w, string(body))
}

If the result is then requested using Postman again, the results now display appropriately!

5

Response Status Codes!

HTTP Status codes fit in to a set of ranges for various categories of responses. The most common code is of course the success code, which is 200 “Status OK”. Another common one is status code 500, which is a generic catch all for “Server Error”. The ranges are as follows:

  • Informational responses (100–199)
  • Successful responses (200–299)
  • Redirects (300–399)
  • Client errors (400–499)
  • and Server errors (500–599)

For the next function, to get an example working of how to set this status code, I added the following function.

func ResponseExampleHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(200)
    fmt.Fprintln(w, "Testing status code. Manually added a 200 Status OK.")
    fmt.Fprintln(w, "Another line.")
}

With that, add a handler to the main function.

http.HandleFunc("/response", ResponseExampleHandler)

Now we’re ready to try that out. In the upper right of Postman, the 200 status is displayed. The other data is shown in the respective body & header details of the response.

6

Next up, let’s just write a function specifically to return an error. We’ll use the standard old default 500 status code.

func ErrorResponseHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(500)
    fmt.Fprintln(w, "Server error.")
}

Then in main, as before, we’ll add the http handle for the function handler.

http.HandleFunc("/errexample", ErrorResponseHandler)

Now if the server is run again and an HTTP request is sent to the end point, the status code changes to 500 and the message “Server error.” displays on the page.

7

Summary

That’s a quick intro to writing an HTTP server with Go. From here, we can take many next steps such as writing tests to verify the function handlers, or setup a Docker image in which to deploy the server itself. In subsequent blog entries I’ll write up just those and many other step by step procedures. For now, a great next step is to expand on trying out the different functions and features of the http library.

That’s it for now. However, if you’re interested in joing me to write some JavaScript, Go, Python, Terraform, and more infrastructure, web dev, and coding in general I stream regularly on Twitch at https://twitch.tv/adronhall, post the VOD’s to YouTube along with entirely new tech and metal content at https://youtube.com/c/ThrashingCode. Feel free to check out a coding session, ask questions, interject, or just come and enjoy the tunes!

Cobra + Viper Go CLI Sample

I just released a Go Cobra + Viper library CLI App sample app here. Here are the main up to date details as of release today.

The app example is using the Cobra and Viper libraries for Go. This application provides commands (per Cobra) to read, write, update, and delete configuration records from a configuration file using Viper.

Using the example CLI App.

<cli> config -h provides the following documentation for using this CLI App.

Available Commands:
  add         The 'add' subcommand will add a passed in key value pair to the application configuration file.
  delete      The 'delete' subcommand removes a key value pair from the configuration file. 
  update      The 'update' subcommand will update a passed in key value pair for an existing set of data to the application configuration file.
  view        The 'view' subcommand will provide a list of keys and a map of the values.

Flags:
  -h, --help           help for config
  -k, --key string     The key for the key value set to add to the configuration.
  -v, --value string   The value for the key value set to add to the configuration.

Currently I got this to work but it isn’t in the best shape. Check out the code here.

Get Involved, Add Samples, Make Requests

If you’d like to get involved and add samples or make request for additional samples please file an issue here!

If you’ve found any bugs or issues with the code please file a bug report here!

Examples, The CRUD!

./cli config add -k "blog" -v "https://compositecode.blog/" example writes a record to the configuration file with a key of “blog” and a value of “https://compositecode.blog/“.

./cli config view displays the contents of the configuration file and CLI specific environment variables. These are the configuration files located in the .cobrae-cli-samples.yml and environment variables prefaced with COBRACLISAMPLES. The list of keys is displaced first and then the keys and values are displayed below that.

./cli config update -k "blog" -v "not found" will update the blog entry in the configuration to read not found for the value.

./cli config delete ... will delete the key and value from the configuration file.

Building the Project

Following a fairly standard clone, one can build this project with a single step using the ./build.sh file. If you’d like to contribute the same for Windows, feel free I’d be happy to pull that PR in. Once the project is built use the CLI as defined above.

Installing the CLI App

This application can be installed as a CLI app by referencing it’s location in your bash (powershell? etc) startup script.

Schedule Updates: Coding, Meetups, Twitch Streams, Etc.

Here is my updated schedule for the two weeks starting August 12th for my Twitch streams, the basic topics, meetups (a little beyond two weeks), and related events coming up.

Series: Thrashing Code General Calamity

This is going to be a mix of tech this week. Probably some database hacking, code hacking, samples, and setup of even more examples for use throughout your coding week!

SeriesMeetups

This is a little further out than the next few weeks, but in August we’re having another meetup!

Series: Emerald City Tech Conversations

Join me and several guests, on several different days to get into particulars about programming, coding, hacking, ops, devops, or whatever comes up. Ask questions, inquire, infer, or inform we’re here for the conversation. Join my guests and I!

SeriesDataStax’s Apache Cassandra & C# Hour (C# Schema Migration builder, C# driver, and additional content). Navigate through to the DataStax Developers Channel to check out the event times.

The regularly occurring DataStax Devs series are continuing too, so join me, Cristina, Eric, and others on the team for these episodes and more. Check out the channel here!