Author Archives: Adron

About Adron

See: https://compositecode.blog/about

Learning Go Episode 2 – Further into packages, dependencies, application creation, and IDE’s

Episode Post & Video Links:  1, 2 (this post), 3, 4 (almost done)

In episode two I went over a lot of the material that I covered in the first episode, but added more context, historical reasons certain things are the way they are in Go and the stack, and went over a number of new elements of information too. One thing I got further into this episode is the package and dependency management with Go Dep and also how to create a package, or dependency library for use in other Go libraries or applications. It’s just a small introduction in this episode, but pivotal to future episodes, as I’ll be jumping further into library creation and related details.

In this post I’ve got the time point breakdown like usual,  but also a few additional bits of information and code examples, plus links to the repository I’ve setup for this particular episode. The quick links to those references are below, and also I’ll link at particular call out points within the time points.

Quick Links:

Key Topics Covered

Data Types, Packages, and Dependency Management

2:52 – Fumbling through getting started. It is after all Saturday morning!
3:00 – Recap of what we covered in the first session. Includes a quick review of the previous session code too, such as the random data generation library we setup and used.
6:40 – Covering some specifics of the IDE’s, the stories of the benefits of Go having a specific and somewhat detailed convention to the way syntax, variables, and related features are used.
7:40 – Covering gofmt and what it gives us.
9:45 – Looking at the gofmt plugins and IDE features around the conventions.
14:06 – New example time! In this one, I work through an example showing how to find duplicate lines in passed in text.

Duplicate Line Finder

I went through the various steps of creating the code, but then took a little bit of a detour from the example in the book. Instead of lines by the CLI it takes in content from a text file. The code in main.go ended up like this.

Then if you’d like to check out the text file and remaining content in that project, check out the master branch of the episode 2 repo.

36:34 – Here I take a thorough step through committing this project to github, which is the included repo in this post. However I step through the interface of using Jetbrains Goland to do the commit, how it enables gofmt and other features to improve the condition of code and ensure it meets linter demands and related crtieria. I also cover the .gitignore file and other elements to create a usable repository.
44:30 – Setting up the repository for today’s code at https://github.com/Adron/learning-go-…
50:00 – Setup of the key again for using Github. How to setup your ssh keys using ssh-keygen.
56:00 – Going beyond just the language, and building out a Go build on Travis CI.
1:10:16 – Creating a new branch for the next code examples and topics. At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

The tempApp Branch is available in the repository here.

At this point I shift into type declarations. Working through some constants, very basic function declarations, and related capabilities to calculate temperatures between Fahrenheit and Celsius.

During this point, we take a look at our first package. This package ended up looking like this.

In the main.go file, I showed how you can use this package by adding a respective import shown in this code.

1:17:54 – At this point, to increase readability of font sizes I get into the various Goland IDE options.
1:38:12 – Creating the final branch for this session to pull in a public package and use it in project. For this, I pull in a random data generation package to use in some application code.

1:44:50 – Further discussion and explanation of what to include in .gitignore files to manage projects, but also what is and isn’t included for dependencies and other details around all of this.
2:13:22 – The wicked awesome hacker outtro.

Fixing Windows 10 Drives for Linux Mounting

I mentioned, since I’d run into this problem, a while back in my post “Oh, exFAT Doesn’t Work on Linux” which then includes a fix. Here’s a slightly more updated fix for the same and another fix for mounting just the Windows 10 partition if you have a Windows 10 System Drive (i.e. the drive with the OS on it). First, the exFAT install can simply be done with the following, no need to go get it with git via a clone.

sudo apt-get install exfat-fuse exfat-utils

The other trick is to get a Windows 10 System Drive mounted. To do that, open up the disks application and click the gears on the drive. Then make the following settings by clicking off for “User Session Defaults”. Then you can set “Mount at system startup” and add “remove_hiberfile” to the end of the command parameters as shown. Now the drive can be mounted, just click the mount button and you should be set.

drive-setup-for-mount-windows-10-drive.png

Windows 10’s Path of Destruction and Garbage Support for Linux from Dell

At this point, I’d say do not buy a Dell XPS 15 unless you are only going to use Windows 10 apps and such on the machine. Linux runs like garbage on the machine. The generic drivers that allow one to sort of load Linux onto the machine work mediocre at best, with lot’s of crashes, kernel panics, and related wrecks within seconds and minutes (if you get that far) after starting the machine with Linux. So that the situation there.

Docker, Hyper-V, and VirtualBox Conflict

As I’ve used Windows 10 I keep hitting hard roadblocks on getting things done. After years of MacOS and Linux usage it’s kind of an ongoing, slow motion, insulting train wreck how monopolistic a lot of their applications are still built to run. For example, the latest application that I’ve tried to use that is effectively disabled is Virtual Box. Upon further research I realized I can’t used Virtual Box if I have Hyper-V installed. But that means if I want to use the latest and greatest Docker software then I either have to enable Hyper-V and use Docker or disable Docker and Hyper-V and use Virtual Box. Which means I could go back a version or two of this and that and use Docker-machine with Virtual Box, but, seriously, that’s ridiculous.

The way, from what I understand so far, is that they’ve built the operation of Hyper-V to work on Windows 10 in a way that it effectively prevents Virtual Box, and I suppose VMWare, from actually operating. So one has to go and disable Hyper-V in order to be able to run Virtual Box.

Enabling / Disabling VirtualBox and Hyper-V

Ok, here’s one way to do this with Powershell. Open up Powershell with the “Run As Administrator” option. You’ll have to do that by clicking the start menu (or whatever it’s called these days) and typing “power” which brings up the Powershell application icon. Right click on it and select “Run as Administrator”.

dism_upd

hyper-vHere’s the dialog where one can disable Hyper-V to run VirtualBox too. One this is run, or disabled, Windows 10 will likely want you to restart your computer. You’ll actually need to do this too, because VirtualBox will still die out without a clean OS launch for the base underlying operating system.

Once the restart is complete however, VirtualBox should start up and run virtual machines just fine. Emphasis as usual for Windows on the keyword of should.

Virtual Machine SITREP Resolved Poorly

At this point I’m moderately ok with this solution. It just means that I need to re-create all of my images in Hyper-V and nuke VirtualBox though. I’m really indifferent on which one I need to use, but would have liked to have RTFM’ed the docs before so I had realized upgrading to Windows 10 and using Hyper-V for Docker would kill off my VirtualBox images that I’ve been using for months. Oh well, onward!

The other thing that I find horrifying about this though, is as I’ve learned why and how Hyper-V disabled VirtualBox from running side by side, it seems like 90’s era monopolistic Microsoft building something in a way that disables competition. Ok, so I know it isn’t really because of that, but it’s close enough that it just seem dirty, nasty, and all around kind of disingenuous on Microsoft’s part. Especially considering they neglected Hyper-V for so many years and of course now are all like, “hey, Hyper-V is the way the future, nobody uses it outside of the Windows camp, but it’s the way of the future so bow down and use it Windows users!!” But whatever, I’m at least back in business to get some things done and get some development images put together in Hyper-V.

Hyper-V plus Docker for Windows 10

At this point, since Linux has such poor support on the Dell XPS 15 I’m just sticking with Windows 10 and going the Hyper-V + Docker + Virtual Machines path for what I’m working on. More on this soon, and I’ll be mentioning various things on Twitch soon too. When I get back around to getting the Linux + XPS 15 situation sorted out and tried out again I’ll be sure to blog that too!

 

 

Distributed Database Things to Know: Snitches

Snitches. What a great name for a feature right? I’d bring up the Harry Potter thing, but I’m gonna let that one fly. (get it, it flies!)

A snitch determines where nodes go among the racks and datacenters. This is the Cassandra specific racks and datacenters however, so check out my previous post on datacenters and racks for more detail on the specifics about what they are in relation to Cassandra and DataStax Enterprise (DSE). Snitches tell the database about the network topology of the system. Requests can then be routed efficiently and enables Cassandra and DSE to distribute replicas by grouping the machines accordingly. Of the nodes, all within a cluster must use this same snitch in the logic of distribution among the system.

Snitch Options

The following are the feature options we have to determine how the snitches determine node placement.

  • DseSimpleSnitch – This is the default snitch and is intended only for development deployments. It doesn’t recognize datacenter or rack information, and simply needs a a keyspace defined to use SimpleStrategy and set a replication factor. It’s use makes it a bit easier to setup a cluster for development.
  • GossipingPropertyFileSnitch – This snitch is usable for production. Rack and datacenter information for the local node is defined in the cassandra-rackdc.properties file, which then propagates this to other nodes via gossip.
  • Ec2Snitch – This is a great snitch for simple cluster deployments that reside in a single region. For this snitch, the region name is used as the datacenter name and availability zones are setup as racks. That gives us a setup that matches datacenter and racks to region and zones, making it pretty easy to remember which is where then. Since this maps this way, as the way Ec2 works, this snitch isn’t usable among multi-region clusters.
  • Ec2MultiRegionSnitch – This snitch can be used for multi-region deployments. To use this snitch settings need to be made in both the cassandra.yaml file and cassandra-rackdc.properties file. The way this snitch works is by using the public IP designated in the broadcast_address to allow this multi-region connection.
  • GoogleCloudSnitch – This snitch, as is somewhat obvious by the name, is for DSE deployments on Google Cloud Platform (GCP). This snitch uses datacenters and racks similarly mapped as the Ec2Snitch with datacenters mapped to regions and racks mapped to zones.
  • CloudstackSnitch – This snitch is for Apache Cloudstack. Zone naming is free-form in Cloudstack so this snitch uses <country> <location> <az> notation.
  • PropertyFileSnitch – The way this snitch works is by proximity, determined by rack and datacenter. It uses network details configured in cassandra-topology.properties file, with the datacenter names defined using standard convention. These need to correlated to the name of the actual datacenters in the keyspace definition. Then nodes in the cluster are described in the cassandra-topology.properties file and must be exactly the same on every node in the cluster.
  • RackInferringSnitch – This snitch is kind of funny, because it’s a usable snitch, but it’s also an example snitch. It determines the proximity of nodes by datacenter and rack too. However it assumes these to correspond to the second and third octet of the node’s IP address. It is best used as an example for writing custom snitch classes, unless of course this matches your actual deployment conventions.

That’s the basics on snitches. I recently wrote about another important distributed database architectural concept called consistent hashing, it’s an important concept to understand about distributed databases like Cassandra and DataStax Enterprise.

References:

Learning Go Episode 1 – Environment, Go Workspace, GOPATH/GOROOT, Types, and more Introduction

This is episode one of a multi-part series on “The Go Programming Language“. Not necessary, but if you’d like to follow along you can also pick up the book “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan. At the bottom of the description I have a link to the book publisher’s website and the respective book. I’ll be using that as a guideline and using a number of examples from the book. However I’ll also be adding a lot of additional material around Goland IDE from Jetbrains and Visual Studio Code. The video link to this session is at the bottom of the post, scroll all the way down and it’s waiting for you there.

3:28 – Getting started, introducing that the session I’m starting with a completely new Ubuntu Linux load so that I ensure we cover all of the steps to get up and running. These steps, even though they’re on Linux are reproducible on Windows 10 and MacOS, so any operating system is usable to follow along with, with only minor discrepancies.

5:04 – Introducing the book that I’ll be using as a guideline reference so that viewers can also follow along with a physical book. I’m a big fan of multisensory learning, so between a book, the stream, being able to ask questions in the channel, it’ll give viewers a chance to learn and undertake their coding adventures in Go using all sorts of methods.

Book Reference: “The Go Programming Language” by Alan A. A. Donovan and Brian W. Kernighan

6:58 – Discussing where Go is located on the web related to Github and the golang.org site that is useful in that one can even try out little snippets of Go code itself, on the site!

Github: https://github.com/golang/go
Golang: https://golang.org

10:40 – Setting export in the .bashrc file (or .bash_profile on MacOS or environment variables on Windows 10). Speaking of Windows 10 specifically, Linda Gregier wrote up a great blog post on getting Go setup on Windows specifically.

14:50 – Setting up the Go workspace path for GOPATH using the standard Go convention. From here I get into the first “Hello World!” with Go.

15:34 – Mention of setting up Go on a Docker container and how it is easier, but we’re staying focused on setting it up completely from scratch.

18:20 – Starting first code, a standard “Hello World” program.

19:50 – First build of that “Hello World” program.

20:34 – Here I introduce go run and how to execute a singular file instead of building an entire project.

21:32 – Installing some IDE’s to use for developing Go applications. The first two up for installation is Visual Studio Code and JetBrains Goland.

29:00 – A first variable, what is it, and how to declare one in Go in one of the ways one can declare a variable in Go!

31:08 – Introducing the terminal in Visual Studio Code.

37:12 – A little example of OBS, how I’m using it, and how I interact back and forth with chat and related tooling plus the virtual machine itself.

42:36 – Changing themes and adding plugins for Goland. In the plugins I also find the most epic of status bars, the Nyan Cat!

59:00 – Here I start to get back into some more specific Go details. Starting off with a Go command line parsing application. At this point I also cover several additional ways to declare variables, speak more about short declarations, and other ways to declare, assign, and use variables in Go.

At this point I also go through a number of examples to exemplify how to go about declaring variables, build, run, and explore the details of the code. Further along I also get into string formatting, concatenating, and related string manipulation with Go.

Other details include taking a look at extra ways to figure out Go code using autocomplete inside Goland and other exploratory features. Eventually before wrapping up I broach pointers, tuple declaration techniques, and how to declare additional functions beyond func main().

1:58:40 – Adding dependencies and generating random data. At this point I bring in a dependency. But before pulling in the dependency, after introducing it, I show how to go about doing.

2:00:10 – New machine, and I run into git not being available. The go get command uses git to pull dependencies so I go about why this is needed and the steps to install on Ubuntu.

2:09:20 – Introduction to more concepts around dependencies, what go get does versus managing dependencies with Go Dep.

2:10:00 – Installing Go Dep; MacOS, using curl, Linux installation, and a question sort of remains to get it running on Windows. The easiest method is using chocolatey however, so check that out if you’re going the Windows route.

2:15:20 – Setting up Go Dep integration with Goland.

2:23:55 – Showing off Goland’s commit dialog and some of the respective options.

4 Discovered Axioms of #DevRel

The idea of DevRel, or Developer Relations and the position of Developer Advocates in the industry has become more defined in the last decade than it traditionally has been. In getting to this point there are several key points that have come up that are practical axioms in industry. Some people don’t agree with all of these, and I’d infer that they’re probably just wrong, but the vast majority in industry and specifically working in DevRel itself have these axioms that they’d often stand by. If not march up on the hill to fight for!

  1. Developer Advocates and Developer Relations should NOT exist under any marketing hierarchy. Microsoft killed off this organizational structure, Google never let it happen, and AWS also insured this isn’t how this operated. If anything it’s either it’s own branch feeding directly into the executive team under the CTO, or it is a breakout of the engineering group usually under a VP of engineering or related structural organization. Having Developer Advocates under marketing tends to bring out bad habits (forced talks at trade shows that are just the company spiel) or topics that just don’t align to anything (like talks on X feature that nobody uses implemented in a way that is broken). The end product of having Developer Advocates and Developer Relations work and report up to a marketing leadership hierarchy devalues their work, what they can and indeed do provide that is valuable, and can diminish the credibility that advocates have to fight for so diligently in the first place. For further ideas around this axiom, Francine Hardaway also wrote a great post on just this issue, asking where DevRel should exist.
  2. DevRel & Developer Advocates need to be self-disciplined, build, show, and be technically inclined as much as any software engineer, coder, hacker, or related individual is expected to be. I’m not talking about “make nonsense deadlines and work to death” like some development teams get stuck with, but we advocates do need to build solutions that parallel or innovate on the designs that are in place, in production, and giving us value today. Developer Relations at its core is there to bring value and show value in what X solution can do but needs to provide example and take what exists in industry and build on it.
  3. Developer Advocates serve a two way street of communication, one to developers and users and one back to the internal engineers, product, and leadership working on building products and services. Advocates collect, or as I sometimes call it, perform reconnoiter or reconnaissance, and bring that data back to the various teams within company to determine actions to take. I personally love this part of the job, since I like to make sure that the development teams have the information they need to build products and services that are really needed, valuable, and will get the most bang for the buck. I’ve also never met a developer that doesn’t want to know the direction their developing in is the right direction. This kind of direct data is an invaluable information base for the development teams.
  4. Developer Advocates do not always work directly with customers, but we do indeed and should be communicating with them on a regular basis. Helping to organize discussions, conversations, and future directions of research for product and services usage is very important. We can act as that individual or team for companies that often don’t have enough time to put somebody on a research project, where as we can do that, and provide general information deducting what is or isn’t’ the right path to travel. As developer advocates we have the freedom to often take the path of risky research. We provide an extremely valuable service to the companies we work for, the customers we communicate with, and the industry as a whole by doing this research and making it available (i.e. blog it!)

Got anymore axioms you see in industry around DevRel work? I’d be happy to put together a larger list, this is just the beginning so far as I begin the first steps of a journey into understanding future directions and detailed specifics about how advocacy can increase its value for company, customer, and personal efforts.

Let’s Talk Top 7 Options for Database Gumbo

When one starts to dig into databases things get really complex really fast. There’s not only a whole plethora of database companies and projects, but database types, storage engines, and other options and functionality to choose from. One place to get a start is just to take a look at the crazy long list of databases on db-engines. In this post I’m going to take a look at a few of the top database engines to create a starting point – which I’ll reference – for future video streaming coding sessions (follow me @ twitch.tv/adronhall).

My Options for Database Gumbo

  1. Apache Cassandra / DataStax Enterprise
  2. Postgresql
  3. SQL Server
  4. Elasticsearch
  5. Redis
  6. SQLite
  7. Dynamo DB

The Reasons

Ok, so the list is as such, and as stated it’s my list. There are a lot of databases, and of course some are still more used such as Oracle. However here’s some of the logic and reasoning behind my choices above.

Oracle

First off I feel like I need to broach the Oracle topic. Mostly because of their general use in industry. I’m not doing anything with Oracle now, nor have I for years for a long, long, LONG list of reasons. Using their software tends to be buried in bureaucratic, oddly broken and unnecessary usage today anyway. They use predatory market tactics, completely dishonorable approach to sales and services, as well as threatening and suing people for doing benchmarks, and a host of other practices. In face to face experiences, Oracle tends to give off experiences, that Lawrence from Office Space would say, “naw man, I think you’d get your ass kicked for that!” and I agree. Oracle’s practices are too often disgusting. But even from the purely technical point of view, the Oracle Database and ecosystem itself really isn’t better than other options out there. It is indeed a better, more intelligently strategic and tactical option to use a number of alternatives.

Apache Cassandra / DataStax Enterprise

This combo has multiple reasons and logic to be on the list. First and foremost, much of my work today is using DataStax Enterprise (DSE) and Apache Cassandra since I work for DataStax. But it’s important to know I didn’t just go to DataStax because I needed a job, but because I chose them (and obviously they chose me by hiring me) because of the team and technology. Yes, they pay me, but it’s very much a two way street, I advocate Cassandra and DSE because I personally know the tech is top tier and solid.

On the fact that Apache Cassandra is top tier and solid, it is simply the remaining truly masterless distributed database that provides a linear path of scalability on the market that you can use, buy support for, and is actually actively and knowingly maintained not just by DataStax but by members of the community. One could make an argument for MongoDB but I’ll maybe elaborate on that in the future.

In addition to being a solid distributed database there are capabilities inherent in Apache Cassandra because of the data types and respective the CQL (Cassandra Query Language) that make it a great database to use too. DataStax Enterprise extends that to provide spatial (re: GIS/Geo Data/Queries), graph data, analytics engine, and more built on other components like SOLR and related technology. Overall a great database and great prospective combinations with the database.

Postgresql

Postgres is a relational database that has been around for a long time. It’s got some really awesome features like native JSON support, which I’m a big fan of. But I digress, there’s tons of other material that lays out thoroughly why to use Postgres which I very much agree with.

Just from the perspective of the extensive and rich data types Postgres is enough to be put on this list, but considering there are a lot of reasons around multi-tenancy, scalability, and related characteristics that are mostly unique to Postgres it’s held a solid position.

SQL Server

This one is on my list for a few reasons that have nothing to do with features or capabilities. This is the first database I was responsible for in its entirety. Administration, queries, query tuning, setup, and developer against with the application tier. I think of all my experience, this database I’ve spent the most time with, with Apache Cassandra being a close second, then Postgres and finally Riak.

Kind of a pattern there eh? Relational, distributed, relational, distributed!

The other thing about SQL Server however is the integrations, tooling, and related development ecosystem around SQL Server is above and beyond most options out there. Maybe, with a big maybe, Oracle’s ecosystem might be comparable but the pricing is insanely different. In that SQL Server basically can carry the whole workload, reporting, ETL, and other feature capabilities that the Oracle ecosystem has traditionally done. Combine SQL Server with SSIS (SQL Server Integration Services), SSRS (SQL Server Reporting Services), and other online systems like Azure’s SQL Database and the support, tooling, and ecosystem is just massive. Even though I’ve had my ins and outs with Microsoft over the years, I’ve always found myself enjoying working on SQL Server and it’s respective tooling options and such. It’s a feature rich, complete, solidly, and generally well performing relational database, full stop.

Elasticsearch

Ok, this is kind of a distributed database of sorts but focused more exclusively (not totally since it’s kind of expanded its roles) search engine. Overall I’ve had good experiences with Elasticsearch and it’s respective ELK (or Elastic ecosystem) of tooling and such, with some frustrating flakiness here and there over the years. Most of my experience has come from an operational point of view with Elasticsearch. I’ve however done a fair bit of work over the years in supporting teams that are doing actual software development against the system. I probably won’t write a huge amount about Elasticsearch in the coming months, but I’ll definitely bring it up at certain times.

Redis / SQLite / DynamoDB

These I’ll be covering in the coming months. For Redis and DynamoDB I have wanted to dig in for some comparison analysis from the perspective of implementing data tiers against these databases, where they are a good option, and determining where they’re just an outright bad option.

For SQLite I’ve used it on and off for many years, but have wanted to sit down and just learn it and try out some of its features a bit more.