Learning Go Episode 5 – Functions (and Methods and lots of other things)

Episode Post & Video Links:  1, 2, 3, 4, 5 (this post), 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

In this session we covered a host of topics around Go Functions. Along with some troubleshooting, debugging, and other features in Jetbrains Goland IDE.

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan as a starting point. I’m using it as a simple guideline, but also doing a lot more in each stream that includes ecosystem, dependency management with godep, IDE use of Goland from Jetbrains, and more. In this session I get specifically into: functions, signatures, declarations, recursion, return values, and more.

Video Time Points & Topics

2:50 – Introduction to the snowy wonderland of Seattle and episode 5 of the Learning Go series. Introduction to the various screen transitions and such.
6:40 – Getting started, opening up JetBrains Goland and creating a new project. The project exists on Github as https://github.com/adron/learning-go-….
12:18 – Starting with functions in Go. See the blog entry I wrote on the topic for more additional information around this first code session within the episode 5 session.

Code – This first example I setup a basic function in Go that is called by the main function. The sample function below I’ve named exampleExecutor, and the signature is made up of an int parameter called this, a string parameter called that, and a return parameter of type int and one of type string. In summary for the function signature we have two input parameters going in and two return parameters coming out.

The function does very little besides print the parameters passed in and then return the parameters back out as the return parameters.


package main
import "fmt"
func main() {
var this, result int
var that, message string
this = 2
that = "42"
result, message = exampleExecutor(this, that)
fmt.Printf("%s\n%d", message, result)
}
func exampleExecutor(this int, that string) (int, string) {
fmt.Printf("Numbers: %d, %s\n", this, that)
return this, "This is the result: " + that
}

Recursion with Go & HTML Parsing

28:10 – Here I get into recursion and the application example, largely taken from the book but with some very distinctive modifications, that parses HTML and the various nodes within an HTML document.

For the recursion section I use an example from the book with an expanded sample set of HTML. The HTML is included in the repo under the function-recursion branch. For this example I setup a set of types and variables up that are needed throughout the code.

First a type setup called NodeType of type int32. A constant array of ErrorNode, TextNode, DocumentNode, ElementNode, CommentNode, and DoctypeNode for determining the different nodes within an HTML document. Then a general struct called Node with Type, Data, Attr for attribute, and FirstChild with NextSibling setup as a pointer to *Node, which gives a type of memory recursion to the underlying type. Then finally the Attribute struct with a Key a Value.


type NodeType int32
const (
ErrorNode NodeType = iota
TextNode
DocumentNode
ElementNode
Commentnode
DoctypeNode
)
type Node struct {
Type NodeType
Data string
Attr []Attribute
FirstChild, NextSibling *Node
}
type Attribute struct {
Key, Val string
}

One of the first functions I then end up with is the visit function. It turns out as shown below. Here the function takes a links parameter that is of type string array, a parameter name n that is a pointer reference to an a node within html, and then the function returns a parameter of type string array.


func visit(links []string, n *html.Node) []string {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key == "href" {
links = append(links, a.Val)
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
links = visit(links, c)
}
return links
}

view raw

visit_func.go

hosted with ❤ by GitHub

After that function I worked through and created two additional functions, one for countsWordsAndImages and one called CountWordsAndImages. The casing being specific to scope and use of the functions, and they respectively look like this in completion.


func CountWordsAndImages(url string) (words, images int, err error) {
resp, err := http.Get(url)
if err != nil {
return
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
err = fmt.Errorf("parsing HTML: %s", err)
}
words, images = countWordsAndImages(doc)
return
}
func countWordsAndImages(n *html.Node) (words, images int) {
if n.Type == html.TextNode {
words += len(strings.Split(n.Data, " "))
return
}
if n.Data == "img" {
images++
} else {
if n.FirstChild != nil {
w, i := countWordsAndImages(n.FirstChild)
words += w
images += i
}
if n.NextSibling != nil {
w, i := countWordsAndImages(n.NextSibling)
words += w
images += i
}
}
return
}

view raw

counting.go

hosted with ❤ by GitHub

Then all that is wrapped up, with recursive calls and more, in the main function for program execution.


func main() {
htmlContent, err := ioutil.ReadFile("compositecode.html")
if err != nil {
fmt.Println(err)
}
htmlData := string(htmlContent)
r := strings.NewReader(htmlData)
doc, err := html.Parse(r)
if err != nil {
fmt.Fprintf(os.Stderr, "find links: %v\n", err)
os.Exit(1)
}
for _, link := range visit(nil, doc) {
fmt.Println(link)
}
w, i, _ := CountWordsAndImages("https://compositecode.blog")
fmt.Printf("Words: %d\nImages: %d\n", w, i)
}

Starting Error Handling && Anonymous Functions

1:32:40 – At this point in the episode 5 session I get into a simple Error handling function, and further into function signatures and how to set them up.
1:52:24 – Setting up some anonymous functions and reviewing what they are.
1:59:00 – Introduction to panics in Go. After this short introduction I also discuss some of the pedantic specifics of methods vs functions and related verbiage around the Go language. Additionally I provide more examples around these specifics for declaring functions, various scope, and other types for function calls and related usage.

With that done the wrap up of the session is then a short introduction to anonymous functions.

Cassie Schema Migrator >> CaSMa

A few weeks back I started working on a schema migration tool for Apache Cassandra and DataStax Enterprise. Just for context, here are the short definitions of what each of the elements of CaSMa are.

  • cstar-iconApache Cassandra
    • Definition: Apache Cassandra is a free and open-source, distributed, wide column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure.
    • History: Avinash Lakshman, one of the authors of Amazon’s Dynamo, and Prashant Malik initially developed Cassandra at Facebook to power the Facebook inbox search feature. Facebook released Cassandra as an open-source project on Google code in July 2008. In March 2009 it became an Apache Incubator project. On February 17, 2010 it graduated to a top-level project. Facebook developers named their database after the Trojan mythological prophet Cassandra, with classical allusions to a curse on an oracle.
  • dse-logoDataStax Enterprise
    • Definition: DataStax Enterprise, or routinely just referred to as DSE, is an extended version of Apache Cassandra with multi-model capabilities around graph, search, analytics, and other features like security capabilities and a core data engine 2x speed improvement.
    • History: DataStax was formed in 2009 by Jonathan Ellis and Matt Pfeil and originally named Riptano. In 2011 Riptano changes names to DataStax. For more history check out the Wikipedia page or company page for a timeline of events.
  • command-toolsSchema Migration
    • Definition:In software engineering, schema migration (also database migration, database change management) refers to the management of incremental, reversible changes to relational database schemas. A schema migration is performed on a database whenever it is necessary to update or revert that database’s schema to some newer or older version. Migrations are generally performed programmatically by using a schema migration tool. When invoked with a specified desired schema version, the tool automates the successive application or reversal of an appropriate sequence of schema changes until it is brought to the desired state.
    • Addition reference and related materials:

iconmonstr-twitch-5Over the next dozen weeks or so as I work on this application via the DataStax Devs Twitch stream (next coding session events list) I’ll also be posting some blog posts in parallel about schema migration and my intent to expand on the notion of schema migration specifically for multi-model databases and larger scale NoSQL systems; namely Apache Cassandra and DataStax Enterprise. Here’s a shortlist for the next three episodes;

The other important pieces include the current code base on Github, the continuous integration build, and the tasks and issues.

Alright, now that all the collateral and context is listed, let’s get into at a high level what this is all about.

CaSMa’s Mission

Schema migration is a powerful tool to get a project on track and consistently deployed and development working against the core database(s). However, it’s largely entrenched in the relational database realm. This means it’s almost entirely focused on a schema with the notions of primary and foreign keys, the complexities around many to many relationships, indexes, and other errata that needs to be built consistently for a relational database. Many of those things need to be built for a distributed columnar store, key value, graph, time series, or a million other possibilities too. However, in our current data schema world, that tooling isn’t always readily available.

The mission of CaSMa is to first resolve this gap around schema migration, first and foremost for Apache Cassandra and prospectively in turn for DataStax Enterprise and then onward for other database systems. Then the mission will continue around multi-model systems that should, can, and ought to take advantage of schema migration for graph, and related schema modeling. At some point the mission will expand to include other schema, data, and state management focused around software development and data needs within that state

As progress continues I’ll publish additional posts here on the different data model concepts and nature behind various multi-model database options. These modeling options will put us in a position to work consistently, context based, and seamlessly with ongoing development efforts. In addition to all this, there will be the weekly Twitch sessions where I’ll get into coding and reviewing what coding I’ve done off camera too. Check those out on the DataStax Devs Channel.

If you’d like to get into the project and help out just ping me via Twitter @Adron or message me here.

Learning Go Episode 4 – Composite Types, Slices, Arrays, Etc.

Episode Post & Video Links:  1, 2, 3, 4 (this post), 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan. I use it throughout these sessions to provide a guideline. I however add a bunch of other material about IDE’s, development tips n’ tricks and other material.

8:00 Starting the core content with some notes. Composite types, arrays, slices, etc.
11:40 Announcement of my first reload – http://compositecode.blog/2019/02/11/… – second successful time blogged here) of the XPS 15 I have, which – https://youtu.be/f0z1chi4v1Q – actually ended in catastrophe the first time!
14:08 Starting the project for this session.
16:48 Setting up arrays, the things that could be confusing, and setup of our first code for the day. I work through assignment, creation, new vs. comparison, and various other characteristics of working with arrays during this time.


fmt.Println("Hello, let's talk composite types.")
basketOfStuff := [3]string{"The first string","second","This string."}
var zeeValues [2]int
for i, v := range basketOfStuff {
fmt.Printf("Value %d: %s\n", i, v)
}
fmt.Println(zeeValues)
if zeeValues[0] == zeeValues[1] {
fmt.Println("The values are the same, this doesn't instantiate like the `new` keyword.")
} else {
fmt.Println("The way go appears to instantiate unset variable values, such as in this array is like the `new` keyword instantiation.")
}
zeeValues[0] = 1 + 52 * 3
zeeValues[1] = 9
fmt.Println(zeeValues[len(zeeValues) – 1])

29:36 Creation of a type, called Currency, of type int, setting up constants, and using this kind of like an enumerator to work with code that reads cleaner. Plus of course, all the various things that you might want to, or need for a setup of types, ints, and related composite types like this.


type Currency int
const (
USD Currency = iota
CAN
EUR
GBP
JPY
NOK
SEK
DKK
)
symbol := […]string{USD: "$", CAN: "$", EUR: "€", GBP: "£", JPY:"¥", NOK:"kr", SEK:"kr",DKK:"kr"}
fmt.Println(EUR, symbol[EUR])
fmt.Println(JPY, symbol[JPY])
r := […]int{99: -1}
r[36] = 425
r[42] = 42
fmt.Println(r[36] + r[42])
fmt.Println(strconv.Itoa(r[36]))

view raw

currency.go

hosted with ❤ by GitHub

43:48 Creating an example directly from the aforementioned book enumerating the months of the year. This is a great example I just had to work through it a bit for an example.


months := […]string{1: "January", 2:"February", 3: "March", 4:"April", 12:"December"}
for _, s := range months {
fmt.Printf("The month: %s\n", s)
}
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}

view raw

the_months.go

hosted with ❤ by GitHub

52:40 Here I start showing, and in the process, doing some learning of my own about runes. I wasn’t really familiar with them before digging in just now!


… the rest of the main.go file is here…
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}
}
func appendInt(x []int, i int) []int {
var z []int
zlen := len(x) + 1
if zlen <= cap(x) {
z = x[:zlen]
} else {
zcap := zlen
if zcap < 2* len(x) {
zcap = 2 * len(x)
}
z = make([]int, zlen, zcap)
copy(z, x)
}
return z
}

view raw

main.go

hosted with ❤ by GitHub

1:09:40 Here I break things down and start a new branch for some additional examples. I also derail off into some other things about meetups and such for a short bit. Skip to the next code bits at the next time point.
1:23:58 From here on to the remainder of the video I work through a few examples of how to setup maps, how make works, and related coding around how to retrieve, set, and otherwise manipulate the maps one you’ve got them.


package main
import "fmt"
func main() {
ages := map[string]int{
"Peterson": 52,
"Sally": 22,
"Javovia": 15,
"Ben": 42,
}
jobAssociation := make(map[string]string)
jobAssociation["Peterson"] = "Engineer"
jobAssociation["Sally"] = "CEO"
jobAssociation["Jovovia"] = "Gamer"
jobAssociation["Ben"] = "Programmer"
printAges(ages)
printJobAssociations(jobAssociation)
fmt.Println(ages)
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"])
fmt.Println(jobAssociation["Frank"]) // Blank! 😮
fmt.Println(ages["Sally"])
delete(ages, "Sally")
fmt.Println(ages)
fmt.Println(ages["Sally"])
delete(jobAssociation, "Jovovia")
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"]) // Blank.
ages2 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
ages3 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
if equal(ages, ages2) {
fmt.Println("Naw, not really equal.")
} else {
fmt.Println("This is correct, not equal.")
}
if equal(ages2, ages3) {
fmt.Println("True, these are effectively the same map values and keys.")
}
}
func printJobAssociations(associations map[string]string) {
for name, job := range associations {
fmt.Printf("%s\t%s\n", name, job)
}
}
func printAges(ages map[string]int) {
for name, age := range ages {
fmt.Printf("%s\t%d\n", name, age)
}
}
func equal(x, y map[string]int) bool {
if len(x) != len(y) {
return false
}
for k, xv := range x {
if yv, ok := y[k]; !ok || yv != xv {
return false
}
}
return true
}

view raw

map_types.go

hosted with ❤ by GitHub

That’s it for this synopsis. Until next episode, happy code thrashing and go coding!

DevRel Data: Presentation & Deductions

Before diving into conclusions, let’s take a look at some answers to questions asked. This is a slice of answers, with totals for the charts and such. After a few months of answers I’ll have another follow up to see how things may or may not change.

Do you like video material?

chart

What specifically do you, or would you like to watch in video? Screencasts, short videos, conversational, or some other type of videos?

  • Screencasts/tutorials
  • I love both screencasts going through big topics and short videos that cover smaller tips and gotchas.
  • Videos with a specific outcome as the goal, whether achieved or not. Showing the process of something.. like hey, here’s how you building out a Postgres cluster using streaming replication and repmgr and pgpool… Kind of thing.
  • Bite sized content, maybe 2 minutes, to teach me one thing.
  • Editing. No jokes, no “hey what’s up guys” with 60 second intros. Discuss the problem, then solve it.
  • Demos, learning a new way of doing something
  • Doesn’t matter short or long, but has to be deeply technical with code examples that I can actually apply
  • I watch videos mostly for fun.
  • Screencast
  • Short videos of say 5-10 minutes each covering different concept of the subject matter
  • (videos work best in a classroom setting where time/attention is precommitted, or as part of a tutorial)
  • conversational
  • Short videos.
  • If it’s too long, it ends up on my todo list forever (not good). So shorter is better. And something that benefits from visuals, rather than something that could just be written.
  • I also watch LinkedIn Learning when just starting a new tech. to get a general overview and pick up a tip or tow, then I read books and the Internet from there.
  • short videos

What kind of written material do you like?

chart2

Do you like other material mixed in that details the reason for the tech, the story, or such?

chart3

Is there anything that comes to mind, that you’d like to have me or the team I’m working with (@ DataStax) put together that you’d find useful, entertaining, or related.

  • Place priorities on designing materials for more depth (i.e., more linked material) as well as less attention-nuisance. That’s no criticism of your work, merely the gestalt of where we work — so less noise is a better way to stand out and make materials useful.
  • Maybe focus more on written material – code & architecture material (books, articles) rather than videos and twitch. It is much easier to consume and is easily googlable. Also I’d suggest making blog posts target a specific common issue or question – sometimes I see posts that I don’t really care about or the problem is so narrow that I don’t want to read about it. I’d read about building resilient and highly available architectures in various configurations.
  • Database reliability, scalability, migrations and such stuff is interesting.
  • Anything to do with machine learning.
  • Data model examples, starting up a Cassandra node, configuring YAML, etc

Deductions

I’m going to go backwards through the questions and discuss what I’ve deducted, and in some ways what has surprised me among the answers!

First there’s the “Is there anything that comes to mind, that you’d like to have me or the team I’m working with (@ DataStax) put together that you’d find useful, entertaining, or related.” request and questions.

The answers here didn’t surprise me much at all. Within DevRel from Microsoft to DataStax to Google to many other organizations we have this ongoing battle between “write a whole book on it” or “make it 2 minutes short”. It’s wildly difficult to determine what format, what timing, and what structure material needs to be in for it to be most useful to people. So when I saw the answer that reads, “Place priorities on designing materials for more depth (i.e., more linked material) as well as less attention-nuisance.” I immediately thought, “yeah, for real, but ugh…” it’s difficult. However, I’m working on more thorough material, some of it will be paywalled via LinkedIn Learning or Pluralsight and other material may be available by book in the coming months. But there will be other material that will indeed be long form how to material on how to really put things together – from scratch and from the basis of “we have X thing and need to hack it so we can add a feature”.

The next answer I got in this section that I completely agree with is increasing the focus on written material. I’m making tons of video, and I’ve got that down to the point where it’s actually easier and faster to do most of it then it is to write things down. However I realized, especially from my own point of view, that written material actually ends up being vastly more useful than video material. That’s also why, even with the video material, when I’m covering specifics I aim to provide a linkable timeline and a blog entry with the code and other changes shown in the video. Thanks for reinforcing these efforts and giving me that indirect encouragement to make this process and the results even better. More written material is on its way!

As for the database reliability, scalability, migrations, machine learning, data modeling, Cassandra node starting, and all that it’s in the queue and I’m getting to it as fast as I possibly can.

Next question I asked is, “Do you like other material mixed in that details the reason for the tech, the story, or such?

It appears, albeit not a huge contingent of people, some people are curious about biking, train coding, and making good grub! Hey, that’s groovy cuz I’ve got a show coming out which is basically the behind the scenes videos about all those topics that make the coding and technology hacking possible!

The one outlier in this set however is clearly the request for “Ways to simplify life to dig through those algorithms faster, easier, better?” which I didn’t suspect would be any different then the other answers for this questions. Which left me surprised and ill-prepared on what to do about fulfilling what is clearly a demand. I’ll have to up level my blog posts around algorithms. I did do a couple a long while ago now in “Algorithms 101: Big Sums” which I completed in Go and another I wrote up “Algorithms 101: Roads & Town Centers” which I have 90% of the answer complete but I’ve never finished the blog entry! I guess it’s time to get the algorithm train coupled up and ready to depart!

Then the question, “What kind of written material do you like?

Two options lead by a healthy margin for this question: Demos w/ Write Up and Blog Articles. With this coupled up to the first question it’s clear that written material via blog and demoes via blog should and ought to be top priority. They are, however they’re a whole helluva lot of work, so I only get them produced but so fast. Got some gems coming on Go, Bash, Cassandra, and a few other demo, tech, and historical information.

Next up was single page cheat sheets and documentation, followed closely by books. I kind of expected documentation and books, but wow that single page cheat sheets option is higher rated than I would have suspected and by proxy I’ve immediately added that to my produce this type of material list! I put it in as a very secondary thought but it’s going to get into that increased focus queue.

The last one with some semblance of demand is pamphlet size short form. This one almost seems like a fluke, but I’ll ponder putting together some of these. I know O’Reilly has their short novelette size books which cover a particular topic. They hand these out for free at conferences and seem pretty solid. Maybe I’ll work one of those into the queue? Maybe.

The other three options scraped by with 1%, so somebody was choosing them. So the vi mug isn’t a priority nor the short explainer videos. Which seems in contention with video content demands around shorter content. I guess, explainer videos just doesn’t sound useful!

The next question I just put together a top three of the results, “What specifically do you, or would you like to watch in video? Screencasts, short videos, conversational, or some other type of videos?

  1. Make screen casts.
  2. Make screen casts generally short.
  3. Make screen cases that are short and on a specific and deep dive into a topic.

This seems kind of in conflict with itself, but I’m going to aim for it and try to hone the skill further. So that I can produce screen casts, screen casts that are generally short, and make sure that these screen casts that are short are on a specific and deep dive into a topic. Whew, got it.

Finally, “do you like video material?

chart

At this time, 53.8% of you have said yes. I had guessed it would be around 50%.

I had guessed no would be about 25%, and at 23.1% I wasn’t to far off.

The other respective mishmash of answers made for interesting depth to the questions that followed this question.

Article Summary & TLDR

Produce more topic specific, detailed material around screen casts and blog entries!

End of story.

For more on this information, why I asked, and what I do check out my article titled “Evangelism, Advocacy, and Activism in The Technology Industry” and for some of the big victories for big corporations check out “The Developer Advocates – Observations on Microsoft’s New Competence“.

Let’s Really Discuss Lock In

For to long lock-in has been referred to with an almost entirely negative connotation even though it can be inferred in positive and negative situations. The fact is that there’s a much more nuanced and balanced range to benefits and disadvantages of lock-in. Often this may even be referred to as this or that dependency, but either way a dependency often is just another form of lock in. Weighing those and finding the right balance for your projects can actually lead to lock-in being a positive game changer or something that simply provides one a basis in which to work and operate. Sometimes lock-in actually will provide a way to remove lock-in by providing more choices to other things, that in turn may provide another variance of lock-in.

Concrete Lock-in Examples

The JavaScript Lock-In

IT Security icons. Simplus seriesTake the language we choose to build an application in. JavaScript is a great example. It has become the singular language of the web, at least on the client side. This was long ago, a form of lock-in that browser makers (and standards bodies) chose that dictated how and in which direction the web – at least web pages – would progress.

JavaScript has now become a prominent language on the server side now too thanks to Node.js. It has even moved in as a first class language in serverless technology like AWS’s Lambda. JavaScript is a perfect example of a language, initially being a source of specific lock-in, but required for the client, that eventually expanded to allow programming in a number of other environments – reducing JavaScript’s lock in – but displacing lock in through abstractions to other spaces such as the server side and and serverless functions.

The .NET Windows SQL Server Lock In

IT Security icons. Simplus seriesJavaScript is merely one example, and a relatively positive one that expands one’s options in more ways than limits one’s efforts. But let’s say the decision is made to build a high speed trading platform and choose SQL Server, .NET C#, and Windows Server. Immediately this is a technology combination that has notoriously illuminated in the past * how lock-in can be extremely dangerous.

This application, say it was built out with this set of technology platforms and used stored procedures in SQL Server, locking the application into the specific database, used proprietary Windows specific libraries in .NET with the C# code, and on Windows used IIS specific advances to make the application faster. When it was first built it seemed plenty fast and scaled just right according to the demand at the time.

Fast forward to today. The application now has a sharded database when it hit a mere 8 Terabytes, loaded on two super pumped up – at least for today – servers that have many cores, many CPUs, GPUs, and all that jazz. They came in around $240k each! The application is tightly coupled to a middle tier, that is then sort of tightly coupled to those famous stored procedures, and the application of course has a turbo capability per those IIS Servers.

But today it’s slow. Looking at benchmarks and query times the database is having a hard time dealing with things as is, and the application has outages on a routine basis for a whole variation of reasons. Sometimes tracing and debugging solves the problems quickly, other times the servers just oversubscribe resources and sit thrashing.

Where does this application go? How does one resolve the database loading issues? They’ve already sunk a half million on servers, they’re pegged out already, horizontally scaling isn’t an option, they’re tightly coupled to Window Servers running IIS removing the possibility of effectively scaling out the application servers via container technologies, and other issues. Without recourse, this is the type of lock in that will kill the company if something is changed in a massive way very soon.

To add, this is the description of an actual company that is now defunct. I phrased it as existing today only to make the point. The hard reality is the company went under, almost entirely because of the costs of maintaining and unsustainable architecture that caused an exorbitant lock in to very specific tools – largely because the company drank the cool aid to use the tools as suggested. They developed the product into a corner. That mistake was so expensive that it decimated the finances of the company. Not a good scenario, not a happy outcome, and something to be avoided in every way! This is truly the epitomy of negative lock in.

Of course there’s this distinctive lock in we have to steer clear from, but there’s the lock in associated with languages and other technology capabilities that will help your company move forward faster, easier, and with increasing capabilities. Those are the choices, the ties to technology and capabilities that decision makers can really leverage with fewer negative consequences.

The “Lock In” That Enables

IT Security icons. Simplus seriesOne common statement is, “the right tool for the job”. This is of course for the ideal world where ideal decisions can be made all the time. This doesn’t exist and we have to strive for balance between decisions that will wreck the ship or decisions that will give us clear waters ahead.

For databases we need to choose the right databases for where we want to go versus where we are today. Not to gold plate the solution, but to have intent and a clear focus on what we want our future technology to hold for us. If we intend to expand our data and want to maintain the ability to effectively query – let’s take the massive SQL Server for example – what could we have done to prevent it from becoming a debilitating decision?

A solution that could have effectively come into play would have been not to shard the relational database, but instead to either export or split the data in a more horizontal way and put it into a distributed database store. Start building the application so that this system could be used instead of being limited by the relational database. As the queries are built out and the tight coupling to SQL Server removed, the new distributed database could easily add nodes to compensate for the ever growing size of the data stored. The options are numerous, that all are a form of lock-in, but not the kind that eventually killed this company that had limited and detrimentally locked itself into use of a relational database.

At the application tier, another solution could have been made to remove the ties to IIS and start figuring out a way to containerize the application. One way years ago would have been to move away from .NET, but let’s say that wasn’t really an option for other reasons. The idea to mimic containerization could have been done through shifting to a self-contained web server on Windows that would allow the .NET application to run under a singular service and then have those services spin off the application as needed. This would decouple from IIS, and enable spreading the load more quickly across a set number of machines and eventually when .NET Core was released offer the ability to actually containerize and shift entirely off of Windows Server to a more cost efficient solution under Linux.

These are just some ideas. The solutions of course would vary and obviously provide different results. Above all there are pathways away from negative lock in and a direction toward positive lock in that enables. Realize there’s the balance, and find those that leverage lock in positively.

Nuanced Pedantic Notes:

  • Note I didn’t say all examples, but just that this combo has left more than a few companies out on a limb over the years. There are of course other technologies that have put companies (people actually) in awkward situations too. I’m just using this combo here as an example. For instance, probably some of the most notorious lock in comes from the legal ramifications of using Oracle products and being tied into their sales agreements. On the opposite end of the spectrum, Stack Overflow is a great example of how choosing .NET and scaling with it, SQL Server, and related technologies can work just fine.