Tag: software development

Schedule Updates: Databases, Coding, Meetups, & More

Here is my updated schedule for the two weeks starting the 23rd for my Twitch streams, the basic topics, meetups (a little beyond two weeks), and related events coming up.

Series: Thrashing Code General Calamity

This is going to be a mix of tech this week. Probably some database hacking, code hacking, samples, and setup of even more examples for use throughout your coding week!

Series: Bunches of Databases in Bunches of Weeks

Focusing on DataStax Enterprise setup and configuration over the next few weeks.

Series: Meetups

This is a little further out than the next few weeks, but in August we’re having another meetup!

Series: Building the Geo App Trux (w/ Vue.js, Go, and DataStax Enterprise (Apache Cassandra). Navigate through to the DataStax Developers Channel to check out the event times.

Series: DataStax’s Apache Cassandra & C# Hour (C# Schema Migration builder, C# driver, and additional content). Navigate through to the DataStax Developers Channel to check out the event times.

Learning Go Episode 5 – Functions (and Methods and lots of other things)

Episode Post & Video Links:  1, 2, 3, 4, 5 (this post), 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

In this session we covered a host of topics around Go Functions. Along with some troubleshooting, debugging, and other features in Jetbrains Goland IDE.

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan as a starting point. I’m using it as a simple guideline, but also doing a lot more in each stream that includes ecosystem, dependency management with godep, IDE use of Goland from Jetbrains, and more. In this session I get specifically into: functions, signatures, declarations, recursion, return values, and more.

Video Time Points & Topics

2:50 – Introduction to the snowy wonderland of Seattle and episode 5 of the Learning Go series. Introduction to the various screen transitions and such.
6:40 – Getting started, opening up JetBrains Goland and creating a new project. The project exists on Github as https://github.com/adron/learning-go-….
12:18 – Starting with functions in Go. See the blog entry I wrote on the topic for more additional information around this first code session within the episode 5 session.

Code – This first example I setup a basic function in Go that is called by the main function. The sample function below I’ve named exampleExecutor, and the signature is made up of an int parameter called this, a string parameter called that, and a return parameter of type int and one of type string. In summary for the function signature we have two input parameters going in and two return parameters coming out.

The function does very little besides print the parameters passed in and then return the parameters back out as the return parameters.


package main
import "fmt"
func main() {
var this, result int
var that, message string
this = 2
that = "42"
result, message = exampleExecutor(this, that)
fmt.Printf("%s\n%d", message, result)
}
func exampleExecutor(this int, that string) (int, string) {
fmt.Printf("Numbers: %d, %s\n", this, that)
return this, "This is the result: " + that
}

view raw

func_main_plus_example_executor.go

hosted with ❤ by GitHub

Recursion with Go & HTML Parsing

28:10 – Here I get into recursion and the application example, largely taken from the book but with some very distinctive modifications, that parses HTML and the various nodes within an HTML document.

For the recursion section I use an example from the book with an expanded sample set of HTML. The HTML is included in the repo under the function-recursion branch. For this example I setup a set of types and variables up that are needed throughout the code.

First a type setup called NodeType of type int32. A constant array of ErrorNode, TextNode, DocumentNode, ElementNode, CommentNode, and DoctypeNode for determining the different nodes within an HTML document. Then a general struct called Node with Type, Data, Attr for attribute, and FirstChild with NextSibling setup as a pointer to *Node, which gives a type of memory recursion to the underlying type. Then finally the Attribute struct with a Key a Value.


type NodeType int32
const (
ErrorNode NodeType = iota
TextNode
DocumentNode
ElementNode
Commentnode
DoctypeNode
)
type Node struct {
Type NodeType
Data string
Attr []Attribute
FirstChild, NextSibling *Node
}
type Attribute struct {
Key, Val string
}

view raw

types_nodes_etc.go

hosted with ❤ by GitHub

One of the first functions I then end up with is the visit function. It turns out as shown below. Here the function takes a links parameter that is of type string array, a parameter name n that is a pointer reference to an a node within html, and then the function returns a parameter of type string array.


func visit(links []string, n *html.Node) []string {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key == "href" {
links = append(links, a.Val)
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
links = visit(links, c)
}
return links
}

view raw

visit_func.go

hosted with ❤ by GitHub

After that function I worked through and created two additional functions, one for countsWordsAndImages and one called CountWordsAndImages. The casing being specific to scope and use of the functions, and they respectively look like this in completion.


func CountWordsAndImages(url string) (words, images int, err error) {
resp, err := http.Get(url)
if err != nil {
return
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
err = fmt.Errorf("parsing HTML: %s", err)
}
words, images = countWordsAndImages(doc)
return
}
func countWordsAndImages(n *html.Node) (words, images int) {
if n.Type == html.TextNode {
words += len(strings.Split(n.Data, " "))
return
}
if n.Data == "img" {
images++
} else {
if n.FirstChild != nil {
w, i := countWordsAndImages(n.FirstChild)
words += w
images += i
}
if n.NextSibling != nil {
w, i := countWordsAndImages(n.NextSibling)
words += w
images += i
}
}
return
}

view raw

counting.go

hosted with ❤ by GitHub

Then all that is wrapped up, with recursive calls and more, in the main function for program execution.


func main() {
htmlContent, err := ioutil.ReadFile("compositecode.html")
if err != nil {
fmt.Println(err)
}
htmlData := string(htmlContent)
r := strings.NewReader(htmlData)
doc, err := html.Parse(r)
if err != nil {
fmt.Fprintf(os.Stderr, "find links: %v\n", err)
os.Exit(1)
}
for _, link := range visit(nil, doc) {
fmt.Println(link)
}
w, i, _ := CountWordsAndImages("https://compositecode.blog")
fmt.Printf("Words: %d\nImages: %d\n", w, i)
}

view raw

TheCountingRecursionOfThings.go

hosted with ❤ by GitHub

Starting Error Handling && Anonymous Functions

1:32:40 – At this point in the episode 5 session I get into a simple Error handling function, and further into function signatures and how to set them up.
1:52:24 – Setting up some anonymous functions and reviewing what they are.
1:59:00 – Introduction to panics in Go. After this short introduction I also discuss some of the pedantic specifics of methods vs functions and related verbiage around the Go language. Additionally I provide more examples around these specifics for declaring functions, various scope, and other types for function calls and related usage.

With that done the wrap up of the session is then a short introduction to anonymous functions.

Cassie Schema Migrator >> CaSMa

A few weeks back I started working on a schema migration tool for Apache Cassandra and DataStax Enterprise. Just for context, here are the short definitions of what each of the elements of CaSMa are.

  • cstar-iconApache Cassandra
    • Definition: Apache Cassandra is a free and open-source, distributed, wide column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure.
    • History: Avinash Lakshman, one of the authors of Amazon’s Dynamo, and Prashant Malik initially developed Cassandra at Facebook to power the Facebook inbox search feature. Facebook released Cassandra as an open-source project on Google code in July 2008. In March 2009 it became an Apache Incubator project. On February 17, 2010 it graduated to a top-level project. Facebook developers named their database after the Trojan mythological prophet Cassandra, with classical allusions to a curse on an oracle.
  • dse-logoDataStax Enterprise
    • Definition: DataStax Enterprise, or routinely just referred to as DSE, is an extended version of Apache Cassandra with multi-model capabilities around graph, search, analytics, and other features like security capabilities and a core data engine 2x speed improvement.
    • History: DataStax was formed in 2009 by Jonathan Ellis and Matt Pfeil and originally named Riptano. In 2011 Riptano changes names to DataStax. For more history check out the Wikipedia page or company page for a timeline of events.
  • command-toolsSchema Migration
    • Definition:In software engineering, schema migration (also database migration, database change management) refers to the management of incremental, reversible changes to relational database schemas. A schema migration is performed on a database whenever it is necessary to update or revert that database’s schema to some newer or older version. Migrations are generally performed programmatically by using a schema migration tool. When invoked with a specified desired schema version, the tool automates the successive application or reversal of an appropriate sequence of schema changes until it is brought to the desired state.
    • Addition reference and related materials:

iconmonstr-twitch-5Over the next dozen weeks or so as I work on this application via the DataStax Devs Twitch stream (next coding session events list) I’ll also be posting some blog posts in parallel about schema migration and my intent to expand on the notion of schema migration specifically for multi-model databases and larger scale NoSQL systems; namely Apache Cassandra and DataStax Enterprise. Here’s a shortlist for the next three episodes;

The other important pieces include the current code base on Github, the continuous integration build, and the tasks and issues.

Alright, now that all the collateral and context is listed, let’s get into at a high level what this is all about.

CaSMa’s Mission

Schema migration is a powerful tool to get a project on track and consistently deployed and development working against the core database(s). However, it’s largely entrenched in the relational database realm. This means it’s almost entirely focused on a schema with the notions of primary and foreign keys, the complexities around many to many relationships, indexes, and other errata that needs to be built consistently for a relational database. Many of those things need to be built for a distributed columnar store, key value, graph, time series, or a million other possibilities too. However, in our current data schema world, that tooling isn’t always readily available.

The mission of CaSMa is to first resolve this gap around schema migration, first and foremost for Apache Cassandra and prospectively in turn for DataStax Enterprise and then onward for other database systems. Then the mission will continue around multi-model systems that should, can, and ought to take advantage of schema migration for graph, and related schema modeling. At some point the mission will expand to include other schema, data, and state management focused around software development and data needs within that state

As progress continues I’ll publish additional posts here on the different data model concepts and nature behind various multi-model database options. These modeling options will put us in a position to work consistently, context based, and seamlessly with ongoing development efforts. In addition to all this, there will be the weekly Twitch sessions where I’ll get into coding and reviewing what coding I’ve done off camera too. Check those out on the DataStax Devs Channel.

If you’d like to get into the project and help out just ping me via Twitter @Adron or message me here.

Learning Go Episode 4 – Composite Types, Slices, Arrays, Etc.

Episode Post & Video Links:  1, 2, 3, 4 (this post), 5, 6, 7, and 8. Non-linked are in the works! Videos available now on Youtube however, so check em’ out!

If you’d like to go through this material too in book form, I highly suggest “The Go Programming Language” by Alan A.A. Donovan & Brian W. Kernighan. I use it throughout these sessions to provide a guideline. I however add a bunch of other material about IDE’s, development tips n’ tricks and other material.

8:00 Starting the core content with some notes. Composite types, arrays, slices, etc.
11:40 Announcement of my first reload – http://compositecode.blog/2019/02/11/… – second successful time blogged here) of the XPS 15 I have, which – https://youtu.be/f0z1chi4v1Q – actually ended in catastrophe the first time!
14:08 Starting the project for this session.
16:48 Setting up arrays, the things that could be confusing, and setup of our first code for the day. I work through assignment, creation, new vs. comparison, and various other characteristics of working with arrays during this time.


fmt.Println("Hello, let's talk composite types.")
basketOfStuff := [3]string{"The first string","second","This string."}
var zeeValues [2]int
for i, v := range basketOfStuff {
fmt.Printf("Value %d: %s\n", i, v)
}
fmt.Println(zeeValues)
if zeeValues[0] == zeeValues[1] {
fmt.Println("The values are the same, this doesn't instantiate like the `new` keyword.")
} else {
fmt.Println("The way go appears to instantiate unset variable values, such as in this array is like the `new` keyword instantiation.")
}
zeeValues[0] = 1 + 52 * 3
zeeValues[1] = 9
fmt.Println(zeeValues[len(zeeValues) – 1])

view raw

testing-out-composite-types.go

hosted with ❤ by GitHub

29:36 Creation of a type, called Currency, of type int, setting up constants, and using this kind of like an enumerator to work with code that reads cleaner. Plus of course, all the various things that you might want to, or need for a setup of types, ints, and related composite types like this.


type Currency int
const (
USD Currency = iota
CAN
EUR
GBP
JPY
NOK
SEK
DKK
)
symbol := […]string{USD: "$", CAN: "$", EUR: "€", GBP: "£", JPY:"¥", NOK:"kr", SEK:"kr",DKK:"kr"}
fmt.Println(EUR, symbol[EUR])
fmt.Println(JPY, symbol[JPY])
r := […]int{99: -1}
r[36] = 425
r[42] = 42
fmt.Println(r[36] + r[42])
fmt.Println(strconv.Itoa(r[36]))

view raw

currency.go

hosted with ❤ by GitHub

43:48 Creating an example directly from the aforementioned book enumerating the months of the year. This is a great example I just had to work through it a bit for an example.


months := […]string{1: "January", 2:"February", 3: "March", 4:"April", 12:"December"}
for _, s := range months {
fmt.Printf("The month: %s\n", s)
}
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}

view raw

the_months.go

hosted with ❤ by GitHub

52:40 Here I start showing, and in the process, doing some learning of my own about runes. I wasn’t really familiar with them before digging in just now!


… the rest of the main.go file is here…
var runes []rune
for _, r := range "Language: 走" {
runes = append(runes, r)
}
fmt.Printf("%q \n", runes)
var x, y []int
for i := 0; i < 10; i++ {
y = appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x = y
}
}
func appendInt(x []int, i int) []int {
var z []int
zlen := len(x) + 1
if zlen <= cap(x) {
z = x[:zlen]
} else {
zcap := zlen
if zcap < 2* len(x) {
zcap = 2 * len(x)
}
z = make([]int, zlen, zcap)
copy(z, x)
}
return z
}

view raw

main.go

hosted with ❤ by GitHub

1:09:40 Here I break things down and start a new branch for some additional examples. I also derail off into some other things about meetups and such for a short bit. Skip to the next code bits at the next time point.
1:23:58 From here on to the remainder of the video I work through a few examples of how to setup maps, how make works, and related coding around how to retrieve, set, and otherwise manipulate the maps one you’ve got them.


package main
import "fmt"
func main() {
ages := map[string]int{
"Peterson": 52,
"Sally": 22,
"Javovia": 15,
"Ben": 42,
}
jobAssociation := make(map[string]string)
jobAssociation["Peterson"] = "Engineer"
jobAssociation["Sally"] = "CEO"
jobAssociation["Jovovia"] = "Gamer"
jobAssociation["Ben"] = "Programmer"
printAges(ages)
printJobAssociations(jobAssociation)
fmt.Println(ages)
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"])
fmt.Println(jobAssociation["Frank"]) // Blank! 😮
fmt.Println(ages["Sally"])
delete(ages, "Sally")
fmt.Println(ages)
fmt.Println(ages["Sally"])
delete(jobAssociation, "Jovovia")
fmt.Println(jobAssociation)
fmt.Println(jobAssociation["Jovovia"]) // Blank.
ages2 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
ages3 := map[string]int{
"Frank": 52,
"Johnson": 22,
"Smith": 15,
"Jezebelle": 42,
}
if equal(ages, ages2) {
fmt.Println("Naw, not really equal.")
} else {
fmt.Println("This is correct, not equal.")
}
if equal(ages2, ages3) {
fmt.Println("True, these are effectively the same map values and keys.")
}
}
func printJobAssociations(associations map[string]string) {
for name, job := range associations {
fmt.Printf("%s\t%s\n", name, job)
}
}
func printAges(ages map[string]int) {
for name, age := range ages {
fmt.Printf("%s\t%d\n", name, age)
}
}
func equal(x, y map[string]int) bool {
if len(x) != len(y) {
return false
}
for k, xv := range x {
if yv, ok := y[k]; !ok || yv != xv {
return false
}
}
return true
}

view raw

map_types.go

hosted with ❤ by GitHub

That’s it for this synopsis. Until next episode, happy code thrashing and go coding!

DevRel Data: Presentation & Deductions

Before diving into conclusions, let’s take a look at some answers to questions asked. This is a slice of answers, with totals for the charts and such. After a few months of answers I’ll have another follow up to see how things may or may not change.

Do you like video material?

chart

What specifically do you, or would you like to watch in video? Screencasts, short videos, conversational, or some other type of videos?

  • Screencasts/tutorials
  • I love both screencasts going through big topics and short videos that cover smaller tips and gotchas.
  • Videos with a specific outcome as the goal, whether achieved or not. Showing the process of something.. like hey, here’s how you building out a Postgres cluster using streaming replication and repmgr and pgpool… Kind of thing.
  • Bite sized content, maybe 2 minutes, to teach me one thing.
  • Editing. No jokes, no “hey what’s up guys” with 60 second intros. Discuss the problem, then solve it.
  • Demos, learning a new way of doing something
  • Doesn’t matter short or long, but has to be deeply technical with code examples that I can actually apply
  • I watch videos mostly for fun.
  • Screencast
  • Short videos of say 5-10 minutes each covering different concept of the subject matter
  • (videos work best in a classroom setting where time/attention is precommitted, or as part of a tutorial)
  • conversational
  • Short videos.
  • If it’s too long, it ends up on my todo list forever (not good). So shorter is better. And something that benefits from visuals, rather than something that could just be written.
  • I also watch LinkedIn Learning when just starting a new tech. to get a general overview and pick up a tip or tow, then I read books and the Internet from there.
  • short videos

What kind of written material do you like?

chart2

Do you like other material mixed in that details the reason for the tech, the story, or such?

chart3

Is there anything that comes to mind, that you’d like to have me or the team I’m working with (@ DataStax) put together that you’d find useful, entertaining, or related.

  • Place priorities on designing materials for more depth (i.e., more linked material) as well as less attention-nuisance. That’s no criticism of your work, merely the gestalt of where we work — so less noise is a better way to stand out and make materials useful.
  • Maybe focus more on written material – code & architecture material (books, articles) rather than videos and twitch. It is much easier to consume and is easily googlable. Also I’d suggest making blog posts target a specific common issue or question – sometimes I see posts that I don’t really care about or the problem is so narrow that I don’t want to read about it. I’d read about building resilient and highly available architectures in various configurations.
  • Database reliability, scalability, migrations and such stuff is interesting.
  • Anything to do with machine learning.
  • Data model examples, starting up a Cassandra node, configuring YAML, etc

Deductions

I’m going to go backwards through the questions and discuss what I’ve deducted, and in some ways what has surprised me among the answers!

First there’s the “Is there anything that comes to mind, that you’d like to have me or the team I’m working with (@ DataStax) put together that you’d find useful, entertaining, or related.” request and questions.

The answers here didn’t surprise me much at all. Within DevRel from Microsoft to DataStax to Google to many other organizations we have this ongoing battle between “write a whole book on it” or “make it 2 minutes short”. It’s wildly difficult to determine what format, what timing, and what structure material needs to be in for it to be most useful to people. So when I saw the answer that reads, “Place priorities on designing materials for more depth (i.e., more linked material) as well as less attention-nuisance.” I immediately thought, “yeah, for real, but ugh…” it’s difficult. However, I’m working on more thorough material, some of it will be paywalled via LinkedIn Learning or Pluralsight and other material may be available by book in the coming months. But there will be other material that will indeed be long form how to material on how to really put things together – from scratch and from the basis of “we have X thing and need to hack it so we can add a feature”.

The next answer I got in this section that I completely agree with is increasing the focus on written material. I’m making tons of video, and I’ve got that down to the point where it’s actually easier and faster to do most of it then it is to write things down. However I realized, especially from my own point of view, that written material actually ends up being vastly more useful than video material. That’s also why, even with the video material, when I’m covering specifics I aim to provide a linkable timeline and a blog entry with the code and other changes shown in the video. Thanks for reinforcing these efforts and giving me that indirect encouragement to make this process and the results even better. More written material is on its way!

As for the database reliability, scalability, migrations, machine learning, data modeling, Cassandra node starting, and all that it’s in the queue and I’m getting to it as fast as I possibly can.

Next question I asked is, “Do you like other material mixed in that details the reason for the tech, the story, or such?”

It appears, albeit not a huge contingent of people, some people are curious about biking, train coding, and making good grub! Hey, that’s groovy cuz I’ve got a show coming out which is basically the behind the scenes videos about all those topics that make the coding and technology hacking possible!

The one outlier in this set however is clearly the request for “Ways to simplify life to dig through those algorithms faster, easier, better?” which I didn’t suspect would be any different then the other answers for this questions. Which left me surprised and ill-prepared on what to do about fulfilling what is clearly a demand. I’ll have to up level my blog posts around algorithms. I did do a couple a long while ago now in “Algorithms 101: Big Sums” which I completed in Go and another I wrote up “Algorithms 101: Roads & Town Centers” which I have 90% of the answer complete but I’ve never finished the blog entry! I guess it’s time to get the algorithm train coupled up and ready to depart!

Then the question, “What kind of written material do you like?”

Two options lead by a healthy margin for this question: Demos w/ Write Up and Blog Articles. With this coupled up to the first question it’s clear that written material via blog and demoes via blog should and ought to be top priority. They are, however they’re a whole helluva lot of work, so I only get them produced but so fast. Got some gems coming on Go, Bash, Cassandra, and a few other demo, tech, and historical information.

Next up was single page cheat sheets and documentation, followed closely by books. I kind of expected documentation and books, but wow that single page cheat sheets option is higher rated than I would have suspected and by proxy I’ve immediately added that to my produce this type of material list! I put it in as a very secondary thought but it’s going to get into that increased focus queue.

The last one with some semblance of demand is pamphlet size short form. This one almost seems like a fluke, but I’ll ponder putting together some of these. I know O’Reilly has their short novelette size books which cover a particular topic. They hand these out for free at conferences and seem pretty solid. Maybe I’ll work one of those into the queue? Maybe.

The other three options scraped by with 1%, so somebody was choosing them. So the vi mug isn’t a priority nor the short explainer videos. Which seems in contention with video content demands around shorter content. I guess, explainer videos just doesn’t sound useful!

The next question I just put together a top three of the results, “What specifically do you, or would you like to watch in video? Screencasts, short videos, conversational, or some other type of videos?”

  1. Make screen casts.
  2. Make screen casts generally short.
  3. Make screen cases that are short and on a specific and deep dive into a topic.

This seems kind of in conflict with itself, but I’m going to aim for it and try to hone the skill further. So that I can produce screen casts, screen casts that are generally short, and make sure that these screen casts that are short are on a specific and deep dive into a topic. Whew, got it.

Finally, “do you like video material?”

chart

At this time, 53.8% of you have said yes. I had guessed it would be around 50%.

I had guessed no would be about 25%, and at 23.1% I wasn’t to far off.

The other respective mishmash of answers made for interesting depth to the questions that followed this question.

Article Summary & TLDR

Produce more topic specific, detailed material around screen casts and blog entries!

End of story.

For more on this information, why I asked, and what I do check out my article titled “Evangelism, Advocacy, and Activism in The Technology Industry” and for some of the big victories for big corporations check out “The Developer Advocates – Observations on Microsoft’s New Competence“.