New Live Coding Streams and Episodes!

I’ve been working away in Valhalla on the next episodes of Thrashing Code TV and subsequent content for upcoming Thrashing Code Sessions on Twitch (follow) and Youtube (subscribe). The following I’ve broken into the main streams and shows that I’ll be putting together over the next days, weeks, and months and links to sessions and shows already recorded. If you’ve got any ideas, questions, thoughts, just send them my way.

Colligere (Next Session)

Coding has been going a little slow, in light of other priorities and all, but it’ll still be one of the featured projects I’ll be working on. Past episodes are available here, however join in on Friday and I’ll catch everybody up, so you can skip past episodes if you aren’t after specific details and just want to join in on future work and sessions.

In this next session, this Friday the 9th at 3:33pm PST, I’m going to be working on reading in JSON, determining what type of structure the JSON should be unmarshalled into, and how best to make that determination through logic and flow.

Since Go needs something to unmarshall JSON into, a specific structure, I’ll be working on determining a good way to pre-read information in the schema configuration files (detailed in the issue listed below) so that a logic flow can be implemented that will then begin the standard Go JSON unmarshalling of the object. So this will likely end up including some hackery around reading in JSON without the assistance of the Go JSON library. Join in and check out what solution(s) I come up with.

The specific issue I’ll be working on is located on Github here. These sessions I’m going to continue working on, but will be a little vague and will start working on the Colligere CLI primarily on Saturday’s at 10am. So you can put that on your schedule and join me then for hacks. If you’d like to contribute, as always, reach out via here, @Adron, or via the Github Colligere Repository and let’s discuss what you’d like to add.

Getting Started with Go

This set of sessions, which I’ve detailed in “Getting Started with Go“, I’ll be starting on January 12th at 4pm PST. You can get the full outline and further details of what I’ll be covering on my “Getting Started with Gopage and of course the first of these sessions I’ve posted details on the Twitch event page here.

  • Packages & the Go Tool – import paths, package declarations, blank imports, naming, and more.
  • Structure – names, declarations, variables, assignments, scope, etc., etc.
  • Basic Types – integers, floats, complex numbers, booleans, strings, and constants.

Infrastructure as Code with Terraform and Apache Cassandra

I’ll be continuing the Terraform, bash, and related configuration and coding of using infrastructure as code practices to build out, maintain, and operate Apache Cassandra distributed database clusters. At some point I’ll likely add Kubernetes, some additional on the metal cluster systems and start looking at Kubernetes Operators and how one can manage distributed systems on Kubernetes using this on the metal environment. But for now, these sessions will continue real soon as we’ve got some systems to build!

Existing episodes of this series you can check out here.

Getting Started with Multi-model Databases

This set of sessions I’ve detailed in “Getting Started with a Multi-model Database“, and this one I’ll be starting in the new years also. Here’s the short run down of the next several streams. So stay tuned, subscribe or follow my Twitch and Youtube and of course subscribe to the Composite Code blog (should be to the left, or if on mobile click the little vertical ellipses button)

  1. An introduction to a range of databases: Apache Cassandra, Postgresql and SQL Server, Neo4j, and … in memory database. Kind of like 7 Databases in 7 Weeks but a bunch of databases in just a short session!
  2. An Introduction – Apache Cassandra and what it is, how to get a minimal cluster started, options for deploying something quickly to try it out.
  3. Adding to Apache Cassandra with DataStax Enterprise, gaining analytics, graph, and search. In this session I’ll dive into what each of these capabilities within DataStax Enterprise give us and how the architecture all fits together.
  4. Deployment of Apache Cassandra and getting a cluster built. Options around ways to effectively deploy and maintain Apache Cassandra in production.
  5. Moving to DataStax Enterprise (DSE) from Apache Cassandra. Getting a DSE Cluster up and running with OpsCenter, Lifecycle Manager (LCM), and getting some queries tried out with Studio.

Baseline Collateral for Useful Video Streaming

This post I’m making just to outline, and provide future reference, of a few things I’ve realized between this post about what you dear readers want and things that would make my video streams more useful, plus what content would be more useful. I’ve taken this survey (which you can still fill out your two cents) and what I’ve learned form it, combined that with what I’ve learned with doing some Twitch and YouTube streaming and prepared to introduce better content, with more details and related samples, along with the streaming I’ve been doing. With that, here’s an outline of the basic collateral I’m going to produce for every single streaming session I do from now on.

Per Stream Per Video Collateral

  1. I’ll post updates with content and objectives overview for each session well before the actual session that I schedule. So that there’s a blog entry with the next dozen or so sessions, plus the events will be listed on Twitch here. So subscribe to my Twitch channel @adronhall if you want to get updates when I go live!
  2. The full unedited video will be on Youtube here. Subscribe to get updates when the stream goes live on Youtube and to be notified when they post. With each video after it is recorded I will provide a timeline of topics covered for each video that goes live on Youtube. Similar to the timeline in this session or this session.
  3. I’ll post a post-stream blog post with the stream link on Youtube, the timeline, and a write up of the code, repositories, and any other collateral that goes along with the stream session. That way everybody will have access to a quick way of getting caught back up, review streams they’ve watched, and reuse or refer to the material after the stream is over for implementation or other use.
  4. I’ll also endeavor to cut out slices of the video into smaller multiple minute segments for specific how-to videos on how I’ve put together certain things, written certain pieces of code, configured things, or otherwise useful little segments. These segments, which I don’t have an example just yet, will be somewhere around 30 seconds to less than 5 minutes. Hopefully quickly watchable and useful.

I’m looking forward to these new sessions and the collateral I’m planning to put together. It ought to be good material and I hope you find it useful. Cheers! \m/

‘bash’ A.K.A. The Solution for Everything – A few of the *Special Files*!

In bash, the shell reads one or more startup files. Here’s the details about what’s what and which is run when.

  1. /etc/profile is executed automatically at login.
  2. The file from the list of ~/.bash_profile, ~/.bash_login, or ~/.profile are then executed at login.
  3. ~/.bashrc is executed by every non-login shell, but if sh is used to invoke bash it reads the $ENV for POSIX compatability.

For reference, the ~ symbol is used in place of the user directory. One way to check this out yourself is to change directory to ~ with a cd ~ in the shell, then type pwd which will give the current directory. You’ll find that it is something like /Users/adron where instead of my name it’d be your user name.

When invoking the shell, you can also skip the ~/.bashrc or otherwise change the way bash starts up with the following options.

  • bash --init-file theFileToUseInstead or --rcfile instead of ~/.bashrc.
  • bash --norc which is similar to invoking with sh, which will use $ENV.
  • bash --noprofile will prevent /etc/profile or any other personal startup files. This will provide a pretty baseline bash shell for use.

Until next time, happy bash code thrashing!

Twitter for Developers, Cutting the Bullshit, Quelling the Trash Tire Fire

It’s been over a decade that Twitter has been an active part of the developer community. It’s grown in popularity from day one, and now holds the uneasy crown as the place for hot takes, trash from politicians, and the general tire fire that is the news. In many ways, that’s what they’ve aimed for. But then there’s us developers, people who make software, who make Twitter, who build all of this technology internet stuff right? We’re here using Twitter still, even amid the backstabbing and Twitter UI’s API’s being yanked from under us. They’ve of course in the past also banned UI’s and somehow here we are still using the service. However, I digress, Twitter’s wrongs against developers are numerous after we effectively built the service. In spite of all this we developers are a large contingent of people on Twitter. It’s still an amazingly useful medium for software developers, and especially new software developers, to get involved with. It’s a very effective tool to strengthen our careers and continue conversations within the developer communities themselves. One just has to avoid the cruft, and that’s what I intent to tackle some of in this article.

This list I’ve put together is of things that I personally have learned, often by stumbling through and discovering myself. These activities on Twitter do have a net positive effect on your career and ability to communicate with the world and local developer communities. First I’ll cover positive use cases of Twitter that are immensely useful as a software developer. These are even compounded if you’re an advocate of open source, cool technologies and libraries, and other miscellaneous things.

1. Twitter as a Communication Tool

First and foremost, Twitter has been and does – mostly – continue to be a communication tool. I make use of Twitter to connect with people for conference organizing, code projects, open source work, to have geek lunch, nerd brunch, and many other things that come up. It can and ought to be one of your primary communication mediums in that it connects many of the key active people within our overall communities. More so than email and other mediums by a large percent. If you intend to have a long term net effect and grow your presence and activities (conferences, meetups, coding groups, etc) you want to foster Twitter has become the de facto medium to be active on.

2. Twitter as a Collector of People

Twitter, even though it does seem to attract some of the most villainous scum (literally, not a figure of speech or hyperbole) and have some pretty horrifying problems (people calling in SWAT’s on people (extremely illegal), death threats, harassment) the net benefit within the community to bring people together has far outclassed pretty much any other system out there. Hacker News doesn’t, Facebook doesn’t, Google+ is cancelled, and about every other social media platform has failed to bring together the develop community in an effective and useful way.

3. Twitter for Answers

Even though I don’t often go to Twitter to find answers, sometimes I do. Often it is a last resort. After all, Twitter is most efficient at providing a place for links, quick blurbs, bumbling and babbling threads from people, and of course cat pictures and hot takes.

The combination powers of Twitter with other services however exponentially increases the ability of Twitter to help with answers. For example, write up a solidly written question on Stackoverflow or one of the branched out services and then post the question on Twitter, maybe inquiring for some retweets and boom, doubling, tripling, and greater multiplier of people looking at the question that can provide a prospective answer!

4. Twitter, Firestarter

One of the things I’ve also found Twitter good for is an outlet for pushing and often straightening out bad behavior in the community. Ever done something racist? Ever known someone to pull some misogynistic action? Yeah, unfortunately I know of these things happening too, and Twitter forces apologies and better behavior among people. But it also is a place people can wreck themselves and be just as destructive as they can learn to better themselves, especially those humans of us that have poor behavior and disrespectful tendencies.

But just as much as individual behaviors among us, Twitter has been used to straighten out some pretty trash behaviors from corporations. Sure, they’re not really people, but the conflagrations of this notion – true or not – make for pressure to be applied to corporations through other means besides the products and services they sell us individual humans, which to often are things we have to buy regardless, and this medium provides us an avenue to induce better behaviors in spite of purchases.

There of course is the positive and negative of this forced societal behavior and in many ways, improving corporate behavior throughout the world, but it’s here. Pressure of the people, often organized and started through Twitter, including against Twitter itself sometimes, is heavily rooted in activity right there on ole’ Twitter itself.

GSD Tactical Twitter

Alright, now to the meat of things. Twitter is great at all these things but how does one make the best use of it without it turning into an outright tire fire trash dump of distraction and stress? Well, it’s moderately easy, but one has to be careful.

1. Find Good and Entertaining People

My personal advice when starting on Twitter is to skip the companies. Don’t follow any of them. Same goes for organizations or any group account of sorts. The key to find good content, good common ground, and useful links, news, and related communities is to follow individuals that are involved in those things you want to be involved in already. The following are some specific examples, and for me, great people to follow.

2. Lift Up Others, Tweet to Others, Get Involved

When on Twitter, one can just lurk. It’s a completely valid thing to do. However lurking isn’t super high value. You just won’t get that much out of it. Instead, get involved. Find a link with something interesting, write up a tweet and post it. See something interesting someone else just tweeted, respond! See something that isn’t right, maybe tweet why it isn’t.

Always a good idea, regardless of the trash that is often on Twitter to still stay courteous, kind, and friendly. Remember, not everyone is from the mold you’ve come from, or seen things the way you have, so tread lightly and friendly and things mostly work out real well. Overall, people are attuned to helping those that help themselves and helping those that we run in social circles with.

All in all, get involved, tweet at, with, and all around your fellow Twitterers. Your return will improve and in the process you’ll add more value for others too.

3. Follow & Prune the Firehose of Tweets

Alright, I’ve written to follow and lift up others. That’s groovy, but also you gotta bring the hammer down sometimes. When that firehose of tweets just gets a little overwhelming check out what tweets are helpful, rate them to yourself, and unfollow some people if it’s not the direction or the tweets you’re getting value from.

Even though it’s difficult when just starting to use Twitter, the ratio will be more followed than followers for you. But as time goes forward and you get past 50 followers, 100, 500, 1000 you’ll need to make sure to keep the list of people you’ve followed just equal to or less than how many people follow you. It’ll help keep your feed manageable and also help you to keep interactions beneficial for you, followers, and followed.

4. The Down-Low on Conferences

If you’re looking to attend a conference, Twitter via hashtags is a great way to get information on conferences. Dig in, dig deep. Talk to people about the conference in particular. If necessary get into direct messages and invoke the whisper net if need be. Sometimes conferences can be exponentially useful and sometimes they end up bothersome cash burning wastes of time. Figure out what you want from a prospective conference and dig in via Twitter, you’ll prevent wasting time and burning cash, and exponentially increase the positives you can get out of a conference.

5. Filter the Trash Fire

Ok, let’s get super serious. One way Twitter has become a trash fire for many or most people these days is because of the political trash dumped in. Much of Twitter for the general public is bot armies from Russia, crazies like the nutty Wohl kid, and other junk nut accounts. One way to notch this down to a minimal trash fire is to throw some filters (i.e. mute certain words) on your Twitter account. For example here’s my list:

filters-muted-words

Now as you’ve read that, remember that my goal has been to focus the stream on tech content with a little heavy metal, a few cats, and other entertainment here and there. For example I’m fine with sports events like baseball and football but really don’t want to get distracted by it in on my Twitter stream. On game day those events just overwhelm the tweets and things that are useful get drowned out.

Now a lot of the other stuff in the list is the horrifying reality of the United States today, reflected on Twitter, and part of something that I don’t want distracting me either. Overall this has made Twitter dramatically more useful for me again.

Top 10 West Coast Confs for 2019

I’ve been putting together a list of conferences that I want to aim to attend this coming year. I made it, then thought, “somebody else could use this list probably” so here it is. If you think of any other specific conferences I ought to add and attend please leave a comment. Enjoy!

March 7-10 is SCALE Southern California Linux Expo in Pasadena, California

March 25-28 is O’Reilly Strata in San Francisco, California

April 26-28 is LinuxFest Northwest in Bellingham, Washington

June 3-5 is Monitorama in Portland, Oregon

June 10-13 is O’Reilly Velocity in San Jose, California

June 10-13 is O’Reilly Software Architecture Conference SACON in San Jose, California

July 15-18 is O’Reilly OSCON in Portland, Oregon

August 21-23 is the Open Source Summit in San Diego, California

September 9-12 is the O’Reilly Artificial Intelligence in San Jose, California

November 18-21 KubeCon 2019 in San Diego, California

Without Dates – Conferences that are really great that don’t currently have a date just yet.

Polyglot Conf in Vancouver BC

Seattle Code Camp

Microsoft Build

GDG DevFest

What others should I add that are awesome Seattle or immediate surrounding area conferences?

 

Cassandra Datacenter & Racks

This last post in this series is Distributed Database Things to Know: Consistent Hashing.

Let’s talk about the analogy of Apache Cassandra Datacenter & Racks to actual datacenter and racks. I kind of enjoy the use of the terms datacenter and racks to describe architectural elements of Cassandra. However, as time moves on the relationship between these terms and why they’re called datacenter and racks can be obfuscated.

Take for instance, a datacenter could just be a cloud provider, an actual physical datacenter location, a zone in Azure, or region in some other provider. What an actual Datacenter in Cassandra parlance actually is can vary, but the origins of why it’s called a Datacenter remains the same. The elements of racks also can vary, but also remain the same.

Origins: Racks & Datacenters?

Let’s cover the actual things in this industry we call datacenter and racks first, unrelated to Apache Cassandra terms.

Racks: The easiest way to describe a physical rack is to show pictures of datacenter racks via the ole’ Google images.

racks.png

A rack is something that is located in a data-center, or even just someone’s garage in some odd scenarios. Ya know, if somebody wants serious hardware to work with. The rack then has a number of servers, often various kinds, within that rack itself. As you can see from the images above there’s a wide range of these racks.

Datacenter: Again the easiest way to describe a datacenter is to just look at a bunch of pictures of datacenter, albeit you see lots of racks again. But really, that’s what a datacenter is, is a building that has lots and lots of racks.

data-center.png

However in Apache Cassandra (and respectively DataStax Enterprise products) a datacenter and rack do not directly correlate to a physical rack or datacenter. The idea is more of an abstraction than hard mapping to the physical realm. In turn it is better to think of datacenter and racks as a way to structure and organize your DataStax Enterprise or Apache Cassandra architecture. From a tree perspective of organizing your cluster, think of things in this hierarchy.

  • Cluster
    • Datacenter(s)
      • Rack(s)
        • Server(s)
          • Node (vnode)

Apache Cassandra Datacenter

An Apache Cassandra Datacenter is a group of nodes, related and configured within a cluster for replication purposes. Setting up a specific set of related nodes into a datacenter helps to reduce latency, prevent transactions from impact by other workloads, and related effects. The replication factor can also be setup to write to multiple datacenter, providing additional flexibility in architectural design and organization. One specific element of datacenter to note is that they must contain only one node type:

Depending on the replication factor, data can be written to multiple datacenters. Datacenters must never span physical locations.Each datacenter usually contains only one node type. The node types are:

  • Transactional: Previously referred to as a Cassandra node.
  • DSE Graph: A graph database for managing, analyzing, and searching highly-connected data.
  • DSE Analytics: Integration with Apache Spark.
  • DSE Search: Integration with Apache Solr. Previously referred to as a Solr node.
  • DSE SearchAnalytics: DSE Search queries within DSE Analytics jobs.

Apache Cassandra Racks

An Apache Cassandra Rack is a grouped set of servers. The architecture of Cassandra uses racks so that no replica is stored redundantly inside a singular rack, ensuring that replicas are spread around through different racks in case one rack goes down. Within a datacenter there could be multiple racks with multiple servers, as the hierarchy shown above would dictate.

To determine where data goes within a rack or sets of racks Apache Cassandra uses what is referred to as a snitch. A snitch determines which racks and datacenter a particular node belongs to, and by respect of that, determines where the replicas of data will end up. This replication strategy which is informed by the snitch can take the form of numerous kinds of snitches, some examples include;

  • SimpleSnitch – this snitch treats order as proximity. This is primarily only used when in a single-datacenter deployment.
  • Dynamic Snitching – the dynamic snitch monitors read latencies to avoid reading from hosts that have slowed down.
  • RackInferringSnitch – Proximity is determined by rack and datacenter, assumed corresponding to 3rd and 2nd octet of each node’s IP address. This particular snitch is often used as an example for writing a custom snitch class since it isn’t particularly useful unless it happens to match one’s deployment conventions.

In the future I’ll outline a few more snitches, how some of them work with more specific detail, and I’ll get into a whole selection of other topics. Be sure to subscribe to the blog, the ole’ RSS feed works great too, and follow @CompositeCode for blog updates. For discourse and hot takes follow me @Adron.

Distributed Database Things to Know Series

  1. Consistent Hashing
  2. Apache Cassandra Datacenter & Racks (this post)

 

DataStax Developer Days

Over the last week I had the privilege and adventure of coming out to Chicago and Dallas to teach about operations and security capabilities of DataStax Enterprise. More about that later in this post, first I’ll elaborate on and answer the following:

  • What is DataStax Developer Day? Why would you want to attend?
  • Where are the current DataStax Developer Day events that have been held, and were future events are going to be held?
  • Possibilities for future events near a city you live in.

What is DataStax Developer Day?

The way we’ve organized this developer day event at DataStax, is focused around the DataStax Enterprise built on Apache Cassandra product, however I have to add the very important note that this isn’t merely just a product pitch type of thing, you can and will learn about distributed databases and systems in a general sense too. We talk about a number of the core principles behind distributed systems such as the pivotally important consistent hash ring, datacenter and racks, gossip, replication, snitches, and more. We feel it’s important that there’s enough theory that comes along with the configuration and features covered to understand who, what, where, why, and how behind the configuration and features too.

The starting point of the day’s course material is based on the idea that one has not worked with or played with a Apache Cassandra or DataStax Enterprise. However we have a number of courses throughout the day that delve into more specific details and advanced topics. There are three specific tracks:

  1. Cassandra Track – this track consists of three workshops: Core Cassandra, Cassandra Data Modeling, and Cassandra Application Development. [more details]
  2. DSE Track – this track consists of three workshops: DataStax Enterprise Search, DataStax Enterprise Analytics, and DataStax Enterprise Graph. [more details]
  3. Bonus Content – This track has two workshops: DataStax Enterprise Overview and DataStax Enterprise Operations and Security.  [more details]

Why would you want to attend?

  • One huge rad awesome reason is that the developer day events are FREE. But really, nothing is ever free right? You’d want to take a day away from the office to join us, so there’s that.
  • You also might want to even stay a little later after the event as we always have a solidly enjoyable happy hour so we can all extend conversations into the evening and talk shop. After all, working with distributed databases, managing data, and all that jazz is honestly pretty enjoyable when you’ve got awesome systems like this to work with, so an extended conversation into the evening is more than worth it!
  • You’ll get a firm basis of knowledge and skillset around the use, management, and more than a few ideas about how Apache Cassandra and DataStax Enterprise can extend your system’s systemic capabilities.
  • You’ll get a chance to go beyond merely the distributed database system of Apache Cassandra itself and delve into graph, what it is and how it works, analytics, and search too. All workshops take a look at the architecture, uses, and what these capabilities will provide your systems.
  • You’ll also have one on one time with DataStax engineers, and other technical members of the team to ask questions, talk about architecture and solutions that you may be working on, or generally discuss any number of graph, analytics, search, or distributed systems related questions.

Where are the current DataStax Developer Day events that have been held, and were future events are going to be held? So far we’ve held events in New York City, Washington DC, Chicago, and Dallas. We’ve got two more events scheduled with one in London, England and one in Paris, France.

Future events? With a number of events completed and a few on the calendar, we’re interested in hearing about future possible locations for events. Where are you located and where might an event of this sort be useful for the community? I can think of a number of cities, but organizing them into order to know where to get something scheduled next is difficult, which is why the team is looking for input. So ping me via @Adron, email, or just send me a quick message from here.