Lena presents “So You Want to Run Data-Intensive Systems on Kubernetes”

If you’re interested in running data-intensive systems (think Apache Cassandra, DataStax Enterprise, Kafka, Spark, Tensorflow, Elasticsearch, Redis, etc) in Kubernetes this is a great talk. @Lenadroid covers what options are available in Kubernetes, how architectural features around pods, jobs, stateful sets, and replica sets work together to provide distributed systems capabilities. Other features she continues and delves into include custom resource definitions (CRDs), operators, and HELM Charts, which include future and peripheral feature capabilities that can help you host various complex distributed systems. I’ve included references below the video here, enjoy.

References:

New Live Coding Streams and Episodes!

I’ve been working away in Valhalla on the next episodes of Thrashing Code TV and subsequent content for upcoming Thrashing Code Sessions on Twitch (follow) and Youtube (subscribe). The following I’ve broken into the main streams and shows that I’ll be putting together over the next days, weeks, and months and links to sessions and shows already recorded. If you’ve got any ideas, questions, thoughts, just send them my way.

Colligere (Next Session)

Coding has been going a little slow, in light of other priorities and all, but it’ll still be one of the featured projects I’ll be working on. Past episodes are available here, however join in on Friday and I’ll catch everybody up, so you can skip past episodes if you aren’t after specific details and just want to join in on future work and sessions.

In this next session, this Friday the 9th at 3:33pm PST, I’m going to be working on reading in JSON, determining what type of structure the JSON should be unmarshalled into, and how best to make that determination through logic and flow.

Since Go needs something to unmarshall JSON into, a specific structure, I’ll be working on determining a good way to pre-read information in the schema configuration files (detailed in the issue listed below) so that a logic flow can be implemented that will then begin the standard Go JSON unmarshalling of the object. So this will likely end up including some hackery around reading in JSON without the assistance of the Go JSON library. Join in and check out what solution(s) I come up with.

The specific issue I’ll be working on is located on Github here. These sessions I’m going to continue working on, but will be a little vague and will start working on the Colligere CLI primarily on Saturday’s at 10am. So you can put that on your schedule and join me then for hacks. If you’d like to contribute, as always, reach out via here, @Adron, or via the Github Colligere Repository and let’s discuss what you’d like to add.

Getting Started with Go

This set of sessions, which I’ve detailed in “Getting Started with Go“, I’ll be starting on January 12th at 4pm PST. You can get the full outline and further details of what I’ll be covering on my “Getting Started with Go” page and of course the first of these sessions I’ve posted details on the Twitch event page here.

  • Packages & the Go Tool – import paths, package declarations, blank imports, naming, and more.
  • Structure – names, declarations, variables, assignments, scope, etc., etc.
  • Basic Types – integers, floats, complex numbers, booleans, strings, and constants.

Infrastructure as Code with Terraform and Apache Cassandra

I’ll be continuing the Terraform, bash, and related configuration and coding of using infrastructure as code practices to build out, maintain, and operate Apache Cassandra distributed database clusters. At some point I’ll likely add Kubernetes, some additional on the metal cluster systems and start looking at Kubernetes Operators and how one can manage distributed systems on Kubernetes using this on the metal environment. But for now, these sessions will continue real soon as we’ve got some systems to build!

Existing episodes of this series you can check out here.

Getting Started with Multi-model Databases

This set of sessions I’ve detailed in “Getting Started with a Multi-model Database“, and this one I’ll be starting in the new years also. Here’s the short run down of the next several streams. So stay tuned, subscribe or follow my Twitch and Youtube and of course subscribe to the Composite Code blog (should be to the left, or if on mobile click the little vertical ellipses button)

  1. An introduction to a range of databases: Apache Cassandra, Postgresql and SQL Server, Neo4j, and … in memory database. Kind of like 7 Databases in 7 Weeks but a bunch of databases in just a short session!
  2. An Introduction – Apache Cassandra and what it is, how to get a minimal cluster started, options for deploying something quickly to try it out.
  3. Adding to Apache Cassandra with DataStax Enterprise, gaining analytics, graph, and search. In this session I’ll dive into what each of these capabilities within DataStax Enterprise give us and how the architecture all fits together.
  4. Deployment of Apache Cassandra and getting a cluster built. Options around ways to effectively deploy and maintain Apache Cassandra in production.
  5. Moving to DataStax Enterprise (DSE) from Apache Cassandra. Getting a DSE Cluster up and running with OpsCenter, Lifecycle Manager (LCM), and getting some queries tried out with Studio.

Baseline Collateral for Useful Video Streaming

This post I’m making just to outline, and provide future reference, of a few things I’ve realized between this post about what you dear readers want and things that would make my video streams more useful, plus what content would be more useful. I’ve taken this survey (which you can still fill out your two cents) and what I’ve learned form it, combined that with what I’ve learned with doing some Twitch and YouTube streaming and prepared to introduce better content, with more details and related samples, along with the streaming I’ve been doing. With that, here’s an outline of the basic collateral I’m going to produce for every single streaming session I do from now on.

Per Stream Per Video Collateral

  1. I’ll post updates with content and objectives overview for each session well before the actual session that I schedule. So that there’s a blog entry with the next dozen or so sessions, plus the events will be listed on Twitch here. So subscribe to my Twitch channel @adronhall if you want to get updates when I go live!
  2. The full unedited video will be on Youtube here. Subscribe to get updates when the stream goes live on Youtube and to be notified when they post. With each video after it is recorded I will provide a timeline of topics covered for each video that goes live on Youtube. Similar to the timeline in this session or this session.
  3. I’ll post a post-stream blog post with the stream link on Youtube, the timeline, and a write up of the code, repositories, and any other collateral that goes along with the stream session. That way everybody will have access to a quick way of getting caught back up, review streams they’ve watched, and reuse or refer to the material after the stream is over for implementation or other use.
  4. I’ll also endeavor to cut out slices of the video into smaller multiple minute segments for specific how-to videos on how I’ve put together certain things, written certain pieces of code, configured things, or otherwise useful little segments. These segments, which I don’t have an example just yet, will be somewhere around 30 seconds to less than 5 minutes. Hopefully quickly watchable and useful.

I’m looking forward to these new sessions and the collateral I’m planning to put together. It ought to be good material and I hope you find it useful. Cheers! \m/

‘bash’ A.K.A. The Solution for Everything – A few of the *Special Files*!

In bash, the shell reads one or more startup files. Here’s the details about what’s what and which is run when.

  1. /etc/profile is executed automatically at login.
  2. The file from the list of ~/.bash_profile, ~/.bash_login, or ~/.profile are then executed at login.
  3. ~/.bashrc is executed by every non-login shell, but if sh is used to invoke bash it reads the $ENV for POSIX compatability.

For reference, the ~ symbol is used in place of the user directory. One way to check this out yourself is to change directory to ~ with a cd ~ in the shell, then type pwd which will give the current directory. You’ll find that it is something like /Users/adron where instead of my name it’d be your user name.

When invoking the shell, you can also skip the ~/.bashrc or otherwise change the way bash starts up with the following options.

  • bash --init-file theFileToUseInstead or --rcfile instead of ~/.bashrc.
  • bash --norc which is similar to invoking with sh, which will use $ENV.
  • bash --noprofile will prevent /etc/profile or any other personal startup files. This will provide a pretty baseline bash shell for use.

Until next time, happy bash code thrashing!

Twitter for Developers, Cutting the Bullshit, Quelling the Trash Tire Fire

It’s been over a decade that Twitter has been an active part of the developer community. It’s grown in popularity from day one, and now holds the uneasy crown as the place for hot takes, trash from politicians, and the general tire fire that is the news. In many ways, that’s what they’ve aimed for. But then there’s us developers, people who make software, who make Twitter, who build all of this technology internet stuff right? We’re here using Twitter still, even amid the backstabbing and Twitter UI’s API’s being yanked from under us. They’ve of course in the past also banned UI’s and somehow here we are still using the service. However, I digress, Twitter’s wrongs against developers are numerous after we effectively built the service. In spite of all this we developers are a large contingent of people on Twitter. It’s still an amazingly useful medium for software developers, and especially new software developers, to get involved with. It’s a very effective tool to strengthen our careers and continue conversations within the developer communities themselves. One just has to avoid the cruft, and that’s what I intent to tackle some of in this article.

This list I’ve put together is of things that I personally have learned, often by stumbling through and discovering myself. These activities on Twitter do have a net positive effect on your career and ability to communicate with the world and local developer communities. First I’ll cover positive use cases of Twitter that are immensely useful as a software developer. These are even compounded if you’re an advocate of open source, cool technologies and libraries, and other miscellaneous things.

1. Twitter as a Communication Tool

First and foremost, Twitter has been and does – mostly – continue to be a communication tool. I make use of Twitter to connect with people for conference organizing, code projects, open source work, to have geek lunch, nerd brunch, and many other things that come up. It can and ought to be one of your primary communication mediums in that it connects many of the key active people within our overall communities. More so than email and other mediums by a large percent. If you intend to have a long term net effect and grow your presence and activities (conferences, meetups, coding groups, etc) you want to foster Twitter has become the de facto medium to be active on.

2. Twitter as a Collector of People

Twitter, even though it does seem to attract some of the most villainous scum (literally, not a figure of speech or hyperbole) and have some pretty horrifying problems (people calling in SWAT’s on people (extremely illegal), death threats, harassment) the net benefit within the community to bring people together has far outclassed pretty much any other system out there. Hacker News doesn’t, Facebook doesn’t, Google+ is cancelled, and about every other social media platform has failed to bring together the develop community in an effective and useful way.

3. Twitter for Answers

Even though I don’t often go to Twitter to find answers, sometimes I do. Often it is a last resort. After all, Twitter is most efficient at providing a place for links, quick blurbs, bumbling and babbling threads from people, and of course cat pictures and hot takes.

The combination powers of Twitter with other services however exponentially increases the ability of Twitter to help with answers. For example, write up a solidly written question on Stackoverflow or one of the branched out services and then post the question on Twitter, maybe inquiring for some retweets and boom, doubling, tripling, and greater multiplier of people looking at the question that can provide a prospective answer!

4. Twitter, Firestarter

One of the things I’ve also found Twitter good for is an outlet for pushing and often straightening out bad behavior in the community. Ever done something racist? Ever known someone to pull some misogynistic action? Yeah, unfortunately I know of these things happening too, and Twitter forces apologies and better behavior among people. But it also is a place people can wreck themselves and be just as destructive as they can learn to better themselves, especially those humans of us that have poor behavior and disrespectful tendencies.

But just as much as individual behaviors among us, Twitter has been used to straighten out some pretty trash behaviors from corporations. Sure, they’re not really people, but the conflagrations of this notion – true or not – make for pressure to be applied to corporations through other means besides the products and services they sell us individual humans, which to often are things we have to buy regardless, and this medium provides us an avenue to induce better behaviors in spite of purchases.

There of course is the positive and negative of this forced societal behavior and in many ways, improving corporate behavior throughout the world, but it’s here. Pressure of the people, often organized and started through Twitter, including against Twitter itself sometimes, is heavily rooted in activity right there on ole’ Twitter itself.

GSD Tactical Twitter

Alright, now to the meat of things. Twitter is great at all these things but how does one make the best use of it without it turning into an outright tire fire trash dump of distraction and stress? Well, it’s moderately easy, but one has to be careful.

1. Find Good and Entertaining People

My personal advice when starting on Twitter is to skip the companies. Don’t follow any of them. Same goes for organizations or any group account of sorts. The key to find good content, good common ground, and useful links, news, and related communities is to follow individuals that are involved in those things you want to be involved in already. The following are some specific examples, and for me, great people to follow.

2. Lift Up Others, Tweet to Others, Get Involved

When on Twitter, one can just lurk. It’s a completely valid thing to do. However lurking isn’t super high value. You just won’t get that much out of it. Instead, get involved. Find a link with something interesting, write up a tweet and post it. See something interesting someone else just tweeted, respond! See something that isn’t right, maybe tweet why it isn’t.

Always a good idea, regardless of the trash that is often on Twitter to still stay courteous, kind, and friendly. Remember, not everyone is from the mold you’ve come from, or seen things the way you have, so tread lightly and friendly and things mostly work out real well. Overall, people are attuned to helping those that help themselves and helping those that we run in social circles with.

All in all, get involved, tweet at, with, and all around your fellow Twitterers. Your return will improve and in the process you’ll add more value for others too.

3. Follow & Prune the Firehose of Tweets

Alright, I’ve written to follow and lift up others. That’s groovy, but also you gotta bring the hammer down sometimes. When that firehose of tweets just gets a little overwhelming check out what tweets are helpful, rate them to yourself, and unfollow some people if it’s not the direction or the tweets you’re getting value from.

Even though it’s difficult when just starting to use Twitter, the ratio will be more followed than followers for you. But as time goes forward and you get past 50 followers, 100, 500, 1000 you’ll need to make sure to keep the list of people you’ve followed just equal to or less than how many people follow you. It’ll help keep your feed manageable and also help you to keep interactions beneficial for you, followers, and followed.

4. The Down-Low on Conferences

If you’re looking to attend a conference, Twitter via hashtags is a great way to get information on conferences. Dig in, dig deep. Talk to people about the conference in particular. If necessary get into direct messages and invoke the whisper net if need be. Sometimes conferences can be exponentially useful and sometimes they end up bothersome cash burning wastes of time. Figure out what you want from a prospective conference and dig in via Twitter, you’ll prevent wasting time and burning cash, and exponentially increase the positives you can get out of a conference.

5. Filter the Trash Fire

Ok, let’s get super serious. One way Twitter has become a trash fire for many or most people these days is because of the political trash dumped in. Much of Twitter for the general public is bot armies from Russia, crazies like the nutty Wohl kid, and other junk nut accounts. One way to notch this down to a minimal trash fire is to throw some filters (i.e. mute certain words) on your Twitter account. For example here’s my list:

filters-muted-words

Now as you’ve read that, remember that my goal has been to focus the stream on tech content with a little heavy metal, a few cats, and other entertainment here and there. For example I’m fine with sports events like baseball and football but really don’t want to get distracted by it in on my Twitter stream. On game day those events just overwhelm the tweets and things that are useful get drowned out.

Now a lot of the other stuff in the list is the horrifying reality of the United States today, reflected on Twitter, and part of something that I don’t want distracting me either. Overall this has made Twitter dramatically more useful for me again.