Tag Archives: node

Cassandra Datacenter & Racks

The last post in this series is Distributed Database Things to Know: Consistent Hashing.

Let’s talk about the analogy of Apache Cassandra Datacenter & Racks to actual datacenter and racks. I kind of enjoy the use of the terms datacenter and racks to describe architectural elements of Cassandra. However, as time moves on the relationship between these terms and why they’re called datacenter and racks can be obfuscated.

Take for instance, a datacenter could just be a cloud provider, an actual physical datacenter location, a zone in Azure, or region in some other provider. What an actual Datacenter in Cassandra parlance actually is can vary, but the origins of why it’s called a Datacenter remains the same. The elements of racks also can vary, but also remain the same.

Origins: Racks & Datacenters?

Let’s cover the actual things in this industry we call datacenter and racks first, unrelated to Apache Cassandra terms.

Racks: The easiest way to describe a physical rack is to show pictures of datacenter racks via the ole’ Google images.

racks.png

A rack is something that is located in a data-center, or even just someone’s garage in some odd scenarios. Ya know, if somebody wants serious hardware to work with. The rack then has a number of servers, often various kinds, within that rack itself. As you can see from the images above there’s a wide range of these racks.

Datacenter: Again the easiest way to describe a datacenter is to just look at a bunch of pictures of datacenter, albeit you see lots of racks again. But really, that’s what a datacenter is, is a building that has lots and lots of racks.

data-center.png

However in Apache Cassandra (and respectively DataStax Enterprise products) a datacenter and rack do not directly correlate to a physical rack or datacenter. The idea is more of an abstraction than hard mapping to the physical realm. In turn it is better to think of datacenter and racks as a way to structure and organize your DataStax Enterprise or Apache Cassandra architecture. From a tree perspective of organizing your cluster, think of things in this hierarchy.

  • Cluster
    • Datacenter(s)
      • Rack(s)
        • Server(s)
          • Node (vnode)

Apache Cassandra Datacenter

An Apache Cassandra Datacenter is a group of nodes, related and configured within a cluster for replication purposes. Setting up a specific set of related nodes into a datacenter helps to reduce latency, prevent transactions from impact by other workloads, and related effects. The replication factor can also be setup to write to multiple datacenter, providing additional flexibility in architectural design and organization. One specific element of datacenter to note is that they must contain only one node type:

Depending on the replication factor, data can be written to multiple datacenters. Datacenters must never span physical locations.Each datacenter usually contains only one node type. The node types are:

  • Transactional: Previously referred to as a Cassandra node.
  • DSE Graph: A graph database for managing, analyzing, and searching highly-connected data.
  • DSE Analytics: Integration with Apache Spark.
  • DSE Search: Integration with Apache Solr. Previously referred to as a Solr node.
  • DSE SearchAnalytics: DSE Search queries within DSE Analytics jobs.

Apache Cassandra Racks

An Apache Cassandra Rack is a grouped set of servers. The architecture of Cassandra uses racks so that no replica is stored redundantly inside a singular rack, ensuring that replicas are spread around through different racks in case one rack goes down. Within a datacenter there could be multiple racks with multiple servers, as the hierarchy shown above would dictate.

To determine where data goes within a rack or sets of racks Apache Cassandra uses what is referred to as a snitch. A snitch determines which racks and datacenter a particular node belongs to, and by respect of that, determines where the replicas of data will end up. This replication strategy which is informed by the snitch can take the form of numerous kinds of snitches, some examples include;

  • SimpleSnitch – this snitch treats order as proximity. This is primarily only used when in a single-datacenter deployment.
  • Dynamic Snitching – the dynamic snitch monitors read latencies to avoid reading from hosts that have slowed down.
  • RackInferringSnitch – Proximity is determined by rack and datacenter, assumed corresponding to 3rd and 2nd octet of each node’s IP address. This particular snitch is often used as an example for writing a custom snitch class since it isn’t particularly useful unless it happens to match one’s deployment conventions.

In the future I’ll outline a few more snitches, how some of them work with more specific detail, and I’ll get into a whole selection of other topics. Be sure to subscribe to the blog, the ole’ RSS feed works great too, and follow @CompositeCode for blog updates. For discourse and hot takes follow me @Adron.

Distributed Database Things to Know Series

  1. Consistent Hashing
  2. Apache Cassandra Datacenter & Racks (this post)

 

The Method I Use Setting Up a Dev Machine for Node.js

UPDATED: April 4th, 2019

It seems every few months setup of whatever tech stack is always tweaked a bit. This is a collection of information I used to setup my box recently. First off, for the development box I always use nvm as it is routine to need a different version of Node.js installed for various repositories and such. The best article I’ve found that is super up to date for Ubuntu 18.04 is Digital Ocean’s article (kind of typical for them to have the best article, as their blog is exceptionally good). In it the specific installation of nvm I’ve noticed has changed since I last worked with it some many months ago. Continue reading

__2 “Starting a Basic Loopback API & Continuous Integration”

In this article Keartida is going to dive into setting up a basic Loopback API project and get a build of that project running on a continuous integration service. In this example she’s going to get the project setup with Codeship.

Prerequisites:

  • Be sure, whichever system you are using, to have a C++ compiler installed. For Windows that usually means installing Visual Studio or something, on OS-X install XCode and the Developer Tools. On Ubuntu the GCC compiler and other options exist. For instructions on OS-X and Linux check out installing compiler tools.
  • Ubuntu
  • OS-X
  • For windows, I’d highly suggest setting up a VM of Ubuntu to do any work with Loopback, Node.js, or follow along with this material. It’s possible on Windows, but there are a number of things that are lacking. If you still want to make a go of using Windows, here are some initial setup steps here.

Nice to Haves:

  • git-flow – works on any bash, handles the branching and merging. Very nice scripts to have.
  • bashit – Adding more information to the bash prompt (works on OS-X, not Ubuntu or Windows Bash)

Continue reading

__1 “Getting Started, Kanban & First Steps for a Sharing App”

This is the first (of course the precursor to this entry was the zero day team introduction article) of an ongoing series I’m going to put together. I’m going to write this series from the context of a team building a product. I’ll have code samples and more as I work along through the material.

The first step included Oi Elffaw having a discussion with the team to setup the first week’s working effort. Oi decided to call it a sprint and the rest of the team decided that was cool too. This was week one after all and there wasn’t going to be much else besides testing, research, and setup that took place.

Prerequisites

Before starting everything I went ahead and created a project repository on github for Oi to use waffle.io with. Waffle.io is an online service that works with github issues to provide a kanban style inferface to the issues. This provides an easier view, especially for leads and management, to get insight into where things are and what’s on the plate for the team for the week. I included the default node.js .gitignore file and an Apache 2.0 license when I created the repository. Github then seeds the project with a .gitignore, README.md and the license files.

After setting up the repository in github I pinged Oi and he set to work after the team’s initial meet to discuss what week one would include. Continue reading

Nordic.js and .NET Fringe

Ok, so many of the conferences out there you’re going to get fed the company line. You’ll probably experience some odd behaviors and people pushing product on you. If you’ve got the same feeling about conferences as me, and you’d like to experience these things at a conference:

  • A diverse audience of many different people from many different places.
  • You’d like to talk to others that are passionate about the future direction of technology and what we can create with that technology!
  • Listen and watch presenters provide insight to technology, ideas, and spaces that I don’t regularly get to hear about or discuss.
  • Meet many new friends, build my cohort of coders, and learn from each other.
  • Have a good time, relaxed, and not under the pressure of being sold things.

…then these conferences are for you. Seriously, I wouldn’t and won’t ever direct anybody to corporate conferences anymore except maybe in super rare occasions. The conferences to attend are the grassroots, community organized conferences like these two! There are too many other truly awesome conferences where the future is being discussed and made RIGHT NOW! There’s a few lined up that I’ll be attending and am currently working with as an organizer. Here’s the top two RIGHT NOW!

Continue reading

Update 2 – Speaking at Node Summit and Node.js Club SF

I’ll be puddle jumping down to San Francisco on the 9th to teach a workshop at Node Summit titled “Node Continuous Integration to Delivery” and then I’ll swing into the Node.js Club SF to give a short (45 minute) deep dive into who, what, where, when, and why of “Node Continuous Integration and Delivery”.

Here’s a short description of the workshop, “In this workshop I’ll show the steps to get from minimal project, to fully deployed web site with a fully integrated and operative continuous integration and delivery pipeline. I’ll also dive into some basic practices around test or behavior driven development within JavaScript to build a full server to client website and have that website deployed, tested – on both client and server, and deployed auto-magically to a live site. During this workshop I’ll break this out into segments, so that over the course of the workshop there will be time to stop, ask questions, and then move to the next stage of the project.

As I get more specific details about this, I’ll add them to this post. But currently check out Node Summit and Node.js Club SF for more information.

UPDATED January 21, 2015 @ 4:16pm

I’m now listed on the speakers section of the Node Summit Conference Site. The workshop I’ll be providing will be on Monday the 9th, the day before the actual conference from. My workshop will be 3-5pm, in the Fisher East Banquet Room in the event location at:  Mission Bay Conference Center at UCSF, 1675 Owens St, San Francisco, CA 94158

If you’d like to sign up for Node Summit and get 25% off of the registration fee, then follow this link to register for the conference!

iojs

I’m so mad!!! My response to the fork of node.js to io.js!

I’m so mad!!!

No, actually I’m not. This is cool.

It’s so confusing! No, it’s actually not. Use your learnings and read this Node Forward and read Max’s @maxogdenWhat you need to know.” gist.

It’ll be so hard to test this and test with node.js. Not really, again, read the threads there are a million different ways that you’ll be able to setup a clean build against either while keeping both around on your dev machine. Again, repeating myself, this is cool.

Summary for Devs: This is cool. It’s not going to wreck your projects. Just read up on it and it’ll all be a most excellent journey.

Summary for Decision Makers: Read @eranhammer‘s blog post here. TLDR; is, don’t second guess your decision to go with Node.js, don’t flip out about a Node.js or io.js investment, this isn’t anything more than a healthy ecosystem at work. It’ll be cool.