Often a programmer sits back and reflects on, “how can I get better at my job?” A number of months ago I found myself sitting with that same reflection. One of the many ideas that popped into my head was to put together focused material around a topic. Training material, a blog series or something else. I wasn’t sure just yet what it would be, but I knew that was one of the many things I wanted to do. Fast forward a few months.
Fellow Tier 3 coder Richard Seroter, who I’d known for some time via his blog “Richard Seroter’s Architecture Musings” and I had a conversation about what we do, respectively, to keep our skills honed in tech. He brought up he’d be putting together material for Pluralsight for a while now and enjoyed that. It struck me as something I’d like to do too, considering my past reflection, so he made an intro. The rest is history!
Pluralsight Author, Achievement Reached
For now, check out my Riak Fundamentals Course and some of the other great material that Pluralsight has available. They’ve just acquired TekPub, Peepcode and a number of other companies too, so when you subscribe you don’t just get all the Pluralsight material but also access to all the material at these excellent course creating companies! In the coming days and weeks I’ll have some reviews of other courses I enjoyed. As always, enjoy, subscribe to my blog, subscribe to Pluralsight and cheers!
…and another update on Junction.
The review slipped past me this week. A little food poisoning will do that to a person. But I’m kicking again.
A Quick Summary
The RSS/News Feed section of the app has been built and put into action. So if you pull the latest code and run the application, navigate into the news section you’ll get the Basho Blog feed. This definitely needs cleaned up a bit from the UI perspective but the main elements are there.
Next steps are…
One of the things the team is aiming to knock out next is to get some MVVM (Mode View ViewModel) architecture setup to build against versus what we’ve started with, which is just the basic skeleton of things thrown together. It works, but it’ll be nice to have some clean architecture behind the application to work with.
I’m aiming to put together a blog entry on troubleshooting the build server for Junction and also the how-to on setting up the RSS/News Feed Reader section of the app in the coming week also. Subscribe to keep up with the latest in Junction news and also all the other tidbits on the blog. Cheers!
It’s time for another Tuesday Bi-weekly Review! We’ve been making some progress and so far we’ve tackled a few elements of the project. The first big task was to get more information out there for the community & team working on the project. I’ve spent some time along with the contributors on github and via other means to make more information available to what the intent is and how people can contribute. So if you’re interested in helping with an entire domain space or merely a small element of the application, ping me and I’ll work with you to make it as easy as possible to contribute. With that, let’s jump into what’s what and what’s new. Cheers!
We Have a Build Server, More on This Soon, but for now…
I’ll have a post on how to setup Team City and quick tour of what is setup for the Junction Project. So stay tuned and I’ll have that and other news posted as it happens this coming week along with Team City & other tutorials related to the project itself. For a quick sneak peek feel free to take a look at the build server located at: http://teamcity.cascadiahacks.org/. Just login with “guest” and no password.
More Items Listed and Working on First Feature Commits and Comments For…
We also got a conversation started among a few of us “What would teams that use Riak like to see in a Riak Admin Application?” Jump into and add your two cents regardless of whether you’re diving into the project or not.
Until later, happy coding!
So every two weeks I intend to provide an update for the Junction Project. Who might have joined, what was worked on, where we are and generally any other bits of news related to the project. This is the first “Junction Two Weeks on Tuesday Review” so enjoy! 🙂
- Two weeks ago today I wrote the entry “Introducing Junction” to kick off the project. Everything is hosted on github via github pages at http://adron.github.io/junction/ and the git repository at https://github.com/Adron/junction. The video in which I described at a high level each of the sections of the application is located here: http://vimeo.com/adronhall/junction.
- Clive Boulton @cliveb, Jared Wray @jaredwray, Kristen Mozian @kmozian and OJ Reeves @OJ joined the project to help out.
- Issues, as stories and tasks were added to get started with the project. Here’s a first draft of the things we’re all working on. If you’d like to jump in, feel free to ping me and I’ll add you to the project, you can submit a PR (Pull Request) or talk to me about organizing a hackathon to help move the project forward.
Github Issues – Working Items
The easiest way to view these is to log into the Huboard Kanban Board and give a look see of what is in progress and who’s working on what. Currently I’ve outlined the big items that we’re working on and would love a fellow coder to jump in on. If you’re interested, ping me @adron or just jump into the issues list on Github (or view by milestone – i.e. functional area) and comment on the issue you want to dive into, I’ll add you so you can get started!
For the “Call the Doctor (Administration and Maintenance)” part of the application there are a number of questions to answer. How should we connect to Riak to ensure a secure SSH connection? Should we even use SSH? Is there another way to connect to the Riak Cluster for a secure way to administer the cluster?
In the “Golfing With Your Data (Query, Put, Deletes, Etc. Handling the CRUD)” one could dive into creating a functional query space to pull data out of a Riak Cluster. A lot of UI work needs to be done in this space, so if you’re up for putting together some awesome windows 8 interfaces, I’d love to hear from you!
At this point we’re moving forward. We’re always looking forward to new participants so reach out if you’re up for helping out! So until the next two weeks are up, see ya at the Junction!
The “Client Round Robin Anti-Pattern”
One of the features that is often available in Riak Client software (including the CorrguatedIron .NET Client, the riak-js client and others) is the ability to send requests to the Riak Cluster through a round robin style approach. What this means is each IP, of each node within the Riak Cluster is entered into a config file for the client. The client then goes through that list to send off requests to read, write or delete data in the database.
The client being responsible and knowledgeable about the data tier of the application in an architecture is an immediate red flag! The concept around SoC (Separation of Concerns) dictates that
“SoC is a principle for separating a computer program into distinct sections, such that each section addresses a separate concern.“
Having the client provide a network tier layer to round robin communication with the database leaves us in a scenario that should be separated into individual concerns. Below is some basic guidance on eliminating this SoC issue.
- Client ONLY sends and receives communication: The client, especially in the situation with a distributed system like Riak should only be dealing with sending and receiving information from the cluster or a facade that provides an interface for that cluster.
- Another layer should deal with the network communication and division of nodes and node communication. Ideally, in the case or Riak, and most distributed systems this should be dealt with at the network device layer (router).
- The network device (router) layer would ideally be able to have (through software likely) a way to automate the failure, inclusion or exclusion of nodes with the cluster system. If a node goes down, the network device should handle the immediate cessation of communication with that node from all clients, routing the communication accordingly to an active node.
- The node itself needs to maintain a continual information state available to the network. Ideally the network state would identify any addition or removal of a node and if possible the immediate failure of a node. Of course it isn’t always possible to be informed of a failure, but the first line of defense should start within the cluster itself among the nodes.
Having the client handle all of these parts of the functional architecture leads to a number of problems, not merely that the guidance of the SoC concept is broken. With the client attempting to track and be aware of the individual nodes in the cluster, it sets the client with a huge responsibility.
Take for instance the riak-js client. If a node goes down the client will need to be aware of which node has gone down. For a few seconds (yes, you have to wait entire seconds at this level) the node will be gone and the client won’t know it is down. The client would just have to reasonably wait. When the communication times out, the client would then have to have the responsibility of marking that particular node as down. At this point the client must track which node it is in some type of data repository local to the client. The client must also set a time or some way to identify when the node comes back up. Several questions start to come up such as;
- Does the client do an arbitrary test to determine when the node comes back up?
- When the node comes back up is it considered alive or damaged?
- How would the client manage the IP (or identifier) of the node that has gone down?
- How long would the client store that the node is down?
The list of questions can get long pretty quick, thus the bad karma of not following a good practice around separating your concerns appropriately! One has to be careful, a god class might be right around the corner otherwise! That’s it for this quick journey into some distributed database usage guidelines. Until next, happy data sciencing. 😉
Jeremiah’s Demo Works, IT WORKS IT WORKS!
A few days ago Troy Howard, Jeremiah Peschka and I all traveled via Amtrak Cascades up to Seattle. The mission was simple, Jeremiah was presenting “Riak in a .NET World”, I was handling logistics and Troy was handling video.
So I took the video that Troy shot, I edited it, put together some soundtrack to it and let Jeremiah’s big data magic shine. He covers the basics around RDBMSes, SQL Server in this case but easily it applies to any RDBMS in large part. These basics bring us up to where and why an architecture needs to shift from an RDBMS solution to a distributed solution like Riak. After stepping through some of the key reasons to move to Riak, Jeremiah walks through a live demo of using CorrugatedIron, the .NET Client for Riak (Github repo). During the walk through he covers the specific characteristics of how CorrugatedIron interacts with Riak through indexs, buckets and during puts and pulls of data.
Toward the end of the video Joseph Blomstedt @jtuple, Troy Howard @thoward37, Jeremiah Peschka @peschkaj, Clive Boulton @iC and Richard Turner @bitcrazed. Also note, I’ve enabled download for this specific video since it is actually a large video (1.08GB total). So you may want to download and watch it if you don’t have a super reliable high speed internet connection.
Also for more on Jeremiah’s work check out http://www.brentozar.com/articles/riak/ and contact him at http://www.brentozar.com/contact/
Whew, it’s been a total blast working at Basho. I’ve accomplished a ton of things. Riak is a solid distributed database system and I’m glad to have worked with the team on advocating its use, teaching distributed systems ideas and concepts and generally spreading the knowledge. I’ve seen some truly great things that people are hacking together, setting up for projects and redesigning old systems to utilize newer, better, faster and more capable distributed systems concepts and ideas. Some of the things I’m happy to have contributed to in my time at Basho.
- Wrote a couple blog entries on the Basho blog
- …and a few right here related to my work at Basho
- I helped negotiate and get an effort started that came to fruition with Tier 3 releasing a Riak CS backed object store for their customers. A very cool feature added to their already formidable enterprise cloud offerings. Read more about that here “Tier 3 Object Storage” and here “Tier 3 Launches Global Object Storage“. Their implementation is really nice, including many geographic regions of accessibility, S3 API compatibility and high end storage capabilities that offer a bigger punch of performance than your average object storage in the cloud!
- I launched both the Seattle Riak (up to 52 members now!) and Portland Riak (up to 74 members now!!) groups started, which you should join, they’re a good time, good conversation and great information.
- I partnered with Troy Howard @thoward37 to run the second year of Node PDX. Basho was excellent enough to contribute not just a few bucks but also sent Chris Meiklejohn @cmeik out to speak at the conference.
- I worked with the Google team to get the awesome Tyler Hannan over to the Google I/O Conference onto the Google I/O Data Panel. The panel was solid, thanks to Tyler @tylerhannan, @juliaferraioli and the Google Team for making this happen!
- I got to work directly with a number of people at Windows Azure, AWS and EngineYard in deploying Riak, testing out how the respective images (azure VM Depot & AWS AMI) and deployments (Riak EDS) would work. In the end, this has been a great opportunity to learn more about the latest and greatest of each of these services. I’ve been impressed as they’ve each been doing a seriously kick ass job lately!
- Had a blast working on the New Relic Riak Plugin with Drew Kerrigan for the recently released New Relic plugin ecosystem. Code is also available via the Github Basho Riak Plugin repo.
…and there has been a whole lot more. Suffice it to say, Basho has provided me with some sweet opportunities to work on some extremely interesting data projects from a very data sciency point of view (yeah I know sciency aint a word). There may be more Riak work and Riak meetups and Riak hacks and Riak who knows what coming from me, but the meetups & such are now at the hands of the core Riak crew and…
Where Am I Headed?
Right now, I’m moving 20 blocks away from where I currently live, setting up a couch to hack on and grabbing a beer. I’ve got a few personal projects I’ve been wanting to work on. Then I’m taking a few weeks to do some side projects that have been on the burner. Keep an eye out, I’ll be kicking off one, maybe two of these open source projects in the next few days. As @tsantero twitted…
…I’m going to attack my own notebook of ideas. Maybe I’ll even work on that Riak CS Video object store that Tom and I spoke about 10 months ago? Either way, whatever the projects are, I’ll have them posted right here. Until then…
Cheers & Happy Hacking!