Industry Introspection

farnsworth-in-chair.jpgEvery few days I like to sit down like an old academic professor and just read and ponder what’s going on in the computer hardware and software industries. As of late it’s been interesting to dive into whatever Elon Musk is working on also; electric cars, solar energy, rockets, or even boring machines. Another thing I’ve had a curiosity in and continue to find interesting is machine learning, or as some call it in the mainstream movie media artificial intelligence.

I’ll tackle some of these technologies real quick so we all have a current situational report. Each of these topics seems to be a area of work that could use some extensive clarification. I’ll provide some of my own thoughts along with a few links of reference to dig in deeper, so that one doesn’t fall into the trap of uniformed oblivious media consumer.

Rockets (i.e. Space)

My History: I haven’t always paid attention to this space because I never had intention of working in the area, however who isn’t interested in rockets in some way. Well, considering my desire for other work, I was extremely fortunate where I grew up to be involved in rocketry. I grew up in Picayune, Mississippi which is a mere ~15-20 miles away from John C. Stennis Space Center.

I had the joy of experiencing rocket tests at John C. Stennis and seeing research into rockets as a child, and on some of my first paid computer related gigs. As one might suspect, they use computers to do rocket research and help with launches. Shocker right!

My Thoughts: The space we’re in right now is impressive. The market is actually getting involved in launching it’s own rockets, which means we’re likely only years or maybe a decade or two of having an economically sustainable rocket program. Hopefully NASA can work on more focused deep space missions now while Musk’s Space-X and others refine and perfect orbital rockets for satellites and all that mess.

SITREP:

Machine Learning

farnsworth-future-of-ai-mlMy History: I’ve toyed with machine learning on and off again, working on pathing algorithms for objects to decide travel patterns to supervised learning algorithms. In the end I’ve generally ended up working on other things in my day to day work but I know this will be changing in the near future (next year or three). It’s an extremely interesting space of work and research.

My Thoughts: First, getting AI & ML (That’s artificial integlligence and machine learning) conflated, especially in the media, is starting to reflect the popularity of said space in the software industry. However, for the most part these two things are effectively the same thing. It’s just different words describing the industry space where we’re trying to make machines make decisions we deem intelligent based on available data.

That actually leads to many other discussions. What do we as humans deem intelligent and what happens when available data isn’t enough? But more words on that for another day. I know the questions are burning in the mind of every chief executive of something that wants this mythical AI they keep hearing about and paying voluminous amounts of money to their big data bad ass data science ninja architects to implement but rarely have answers for all of it.

The SITREP:

Overall there’s a ton of material ending up on the web related to AI/ML, and my top suggestion is to start googling so you can pick and choose which aspects you want to read about in this space. One could dive in via the super technical aspect of how systems work that are being used for processing, how the algorithms work, or even working with data to model good decision results from data sets (i.e. diving into training). But it’s really a space that is awash in resources. Dive in!

End Words

That’s it for my industry introspections for now. If you’re interested in reading about this, programming material, and related topics subscribe, follow, or RSS feed read the ole’ blog here.

Using Bosh to Bootstrap Cloud Foundry via Stark & Wayne Consulting

I finally sat down and really started to take a stab at Cloud Foundry Bosh. Here’s the quick lowdown on installing the necessary bits and getting an initial environment built. Big thanks out to Dr Nic @drnic, Luke Bakken & Brain McClain @brianmmcclain for initial pointers to where the good content is. With their guidance and help I’ve put together this how-to. Enjoy…  boshing.

Prerequisites

Step: Get an instance/machine up and running.

To make sure I had a totally clean starting point I started out with an AWS EC2 Instance to work from. I chose a micro instance loaded with Ubuntu. You can use your local workstation if you want to or whatever, it really doesn’t matter. The one catch, of course is you’ll have to have a supported *nix based operating system.

Step: Get things updated for Ubuntu.

[sourcecode language=”bash”]
sudo apt-get update
[/sourcecode]

Step: Get cURL to make life easy.

[sourcecode language=”bash”]
sudo apt-get install curl
[/sourcecode]

Step: Get Ruby, in a proper way.

[sourcecode language=”bash”]
\curl -L https://get.rvm.io | bash -s stable
source ~/.rvm/scripts/rvm
rvm autolibs enable
rvm requirements
[/sourcecode]

Enabling autolibs sets up so that rvm will install all the requirements with the ‘rvm requirements’ command. It used to just show you what you needed, then you’d have to go through and install them. This requirements phase includes some specifics, such as git, gcc, sqlite, and other tools needed to build, execute and work with Ruby via rvm. Really helpful things overall, which will come in handy later when using this instance for whatever purposes.

Finish up the Ruby install and set it as our default ruby to use.

[sourcecode language=”bash”]
rvm install 1.9.3
rvm use 1.9.3 –default
rvm rubygems current
[/sourcecode]

Step: Get bosh-bootstrap.

bosh-bootstrap is the easiest way to get started with a sample bosh deployment. For more information check out Dr Nic’s Stark and Wayne repo on Github. (also check out the Cloud Foundry Bosh repo.)

[sourcecode language=”bash”]
gem install bosh-bootstrap
gem update –system
[/sourcecode]

Git was installed a little earlier in the process, so now set the default user name and email so that when we use bosh it will know what to use for cloning repositories it uses.

[sourcecode language=”bash”]
git config –global user.name "Adron Hall"
git config –global user.email plzdont@spamme.bro
[/sourcecode]

Step: Launch a bosh deploy with the bootstrap.

[sourcecode language=”bash”]
bosh-bootstrap deploy
[/sourcecode]

You’ll receive a prompt, and here’s what to hit to get a good first deploy.

Stage 1: I select AWS, simply as I’ve no OpenStack environment. One day maybe I can try out the other option. Until then I went with the tried and true AWS. Here you’ll need to enter your access & secret key from the AWS security settings for your AWS account.

For the region, I selected #7, which is west 2. That translates to the data center in Oregon. Why did I select Oregon? Because I live in Portland and that data center is about 50 miles away. Otherwise it doesn’t matter which region you select, any region can spool up almost any type of bosh environment.

Stage 2: In this stage, select default by hitting enter. This will choose the default bosh settings. The default uses a medium instance to spool up a good default Cloud Foundry environment. It also sets up a security group specifically for Cloud Foundry.

Stage 3: At this point you’ll be prompted to select what to do, choose to create an inception virtual machine. After a while, sometimes a few minutes, sometimes an hour or two – depending on internal and external connections – you should receive the “Stage 6: Setup bosh” results.

Stage 6: Setup bosh

setup bosh user
uploading /tmp/remote_script_setup_bosh_user to Inception VM
Initially targeting micro-bosh…
Target set to `microbosh-aws-us-west-2′
Creating initial user adron…
Logged in as `admin’
User `adron’ has been created
Login as adron…
Logged in as `adron’
Successfully setup bosh user
cleanup permissions
uploading /tmp/remote_script_cleanup_permissions to Inception VM
Successfully cleanup permissions
Locally targeting and login to new BOSH…
bosh -u adron -p cheesewhiz target 54.214.0.15
Target set to `microbosh-aws-us-west-2′
bosh login adron cheesewhiz
Logged in as `adron’
Confirming: You are now targeting and logged in to your BOSH

ubuntu@ip-yz-xyz-xx-yy:~$

If you look in your AWS Console you should also see a box with a key pair named “inception” and one that is under the “microbosh-aws-us-west-2” name. The inception instance is a m1.small while the microbosh instance is an m1.medium.

That should get you going with bosh. In my next entry around bosh I’ll dive into some of Dr Nic & Brian McClain’s work before diving into what exactly Bosh actually is. As one may expect, from Stark & Wayne we can expect some pretty cool stuff, so keep an eye over there on Stark & Wayne.

A Few Notes on Riak 1.3 RC

Full context – Riak 1.3 RC came out just a couple dozen hours ago. RC stands for release candidate, which in turn basically means that version 1.3 is complete and any other additions will be for quick fixes or any issues that crop up. I’ve just started rolling a few new systems myself with this new version and hope you’ll join me in taking a hack at it. Let’s jump into a few reason why you’d want to leap into 1.3. You can read about the features below via the release notes also, but I’ve turned them into smaller bit size chunks below.

  • Giddyup in action!
    Giddyup in action!

    The first thing with the latest v1.3 has been the massive effort put into testing via the riak_test and the giddyup repos. Ongoing there will be a much easier way to move forward in features & quality. This is one of the reasons I love working for Basho, the whole team isn’t about smoke and mirrors with testing, they readily and diligently work on testing. Which to add context, remember we’re talking about distributed systems here, which aren’t exactly the easiest thing to test. One doesn’t just merely walk in and write unit tests and assume a distributed systems is tested. This moves us forward, and those that want to contribute and get involved more heavily in Riak now have a platform to dive in confidently when using these testing repositories.

  • Active Anti-Entropy – Alright, now we’re getting to the features with bad ass sounding names. Also referred to as AAE, this feature grabs bad replica data and begins a correction through read repair to protect data. It’s one more layer of protection against any type of data loss, disaster, bit rot, etc).
  • MapReduce Sink Backpressure – This one reminds me of tuning when setting up forced induction, AKA a turbo on a car. But I digress, I’ve snagged a description from the release notes for this feature, “Riak Pipe brought inter-stage backpressure to Riak KV’s MapReduce system. However, prior to Riak 1.3, that backpressure did not extend to the sink. It was assumed that the Protocol Buffers or HTTP endpoint could handle the full output rate of the pipe. With Riak 1.3, backpressure has been extended to the sink so that those endpoint processes no longer become overwhelmed. This backpressure is tunable via a soft cap on the size of the sink’s buffer, and a period at which a worker should check that cap. These can be configured at the Riak console by setting application environment variables” ….suffice it to say this helps out with map reduce in certain situations.
  • Additional IPv6 Support – Riak Handoff and Protocol Buggers listen ala IPv6 now. Nuff’ said.
  • Luke removal – Luke is completely and utterly gone now. Dead. Don’t look for Luke here.
  • Riaknostic – This is now part of the default featureset instead of separate tooling.
  • SmartOS 1.8 Packages – They’re available.
  • Health Check – This is a pretty awesome system that’s been added. Basically it watches the system and enables and disables services based on conditions. It’s super easy, just flick the switch in the app.config.
    [sourcecode language=”yml”]
    {enable_health_checks, true}
    [/sourcecode]
  • Reset Bucket Properties – A quickie definition from the release notes “The HTTP interface now supports resetting bucket properties to their default values. Bucket properties are stored in Riak’s ring structure that is gossiped around the cluster. Resetting bucket properties for buckets that are no longer used or that are using the default properties can reduce the amount of gossiped data.”

There were also a lot of PRs and more that you can check out on Github. These are the main key features that are now available and ready for use in 1.3. Check em’ out, feel free to contact me or any of the team to ask questions, let us know your 2 cents or otherwise banter about. Cheers! Sometime in the coming days I’ll have a quick start, akin to what’s in the docs, but with some specific ops on some IaaS Providers. So keep reading, coming up soon.

Happy hacking!  \m/   \m/

ORMs Suck, I’m Asking & I’m Telling

Here’s a thing that’s come up already. ORMs, or Object Relational Mapper, are a RDBMS based thing for devs that want, in essence a statically typed object to deal with when writing code (yes, I know there’s a ton of other things an ORM can do or be used for, but I’m going with a simple explanation here, here’s more info). For some situations, I can see where that’s a productivity booster, but in other situations it is a very broken ideal. Especially when it comes to needs around highly complex query needs, performance and a host of other functionalities when you get into the higher end demands of Enterprise or *web scale* type systems.

Two things that has brought up this question in a new light, is:

  • One: Why would you use an ORM in a dynamic language like JavaScript, that has a simple native format like JSON? It tends to make far less sense to do this. What’s your take? Like the idea? Hate the idea?
  • Two: In a schemaless database, forcing an ORM on something that is built specifically to not have this featureset (for a number of reasons) seems to break the design ideas of the systems. Whether it is Mongo, Riak, Cassandra or whatever, why would you actually want to or not want to use an ORM? I’ve got my own thoughts but would like to know what people think about this notion.

-Please comment, I’ll be diving into any feedback too.

Doing a Video Show, Killing Podcasts, Lighting Fires, Code, More Code, New Technology and Making Things Happen!

Video Production is starting soon on a new hard core coder & tech video show. Done on the nitty gritty. It’s going to be about purely tech, code, more code, testing, coding, entrepreneurship in technology (not in general) and more of the hard core, nitty gritty, total hands down low down on the technology sector and technology scene. We’ll be diving into everything from enterprise technology to startup technology, who’s innovating and who’s stagnating, who’s kicking ass and who is enjoying the ride.

What I’d love from you, dear reader, is help with this question. What kind of content would you like?