As I have stepped full speed into work and research at DataStax there were a few things I needed as soon as I could possibly get them put together. Before even diving into development, use case examples, or reference application development I needed to have some clusters built up. The Docker image is great for some simple local development, but beyond that I wanted to have some live 3+ node clusters to work with. The specific deployed and configured use cases I had included:
I wanted to have a DataStax Enterprise 6 Cassandra Cluster up and running ASAP. A cluster that would be long lived that I could developer sample applications against, use for testing purposes, and generally develop against from a Cassandra and DSE purpose.
I wanted to have an easy to use cluster setup for Cassandra – just the OSS deployment – possibly coded and configured for deployment with Terraform and related scripts necessary to get a 3 node cluster up and running in Google Cloud Platform, Azure, or AWS.
I wanted a DataStax Enterprise 6 enabled deployment, that would showcase some of the excellent tooling DataStax has built around the database itself.
I immediately set out to build solutions for these three requirements.
The first cluster system I decided to aim for was figuring out a way to get some reasonably priced hardware to actually build a physical cluster. Something that would make it absurdly easy to just have something to work with anytime I want without incurring additional expenses. Kind of the ultimate local development environment. With that I began scouring the interwebs and checking out where or how I could get some boxes to build this cluster with. I also reached out to a few people to see if I could be gifted some boxes from Dell or another manufacturer.
.NET Fringe took place this last week. It’s been a rather long time since my last actual conference that I actually got to really attend, meet people, and talk to people about all the different projects, aspirations, goals, and ideas about what’s next for the future. This conference was perfect to jump into, first and foremost, I knew it was an effort in being inclusive of the existing community and newcomers. We’d reached out to many brave souls to come and attend this conference about pushing technology into the future.
I met some truly excellent people. Smart, focused, intent, and a whole lot of great conversations followed meeting these people. Here’s a few people you’ll want to keep an eye on based on the technology they’re working on. I got to sit down and talk to every one of these coders and they’re in top form, smart, inventive, witty and full of great humor to boot!
I met Maria and one of the first things I saw was her crafty and most excellent art sketches around lifestyles, heroes, and more. I love art like this, and was really impressed with what Maria had done with her’s.
I was able to hang out with Maria a bit more and had some good conversation time talking about evangelism, tech fun and nonsense all around. I also was able to attend her talk on “Legacy… What?” which was excellent. The question she posed in the description states a common question posed, “When students think about .Net they think: legacy , enterprise , retired, and what is that?” which I too find to be a valid thought. Is .NET purely legacy these days? For many getting into the field it generally isn’ the landscape of greenfield applications and is far more commonly associated with legacy applications. Hearing her vantage point on this as an evangelist was eye opening. I gained more ideas, thoughts, and was pushed to really get that question answered for students in a different way… which I’ll add to sometime in the future in another blog entry.
I spoke to Kathleen while we took a break across the street from the conference at Grendal’s Coffee Shop. We talked a lot about education and what is effective training, diving heavily into what works around video, samples, and related things. You see, we’re both authors at Pluralsight too and spend a lot of time thinking about these things. It was great to be able to sit down and really discuss these topics face to face.
We also dived into a discussion about city livability and how Portland’s transit system works, what is and isn’t working in the city and what it’s like to live here. I was, of course, more than happy to provide as much information as I could.
We also discussed her interest in taking legacy shops (i.e. pre-C# even, maybe Delphi or whatever might exist) and helping them modernize their shop. I found this interesting, as it could be a lot of fun figuring out large gaps in technology like that and helping a company to step forward into the future.
Kathleen gave two presentations at the conference – excellent presentations. One was the “Your Code, Your Brain” presentation, talking about exactly the topic of legacy shops moving forward without disruption.
If you’re interested in Kathleen’s courses, give a look here.
Amy had a wicked great slides and samples that were probably the most flawless I’ve seen in a while. Matter of fact, a short while after the conference Amy put together a blog entry about those great slides and samples “Super Smooth Technical Demoes“.
An intent and listening audience.
An intent and listening audience.Amy’s talked at the conference was titled “Space, Time, and State“. It almost sounds like we could just turn that into an acronym. The talk was great, touched on the aspects of reactiveness and the battle of state that we developers fight every day while building solutions.
We also got to talk a little after the presentation, the horror of times zones, and a slew of good conversation.
AAAAAaggghhhhhh! I missed half of Tomasz’s talk! It always happens at every conference right! You get to talking to people, excited about this topic or that topic and BOOM, you’ve missed half of a talk that you fully intended to attend. But hey, the good part is I still got to see half the talk!
If you’re not familiar with Tomasz’ work and you do anything with Node.js you should pay close attention. Tomasz has been largely responsible for the great work behind Edge.js and influencing the effort to get Node.js running (and running damn well might I add) on Windows. For more on Edge.js check out Act I and Act II and the Github repository.
The Big Hit for Me, Distributed Systems
First some context. About 4 years ago I left the .NET Community almost entirely. Even though I was still doing a little work with C# I primarily switched stacks to other things to push forward with Riak, distributed systems usage, devops deployment of client apps, and a whole host of other things. At the time I basically had gotten real burned out on where the .NET Community had ended up worldwide, while some pushed onward with the technologies I loved to work with, I was tired of waiting and dived into some esoteric stuff and learned strange programming techniques in JavaScript, Ruby, Erlang and dived deeper into distributed technologies for use in application construction.
However some in the community didn’t stop moving the ball forward, and at this conference I got a great view into some of that progress! I’m stoked to see this technology and where it is now, because there is a LOT of potential for a number of things. Here’s the two talks and two more great people I got to see speak. One I knew already (great to see you again and hang out Aaron!) and one I had the privilege & honor to meet (it was most excellent hanging out and seeing your presentation Lena).
Aaron I’d met back when Troy & I put together the first Node PDX. Aaron had swung into Portland to present on “Building Node.js Applications on Windows Azure“. At .NET Fringe however Aaron was diving into a topic that was super exciting to me. The first line of the description from the topic really says it all “Distributed computing in .NET isn’t something you often hear about, but it’s becoming an increasingly important area for growing .NET businesses around the globe. And frankly it’s an area where .NET has lagged behind other runtimes and platforms for years – but this is changing!“. Yup, that’s my exact pain point, it’s awesome to know Aaron & Petabridge are kicking ass in this space now.
Aaron’s presentation was solid, as to be expected. We also had some good conversations after and before the presentation about the state of distributed compute and systems within the Microsoft and Windows ecosystem. To check out more about Akka .NET that Aaron & Andrew Skotzko … follow @AkkaDotNet, @aaronontheweb, @petabridge, and @askotzko.
…Lena traveled all the way from Kiev in the Ukraine to provide the .NET Fringe crowd with some serious F# distributed and parallel compute knowledge in “Embracing the Cloud“! (Slides here)
Here’s a short dive into F# here if you’re unfamiliar, which you can install on OS-X, Windows or whatever. So don’t use the “well, I don’t use windows” excuse to not give it a try! Here’s info about MBrace that Lena also used in her demo. Also dive into brisk from elastacloud…
In addition to the excellent talk that Lena gave I also got to hang out with her, Phil Haack, Ryan Riley, and others over food at Biwa on the last day of the conference. After speaking with Lena about the Ukraine, computing, coding and other topics around hacking and the OSS Community she really inspired me to take a dive into these tools for some of the work that I’m working on now and what I’ll be doing in the near future.
All The Things
Now of course, there were a ton of other people I got to meet, people I got to catch up with I haven’t seen in ages and others I didn’t get to write about. It was a really great conference with great content. I’m looking forward to round 2 and spending more time with everybody in the future!
The whole bunch of us at the end of the conference!
Cheers everybody! \m/
An Aside of Blog Entries on .NET Fringe
Here are some additional blog entries that others wrote about the event. In addition to these blog entries I’ll be updating this entry with any additional entries that I see pop up – so if you post one let me know, and I’ll also update these talks above that I’ve discussed with videos when they’re posted live.
NOTE: If you just want to check out the code bits, scroll down to the sub-title #symphonize #hacking. Also important to note I’m putting the library through a fairly big refactor at the moment so that everything aligns with the documentation that I’ve recently created. So many things may not be implemented, but we’re moving toward v0.1.0, which will be a functional implementation of the library available via npm based entirely on the documentation and specs that I outline after the history.
There are two main reasons why I chose Orchestrate.io and a data generation library as the two things I wanted to combine. The first, is I knew the orchestrate.io team and really dug what they were building. I wanted to work with it and check out how well it would work for my use cases in the future. The ability to go sit down, discuss with them what they were building was great (which I interviewed Matt Heitzenroder @roder that you can watch Orchestrate.io, Stop Dealing With the Database Infrastructure!) The second reason is that my own startup that I’m co-founding with Aaron Gray (@agray) needed to use key value and graph data storage of some type, somewhere. Orchestrate.io looked like a perfect fit. After some research, giving it a go, it fit very well into what we are building.
December then rolled into the standard holiday doldrums and slowdowns. So fast forward to January post a few rounds of beer and good tidings and I got the 3rd in the series published titled Getting Serious With Symphony.js – JavaScript TDD/BDD Coding Practices (3/3). The post doesn’t speak too much to symphony.js usage but instead my efforts to use TDD or BDD practices in trying to write the library.
Slowly I made progress in building the library and finally it’s in a mostly releasable state now. I use this library daily in working with the code base for Deconstructed and imagine I’ll use it ongoing for many other projects. I hope others might be able to find uses for it too and maybe even add capabilities or ideas. Just ping me via Twitter @adron or Github @adron, add an issue on Github and I’ll be happy to accept pull requests for new features, code refactoring, add you to the project or whatever else you’re interested in.
#symphonize #hacking
Now for the nitty gritty. If you’re up for using or contributing to the project check out the symphonize.js github pages site first. It’s got all the information to help get you kick started. However, you can keep reading as I’ve included much of the information there along with the examples from the README.md below.
NOTE: As I mentioned at the top of this blog entry, the funcitonal implementation of code isn’t available via npm just yet, myself and some others are ripping through a good refactor to align the implementation fo the library with the rewritten and newly available documentation – included blow and at the github pages.
[sourcecode language=”javascript”]
git clone git@github.com:YourUserName/symphonize.git
cd symphonize
npm install
[/sourcecode]
Using The Library
The intended usage is to invocate the JavaScript object and then call generate. That’s it, a super simple process. The code would look like this:
[sourcecode language=”javascript”]var Symphonize = require(‘../bin/symphonize’);
var symphonize = new Symphonize();
[/sourcecode]
The basic constructor invocation like this utilizes the generate.json file to generate data from. To inject the json configuration programmatically just inject the json configuration information via the constructor.
[sourcecode language=”javascript”]
var configJson = {"schema":"keyvalue"};
var Symphonize = require(‘../bin/symphonize’);
var symphonize = new Symphonize();
[/sourcecode]
Once the Symphonize data generator has been created call the generate() method as shown.
That’s basically it. But you say, it’s supposed to do X, Y or Z. Well that’s where the json configuration data comes into play. In the configuration data you can set the data fields and what they’ll generate, what type of data will be generated, the specific schema, how many records to create and more.
generate.json
The library comes with the generate.json file already setup with a working example. Currently the generation file looks like this:
[sourcecode language=”javascript”]
{
"schema": "keyvalue", /* keyvalue, graph, event, geo */
"count": 20, /* X values to generate. */
"write_source": "console", /* console, orchestrateio and whatever other data sources that might come up. */
"fields": {
/* generates a random name. */
"fieldName": "name",
/* generates a random dice roll of a d20. */
"fieldTwo": "d20",
/* A single lorum ipsum random statement is genereated. */
"fieldSentence": "sentence",
/* A random guid is generated. */
"fieldGuid": "guid" }
}
[/sourcecode]
Configuration File Definitions
Each of the configuration options that are available have a default in the configuration file. The default is listed in italics with each definition of the configuration option listed below.
“schema” : This is used to select what type of data structure type is going to be generated. The default iskeyvalue for this option.
“count” : This provides the total records that are to be generated by the library. The default is 1 for this option.
“write_source” : This provides the location to output the generated data to. The default is console for this option.
“fields” : This is a JSON field within the JSON configuration file that provides configuration options around the fields, number of fields and their respective data to generate. The default is one field, with a default data type of guid. Each of the respective entries in this JSON option is a self contained JSON name and value pair. This then looks simply like this (which is also shown above in part):[sourcecode language=”javascript”]{
"someBoolean": "boolean",
"someChar": "character",
"aFloat": "float",
"GetAnInt": "integer",
"fieldTwo": "d20",
"diceRollD10": "d10",
"_string": {
"fieldName": "NameOfFieldForString",
"length": 5,
"pool": "abcdefgh"
},
"_sentence": {
"fieldName": "NameOfFiledOfSentences",
"sentence": "5"
},
"fieldGuid": "guid"
}
[/sourcecode]
Fields Configuration: For each of the fields you can either set the field to a particular data type or leave it empty. If the field name and value pair is left empty then the field defaults to guid. The types of data to generate for fields are listed below. These listed are all simple field and data generation types. More complex nested generation types are listed below under Complex Field Configuration below the simple section.
“boolean“: This generates a boolean value of true or false.
“character“: This generates a single character, such as ‘1’, ‘g’ or ‘N’.
“float“: This generates a float value, similar to something like -211920142886.5024.
“integer“: This generates an integer value, similar to something like 1, 14 or 24032.
“d4“: This generates a random integer value based on a dice roll of one four sided dice. The integer range being 1-10.
“d6“: This generates a random integer value based on a dice roll of one six sided dice. The integer range being 1-10.
“d8“: This generates a random integer value based on a dice roll of one eight sided dice. The integer range being 1-10.
“d10“: This generates a random integer value based on a dice roll of one ten sided dice. The integer range being 1-10.
“d12“: This generates a random integer value based on a dice roll of one twelve sided dice. The integer range being 1-10.
“d20“: This generates a random integer value based on a dice roll of one twenty sided dice. The integer range being 1-20.
“d30“: This generates a random integer value based on a dice roll of one thirty sided dice. The integer range being 1-10.
“d100“: This generates a random integer value based on a dice roll of one hundred sided dice. The integer range being 1-10.
“guid“: This generates a random globally unique identifier. This value would be similar to ‘F0D8368D-85E2-54FB-73C4-2D60374295E3’, ‘e0aa6c0d-0af3-485d-b31a-21db00922517’ or ‘1627f683-efeb-4db8-8174-a5f2e3378c87’.
Complex Field Configuration: Some fields require more complex configuration for data generation, simply because the data needs some baseline of what the range or length of the values need to be. The following list details each of these. It is also important to note that these complex field configurations do not have defaults, each value must be set in the JSON configuration or an error will be thrown detailing that a complex field type wasn’t designated. Each of these complex field types is a JSON name and value parameter. The name is the passed in data type with a preceding underscore ‘_’ to generate with the value having the configuration parameters for that particular data type.
“_string“: This generates string data based on a length and pool parameters. Required fields for this include fieldName, length and pool. The JSON would look like this:[sourcecode language=”javascript”]"_string": {
"fieldName": "NameOfFieldForString",
"length": 5,
"pool": "abcdefgh"
}
[/sourcecode]
Samples of the result would look like this for the field; ‘abdef’, ‘hgcde’ or ‘ahdfg’.
“_hash“: This generates a hash based on the length and upper parameters. Required fields for this included fieldName, length and upper. The JSON would look like this:[sourcecode language=”javascript”]"_hash": {
"fieldName": "HashFieldName",
"length": 25,
"casing": ‘upper’
}
[/sourcecode]
Samples of the result would look like this for the field: ‘e5162f27da96ed8e1ae51def1ba643b91d2581d8’ or ‘3F2EB3FB85D88984C1EC4F46A3DBE740B5E0E56E’.
“_name”: This generates a name based on the middle, *middleinitial* and prefix parameters. Required fields for this included fieldName, middle, middle_initial and prefix. The JSON would look like this:[sourcecode language=”javascript”]"_name": {
"fieldName": "nameFieldName",
"middle": true,
"middle_initial": true,
"prefix": true
}
[/sourcecode]
Samples of the result would look like this for the field: ‘Dafi Vatemi’, ‘Nelgatwu Powuku Heup’, ‘Ezme I Iza’, ‘Doctor Suosat Am’, ‘Mrs. Suosat Am’ or ‘Mr. Suosat Am’.
So that covers the kick start of how eventually you’ll be able to setup, use and generate data. Until then, jump into the project and give us a hand.
Several people got together yesterday to start planning things for 2014 in PDX. It ranged from coding workshops to PDX Node to Node PDX to what kind of food to eat at for lunch. Ya know, daily tactical things that come along with the big picture items. 😉
bridge.js badge.
Two things that I want to bring up to the community out there. One is a workshop that I’ll likely lead efforts to organize and the other is something I’ll just call pdxbridge.js for now. The workshop will cover the topics of which and what databases to use for what data and how to implement. The pdxbridge.js project is about determining the raised or lowered state of the bridges here in Portland.
Some of the other projects, workshops and other topics we discussed included getting a workshop put together around unit, integration and testing code from a behavioral, test driven development or other approaches. This workshop we don’t have anyone to teach, but we’d (ok, so I really really would love to attend a workshop on this) really like to find somebody who would be willing to teach a workshop of this sort, with a focus on Javascript as the language. On that same topic however, if you’re into Java, Erlang, Scala, Haskell or others and would like to teach a TDD, BDD or related testing workshop please get in touch with me. We will work on making that happen! Ping me at adron at composite code dot com. 😉
This is a course I’ll lead and others will work with me on to put something extra useful together. We will then teach the workshop as a group, kind of a team paired programming teaching workshop. If there is anything in particular that you’d like to learn about, any questions that you have about data and usage in applications or otherwise add your two cents on this blog entries comments. Over the next month we’ll be putting together the material and have the course available sometime early this year. So if you’d like to attend, jump in at any time with the conversation or just keep a read here and I’ll have more information about the course as we get it put together.
Let’s Make pdxbridge.js Happen!
The pdxbridge.js project is all about determining if a bridge in Portland is up or down. Right now there are several bridges that matter, that are on this list;
If we add other information to track about the bridges we might add the other 3 that exist and the new bridge that is being built. however the five listed are the only bridges that have a raised and lowered state, and in one case the Steel Bridge has a lowered, partially raised and fully raised state. As shown on the pdxbridge.js badge I threw together (shown above).
This got me thinking about a number of things and I started to write her an email specifically, but realized I should really just blog it. After all, the topic is actually part of what should be the public conversation. It’s about the changing world of technology, which we’re all part of…
First Topic: Usage of PaaS
Barb, just shortly after the tweet above was posted, this other tweet altered what information I might provide her. @TheSteve0 had responded with some items, which @GigaBarb then responded with
@TheSteve0 those names are the same … looking for more…thanks tho
Now, not to pick on OpenShift & Red Hat (the effort @TheSteve0 is working with), because they have a great open source effort going on around this PaaS Technology, but Barb had a point. If Cloud Foundry responded with something like this, she’d still have a point. The only companies that continually sign up new companies is AWS & Beanstalk (ok, so they don’t call it PaaS, it gets you to the same place – arguably better than most of the others), a little bit at Windows Azure and a few companies pop up every once in a long while that might take Cloud Foundry or OpenShift and run with it. Most of the early adopters are already on board and most that might get on board are still mostly just waiting in the sidelines.
This fact is frustrating for those
in the space that want to see more penetration, but for those that arent’ technically in the space, it seems kind of like ASP. Oh wait, I should add context now, ASPs as in Application Service Providers. The technology from the beginning of the 21st century similar in many ways to what is dubbed SaaS now. At the time it could have been revolutionary. However at the time nobody picked it up either. This is similar to what PaaS is seeing. However…
A Hypothesis of What Will Happen to PaaS Tech
I have a theory of what will happen to PaaS Tech, it is similar to ASP Tech. PaaS will keep plundering along in odd ways, and eventually one day, it will become a mainstream tech. Right now however it will remain limited. In that same turn, by the time it becomes a common tech, it’ll be called something else.
Here’s a few reasons. One, is that many developers see PaaS and their response, especially if they’re seasoned developers with more than a few years under their belt, is to respond will immediate apprehension to the tech. It removes key elements of what they want to control. It hides things they can’t actually get to and it abstracts in ways that don’t always make sense. The result is that many senior devs stay away from pure PaaS offerings and instead use it only for prototyping, but production gets something totally different. I’ve been there more than a few times myself.
However, the result of what most senior devs end up with, when they get their continuous integration and development environments running at full tilt, is exactly what PaaS is attempting to promise. There are some companies, with senior devs, and extremely intelligent members that have taken PaaS and effectively implemented it into their continuous integration and delivery environment giving them strengths that most companies can only imagine to have.
One of those companies is lucky and smart enough to have Jonathan Murray @adamalthus heading up efforts. On his team he also has Dave McCrory @mccrory and Brian McClain @brianmmclain. To boot, they are close to the Cloud Foundry team (and @wattersjames, who cuts a path when there are issues) and keep a solid effort going working with key partners such as @Tier3 (now part of CenturyLink) and other companies that help bring together one of the most strategically and tactically relevant PaaS deployments to date.
Other PaaS deployments are questionable for various reasons, they’re trying, but they aren’t there. At least not the types of companies and efforts that Barb was looking for. So really, if there is another out there that’s hiding, but wants serious street cred. A boost to hiring serious A grade talent, and to push forward past competitors, please let us know. Let me know, let Barb know and let’s hear about what you’re doing. If a company is hiding their implementation and doesn’t want to be part of the community, then fine, they can stay hidden and not gain the benefit of the community that presses forward beyond them. But I would love to hear from those that I might have missed, that want to push forward, so ping me. Ping Barb, we’ll get word out there and get developers checking out and making sure your company is getting it done! 😉
Second Topic: PaaS on PaaS and Start Docker
PaaS is nice. If your company can get it deployed and use it effectively, the you’re going to push forward fast in many regards. Deployments, savings, code cleanliness, effective separation of concerns and abstraction at a systems level are some of the things you can expect from a good PaaS implementation. Sometimes however, as the senior devs I mentioned pointed out, you give up control and certain levels of abstraction. However almost all senior devs understand that they want the ability to abstract at the levels that PaaS enables. They want to break apart the app cleanly at the system level from the software level. No reason for an app to know where or what a hard drive is doing right? That’s a rhetorical question, onward with the topic…
Docker has entered the market with a BOOM, part of the abstraction level that enables PaaS tooling in the first place. This tool enables a team to jump into the code or to just deploy the tool to abstract at a PaaS level, but to build the elements that they need specifically. The components are able to be brought together in a composite way that provides all the advantages of PaaS, while put together specifically for the problem space that the team is attacking. For environments that don’t make cookie cutter apps that fit perfectly to PaaS tooling as it is, that needs that little bit extra control of the environment, Docker is the perfect tool to bring those pieces together.
So really, is Docker and containerization that new word (from a technically old tech! lolz), that new tech, that’s going to bring PaaS into the mainstream as the standard implementation? Is it going to make PaaS become containerization when we developers talk about it? It could very well be the next big step. It could be that last mile coverage that devs want to push environments into a PaaS Tech ecosystem and make full use of hardware, software and move to the next stage of application development. Could it? Will it?
Personally I’m ready for the next stage of the whole PaaS thing, are you?
Next up on other thought patterns, WTF are people using Oracle for still when mariadb and postgres mean their freedom to innovate, move forward and surpass their competition.
You must be logged in to post a comment.