A Shiny New Vuejs v3 Web App Using & Deployed to Amplify/AppSync/Cognito

No cruft, let’s just start.

Prerequisites

These details plus yarn and a few other notes are available and derived from the Amplify Docs located here. What I’ve done is take those docs and add some specific details and information for this happy path. It includes additional references for the steps I took, and specifically what I’m running this on for this particular tutorial. As noted below, there is a section where this deviates from those steps and I get into next steps beyond the initial setup of the app, Amplify, and AppSync. I’ll note that part of this tutorial, or you can navigate directly to that part with this anchor thatPartiDeviate.

You’ll need the following for this specific tutorial. If you’re acclimate to various OSes and their respective needs around this software, you can get this sorted yourself and it’s mostly the same for each OS, but for this tutorial I’m rolling with MacOS Big Sur v11.6.2.

  • Your OS, as stated mine is Big Sur for this tutorial.
  • git. Probably any version released in the last decade will work just fine.
  • Node.js. Probably anything since v14 would work great but this tutorial is written against v16.11.1. As of this writing the LTS is 16.13.1 and current is 17.3.0.
  • Vue.js v3. For this tutorial I’m on a version of the v3 Preview. For the CLI a quick yarn global add @vue/cli does the job.
  • Amplify CLI. Version for this tutorial is 7.6.5. One can NPM install it with šŸ¤™šŸ» npm install -g @aws-amplify/cli or get it via cURL šŸ‘šŸ» curl -sL https://aws-amplify.github.io/amplify-cli/install | bash && $SHELL and of course, Windows has gotta be Windowsy with šŸ˜‘ curl -sL https://aws-amplify.github.io/amplify-cli/install-win -o install.cmd && install.cmd.

A few first steps that only need done once. If you’ve already setup your amplify cli then this isn’t needed a second time.

First, get the Vue.js v3 base app skeleton running.

vue create mywhateverproject

Issuing this command will then provide prompts to select Vue.js v3 Preview (or likely just v3 when fully released, which will come along with other tooling as needed). Once this is done, following the standard steps of navigating into the directory cd myWhateverProejct , then executing the yarn command and finally yarn serve --open will open up the running web app in your default browser.

Next initialize the Vue.js App as an Amplify Project and get some defaults set and accepted. Executing amplify init and accepting the defaults will get this done. As displayed when done the Vue.js v3 App will now have multiple defaults and respective items selected.

Amplify Init

With the core Amplify folder and settings set, adding the Amplify libraries for use in user interface components is next up.

yarn add aws-amplify @aws-amplify/ui-components

Now navigate into the src/main.js file and add the Amplify and initial configure in the code, that will do the actual initialization execution when the app launches.

This is replacing this code…

import { createApp } from 'vue'
import App from './App.vue'

createApp(App).mount('#app')

with this code.

import { createApp } from 'vue'
import App from './App.vue'
import Amplify from 'aws-amplify';
import aws_exports from './aws-exports';
import {
	applyPolyfills,
	defineCustomElements,
} from '@aws-amplify/ui-components/loader';

Amplify.configure(aws_exports);
applyPolyfills().then(() => {
	defineCustomElements(window);
});
createApp(App).mount('#app')

This completes the steps we need for a running application. To cover full stack let’s cover the back end build out and schema construction. Then after that I’ll delve into thatPartiDeviate. Next up is getting the Amplify elements added.

npm install aws-amplify @aws-amplify/ui-components

Before even launching I went ahead and added the back end and database, GraphQL API, and related collateral.

amplify add api
amplify add api

Notice in the screenshot, once I selected to edit the schema now, it simply opened the file in the editor of my choice, which is Visual Studio Code for this tutorial. Since I’m executing this from the terminal in Visual Studio Code it simply opened the file in the active editor that I’m in, win win! The file that is opened by default for the schema includes the following GraphQL schema code.

# This "input" configures a global authorization rule to enable public access to
# all models in this schema. Learn more about authorization rules here: https://docs.amplify.aws/cli/graphql/authorization-rules

input AMPLIFY { globalAuthRule: AuthRule = { allow: public } } # FOR TESTING ONLY!

type Todo @model {
	id: ID!
	name: String!
	description: String
}

For now, I’ll just leave the comment, the input AMPLIFY and the Todo type just as it is. It’s important to note that this schema.graphql file is located at app/amplify/backend/schema.graphql. I’ll come back to this later in thatPartiDeviate.

Next I want to push the app, api, and backend to Amplify and AppSync.

amplify push

During this phase a lot of things happen. The GraphQL Schema is turned into an API and this is deployed along with the Database is deployed to DynamoDB.

To get the backend shipped, i.e. deployed to AppSync, issue the amplify push command. Again, following through with the default choices. If amplify console is issued just after this a review of the API can be made.

amplify push

Ok, now it’s auth time. Adding that is somewhat mind boggling minimal, just amplify add auth. For this I chose Default config, then Username for the way users sign in, and then the No, I am done option followed by issuing another amplify push, confirmed that and let it go through its process.

After this next steps included adding the following code to the App.vue file to get the initial interations, security and related things into place for the todo app. Again, I feel it important to note that I’ll be changing all of this later in the post here. But it’s a solid way to start building an application and then looping back around after it is up and running, deployed before moving on to next steps.

<template>
  <amplify-authenticator>
    <div id="app">
      <h1>Todo App</h1>
      <input type="text" v-model="name" placeholder="Todo name">
      <input type="text" v-model="description" placeholder="Todo description">
      <button v-on:click="createTodo">Create Todo</button>
      <div v-for="item in todos" :key="item.id">
        <h3>{{ item.name }}</h3>
        <p>{{ item.description }}</p>
      </div>
    </div>
    <amplify-sign-out></amplify-sign-out>
  </amplify-authenticator>
</template>

<script>
import { API } from 'aws-amplify';
import { createTodo } from './graphql/mutations';
import { listTodos } from './graphql/queries';
import { onCreateTodo } from './graphql/subscriptions';

export default {
  name: 'App',
  async created() {
    this.getTodos();
    this.subscribe();
  },
  data() {
    return {
      name: '',
      description: '',
      todos: []
    }
  },
  methods: {
    async createTodo() {
      const { name, description } = this;
      if (!name || !description) return;
      const todo = { name, description };
      this.todos = [...this.todos, todo];
      await API.graphql({
        query: createTodo,
        variables: {input: todo},
      });
      this.name = '';
      this.description = '';
    },
    async getTodos() {
      const todos = await API.graphql({
        query: listTodos
      });
      this.todos = todos.data.listTodos.items;
    },
    subscribe() {
      API.graphql({ query: onCreateTodo })
        .subscribe({
          next: (eventData) => {
            let todo = eventData.value.data.onCreateTodo;
            if (this.todos.some(item => item.name === todo.name)) return; // remove duplications
            this.todos = [...this.todos, todo];
          }
        });
    }
  }
}
</script>

With this added now I could run yarn serve and check out the site. At this point I signed up just to have an account to use and added a todo item. Everything worked swimmingly at this point!

The final step before getting into a proper deviation from this todo example involves now getting the app properly published to Amplify. This is done by executing amplify add hosting. Accept Hosting with Amplify Console (Managed hosting with custom domains, Continuous deployment) and Manual deployment when prompted. Now, finally, issue the command amplify publish.

Boom, the todo app site is live!

thatPartWhereiDeviate

Now it’s time to get into the nitty gritty of deviations from the easy path!

New GraphQL Schema!

My schema that I want to add is around building out collections for a number of data sets. The first one is a data set that I routinely talk about on a regular basis, and yes, it is indeed centered around trains! If you’re uninterested in the trains part and schema and more interested in the changes skip down to the "Deploying The Changes" section of the post.

Alright, describing the data model that I want to have and use will start with the minimal part of just having a list of railroads. This would be a list, or more specifically a table of railraods, that we can add railroads to and collect peripheral information about them. For this table I’ll add the following fields, AKA columns of data to store. I would want to collect the following for a railroad:

  1. railroad name
  2. wikipedia URI
  3. map URI
  4. peripheral details of an unstructured nature
  5. founding year, month, and day of the railroad
  6. record stamp

In addition, I want to keep a list of trains – specifically named trains – that each railroad operates. This data would include:

  1. train name
  2. active – yes / no
  3. peripheral details of an unstructured type
  4. wikipedia URI
  5. route map URI
  6. time table URI
  7. train URI – i.e. like a website or something that might be dedicated to the particular train.
  8. record stamp

Deploying The Changes

Now it is time to deploy these additional database and schema changes. One of the easiest ways to make these changes is to use Amplify Studio, which has a great section for data modeling, which in turn puts together and ensures your schema is usable. Then it will enable you to deploy that new schema with changes to the database and the active service!

Navigate to the interface from here.

Opening Amplify Studio with the Launch Studio button.

Once I navigated to the interface I built out the additional tables like this.

Building a Schema with Amplify Studio

Then just click on Save and Deploy and then Deploy on the following modal dialog and Amplify will deploy the AppSync schema changes.

Amplify Studio Save & Deploy

With that deployed, in the same Amplify Studio interface I then clicked on the GraphQL API tab section and then on the Resource name for mywahteverproject to open up the AppSync Console.

Opening an AppSync Schema.

Further down in the schema toward the bottom I can then find and confirm my types are in and ready for use. The Todo type is still there, since I didn’t need to really remove it yet and it acts as a good working reference. But more importantly you can see the other types and the correlative relationship that was added via the Amplify data modeling interface.

...more schema

type Todo @aws_iam
@aws_api_key {
	id: ID!
	name: String!
	description: String
	_version: Int!
	_deleted: Boolean
	_lastChangedAt: AWSTimestamp!
	createdAt: AWSDateTime!
	updatedAt: AWSDateTime!
}

type Train @aws_iam
@aws_api_key {
	id: ID!
	train_name: String!
	active: Boolean!
	peripheral_details: AWSJSON
	wikipedia_uri: AWSURL
	route_map_uri: AWSURL
	timetable_uri: AWSURL
	train_uri: AWSJSON
	record_stamp: AWSTimestamp
	_version: Int!
	_deleted: Boolean
	_lastChangedAt: AWSTimestamp!
	createdAt: AWSDateTime!
	updatedAt: AWSDateTime!
	railroads(
		railroadID: ModelIDKeyConditionInput,
		filter: ModelRailroadTrainFilterInput,
		sortDirection: ModelSortDirection,
		limit: Int,
		nextToken: String
	): ModelRailroadTrainConnection
		@aws_iam
@aws_api_key
}

type Railroad @aws_iam
@aws_api_key {
	id: ID!
	railroad: String!
	wikipedia_ur: AWSURL
	map_uri: AWSURL
	peripheral_details: AWSJSON
	founding_year: Int
	founding_month: Int
	founding_day: Int
	record_stamp: AWSTimestamp
	_version: Int!
	_deleted: Boolean
	_lastChangedAt: AWSTimestamp!
	createdAt: AWSDateTime!
	updatedAt: AWSDateTime!
	RailroadTrains(
		trainID: ModelIDKeyConditionInput,
		filter: ModelRailroadTrainFilterInput,
		sortDirection: ModelSortDirection,
		limit: Int,
		nextToken: String
	): ModelRailroadTrainConnection
		@aws_iam
@aws_api_key
}

...more schema

The relationship can be seen via the object connections here in the ModelRailroadTrainConnection and the keys associated.

Next steps to get this updated in the local repository from these changes that were just made out of sync via the Amplify Studio interface requires two quick commands, both of which are displayed on the screen of the GraphQL interface in the studio. It’s best to get the command, as it’ll have the appId already included in a copypasta option on the screen, which looks like this.

amplify pull --appId app-id-which-is-in-studio --envName dev

Executing that will get everything updated and pull in the remote GraphQL Schema to the local schema.graphql file located in the amplify/backend/api/ location. Next run this command.

amplify update api

This will update everything to synchronize things, which will also prompt me for code generation so that I can have the client side code ready for use whenever I build out the user interface later.

Next Up

Some of the things I’ll cover in the next article, as I continue this effort, is what has been done with all these steps from a project perspective. As one can see, some things might be a little confusing at this point, for example the above schema shown in AppSync, but after the synchronization if you look at the schema.graphql file locally it shows this.

type Train @model @auth(rules: [{allow: public}]) {
  id: ID!
  train_name: String!
  active: Boolean!
  peripheral_details: AWSJSON
  wikipedia_uri: AWSURL
  route_map_uri: AWSURL
  timetable_uri: AWSURL
  train_uri: AWSJSON
  railroads: [RailroadTrain] @connection(keyName: "byTrain", fields: ["id"])
  record_stamp: AWSTimestamp
}

type Railroad @model @auth(rules: [{allow: public}]) {
  id: ID!
  railroad: String!
  wikipedia_ur: AWSURL
  map_uri: AWSURL
  peripheral_details: AWSJSON
  founding_year: Int
  founding_month: Int
  founding_day: Int
  record_stamp: AWSTimestamp
  RailroadTrains: [RailroadTrain] @connection(keyName: "byRailroad", fields: ["id"])
}

type Todo @model @auth(rules: [{allow: public}]) {
  id: ID!
  name: String!
  description: String
}

type RailroadTrain @model(queries: null) @key(name: "byRailroad", fields: ["railroadID", "trainID"]) @key(name: "byTrain", fields: ["trainID", "railroadID"]) @auth(rules: [{allow: public}]) {
  id: ID!
  railroadID: ID!
  trainID: ID!
  railroad: Railroad! @connection(fields: ["railroadID"])
  train: Train! @connection(fields: ["trainID"])
}

Obviously this is very different than what is shown from one place to another, so I’ll discuss this and other things. So subscribe (over on the right side of the blog), follow (@Adron), and you’ll be updated on the next post when it’s published.

SITREP (Situational Report)

Alright, what have I wrapped up so far? Here’s a bullet list of the things finished:

  • Vue.js App created.
  • Vue.js Form put together for todo entries.
  • Authentication added with Cognito.
  • An AppSync GraphQL created and published.
  • Additional types added to the AppSync GraphQL API.
  • Updates and code regenerated for our API.

What’s next to do? This is the short list, there will be after that, much more to do!

  • Get the Vue.js app spified up, get some nice design put together for it, add some reasonable CSS, graphics, etc to make the interface pop. But above all, it needs to feel usable and be usable.
  • Add the forms for each of the respective interfaces to manipulate the data. This could amount to lots of different things, adding navigation, routing, and other menues and the like.
  • Add screens that can provide some nice reports on the data that I’m putting together. For example, it’d be nice to get a list of the actual named trains or the railroads and have their images, maps, and other respective elements shown.

…and the list goes on. Until next session, enjoy your thrashing code! šŸ¤˜šŸ»

References

Troubleshooting Node.js Deploys on Beanstalk – The Express v4 node ./bin/www Switch Up

I’ve gotten a ton of 502 errors and related issues that crop up when deploying the Beanstalk. One of the issues that cropped up a few times recently, until I stumbled into a working solution was the 502 NGINX error. I went digging around and ended up just trying to deploy a default, fresh from the ‘express newAppNameHere’ creation and still got the error.

I went digging through the Beanstalk configuration for the app and found this little tidbit.

Node Command (Click for full size image)
Node Command (Click for full size image)

I’ve pointed out the section where I’ve added the command.
[sourcecode language=”bash”]
node ./bin/www
[/sourcecode]

Based on the commands that are executed normally, it seems `npm start` would work work to get the application started. But I have surmised the issue is that the commands are executed sequentially;

[sourcecode language=”bash”]
node server.js
node app.js
npm start
[/sourcecode]

When these are executed in order, errors crop up and the command that should work `npm start` begins with a corrupted and error laden beginning. Leaving the application not running. However by adding the `node ./bin/www` to the text box all the others are skipped and this command is issued, resulting in a running application.

The other thing is to follow the now standard approach of just issue `npm start`, but being sure to replace what I put in the text box above (`node ./bin/www`) with `npm start` so that beanstalk only runs npm start instead of the ordered execution.

Mapping Domain Names with name.com, Elastic Beanstalk, Elastic Load Balancer and AWS Route 53

I finally wrapped up my name server and DNS mapping needs with Name.com,Ā Route 53 and Elastic Beanstalk. Since this was a little confusing I thought a short write up was in order. Thanks to Evan @evandbrown for helping out!

The first thing needed is a delegation set of name servers for your DNS and name server provider. These can be found by creating a hosted zone. The way to do this is open up the AWS Management Console and navigate into the Route 53 management area. The Route 53 icon is under the Compute & Networking section on the management console.

Beanstalk, Route 53 - Click for full size image
Beanstalk, Route 53 – Click for full size image

Upon navigating to the Route 53 console area click on the Create Hosted Zones button.

Create Hosted Zone
Create Hosted Zone – Click for full size image

When the zone is created then the delegation set can be found under the Hosted Zone Details. This delegation set now needs setup as the name servers for whoever, in this case name.com, is the domain provider.

Delegation Set - Click for full size image.
Delegation Set – Click for full size image.

Open up the management console for the name server administration.

Upon adding them the list should look something like this.

Name servers list built from the delegation set of the hosted zone. Click for full size image.
Name servers list built from the delegation set of the hosted zone. Click for full size image.

Once the name servers are setup, those will need time to propagate. Likely this could take a good solid chunk of time, somewhere in the hours range likely, and don’t be surprised if it takes a little bit more than a day.

While the propagation starts navigate back to the AWS Management Console and open up the EC2 section of the console. On the right hand side of the Resources list there is a Load Balancers section. Click it.

Load Balancers - Click for full size image.
Load Balancers – Click for full size image.

In this section there is a listing of all load balancers that have been created manually or by Elastic Beanstalk.

Load Balancers - Click for full size image.
Load Balancers – Click for full size image.

Make note of the Load Balancer Name for selection in Route 53. This is what Route 53 needs in order to point an alias at for incoming traffic to that particular Elastic Beanstalk application. In this particular image above there are 4 load balancers listed, the easiest way to prevent confusion is to take note of the load balancer name at the time of creation, but this is the easiest way to find them otherwise.

Record Set - Click for full size image
Record Set – Click for full size image

Now when going back to the hosted zone to set it up with the appropriate information, create a new record with the appropriate name, in this case I was setting up the admin.deconstructed.io (no it isn’t live yet, I just set it up to test it out) to point to an alias target. Just leave the Type set to A – IPv4 address and click the radio control so that Alias is set to Yes. In the alias target select the appropriate load balancer for the Elastic Beanstalk (or whatever it points to) application.

That’s it, give it a few hours (or a day) and eventually the domain or subdomain will be pointed appropriately at the Elastic Beanstalk load balanced application.

Backup Riak – Learning About Distributed Databases :: Issue 001

I’ve got more than a few series in the queue, so why not another one eh! The intent is, I’ll grab a specific topic to break down and add details to related to distributed systems, primarily around Riak. I will however diverge into other distributed databases too, but I’ll primarily be sticking to Riak. Without more introduction, the first topic is…

Backing Up and Recovery of Riak (Nodes)

I’ve been asked approximately 423,983,321.7 zillion times how this is done. So here’s a quick summary and respective links to the best ways to backup Riak, how to recover nodes.

When backing up Riak there are two key things that need copied to the backup storage; the ring and data directories. Each of these things are specific based on the backend used with Riak. In addition to the core backup containing the ring and data, another good thing to backup is the configuration directory. When recovering this comes in useful.

For the locations of data, it depends slightly based on the operating system being used. The two big variances are OS-X and Linux Distros. On OS-X the data path, ring data and configuration are located at the locations listed below:

  • Bitcask data: ./data/bitcask
  • LevelDB data: ./data/leveldb
  • Ring data: ./data/riak/ring
  • Configuration: ./etc

For each specific distro, there are slight variations on where the locations are, for a full list check out the Basho Riak docs on backups. But on Linux distros the paths are as follows:

Debian and Ubuntu

  • Bitcask data: /var/lib/riak/bitcask
  • LevelDB data: /var/lib/riak/leveldb
  • Ring data: /var/lib/riak/ring
  • Configuration: /etc/riak

Fedora and RHEL

  • Bitcask data: /var/lib/riak/bitcask
  • LevelDB data: /var/lib/riak/leveldb
  • Ring data: /var/lib/riak/ring
  • Configuration: /etc/riak

Other Operating System Paths

Freebsd

  • Bitcask data: /var/db/riak/bitcask
  • LevelDB data: /var/db/riak/leveldb
  • Ring data: /var/db/riak/ring
  • Configuration: /usr/local/etc/riak

SmartOS

  • Bitcask data: /var/db/riak/bitcask
  • LevelDB data: /var/db/riak/leveldb
  • Ring data: /var/db/riak/ring
  • Configuration: /opt/local/etc/riak

Solaris

  • Bitcask data: /opt/riak/data/bitcask
  • LevelDB data: /opt/riak/data/leveldb
  • Ring data: /opt/riak/ring
  • Configuration: /opt/riak/etc

When backing things up, it’s important to note that each node could have slightly inconsistent data. The data however is rebuilt by the Riak read-repair system once it is recovered and brought into use.

Backup Jobs

One of the easiest ways to backup Riak is to setup a cron job with your choice of cp, rsync or tar. Then just get those files onto whatever your choice of backup medium. An example tar cron job to backup a Bitcask backend is shown below (snagged from theĀ documentation) just to give you an idea of where to start.

[sourcecode language=”bash”]tar -czf /mnt/riak_backups/riak_data_`date +%Y%m%d_%H%M`.tar.gz /var/lib/riak/bitcask /var/lib/riak/ring /etc/riak
[/sourcecode]

For a leveldb back end the most important thing to note is that the node must be stopped. The basic workflow of backing up a node in this manner is to stop the node, backup the data, ring and configuration and then start the node back up.

Backup Recovery / Restoring

When recovering data on a node that is replacing an existing node that has the same name (fully qualified or IP) then follow the steps below:

  1. Install Riak
  2. Restore the old node’s configuration, data & ring.
  3. Start the node

Once you’ve got the node started back up it’s a good idea to do a ping or status against the node to verify it is in a good state.

If node names have been changed there are additional steps.

  1. Mark the original instance down[sourcecode language=”bash”]riak-admin down [/sourcecode]
  2. Join the restored clusterĀ Ā [sourcecode language=”bash”]riak-admin join [/sourcecode]
  3. Replace the original withĀ [sourcecode language=”bash”]riak-admin cluster force-replcaeĀ Ā [/sourcecode]
  4. Get the cluster plan builtĀ [sourcecode language=”bash”]riak-admin cluster plan[/sourcecode]
  5. Commit the changesĀ [sourcecode language=”bash”]riak-admin cluster commit[/sourcecode]
  6. Change the -name setting in the vm.args configuration file to match the new name.
  7. Change & verify that the IP reflects the instances IP in the app.config for http and protocol buffer interfaces.

Cluster Backups via Riak Enterprise Multi-Data Center (MDC)

In the above sections I wrote about the traditional backupĀ approaches. This is very similar to the way RDBMS are backed up. However, with a distributed system like Riak there is another great alternative if you’re utilizing multiple datacenters and Enterprise Riak. In this version of Riak, which is basically Riak with additional features and capabilities, one of the possible backup scenarios is to use the Multi-Data Center, or MDC, to replicate a duplicate cluster and use it as an active, real-time and always ready backup.

One workflow that is an exceptionally effective way to provide backups is to setup the “backup” cluster beside the current operative cluster. As an example, if your cluster is operational in AWS and it is running in X region and Y zone then you’d want to put the backup cluster in that same region and zone. Once you’ve setup Riak Enterprise and MDC, then just setup a full sync. Once the full sync is done you can then remove the backup cluster and it provides a point in time backup of the data.

[sourcecode language=”bash”]riak-repl start-fullsync[/sourcecode]

It’s easy to schedule full sync operations to low usage periods and it is also possible to pause and resume full sync operations.

[sourcecode language=”bash”]riak-repl resume-fullsync<br />riak-repl pause-fullsync[/sourcecode]

The variations on backing up data with Riak Enterprise and MDC are pretty expansive. Doing a point in time, maintaining a secondary live copy of the data, using the replication as a data dump to another cluster or even just using the MDC replication to dump all of the data to a single instance.

File System Snapshots

One other technique that is extremely efficient, fast and thorough is snapshotting the file system. The backup workflow for snapshots is extremely easy. First stop Riak, then snapshot, then start Riak again. Of all the methods, snapshotting is one of the easiest of the options. Just like setting up a cron job, automating snapshots based on some pre-defined schedule and meshing that with automated start and stop of Riak provides a very thorough backup.

With these options, have fun strategizing yourĀ stratagemsĀ into strategies for backups.

Diskettes

One of the oldest, tried and true backups is the old diskette. The bestest way to backup with diskettes is to backup each node on three diskettes each. The send one of each diskettes to a geographically dispersed to a bank lock box or other secure facility. Do this for each node, and if need be use as many diskettes for each node as needed. A particularly useful method is to use the sharded zip strategy to stripe a backup across many diskettes. Once each lock box has a copy of the node for each node in the cluster, you’ll have one of the most secure backups in existence. Nothing compares to the diskette backup!

References:

  1. Basho Docs – Backups
  2. Basho Docs – MDC Full Sync

Red Hat, OpenShift PaaS and Cartridges for Riak

Today I participated in the OpenShift Community Day here in Portland at the Doubletree Hotel. One of the things I wanted to research was the possibility of putting together a OpenShift Origin Cartridge for Riak. As with most PaaS Systems this isn’t the most straight forward process. The reason is simple, OpenShift and CloudFoundry have a deployment model based around certain conventions that don’t fit with the multi-node deployment of a distributed database. But there are ways around this and my intent was to create or come up with a plan for a Cartridge to commit these work-arounds.

After reading the “New OpenShift Cartridge Format – Part 1” by Mike McGrath @Michael_Mcgrath I set out to get a Red Hat Enterprise Linux image up and running. The quickest route to that was to spool up an AWS EC2 instance. 30 seconds later I had exactly that up and running. The next goal was to get Riak installed and running on this instance. I wasn’t going to actually build a cluster right off, but I wanted at least a single running Riak node to use for trying this out.

In the article “New OpenShift Cartridge Format – Part 1” Mike skips the specifics of the cartridge and focuses on getting a service up and running that will be turned into a Cartridge. As Mike writes,

What do we really need to do to create an new cartridge? Step one is to pick something to create a cartridge for.

…to which my answer is, “alright, creating a Cartridge for Riak!” Ā šŸ˜‰

However, even though I have the RHEL instance up and running already, with Riak installed, I decided I’d follow along with his exactly example too. So I dove in with

[sourcecode language=”bash”]
sudo yum install httpd
[/sourcecode]

to install Apache. With that done I now have Riak & Apache installed on the RHEL EC2 instance. The goal with both of these services is to get them running as the regular local Unix user in a home directory.

With both Riak and Apache installed, time to create a local user directory for each of the respective tools. However, before that, with this version of Linux on AWS we’ll need to create a local user account.

[sourcecode language=”bash”]
useradd -c "Adron Hall" adron
passwd adron

Changing password for user adron.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[/sourcecode]

Next I switched to the user I created ‘su adron’ and created the following directories in the home path for attempting to get Apache and Riak up and running locally like this. I reviewed the rest of the steps in making the Cartridge w/ Apache and then immediately started running into a few issues with getting Riak setup just like I need it to be able to build a cartridge around it. At least, with my first idea of how I should build aĀ cartridge.

At this point I decided we need to have a conversation around the strategy here. So Bill Decoste, Ryan and some of the other Red Hat team on hand today. After a discussion with Bill it sounds like there are some possibilities to get Riak running via the OpenShift Origin Cartridges.

The Strategy

The plan now is to get a cartridge setup so that the cartridge can launch a single Riak instance. That instance, then with post-launch scripts can then join itself to the overall Riak cluster. The routing can be done via the internal routing and some other capabilities that are inherent to what is in OpenShift itself. It sounds like it’ll take a little more tweaking, but the possibility is there for the near future.

At this point I sat down and read up on the Cartridge a little more before taking off for the day. Overall a good start and interesting to get an overview of the latest around OpenShift.

Thanks to the Red Hat Team, have a great time at the OpenStack Conference and I’ll be hacking on thisĀ CartridgeĀ strategy!

References