Review: In the last blog entry I went through more than a few examples of using cURL to issue GET requests against various end points using Node.js & Restify. I also covered the basics on where to go to find cURL in case it isn’t installed. The last part I covered was a little bit of WebStorm info to boot. In this part of the series I’m now going to dive into the HTTP verbs beyond GET.
POST
The practice around issuing a command via http verb to save data is via a post. When you issue a post via cURL use the -X followed by POST to designate a post verb, then -H to assign the content type parameter. In this particular example I’ve set it to application/json since my payload of data will be JSON format. Then add the final data with a -d option, followed by the actual data.
[sourcecode language=”bash”]curl -X POST -H "Content-Type: application/json" -d ‘{"uuid":"79E5591A-1E54-4562-A276-AFC266F54390","webid":"56E62C3A-D6BC-4F4F-B72A-E6CE081190B6"}’ http://localhost:3000/ident%5B/sourcecode%5D
Other data types can be sent, which the content type can be appropriately set for including; html, json, script, text or html. One example of this same command, issued with jQuery on the client side would actually look like this.
[sourcecode language=”javascript”]
var data = {"uuid":"79E5591A-1E54-4562-A276-AFC266F54390","webid":"56E62C3A-D6BC-4F4F-B72A-E6CE081190B6"};
When building post end points via express one of the things you may run into is the following message being displayed in the console.
[sourcecode language=”bash”]
/usr/local/bin/node app.js
connect.multipart() will be removed in connect 3.0
visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives
connect.limit() will be removed in connect 3.0
[/sourcecode]
The immediate fix for this, until the changes are made (which may or may not mean to just alwasy is to replace this line
var quotes = [
{ author : ‘Audrey Hepburn’, text : "Nothing is impossible, the word itself says ‘I’m possible’!"},
{ author : ‘Walt Disney’, text : "You may not realize it when it happens, but a kick in the teeth may be the best thing in the world for you"},
{ author : ‘Unknown’, text : "Even the greatest was once a beginner. Don’t be afraid to take that first step."},
{ author : ‘Neale Donald Walsch’, text : "You are afraid to die, and you’re afraid to live. What a way to exist."}
];
So often I end up putting together some RESTful services (or the intent is to at least build them with that premise, but we all know how that ends up). The API URIs routing gets put together and one wants to take a crack at the service as soon as possible. Here’s a quick guide for using cURL to take some basic actions against the services and understand what you’re getting back.
The first thing to do is make sure you can run JavaScript, which means you have a computer. The second thing is to get cURL, which means you’re running some variant of Linux or UNIX. In most scenarios one would be running OS-X. The easiest way to determine if it is installed on your computer just open up a terminal and type ‘curl –help’. You should get a result with all the switches, which is almost always a bit of overload.
[sourcecode language=”bash”]$ curl –help
Usage: curl [options…]
Options: (H) means HTTP/HTTPS only, (F) means FTP only
–anyauth Pick "any" authentication method (H)
-a, –append Append to target file when uploading (F/SFTP)
–basic Use HTTP Basic Authentication (H)
–cacert FILE CA certificate to verify peer against (SSL)
–capath DIR CA directory to verify peer against (SSL)
-E, –cert CERT[:PASSWD] Client certificate file and password (SSL)
–cert-type TYPE Certificate file type (DER/PEM/ENG) (SSL)
–ciphers LIST SSL ciphers to use (SSL)
–compressed Request compressed response (using deflate or gzip)
-K, –config FILE Specify which config file to read
–connect-timeout SECONDS Maximum time allowed for connection
-C, –continue-at OFFSET Resumed transfer offset
-b, –cookie STRING/FILE String or file to read cookies from (H)
-c, –cookie-jar FILE Write cookies to this file after operation (H)
–create-dirs Create necessary local directory hierarchy
–crlf Convert LF to CRLF in upload
–crlfile FILE Get a CRL list in PEM format from the given file
-d, –data DATA HTTP POST data (H)
–data-ascii DATA HTTP POST ASCII data (H)
–data-binary DATA HTTP POST binary data (H)
–data-urlencode DATA HTTP POST data url encoded (H)
–delegation STRING GSS-API delegation permission
–digest Use HTTP Digest Authentication (H)
–disable-eprt Inhibit using EPRT or LPRT (F)
–disable-epsv Inhibit using EPSV (F)
-D, –dump-header FILE Write the headers to this file
–egd-file FILE EGD socket path for random data (SSL)
–engine ENGINE Crypto engine (SSL). "–engine list" for list
-f, –fail Fail silently (no output at all) on HTTP errors (H)
-F, –form CONTENT Specify HTTP multipart POST data (H)
–form-string STRING Specify HTTP multipart POST data (H)
–ftp-account DATA Account data string (F)
–ftp-alternative-to-user COMMAND String to replace "USER [name]" (F)
–ftp-create-dirs Create the remote dirs if not present (F)
–ftp-method [MULTICWD/NOCWD/SINGLECWD] Control CWD usage (F)
–ftp-pasv Use PASV/EPSV instead of PORT (F)
-P, –ftp-port ADR Use PORT with given address instead of PASV (F)
–ftp-skip-pasv-ip Skip the IP address for PASV (F)
–ftp-pret Send PRET before PASV (for drftpd) (F)
–ftp-ssl-ccc Send CCC after authenticating (F)
–ftp-ssl-ccc-mode ACTIVE/PASSIVE Set CCC mode (F)
–ftp-ssl-control Require SSL/TLS for ftp login, clear for transfer (F)
-G, –get Send the -d data with a HTTP GET (H)…[/sourcecode]
Don’t get intimidated! It goes on and on and on, but just know it’s installed if you see all these goodies. If you don’t get the results above, then installing cURL is the next step. I’ll leave that to you. Here’s some links to download and get started however.
Next you’ll of course need Node.js and Restify installed. I’ll assume you have Node.js installed. Create a directory and in that directory just run the following command.
Next create a file called server.js in that directory you’ve just installed restify in. Here’s the initial JavaScript code for that file that I’ve used to put together for the first few examples of using cURL.
[sourcecode language=”javascript”]
var restify = require(‘restify’);
function respond(req, res, next) {
res.send(‘hello ‘ + req.params.name);
}
var server = restify.createServer();
server.get(‘/hello/:name’, respond);
server.head(‘/hello/:name’, respond);
Ok, now to run this with node.js just issue the command to launch node.js with this file that was just created.
[sourcecode language=”bash”]
node server.js
restify listening at http://0.0.0.0:8080
[/sourcecode]
Getting Get
Now the service is running on port 8080 against 0.0.0.0. To check out what a standard GET verb will do in a browser, open up a browser and navigate to http://0.0.0.0:8080.
Browsing the GET response via Chrome.
You’ll see this in the browser window. Just straight plain text too. If you look at source, this is all you get back. Now open up a terminal and run the following cURL command to execute a GET against the URI & port. This is the most basic cURL command one can make. It is simply issuing a GET request against the URI and will display the body of the response.
The response will be similar to this for the particular request.
[sourcecode language=”bash”]
{"code":"ResourceNotFound","message":"/ does not exist"}
[/sourcecode]
Your terminal will probably stick the subsequent prompt at the end of the result too, because the result doesn’t end in a newline. Beware of that, your prompt hasn’t disappeared. 😉
To get a little more information you can get the header of the response dumped into the terminal with a -i. The -i option stands for –include, to include the header. Issue the command as either line shown below.
The response will be provide a little bit more about what is going on.
[sourcecode language=”bash”]
HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 56
Date: Wed, 27 Nov 2013 00:27:36 GMT
Connection: keep-alive
{"code":"ResourceNotFound","message":"/ does not exist"}
[/sourcecode]
With this response the actual response error code number is shown. In this case we have a 404, which points us to the problem with this curl request. The server isn’t returning anything to our curl request. If we look at the code, we can see that the ‘get’ route is setup as ‘/hello/:name’ which means that the domain root is only looking at http://url_root/hello/someName for a request to be made in order to return a response.
[sourcecode language=”javascript”]
var server = restify.createServer();
server.get(‘/hello/:name’, respond);
server.head(‘/hello/:name’, respond);
[/sourcecode]
Issue a command against the server now with the following curl request.
The response should come back as an actual response with content.
[sourcecode language=”bash”]
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 13
Date: Wed, 27 Nov 2013 00:34:04 GMT
Connection: keep-alive
"hello Adron"
[/sourcecode]
Here the content is returned as “hello Adron” and the header returns a 200. The content type is application/json format with the length returned as 13. Note also the connection is set to keep-alive. Let’s dive into that.
If we change the connection type, which is important for many scenarios, we have to send extra header information to ask for the response to be returned accordingly. In order to do that we can pass the -H or –header option in with the curl request. If the command is issued with an -i and -H as shown below the result will be as follows.
[sourcecode language=””]
curl -iH "connection: close" http://0.0.0.0:8080/hello/Adron
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 13
Date: Wed, 27 Nov 2013 00:41:07 GMT
Connection: close
"hello Adron"
[/sourcecode]
If we take away the -i we’ll just get the response, which is “hello Adron” and wouldn’t get the header, which now returns Connection: close in the response. By default, curl sets the connection as keep-alive, but in order to make the request return right away the connection needs to be issued a request for it to close. By setting the -H or –header value of connection to close, we get the response immediately. With restify, it is also important to note that it checks if the user agent is curl.
If it is curl the connection header to close and removes the content-length header. However I’ve experienced that restify is not doing this in all circumstances or that the use of curl is being changed in some of my usage. So don’t always assume that this will be the case. The safest bet is to set the connection closed when done. Thus, adding -H or –header and setting connection to close with a “Connection: close”.
Beyond Basic Get
Ok, so that’s a pretty solid use of GET with cURL. Let’s dive into some puts and deletes with a get or two thrown in for comparison. Change the executing code to the code shown in the server.js file below.
[sourcecode language=”javascript”]
var restify = require(‘restify’);
This function is setup to take req, res, and then handle next. The req is the request, the res is the response and the next is for issuing to return and continue with the result. The next bit of code starts the server with the restify.createServer();. Just below that there are several handlers that are setup.
Now at this point I got a little sidetracked writing this blog entry. But I thought to myself, “hell, I’m just figuring out some parts of Webstorm, I ought to blog a little about it!” So, here’s…
A Little Webstorm Love
Webstorm and cURL. Click the image for a full size image.
Before continuing on I wanted to cover a few tidbits of the Jetbrains Webstorm IDE. I often switch back and forth between the Sublime/Terminal combo and the Webstorm IDE. The really cool thing about this IDE is that it actually has a Terminal built in, color coding and autocomplete of the code, refactoring, and file and folder viewer and a whole slew of other features. In the image above that I’ve included there are four neon pointers that are displaying some of the key functionality that I’m using to work through this blog entry with cURL and Restify.
The arrows, from left to right are pointing to the following IDE elements. The first is pointing to the javascript files storgie.js and starter.js which I added specifically to show the git status colors. Each color reflect if the file is new (green), has changes (light blue) or is committed with no changes (white). The second arrow is just pointing to the general folder structure. Here you can see the hidden .* files like the .gitignore and .npmignore and also easy to dig through the node_modules directory. Webstorm also uses the node_modules directory to provide extra information and autocomplete to the code as you work through your coding session. The next arrow is pointing out the terminal in the editor, which is where I’m working up the curl examples in this blog entry. Then of course the color coded starter.js file that is one of the working examples. Webstorm, simply, is pretty sweet. I’m looking to do some more walk throughs and work sessions with the editor in the near future. So if interested, be sure to keep reading and subscribe, I’ll be sure to post any links to wherever the material ends up right here.
Now, back to the cURLing. 😉
After I toyed around with Webstorm and bit to get it work in a way that was efficient for me to use it for developing these APIs I stumbled into an idea. I’d provide a page for the APIs that could be located at the root of the API service such as http://api.blagh.com. The APIs would still be a restful type schema like http://api.blagh.com/thing/create or http://api.blagh.com/thing/destroy but at the very root would be a kind of docs. Maybe this could just be a status page even. Whatever the case, there needs to be something at http://api.blagh.com so I decided right then and there I’d switch to express.js to build the rest of the API services. Restify is fine and all but for this, it seemed like express would have all of the pieces I need for this.
Just to boot, I then read a few articles about express being faster such as this one. But then I read this issue on github and almost thought, “maybe I should keep using restify” but then I thought, “dammit, just get it done the way you want it built” so it was back to express. It’s easy enough to change this later so I just got back to coding, albeit with express now. So keep reading and in the next day or two I’ll have part two of this series on using cURL to hack at your APIs.
A few months ago I posted a poll to ask what courses I should put together next. I just wrapped up and am putting the final edits and finishing touches on a Pluralsight Course on distributed databases, focusing on Riak. On the poll the top three courses, by a decent percentage of votes included the following:
Node.js Distributed Systems – Bringing the Node.js Nodes together for Distributed Noes of Availability and Compute @ 12.14% of the vote.
A Quick Intro to Node.js
Introduction to Relevant Distributed Patterns
How Does Node.js Fit Into the Distribution
Working With Distributed Systems (AKA Avoiding a Big Ball of Mud)
Build a Demo
Distributed Systems Programming with Javascript @ 10.4% of the vote.
Patterns for Distributed Programming
…and I’m figuring the other sections out still for this one… got ideas? It needs to encompass the client side as well as the non-client code side of things. So it’s sort of like the above course, but I’m focusing more on the periphery of what one deals with when dealing with developing on and around distributed systems as well as distributed systems themselves.
Vagrant OS-X, Windows and Linux – how to build, manage and ship machines to use for development and recreation of production environments.
Vagrant, What is it?
OS-X, Linux and Windows
Using Vagrant Machines
Building Vagrant Dev Machines
Vagrant the Universe!
Now I might flip this list, but either way they’re all going to be super cool. So stay tuned and I’ll be working up these into courses. So far here’s the sub-bullets above are the basics of the curriculum I intend to put forward. Am I missing anything? Would you like to see anything specifically? Leave a comment and I’ll be sure to get everything as packed in there as possible!!
Today the first Nodebots Day took place worldwide! Portland held its own event here at the old Urban Airship garage space. With the garage door wide open and the glorious day gleaming daylight into the garage 50+ hardware and coding beginners, learners, coders and hackers of all backgrounds came together. Breadboards were wired up, code was slung, robots moved and twitched to life.
This all started with a super quick organizing effort by some local JavaScript hackers to join the worldwide Nodebots Day (check the repo for infoz). After a meet, some sponsors jumping to our aid, and some hustle by some great people, the event in Portland came together. The turnout was great!
The bits everybody got…Bits up close…
My Own Robot Battle
Oh dear, I was dead zonked when I arrived. It’s been a super long and hard week. I’ve had deadlines to meet, code to write and OSCON to attend, needless to say that leaves basically a few hours each night of the last week to actually recharge. Things were definitely catching up with me… and I’ll admit I made almost zero progress, however I was super excited to see what many others accomplished!
There was the quad copter that Carter @CarterRabasa got up and flying with some aerobatic acrobatic flips.
There was an erector set wheeled robot that was primed for deployment. Nothing like combining the quality and build endurance of erector set gear with that of modern machine and robot automation for fun!
Troy going mad scientist on his bot.
Troy @thoward37 was building a walking bot for world domination… which if he didn’t finish it after my departure I’m looking forward to see it walking and doing full auto-deploy in the near future.
Serial Port for a head!Wires, Connections, Devices & More…
There was musical linkages being made to device and computer alike. With code combining to form knew methods of interaction between device, human and music.
Along with these bots there was much progress among breadboards laced with ideas and blinking LEDs amidst us all. I do believe everybody had a blast and learned a lot.
Erector RobotWorking together… creating the robot society!Codez! Arduino! Wires! Brains!
Johnny 5 and I say “THANKS AND HIGH FIVES!!!”
A huge thanks to so many, I might have missed a few people, apologies (and let me know, I’ll add you to the list of thanks!!) — and also, thanks to EVERYBODY who came out and worked and learned about robots!
Also if anybody has any questions about robots, javascript, node.js, robots in Portland, Portland Bridges or the event or about coming to Portland to hack, code, beer, food or just move here. Let me know and I’ll be more than happy to hook you up with appropriate resources! Peace!
@HackyGoLucky – Cuz yeah, you kicked some ass and herded all us cats together for this! Thanks Tracy!
@nexxylove – Thanks for the code repo o’ lights! I’m sure you enjoyed San Francisco, we missed you in Portland! We’ll hack when you’re back!
@BlaineBublitz – Welcome to Portland again, thanks for traveling into town! I’ll ping you next I’m in Phoenix and we’ll hack the light rail. 😉
@s5fs – Yo, ok, you just front loaded my cortex full of ideas on that last brunch convo. Thanks for helping, being kick ass, and coming out to Nodebots day!
@nickniemeir – Thanks for coming up to Nodebots day PDX!
@thoward37 – What did you do again? You keep showing up everywhere… are you a robot?
@CarterRabasa – Thanks for coming down to the Stumptown from the Emerald City.
@_jden – Yo, Palo Alto to bridge city (another PDX nick name)… welcome back to PDX and to the future with our Robot Overlords!
I wasn’t able to get to Cascadia.js. Sometimes during the course of working smart and hard one misses the smart part and scheduling falls apart. Well, I messed up. I messed up and my scheduling got completed dorked for the last two weeks. What did that result in? I missed Cascadia.js, a codeathon in Spokane that I was putting together and to top it off I was missing Defrag in Denver – which really put me out because I was out of pocket personally for Defrag. Altogether it was a financial, logistical and scheduling nightmare for me.
To all, I apologize for my lapse in scheduling prowess!
Amid all of this mess, I’ve got some great new things coming up in the coming week for the OSS Projects I’m working on, the north west, a little emerald for Seattle, some earthy stuff for Portland and all around interesting tidbits here and there.
Before I go rambling on about those things, I wanted to leave this pre-weekend before Thanksgiving with some shout outs to the Cascadia.js Team & Presenters. Carter, Troy, Luc, Jerry, Laurie, Bobby & the whole lot of the team that put that together – you guys seriously ROCK!
The conference had a number of speakers, who totally rocked it, and here’s a few of my first views. I wasn’t there, as I said, so I was seriously stoked that the team put the videos online.
…and there ARE MORE PRESENTATIONS at Cascadia.js on youtube. Check them out, each is a blast!
Pull Request for People by Chris Williams
@voodootikigod <- but don’t look for him on Twitter… watch the ending keynote of his. It’s really good and we all need to think about what he is saying, seriously think about what he’s saying.
As for some of his questions he asks, I’ll have some answers to that – which I could indeed rattle off quickly, in response. I do say though, I don’t provide these answers to counter what he is saying. I do so only to state and reaffirm what he talked about. I am absolutely, 100% in agreement with what he is saying about the current state of the startup & tech sector.
With that, I’m going to spend some time with friends. Maybe even make some new ones. Cheers! 🙂
…as for Defrag, I’ll have more about that in the coming days too.
You must be logged in to post a comment.