Tag: viper

Cobra + Viper Go CLI Sample

I just released a Go Cobra + Viper library CLI App sample app here. Here are the main up to date details as of release today.

The app example is using the Cobra and Viper libraries for Go. This application provides commands (per Cobra) to read, write, update, and delete configuration records from a configuration file using Viper.

Using the example CLI App.

<cli> config -h provides the following documentation for using this CLI App.

Available Commands:
  add         The 'add' subcommand will add a passed in key value pair to the application configuration file.
  delete      The 'delete' subcommand removes a key value pair from the configuration file. 
  update      The 'update' subcommand will update a passed in key value pair for an existing set of data to the application configuration file.
  view        The 'view' subcommand will provide a list of keys and a map of the values.

Flags:
  -h, --help           help for config
  -k, --key string     The key for the key value set to add to the configuration.
  -v, --value string   The value for the key value set to add to the configuration.

Currently I got this to work but it isn’t in the best shape. Check out the code here.

Get Involved, Add Samples, Make Requests

If you’d like to get involved and add samples or make request for additional samples please file an issue here!

If you’ve found any bugs or issues with the code please file a bug report here!

Examples, The CRUD!

./cli config add -k "blog" -v "https://compositecode.blog/" example writes a record to the configuration file with a key of “blog” and a value of “https://compositecode.blog/“.

./cli config view displays the contents of the configuration file and CLI specific environment variables. These are the configuration files located in the .cobrae-cli-samples.yml and environment variables prefaced with COBRACLISAMPLES. The list of keys is displaced first and then the keys and values are displayed below that.

./cli config update -k "blog" -v "not found" will update the blog entry in the configuration to read not found for the value.

./cli config delete ... will delete the key and value from the configuration file.

Building the Project

Following a fairly standard clone, one can build this project with a single step using the ./build.sh file. If you’d like to contribute the same for Windows, feel free I’d be happy to pull that PR in. Once the project is built use the CLI as defined above.

Installing the CLI App

This application can be installed as a CLI app by referencing it’s location in your bash (powershell? etc) startup script.

Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation

Part 2 of 3 – Coding Session in Go – Cobra + Viper CLI for Parsing Text Files, Retrieval of Twitter Data, Exports to various file formats, and export to Apache Cassandra.

UPDATED PARTS:

  1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
  2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
  3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

Updated links to each part will be posted at bottom of  this post when I publish them. For code, written walk through, and the like scroll down below the video and timestamps.

Hacking Together a CLI Installing Cassandra, Setting Up the Twitter API, ENV Vars, etc.

0:04 Kick ass intro. Just the standard rocking tune.

3:40 A quick recap. Check out the previous write “Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files” of this series.

4:30 Beginning of completion of twitz parse command for exporting out to XML, JSON, and CSV (already did the text export previous session). This segment also includes a number of refactorings to clean up the functions, break out the control structures and make the code more readable.

In the end of refactoring twitz parse came out like this. The completed list is put together by calling the buildTwitterList() function which is actually in the helpers.go file. Then prints that list out as is, and checks to see if a file export should be done. If there is a configuration setting set for file export then that process starts with a call to exportParsedTwitterList(exportFilename string, exportFormat string, ... etc ... ). Then a simple single level control if then else structure to determine which format to export the data to, and a call to the respective export function to do the actual export of data and writing of the file to the underlying system. There’s some more refactoring that could be done, but for now, this is cleaned up pretty nicely considering the splattering of code I started with at first.


var parseCmd = &cobra.Command{
Use: "parse",
Short: "This command will extract the Twitter Accounts form a text file.",
Long: `This command will extract the Twitter Accounts and clean up or disregard other characters
or text around the twitter accounts to create a simple, clean, Twitter Accounts only list.`,
Run: func(cmd *cobra.Command, args []string) {
completedTwittererList := buildTwitterList()
fmt.Println(completedTwittererList)
if viper.Get("fileExport") != nil {
exportParsedTwitterList(viper.GetString("fileExport"), viper.GetString("fileFormat"), completedTwittererList)
}
},
}
func exportParsedTwitterList(exportFilename string, exportFormat string, twittererList []string) {
if exportFormat == "txt" {
exportTxt(exportFilename, twittererList, exportFormat)
} else if exportFormat == "json" {
exportJson(exportFilename, twittererList, exportFormat)
} else if exportFormat == "xml" {
exportXml(exportFilename, twittererList, exportFormat)
} else if exportFormat == "csv" {
exportCsv(exportFilename, twittererList, exportFormat)
} else {
fmt.Println("Export type unsupported.")
}
}
func exportXml(exportFilename string, twittererList []string, exportFormat string) {
fmt.Printf("Starting xml export to %s.", exportFilename)
xmlContent, err := xml.Marshal(twittererList)
check(err)
header := xml.Header
collectedContent := header + string(xmlContent)
exportFile(collectedContent, exportFilename+"."+exportFormat)
}
func exportCsv(exportFilename string, twittererList []string, exportFormat string) {
fmt.Printf("Starting txt export to %s.", exportFilename)
collectedContent := rebuildForExport(twittererList, ",")
exportFile(collectedContent, exportFilename+"."+exportFormat)
}
func exportTxt(exportFilename string, twittererList []string, exportFormat string) {
fmt.Printf("Starting %s export to %s.", exportFormat, exportFilename)
collectedContent := rebuildForExport(twittererList, "\n")
exportFile(collectedContent, exportFilename+"."+exportFormat)
}
func exportJson(exportFilename string, twittererList []string, exportFormat string) {
fmt.Printf("Starting %s export to %s.", exportFormat, exportFilename)
collectedContent := collectContent(twittererList)
exportFile(string(collectedContent), exportFilename+"."+exportFormat)
}
func collectContent(twittererList []string) []byte {
collectedContent, err := json.Marshal(twittererList)
check(err)
return collectedContent
}
func rebuildForExport(twittererList []string, concat string) string {
var collectedContent string
for _, twitterAccount := range twittererList {
collectedContent = collectedContent + concat + twitterAccount
}
if concat == "," {
collectedContent = strings.TrimLeft(collectedContent, concat)
}
return collectedContent
}
func exportFile(collectedContent string, exportFile string) {
contentBytes := []byte(collectedContent)
err := ioutil.WriteFile(exportFile, contentBytes, 0644)
check(err)
}

view raw

parse.go

hosted with ❤ by GitHub

50:00 I walk through a quick install of an Apache Cassandra single node that I’ll use for development use later. I also show quickly how to start and stop post-installation.

Reference: Apache Cassandra, Download Page, and Installation Instructions.

53:50 Choosing the go-twitter API library for Go. I look at a few real quickly just to insure that is the library I want to use.

Reference: go-twitter library

56:35 At this point I go through how I set a Twitter App within the API interface. This is a key part of the series where I take a look at the consumer keys and access token and access token secrets and where they’re at in the Twitter interface and how one needs to reset them if they just showed the keys on a stream (like I just did, shockers!)

57:55 Here I discuss and show where to setup the environment variables inside of Goland IDE to building and execution of the CLI. Once these are setup they’ll be the main mechanism I use in the IDE to test the CLI as I go through building out further features.

1:00:18 Updating the twitz config command to show the keys that we just added as environment variables. I set these up also with some string parsing and cutting off the end of the secrets so that the whole variable value isn’t shown but just enough to confirm that it is indeed a set configuration or environment variable.

1:16:53 At this point I work through some additional refactoring of functions to clean up some of the code mess that exists. Using Goland’s extract method feature and other tooling I work through several refactoring efforts that clean up the code.

1:23:17 Copying a build configuration in Goland. A handy little thing to know you can do when you have a bunch of build configuration options.

1:37:32 At this part of the video I look at the app-auth example in the code library, but I gotta add the caveat, I run into problems using the exact example. But I work through it and get to the first error messages that anybody would get to pending they’re using the same examples. I get them fixed however in the next session, this segment of the video however provides a basis for my pending PR’s and related work I’ll submit to the repo.

The remainder of the video is trying to figure out what is or isn’t exactly happening with the error.

I’ll include the working findem code in the next post on this series. Until then, watch the wrap up and enjoy!

1:59:20 Wrap up of video and upcoming stream schedule on Twitch.

UPDATED SERIES PARTS

    1. Twitz Coding Session in Go – Cobra + Viper CLI for Parsing Text Files
    2. Twitz Coding Session in Go – Cobra + Viper CLI with Initial Twitter + Cassandra Installation (this post)
    3. Twitz Coding Session in Go – Cobra + Viper CLI Wrap Up + Twitter Data Retrieval

 

Building a Data Thrashing CLI Tool in Go

I need a tool just to do some testing against an API end point. I figured I’d throw one together real quick in Go. With a few libraries it’s just a few steps to get the job done. The following is that project. Eventually I’ll create the services that will run in some containers I’ll throw into a Kubernetes cluster, but for now, it’s all CLI. Onward.The first thing I’ll need is Cobra. Continue reading “Building a Data Thrashing CLI Tool in Go” →