Tag: api

Engineering API Governance

I’ve written this post to provide a detailed look at what a role around API Governance, or simply the functions of API Governance, would be centered around.

Why does this role exist?

An API Governance function or role would be responsible for overseeing the governance of APIs within an organization. It would center around ensuring that APIs are developed, implemented, and maintained according to established standards, best practices, and business requirements.

Here are the primary reasons why this type of work and role exist:

  1. Growing Importance of APIs: APIs have become the backbone of modern software development and integration. They enable different applications and systems to communicate and share data seamlessly. As organizations increasingly adopt microservices architecture and cloud-based solutions, the number and complexity of APIs they use grow significantly.
  2. Consistency and Standardization: In large organizations with multiple development teams and projects, maintaining consistency in API design, implementation, and usage becomes challenging. An API governance role ensures that APIs adhere to standardized guidelines, leading to better interoperability and reusability.
  3. Security and Compliance: APIs expose access points to an organization’s systems and data. Ensuring the security of these access points is critical to prevent data breaches, unauthorized access, and other security risks. An API governance role focuses on implementing robust security measures and compliance with relevant regulations.
  4. Efficient Collaboration: When different teams create APIs independently, they may not be aware of existing solutions or may duplicate efforts. An API governance function facilitates collaboration, knowledge sharing, and reuse of APIs, leading to more efficient development processes.
  5. Scalability and Performance: Proper governance includes performance monitoring and optimization of APIs. This helps identify bottlenecks, improve response times, and ensure that APIs can handle increasing workloads as the organization grows.
  6. User Experience: APIs are used by developers, both internally and externally, to build applications. A well-governed API ecosystem includes clear and comprehensive documentation, which enhances the developer experience and enables faster integration with APIs.
  7. Risk Mitigation: By setting up a structured governance process, organizations can identify and address potential risks associated with APIs early on, reducing the chances of costly issues arising in production.
  8. API Lifecycle Management: APIs have a lifecycle that includes planning, development, versioning, and deprecation. Proper governance ensures that APIs are maintained and updated according to their lifecycle stages, preventing the accumulation of outdated or obsolete APIs.
  9. Compliance with Business Strategy: API governance aligns API development and management with the organization’s overall business strategy, ensuring that APIs support the company’s goals and objectives effectively.
  10. Adoption of Best Practices: An API governance role stays updated with industry trends and best practices, continuously improving the organization’s API strategy and development processes.

Engineering API Governance Primary Functions

The API Governance function or role is responsible for overseeing and managing the governance of Application Programming Interfaces (APIs) within an organization. It ensures that APIs are developed, implemented, and maintained according to established standards, best practices, and business requirements. Here are some key aspects of an API Governance function or role:

  1. Defining API Standards: The API Governance team establishes and documents API design standards, naming conventions, versioning practices, security guidelines, documentation requirements, and other relevant policies. These standards promote consistency and facilitate seamless integration among different APIs.
  2. API Lifecycle Management: The API Governance function oversees the entire lifecycle of APIs. This includes planning and design, development, testing, deployment, monitoring, version control, and eventual retirement or deprecation when necessary.
  3. Security and Compliance: Ensuring the security of APIs is a critical aspect of the API Governance role. The team establishes security protocols, access controls, authentication mechanisms, and data protection measures to safeguard APIs and the systems they interact with. Additionally, they ensure compliance with relevant industry regulations and data privacy laws.
  4. Documentation and Communication: API Governance teams create comprehensive and easily accessible documentation for APIs. This documentation helps internal and external developers understand how to use the APIs effectively and provides details about their capabilities, limitations, and potential use cases.
  5. Monitoring and Performance: The API Governance function sets up monitoring systems to track API usage, performance metrics, and error rates. This data helps identify potential issues, bottlenecks, and areas for improvement. It allows the team to optimize APIs to meet service-level requirements and user expectations.
  6. Collaboration and Coordination: The API Governance role involves collaborating with various teams, including development, product management, security, and operations. Effective coordination ensures that APIs are aligned with business objectives and developed in a way that meets the needs of different stakeholders.
  7. Review and Approval Process: The API Governance team establishes a review and approval process for new APIs and changes to existing ones. This process ensures that APIs meet the required standards, security measures, and compliance before they are deployed.
  8. Education and Training: The API Governance function may conduct training sessions for developers and other stakeholders to promote best practices in API development, usage, and maintenance.
  9. Adoption of API Management Tools: API Governance teams often utilize API management tools to facilitate API governance tasks. These tools can assist in monitoring, versioning, security, analytics, and documentation management.
  10. Continuous Improvement: The API Governance function remains updated with industry trends and best practices to continuously improve the organization’s API strategy and governance approach.

In a large enough organization even these individual functions become individually staffed work. However, a single API Governance staff member often would be tasked with these items and would need to delegate and organize the priority of the various functions throughout the organization.

In following posts (which I’ll include here once they’re posted) I’ll write up some scenarios, from working as an individual contributor and leader (i.e. managing staff) as well as hiring for API Governance roles. I’ll get into the nitty gritty of how process, practice, and patterns can be used to elaborate on things like education, adoption of API management tools, continuous improvement and the many other functions.

With that, subscribe for updates, and you’ll get any new posts direct to your inbox! (check side bar for subscribing)

A Hasura Quick Start with Remote Schema, Remote Joins

I’ve been building GraphQL APIs for a number of years now – of along side RESTful, gRPC, XML, and other API styles I won’t even bring up right now – and so far GraphQL APIs have been great to work with. The libraries in different languages form .NET’s Hot Chocolate, Go’s graphql-go, Apollo’s JavaScript based tooling and servers, to Java’s GraphQL for Spring have worked great.

Sometimes you’re in the fortunate situation where you’re using PostgreSQL or SQL Server, or other supported database for a tool like Hasura. Being able to get a full GraphQL (with REST options too) API running in seconds is pretty impressive. From a development perspective it is a massive boost. As Hasura adds more database connectors as they have with Snowflake and Amazon Athena, the server and tooling becomes even more powerful.

With that I wanted to show a N+1 demo where N is day 1 with Hasura. The idea is what do you do immediately after you get a sample service running with Hasura. How do you integrate it with other services, or more specifically how do you integrate your Hasura API along side APIs you’ve written yourself, such as an enterprise GraphQL for Spring based API running against Mongo or other data source? This repo is the basis for several demonstration repositories I am building that will show how you can setup – generally for local development – Hasura + X API with Y Language stack.

This is the Hasura quick start repository here, with migrations and metadata for a local setup. The first demonstration repo for a peripheral GraphQL API will be a Spring based API in this repository. The following steps will get the quick start repository up and running.

  1. Clone this repo git clone git@github.com:Adron/hasura-quick-start.git.
  2. From the root (where the docker-compose.yml file is located) execute docker compose up -d.
  3. Navigate into the hasura directory.
  4. Execute hasura metadata apply, then hasura migrate apply, and then hasura metadata apply. Just do it, it’s a strange workflow thing.
  5. Navigate now into the `hasura` directory and execute hasura console.

These steps are demonstrated in this video from 48 seconds.

What do you get once deployed?

The following are some of the core capabilities of Hasura and showcase what you can get up and running in a matter of seconds, even when you start from a completely empty database! First off you’ll find the database now has 3 tables along with their pertinent schema built out in PostgreSQL and available via Hasura, as shown here under the Data tab of the console.

I also created a schema diagram just to provide a visual of how these tables are designed.

For the remote schema, the Spring API, the following steps will get it cloned and running locally.

  1. Clone this repo git clone git@github.com:Adron/hasura-spring-boot-graphql.git.
  2. Execute ./gradlew build to get the jar file build. It will then be located in the build/libs directory of the project.
  3. Next build the docker image with docker build -t adron/hasura-spring-boot-graphql . to build the docker image locally.
  4. Now you can either start this container with docker compose up -d using the docker-compose.yml in the project or you can run the image with Docker specifically with docker run -p 8081:8080 adron/hasura-spring-boot-graphql.

For a walkthrough of getting the Spring API running, check out 2:28 onward in this video.

Now both of these instances are running locally and you can test each out respectively, but not specifically together. I’ll have probably write up another post on how to get services that spin up separately to run together for localized development. However, with the way things are setup in the two repos, it’s as if one team is the Hasura team building a GraphQL API and another is a Spring Java GraphQL API team, and they’re working autonomously of each other just based on contract of the APIs themselves.

Remote Schema

With that being the scenario, I’ve deployed the Spring API out remotely so that I could show how to put together a remote schema connection and then a remote join query, i.e. nested query in GraphQL speak, across these two APIs.

To add the remote schema, click on the remote schemas tab on the console. Add a name (1), then the URI (2), and optionally if needed add appropriate headers (3) or forward all headers from client requests.

Once that’s added, navigate to the relationships tab of the new remote schema and click on add. Then for this example, select remote database (1), then add a name (4) (Customer in the example) and then for type choose object (3) (per the example).

Then scroll down on that console screen and choose sales_data (1) and default, public, and users (2) under the reference database, schema, and table. Next up choose the source field (3) and reference column (4).

Once added it will look like this in the console.

This creates a relationship to be able to make nested queries against these sources with GraphQL. If it were a single contiguous database the schema would look like this. I’ve color coded the sales_data table as red, to signify it is the table we know is in another database (or, specifically, provided via another hosted API). However, as stated, in a single database the relationships would now look like this. The relationship however, isn’t in a database, but stored in the Hasura metadata between users and sales_data.

Now writing a query across this data would shape up like this. Because of the way the relationship was drawn via the remote schema, the path to get the nested object Customer (2) for the sales data is to start with the sales_data (1) entity. As shown.

sales_data {
  sales_number
  updated_at
  Customer {
    name
  }
}

Now we want to add more details about the particular customer like their email and details. To do this we’ll utilize another nesting level within this query that delves into relationships that are in the PostgreSQL database itself.

sales_data {
  sales_number
  updated_at
  Customer {
    name
    emails {
      email
    }
    details {
      details
    }
  }
}

With this the nested details email (3) and details (4) will be provided, which is foreign key relationships to the primary key table users in the underlying database, made available by Hasura’s relationships in metadata.

Boom! That’s it. Pretty easy setup if the databases and APIs have Hasura available to connect them in this way. Otherwise, this is a huge challenge to develop against if you’re just using solely a tech stack like Apollo, Spring Boot, or Hot Chocolate. Often something along federation and more complexities would come into play. But more on that later, I’ve got a piece coming on federation, stitching, remote schemas, and gateway – among various ways – to get multiple GraphQL, or GraphQL and RESTful APIs together into a singular, or singularly managed, API end point.

Hope that was useful, if you’ve got comments, questions, or curiosities let me know in the comments here, or pop over to the video and leave a comment there.

References:

The full video of setup and how the remote schema & joins work in Hasura.

Gradle Build Tool

A few helpful links and details to where information is on the Gradle Build Tool.

Installation

Via SDKMAN sdk install gradle x.y.z where x.y.z is the version, like 8.0.2.

Via Brew with brew install gradle.

Manually check out the instructions here.

Building a Java Library (or application, Gradle plugin, etc)

Using the init task. From inside a directory with the pertinent project.

gradle init

You’ll be prompted for options.

With the project initialized this is what that initialized folder structure looks like.

At this point add the Java code for the library, similar to this example, and execute a build like this.

./gradlew build

Build Collateral

View the test report via the HTML output file at lib/build/reports/tests/test/index.html.

The JAR file is available in lib/build/libs with the name lib.jar. Verify the archive is valid with jar tf lib/build/libs/lib.jar.

Add the version by setting the version = '0.1.1' in the build.gradle file.

Run the jar task ./gradlew jar and the build will create a lib/build/libs/lib-0.1.1.jar with the expected version.

Add all this to the build by adding the following to the build.gradle file:

tasks.named('jar') {
    manifest {
        attributes('Implementation-Title': project.name,
                   'Implementation-Version': project.version)
    }
}

Verifying this all works, execute a ./gradlew jar and then extract the MANIFEST.MF via jar xf lib/build/libs/lib-0.1.0.jar META-INF/MANIFEST.MF.

Adding API Docs

In the */Library.java file, replace the / in the comment by / * so that we get javadoc markup.

Run the ./gradlew javadoc task. The generated javadoc files are located at lib/build/docs/javadoc/index.html.

To add this as a build task, in build.gradle add a section with the following:

java {
    withJavadocJar()
}

Publish a Build Scan

Execute a build scan with ./gradlew build --scan.

Common Issues + Tips n’ Tricks

gradlew – Permission Denied issue

Let’s say you execute Gradle with ./gradlew with whatever parameter and immediately get a response of “Permission Denied”. The most common solution, especially for included gradlew executables included in repositories, is to just give the executable permission to execute. This is done with a simple addition chmod +x gradelw and you should now be ready to execute!

The Best Collected Details on the GraphQL Specification – Overview & Language

GraphQL, a query language and execution engine is described in this specification based on capabilities and requirements of data models for client-server applications. This article details and elaborates on the specification, the features and capabilities of GraphQL and implementations. I hope this collection of details around the GraphQL Specification can be used as a reference and launch point into learning about GraphQL use, implementation – server and client side – and ongoing references during future specification additions or changes!

Continue reading “The Best Collected Details on the GraphQL Specification – Overview & Language” →

Hasura 2.0 – A Short Story of v1.3.3 to v2.0 Upgrades

Today at Hasura we released Hasura v2.0! This is a pretty major release with a number of new features that will dramatically increase the capabilities for Hasura. For several of my projects, specifically the infrastructure as code projects terrazura (check out the previous blog post w/ video time points and more) and tenancy-bydata I was able to get the upgrade to Hasura v2.0 done in moments! Since I don’t have to pull backups or anything for these projects, it merely involved the following steps.

  1. Upgrade the Hasura CLI. This is super easy, just issue the command hasura update-cli --version v2.0.0-alpha.1. This command will then download and update the CLI.
  2. Next I updated the Terraform file so the container pulls the latest version image = "hasura/graphql-engine:v2.0.0-alpha.1".

Next run an updated terraform apply command, which in my case is this command in the case of the terrazura project for example.

terraform init

terraform apply -auto-approve \
  -var 'server=terrazuraserver' \
  -var 'username='$PUSERNAME'' \
  -var 'password='$PPASSWORD'' \
  -var 'database=terrazuradb' \
  -var 'apiport=8080'

cd migrations

hasura migrate apply

Boom! Everything is now updated to v2.0 and we’re ready for all the upcoming Twitch streams relating back to these particular projects!

For more, be sure to subscribe to the HasuraHQ Twitch Channel and my Twitch Channel Thrashing Code as I’ll be covering more of the new features in the coming days!