Skip to main content

Overview

Purpose

Aqago is modern software infrastructure for edge devices. Aqago was founded in 2023 to make it easy for engineers, teams, and companies to securely get edge-powered products to market faster and manage them more effectively at scale while keeping the infrastructure needed to do so under their control.

Aqago is self hosted to support real-world deployment models ill-suited for commercial SaaS. For example, Aqago supports federating deployments into a VPC per customer, a common requirement when integrating edge-powered devices (like robotics) into enterprise IT/OT environments on the factory floor, medical facilities, and critical infrastructure sites. Aqago is also suited for simpler deployment models more commonly used in consumer/business IoT products, e.g. one publicly available instance for the entire fleet.

Aqago is actively maintained by Aqa Technologies, Inc. Aqago was intially built to support Aqalink, a suite of edge networking products and software platform to scale remote operations in the physical world. Aqalink continues to be built on Aqago, but Aqago's reach has expanded to other teams and products.

Philosophy

Aqago incorporates lessons learned developing plumbing to support multiple connected products in various markets, as well as ongoing learning from active fleets. Its design goals are,

  • Keep it simple
  • Don't reinvent the wheel
  • Prioritize developer and user experience
  • Be customizable, but with sensible defaults
  • Optimize for security, reliablity, and scalability

Aqago minimizes its baseline surface area and leaves it to users to customize and extend its capabilities to fit their infrastructure when necessary. It makes no assumptions about and does not interact with the application domain or product functionality.

Costs

Aqago is designed to minimize cloud provider costs. There are several ways it achieves this.

First, Aqago is conceptually a metadata management system. Cloud providers charge for many things, key among them being data storage, movement, and compute. Because Aqago manages metadata about your devices in your fleet, it stores and processes small volumes of data. A typical Aqago deployment in AWS will incur ~$0.05 per device per month in charges.

Aqago further minimizes cloud provider costs through a series of optimizations specific to the provider. For example, on AWS the Aqago backend targets arm64 serverless runners (Lambda) which are currently 20% cheaper than the x86 equivalent. In addition, Aqago minimzes its devops burden by leveraging zero-maintenance offerings like DynamoDB that provide hassle-free horizontal scaling.

Features

Aqago provides the following core features,

  • Device identity provider to securely and easily integrate on-device workloads with your backend APIs via JWT
  • Flexible backend -> device artifact delivery with signing, deployment, and on-device verification using JWS
  • Realtime device monitoring including process status, heartbeat, logging, and more

Device Identity Provider (dIdP)

One of Aqago's core features is the Device Identity Provider (dIdP). dIdP offers the same benefits as user and application IdPs like Auth0, but with a focus on the specific authentication needs of devices within connected product fleets.

dIdP is a battle-tested alternative to homebrew/roll-your-own PKI and machine identity management that is secure and performant at scale. dIdP provides a JWT token authentication mechanism that can be integrated with JWT-enabled application-specific backend endpoints to authenticate requests originating from devices. In other words, it allows devices to authenticate against JWT-enabled APIs using JWTs.

JWTs simplify connected product authentication infrastructure by jettisoning the need for mTLS integration with application endpoints without falling back on insecure solutions like long-lived device API tokens. JWTs offer additional benefits, including custom claims that can contain authorization scopes, within the token.

dIdP provisions each Aqago backend instance against an ACME CA server hosted by Aqa Technologies, Inc. The ACME CA server verifies ownership of the domain on which the Aqago backend instance is running and issues a certificate that can be used for signing, digital signature verification, and client auth. This certificate is not meant as a replacement for server auth, in other words it can't be used for one-way TLS and Web PKI of your domain (a browser, for example, won't trust it).

Devices running the Aqago dIdP agent communicate with your Aqago backend instance to authenticate and receive JWT credentials. The operating model is similar to an in-browser user IdP client with the added benefit that IoT-specific implementation details like mTLS and public/private keys are abstracted away. You can optionally use the JWT credentials for authentication in your backend application endpoints to simplify your device authentication strategy.

The Aqa Technologies ACME CA implements a two-tier PKI with an offline root certificate. The CA is provided as a convenience and can be replaced by a private ACME CA for customers who prefer to run their own CA or are required to for data governance and compliance.

Provisioning

Devices are provisioned into dIdP by making a request to a provisioning endpoint. The provisioning endpoint is protected using short-lived, single-use provisioning tokens. Tokens are JWTs and can be created by making a request to a provisining token endpoint.

Active life

Devices run the Aqago agent to utilize dIdP. The agent automates the process of requesting and cycling device authentication tokens. Under the hood, the agent utilizes mTLS with x.509 certificates and asymmetric public/private key cryptography for device attestation.

Device heartbeat

The Aqago agent maintains a persistent network connection to dIdP for control plane operations. This connection also acts as a device heartbeat mechanism. The heartbeat can be configured in the dIdP backend to trigger webhooks for integrations into external applications (not currently implemented).

Decomissioning

Devices are decomissioned by making a request to the decomissioning endpoint. After a device is decomissioned dIdP will no longer issue tokens for the device.

History

dIdP maintains history of provisioning, decomissioning, and token requests throughout a device's lifecycle for security audits and compliance.

Integration with application endpoints

dIdP tokens can be integrated with JWT-enabled backend endpoints. dIdP tokens can be verified dynamically using the kid in the JWT header or statically by downloading the dIdP PEM public key.

Signing

Aqago provides signing endpoints that can be used to sign arbitrary payloads up to 4KB in size. This limit is designed to maintain a clean separation of concerns between data and metadata within your connected product stack. This keeps Aqago performant as your fleet scales and allows you to address scaling considerations in your data infrastructure (e.g. hot and cold storage, vertical vs horizontal DB scaling, etc.) independently of Aqago. It also allows Aqago to be used out of the box in systems with strict data governance requirements like medical devices and automotive and/or when handling sensitive intellectual property like trained model weights. Signing is done using the certificate obtained by dIdP.

For example, instead of providing Aqago with an entire compressed OS image or an ONNX or TF weights file for signing, you generate a hash of its contents and have Aqago sign that instead. Aqago can be used in this way to assert the integrity of arbitrary blobs without reading their contents. The Aqago agent can be used to further assert signature validity of blobs on device, e.g. for signed OTA OS or model updates.

Requests to signing endpoints are authenticated with API tokens.

Software delivery

Aqago provides a secure, flexible software delivery mechanism that can be used to distribute opaque blobs to your fleet. For example a rootfs, Docker export, filesystem volume, model weights for computer vision or LLMs, etc.

Artifacts

Artifacts provide a helpful and flexible semantic overlay over opaque blobs for connected product fleets. Artifacts consists of a header, a payload, and a signature. They're implemented as standard JSON Web Signature (JWS) documents.

An artifact header has the following properties,

  • alg and typ to specify signing algorithm used to produce signature
  • jku and kid parameters to verify the signature
  • User-provided tags (optional)

An artifact payload has the following properties,

  • URL to fetch the artifact's data
  • Hash of artifact's data

An artifact signature has the following properties,

  • Signature of the JWS

Artifacts are created in Aqago through a create artifact endpoint. Note that the artifact itself is not encoded in the payload or hosted by Aqago.

Artifact rollouts

Artifacts have a rollout. A rollout describes the basic properties of how the artifact is delivered to devices within your fleet. Artifacts are always pulled by devices. Rollouts describe when and to which devices.

Rollouts are defined by their alias and strategy. An alias is the query that devices make to retrieve the artifact. For example, ?tags=latest&labels=product,aqago-product'. Only one artifact can have an active rollout per alias. When an artifact is created, a default alias` is created for each tag and combination of labels.

A rollout strategy describes how the artifact is rolled out to devices in your fleet. Currently, the available rollout strategy is immediate. This strategy delivers the artifact to any device that polls without restriction.

Artifact lifecycle hooks

Aqago places no restrictions on the content of artifact payloads.

However, by formatting the artifact payload in the below format and adding the tag aqago.tar.gz in the artifact header, the Aqago agent can help you automate decompressing the payload and executing pre and post fetch lifecycle hooks. Note that this feature is currently only supported on Linux systems.

Given the following artifact payload structure,


artifact.aqago.tar.gz

- payload/
- pre/
- post/

The Aqago agent will extract the pre/ directory from the .tar.gz and execute what it contains (either read and execute shell scripts or execute any binary with the execution x bit set). The agent will then do the same for post/. pre/ and post/ hooks enable you to associate arbitrary logic to execute with your artifacts.

This mechanism simplifies payload handling logic. For example, an artifact payload containing an OS image can include the commands needed to install the image on a particular disk partition and reboot the system without requiring custom application code be present in the running image a priori.

On Linux Aqago agent fetches artifacts into the /tmp directory so that they don't survive reboots.

Monitoring

  • Device heartbeat
  • Peripheral characteristics and states
    • Ethernet and WiFi
      • Info including carrier state, PHY, etc.
    • USB
      • Info including carrier state, bus info, etc.
  • systemd service monitoring (status, other metrics)
  • systemd service lifecycle (start/stop/restart)

CI/CD

  • Up to date Infrastructure as Code (IaaC) CI/CD for all hosted services and OS image management
    • Github Actions workflows for backend, frontend
    • Github Actions for OS image generation

Security

  • 2-tier Public Key Infrastructure (PKI) utilizing asymmetric key cryptography and mTLS
  • FIPS 140-2 compliance for data at rest encrpytion
  • Data in transit encrpytion via TLS
  • Hardware security modules (HSM) used for backend instance certificate key storage

Roadmap

(non-exhaustive)

  • Add user-in-the-loop provisioning flows
  • Improve software delivery management tooling (e.g. cohorts, rollout schedules, etc.)
  • Add support for OpenWRT
  • Add MCU support
  • Improve observability (user-defined metrics, logs, etc.)
  • Integrate (optional) VPN providers for automated device SSH access (e.g. Tailscale, Twingate)
  • Add Webhooks for device lifecycle management and fleet operations

Licensing

Aqago Enterprise is source-available and BSL licensed. Aqago Enterprise is for users who intend to or are already deploying products at scale. Aqago Enterprise includes infrastructure as code to deploy Aqago into your Cloud provider and automate the use of managed service connected product building blocks.

For Enterprise licensing terms and pricing, email us at hi@aqago.com.