Agent Control (AC) manages agent packages (and in the future agent types) distributed as OCI (Open Container Initiative) artifacts. The package manager handles download, signature verification, extraction, installation and garbage collection of packages from OCI registries.
The packaged agent must comply with the OCI image spec. The entrypoint can either
be a manifest JSON file or an index JSON file, but for AC we expect the entrypoint to be an index. That’s the way OCI supports multi-arch.
[!NOTE] The OCI spec has some rough edges, at least around the multi-arch topic.
The index file includes a list of manifest files. AC expects:
each manifest entry in the array to have a platform section with the architecture and os fields
Check fields and possible values in the Image Index Property Descriptions section of the spec. However, to know if AC supports a specific architecture and os, we must look at the oci_client crate support and the AC compatibility section. If it appears on both, then it’s supported.
The section is automatically handled by the oci_client, which automatically detects the platform and retrieves the correct manifest from the index.
[!NOTE] The
artifactTypefield in a given manifest entry of theindexis not verified by AC.
Other than that, it must follow the OCI spec.
Example:
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:82677ba32d1276debe264d14ec5f7b1c61e2a9acbc8c6a6dff779d7133ec8487",
"size": 617,
"platform": {
"architecture": "amd64",
"os": "linux"
},
"artifactType": "application/vnd.newrelic.agent.v1"
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:5a16021a5101f7ae0583cddae44ea715ad2cfd618b61b8982de1b847958260da",
"size": 617,
"platform": {
"architecture": "arm64",
"os": "linux"
},
"artifactType": "application/vnd.newrelic.agent.v1"
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:13e6d06647bbaf4f44d4c29bb57e1078c9919da92e2aee3443c122c24b86d3cb",
"size": 502,
"platform": {
"architecture": "amd64",
"os": "windows"
},
"artifactType": "application/vnd.newrelic.agent.v1"
}
]
}
Then we have the manifest file, which includes the package data and metadata.
[!NOTE] AC can handle artifacts regardless of the entrypoint (manifest or index). However, AC suggests using the index approach. If the author of an agent decides to use a manifest as the entrypoint, they must understand that AC might not behave as expected. The referred artifact will be used in any platform.
There, AC expects to find some specific keys and values.
config must exist, have value application/vnd.oci.image.config.v1+json and contain the architecture and os properties
{
"architecture": "amd64",
"os": "linux"
}
This is not required by AC itself. However, it adds support for external tooling. For example, oras.
artifactType must exist and have value application/vnd.newrelic.agent.v1
layers/mediaType must take one of the following values:
application/vnd.newrelic.agent.content.v1.zipapplication/vnd.newrelic.agent.content.v1.tar+gzipapplication/vnd.newrelic.agent-type.content.v1.tar+gzipExample:
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"artifactType": "application/vnd.newrelic.agent.v1",
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:7758599fc4d06bd93a65bf28bc98fbff6c559a9a56be1ec3d75ff6aa8a8cfe6e",
"size": 39
},
"layers": [
{
"mediaType": "application/vnd.newrelic.agent.content.v1.zip",
"digest": "sha256:2e2e87f3a9403e735bee76c166b7139be36c1a76079f786e21ab2ce138cd9a1a",
"size": 21678636,
"annotations": {
"com.newrelic.artifact.type": "package",
"org.opencontainers.image.title": "newrelic-infra-amd64.zip",
"org.opencontainers.image.version": "1.71.3"
}
}
],
"annotations": {
"org.opencontainers.image.created": "2026-01-23T08:07:06Z"
}
}
Package references are constructed from the data configured in the AgentType packages section.
When an agent needs to install or update a package, the package manager leverages the following paths:
temp_package_path: <base>/packages/<agent-id>/__temp_packages/<package-id>/<sanitized-ref>
final_path: <base>/packages/<agent-id>/stored_packages/<package-id>/<sanitized-ref>
The final_path location is where the extracted package will reside after installation and can be referenced
by the agent through the variable ${nr-sub:packages.infra-agent.dir}.
Steps:
PackageType (tar.gz or zip) derived from the MIME typeThe whole operation blocks the sub-agent thread until it terminates. Notice that the old sub-agent (and therefore the binary) is stopped before the new one is downloaded and executed. In the next iterations, we will have a non-blocking implementation to avoid the sub-agent being blocked by this operation.
There is no installation step or script execution, just extraction. We expect to support installation scripts in the future. TODO: not implemented yet
AC supports signature verifications. This assures users that a given agent was uploaded by the expected author and it hasn’t been tampered.
The signature in the OCI repository follows the Simple Signing format (the only supported format) and it’s been created with the external tool process.
[!NOTE] New Relic uses an owned repository. It doesn’t need extra-services like Rekor or Fulcio. That’s the reason why Agent Control uses the external tool process instead of
cosign sign.
As a result of the “external tool process”, the OCI repository will contain two packages. One for the agent and one for the signature. The signature package contains, among other things, the payload that was signed (in json format) and it’s signature in base64. Inside the payload, we find the hash of the signed agent package. This is enough to verify the signature of an artifact, as we will see later.
Public keys are ALWAYS downloaded when verifying a signature. This avoids problems with outdated caches, like using a revoked key. Public keys MUST be in JWKS format.
{
"keys": [
{
"kty": "OKP",
"use": "sig",
"kid": "key/0",
"x": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"crv": "Ed25519"
},
{
"kty": "OKP",
"use": "sig",
"kid": "key/1",
"x": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"crv": "Ed25519"
}
]
}
Verification flow:
sequenceDiagram
participant AC as Agent Control
participant OCI as OCI Repository
participant JWKS as PublicKey server
Note over AC, JWKS: Step 1: Check artifact signature
AC->>JWKS: Download public keys
JWKS-->>AC: Public keys in JWKS format
AC->>OCI: Get artifact package digest
OCI-->>AC: Artifact package digest
AC->>OCI: Download Signature Package
OCI-->>AC: Base64 Signature & Payload
AC->>AC: Verify base64 signature against the payload (holds for at least one public key)
Note over AC, OCI: Step 2: Download signed artifact
AC->>OCI: Download artifact given the package digest
OCI-->>AC: Artifact
Note over AC: Step 3: Integrity Check
AC->>AC: Compute downloaded artifact hash
AC->>AC: Compare computed hash against the hash inside the payload
Note over AC: Result: Package Verified & Trusted
What happens during a key rotation? It depends on the specific use case. AC tries to verify every signature in the signature-manifest layers with every public key published for that package, this approach avoids downtimes on key rotation. Note that a non-revoked key must exist on the list, otherwise signature verification will fail.
An OCI registry mirror is a copy of the New Relic registry where agent packages are stored. Its support is essential for:
Agent Control supports registry mirrors by setting them up in the corresponding configuration section.
Considering the key rotation timeline:
New Key is Added: When New Relic introduces a new signing key (Key B), the public part of it is added to the JWKS file. The old key (Key A) is not immediately removed. The JWKS now contains both Key A and Key B.
New signatures are created: All agent packages are signed with Key B resulting in new artifacts.
In order to ensure that Agent Control can always use packages from your mirror, even after a key rotation, your mirroring infrastructure must satisfy two fundamental requirements at the moment of verification.
Availability of the public keys: the public key urls need to be reachable from the host where Agent Control is installed.
Your mirror must store and serve the original, unaltered agent package and its corresponding signature artifact as they were published by New Relic.
If these requirements cannot be met, it is possible (although not recommended) to disable signature verification for Agent Control through the corresponding configuration field.
AC keeps track of the latest installed package. Each install operation executes an old package purge operation, which retains the latest tracked package (i.e. package currently running) and the new installed package. You can think of it like a FIFO with size 2.
On restart, AC only retains the current package. The previous package will be deleted.
Example:
Installation Failures:
For signature verification, we only fail when BOTH signature verification is enabled (check the Agent Control Configuration agent_packages section) AND a public key is configured (check the example in AgentType packages section), and the signature doesn’t match. If signature verification is disabled OR no public key is configured, we skip verification and log a warning.
If needed, you can run a local OCI registry using zot with
$ ./tools/oci-registry.sh run
This is needed to execute tests with ignore “needs oci registry” and having suffix ‘with_oci_registry’.
Notice that AC is already configured to use HTTP as protocol when connecting to localhost:5001 if executed/built without --release.
TODO not implemented yet