ADR002: Default Software Catalog File Format

Background

Backstage comes with software catalog functionality that you can use to track all your software components and more. It can be powered by data from various sources, and one of them that is included with the package is a custom database backed catalog. It has the ability to keep itself updated automatically based on the contents of little descriptor files in your version control system of choice. Developers create these files and maintain them side by side with their code, and the catalog system reacts accordingly.

This ADR describes the default format of these descriptor files.

Inspiration

Internally at Spotify, a homegrown software catalog system is used heavily and forms a core part of Backstage and other important pieces of the infrastructure. The user experience, learnings, and certain pieces of metadata from that catalog are being carried over to the open source effort.

The file format described herein also draws heavy inspiration from the kubernetes object format.

Core Concepts

There are a number of descriptor files, all of whose locations (e.g., within a version control system) are registered with the software catalog. The method of registration is not covered in this document; it could happen either manually inside Backstage, or by push events from a CI/CD pipelines, or by webhook triggers from the version control system, etc.

Each file describes one or more entities in accordance with the Backstage System Model. All of these entities have a common structure and nomenclature, and they are stored in the software catalog from which they then can be queried.

Entities have distinct names, and they may reference each other by those names.

Format

Descriptor files use the YAML format. They may be written by hand or created using automated tools. Each file may consist of several YAML documents (separated by ---), where each document describes a single entity.

This is an example entity definition with some mocked data.

---
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: frobs-awesome
  description: |
    Backend service that implements the Frobs API, as defined
    in [the Frobs RFC](https://example.com/spec/frob.html).
  labels:
    system: frobs
    lifecycle: production
    example.com/service-discovery-name: frobsawesome
  annotations:
    circleci.com/project-slug: github/example-org/frobs-awesome
spec:
  type: service

The root fields apiVersion, kind, metadata, and spec are part of the envelope, defining the overall structure of all kinds of entities. Likewise, the name, namespace, labels, and annotations metadata fields are of special significance and have reserved purposes and distinct shapes.

See below for details about these fields.

Envelope

The root envelope object has the following structure.

apiVersion and kind

The kind is the high level entity type being described, typically from the Backstage system model. The first versions of the catalog will focus on the Component kind.

The apiVersion is the version of the specification format for that particular entity that this file is written against. The version is used for being able to evolve the format, and the tuple of apiVersion and kind should be enough for a parser to know how to interpret the contents of the rest of the document.

Backstage specific entities have an apiVersion that is prefixed with backstage.io/, to distinguish them from other types of objects that share the same type of structure. This may be relevant when co-hosting these specifications with, e.g., Kubernetes object manifests.

Early versions of the catalog will be using alpha/beta versions, e.g., backstage.io/v1alpha1 to signal that the format may still change. After that, we will be using backstage.io/v1 and up.

metadata

A structure that contains metadata about the entity, i.e., things that aren't directly part of the entity specification itself. See below for more details about this structure.

spec

The actual specification data that describes the entity.

The precise structure of the spec depends on the apiVersion and kind combination, and some kinds may not even have a spec at all. See further down in this document for the specification structure of specific kinds.

Metadata

The metadata root field has the following nested structure.

name

The name of the entity. This name is both meant for human eyes to recognize the entity, and for machines and other components to reference the entity (e.g. in URLs or from other entity specification files).

Names must be unique per kind, within a given namespace (if specified), at any point in time. This uniqueness constraint is also case insensitive. Names may be reused at a later time, after an entity is deleted from the registry.

Names are required to follow a certain format. Entities that do not follow those rules will not be accepted for registration in the catalog. The ruleset is configurable to fit your organization's needs, but the default behavior is as follows.

• Strings of length at least 1, and at most 63
• Must consist of sequences of [a-z0-9A-Z] possibly separated by one of [-_.]

Example: visits-tracking-service, CircleciBuildsDs_avro_gcs

namespace

The name of a namespace that the entity belongs to. This field is optional and currently has no special semantics apart from bounding the name uniqueness constraint if specified. It is reserved for future use and may get broader semantic implication.

Namespaces may also be part of the catalog and are v1 / Namespace entities, i.e., not Backstage specific but the same as in Kubernetes.

description

A human readable description of the entity to be shown in Backstage. Should be kept short and informative, suitable to give an overview of the entity's purpose at a glance. More detailed explanations and documentation should be placed elsewhere.

labels

Labels are optional key/value pairs that are attached to the entity, and their use is identical to kubernetes object labels.

Their main purpose is for references to other entities and for information that is in one way or another classifying for the current entity. They are often used as values in queries or filters.

Both the key and the value are strings, subject to the following restrictions.

Keys have an optional prefix followed by a slash, and then the name part, which is required. The prefix must be a valid lowercase domain name, at most 253 characters in total. The name part must be sequences of [a-zA-Z0-9] separated by any of [-_.], at most 63 characters in total.

The backstage.io/ prefix is reserved for use by Backstage core components. Some keys, such as system also have predefined semantics.

Values are strings that follow the same restrictions as name above.

annotations

An object with arbitrary non-identifying metadata attached to the entity, identical in use to kubernetes object annotations.

Their purpose is mainly, but not limited, to reference into external systems. This could, for example, be a reference to the git ref the entity was ingested from, to monitoring and logging systems, to pagerduty schedules, etc.

Both the key and the value are strings, subject to the following restrictions.

Keys have an optional prefix followed by a slash, and then the name part, which is required. The prefix must be a valid lowercase domain name, at most 253 characters in total. The name part must be sequences of [a-zA-Z0-9] separated by any of [-_.], at most 63 characters in total.

The backstage.io/ prefix is reserved for use by Backstage core components.

Values can be of any length but are limited to being strings.

Component

Field	Value
apiVersion	backstage.io/v1alpha1
kind	Component

The spec object for this kind is as follows:

Field	Type	Required	Description
type	String	Yes	The type of component, e.g. service.