IncrementalEntityProvider

Home > @backstage/plugin-catalog-backend-module-incremental-ingestion > IncrementalEntityProvider

Ingest entities into the catalog in bite-sized chunks.

A Normal EntityProvider allows you to introduce entities into the processing pipeline by calling an applyMutation() on the full set of entities. However, this is not great when the number of entities that you have to keep track of is extremely large because it entails having all of them in memory at once. An IncrementalEntityProvider by contrast allows you to provide batches of entities in sequence so that you never need to have more than a few hundred in memory at a time.

Signature:

export interface IncrementalEntityProvider<TCursor, TContext> 

Properties

Property	Modifiers	Type	Description
eventHandler?		{ onEvent: (params: EventParams) => Promise<IncrementalEntityEventResult>; supportsEventTopics: () => string[]; }	(Optional) If set, the IncrementalEntityProvider will receive and respond to events. This system acts as a wrapper for the Backstage events bus, and requires the events backend to function. It does not provide its own events backend. See https://github.com/backstage/backstage/tree/master/plugins/events-backend.

Methods

Method	Description
around(burst)	Do any setup and teardown necessary in order to provide the context for fetching pages. This should always invoke burst in order to fetch the individual pages.
getProviderName()	This name must be unique between all of the entity providers operating in the catalog.
next(context, cursor)	Return a single page of entities from a specific point in the ingestion.