Search "HOW TO" guides
How to implement your own Search API
The Search plugin provides implementation of one primary API by default: the SearchApi, which is responsible for talking to the search-backend to query search results.
There may be occasions where you need to implement this API yourself, to customize it to your own needs - for example if you have your own search backend that you want to talk to. The purpose of this guide is to walk you through how to do that in two steps.
- Implement the
SearchApi
interface according to your needs.
export class SearchClient implements SearchApi {
// your implementation
}
- Override the API ref
searchApiRef
with your new implemented API in theApp.tsx
usingApiFactories
. Read more about App APIs.
const app = createApp({
apis: [
// SearchApi
createApiFactory({
api: searchApiRef,
deps: { discovery: discoveryApiRef },
factory({ discovery }) {
return new SearchClient({ discoveryApi: discovery });
},
}),
],
});
How to index TechDocs documents
The TechDocs plugin has supported integrations to Search, meaning that it provides a default collator factory ready to be used.
The purpose of this guide is to walk you through how to register the DefaultTechDocsCollatorFactory in your App, so that you can get TechDocs documents indexed.
If you have been through the
Getting Started with Search guide,
you should have the packages/backend/src/plugins/search.ts
file available. If
so, you can go ahead and follow this guide - if not, start by going through the
getting started guide.
- Import the
DefaultTechDocsCollatorFactory
from@backstage/plugin-techdocs-backend
.
import { DefaultTechDocsCollatorFactory } from '@backstage/plugin-techdocs-backend';
- If there isn't an existing schedule you'd like to run the collator on, be sure to create it first. Something like...
import { Duration } from 'luxon';
const every10MinutesSchedule = env.scheduler.createScheduledTaskRunner({
frequency: Duration.fromObject({ seconds: 600 }),
timeout: Duration.fromObject({ seconds: 900 }),
initialDelay: Duration.fromObject({ seconds: 3 }),
});
- Register the
DefaultTechDocsCollatorFactory
with the IndexBuilder.
indexBuilder.addCollator({
schedule: every10MinutesSchedule,
factory: DefaultTechDocsCollatorFactory.fromConfig(env.config, {
discovery: env.discovery,
logger: env.logger,
tokenManager: env.tokenManager,
}),
});
You should now have your TechDocs documents indexed to your search engine of choice!
If you want your users to be able to filter down to the techdocs type when
searching, you can update your SearchPage.tsx
file in
packages/app/src/components/search
by adding techdocs
to the list of values
of the SearchType
component.
<Paper className={classes.filters}>
<SearchType
values={['techdocs', 'software-catalog']}
name="type"
defaultValue="software-catalog"
/>
...
</Paper>
How to limit what can be searched in the Software Catalog
The Software Catalog includes a wealth of information about the components, systems, groups, users, and other aspects of your software ecosystem. However, you may not always want every aspect to appear when a user searches the catalog. Examples include:
- Entities of kind
Location
, which are often not useful to Backstage users. - Entities of kind
User
orGroup
, if you'd prefer that users and groups be exposed to search in a different way (or not at all).
It's possible to write your own Collator to control
exactly what's available to search, (or a Decorator
to filter things out here and there), but the DefaultCatalogCollator
that's
provided by @backstage/plugin-catalog-backend
offers some configuration too!
// packages/backend/src/plugins/search.ts
indexBuilder.addCollator({
defaultRefreshIntervalSeconds: 600,
collator: DefaultCatalogCollator.fromConfig(env.config, {
discovery: env.discovery,
tokenManager: env.tokenManager,
+ filter: {
+ kind: ['API', 'Component', 'Domain', 'Group', 'System', 'User'],
+ },
}),
});
As shown above, you can add a catalog entity filter to narrow down what catalog entities are indexed by the search engine.
How to migrate from Search Alpha to Beta
For the purposes of this guide, Search Beta version is defined as:
- Search Plugin: At least
v0.7.2
- Search Backend Plugin: At least
v0.4.6
- Search Backend Node: At least
v0.5.0
- Search Common: At least
v0.3.0
In the Beta version, the Search Platform's indexing process has been rewritten as a stream pipeline in order to improve efficiency and performance on large sets of documents.
If you've not yet extended the Search Platform with custom code, and have instead taken advantage of default collators, decorators, and search engines provided by existing plugins, the migration process is fairly straightforward:
Upgrade to at least version
0.5.0
of@backstage/plugin-search-backend-node
, as well as any backend plugins whose collators you are using (e.g. at least version0.23.0
of@backstage/plugin-catalog-backend
and/or version0.14.1
of@backstage/plugin-techdocs-backend
), as well as any search-engine specific plugin you are using (e.g. at least version0.3.0
of@backstage/plugin-search-backend-module-pg
or version0.1.0
of@backstage/plugin-search-backend-module-elasticsearch
).Then, make the following changes to your
/packages/backend/src/plugins/search.ts
file:-import { DefaultCatalogCollator } from '@backstage/plugin-catalog-backend'; -import { DefaultTechDocsCollator } from '@backstage/plugin-techdocs-backend'; +import { DefaultCatalogCollatorFactory } from '@backstage/plugin-catalog-backend'; +import { DefaultTechDocsCollatorFactory } from '@backstage/plugin-techdocs-backend'; // ... const indexBuilder = new IndexBuilder({ logger: env.logger, searchEngine }); indexBuilder.addCollator({ defaultRefreshIntervalSeconds: 600, - collator: DefaultCatalogCollator.fromConfig(env.config, { discovery: env.discovery, }), + factory: DefaultCatalogCollatorFactory.fromConfig(env.config, { discovery: env.discovery, }), }); indexBuilder.addCollator({ defaultRefreshIntervalSeconds: 600, - collator: DefaultTechDocsCollator.fromConfig(env.config, { + factory: DefaultTechDocsCollatorFactory.fromConfig(env.config, { discovery: env.discovery, logger: env.logger, }), });
Any custom collators, decorators, or search engine implementations will require minor refactoring. Continue on for details.
Rewriting alpha-style collators for beta
In alpha versions of the Backstage Search Platform, collators were classes that
implemented an execute
method which resolved an IndexableDocument
array.
In beta versions, the logic encapsulated by the aforementioned execute
method
is contained within an object-mode Readable
stream where each
object pushed onto the stream is of type IndexableDocument
. Instances of this
stream are instantiated by a factory class conforming to the
DocumentCollatorFactory
interface.
The optimal conversion strategy will vary depending on the collator's logic, but the simplest conversion can follow a process like this:
- Rename your collator class to something like
YourCollatorFactory
and update it to implementDocumentCollatorFactory
instead ofDocumentCollator
. - Update its
execute
method so that it resolvesAsyncGenerator<YourIndexableDocument>
instead ofYourIndexableDocument[]
. - Implement
DocumentCollatorFactory
'sgetCollator
method which resolves toReadable.from(this.execute())
(which is a utility for creating readable streams from async generators).
import { DocumentCollatorFactory } from '@backstage/plugin-search-backend-node';
import { Readable } from 'stream';
export class YourCollatorFactory implements DocumentCollatorFactory {
public readonly type: string = 'your-type';
async *execute(): AsyncGenerator<YourIndexableDocument> {
const widgets = await this.client.getWidgets();
for (const widget of widgets) {
yield {
title: widget.name,
location: widget.url,
text: widget.description,
};
}
}
async getCollator() {
return Readable.from(this.execute());
}
}
Note: it may be possible to simplify your collator dramatically! If your custom collator was previously using streams under the hood (for example, by reading newline delimited JSON from a local or remote file), you could just expose the stream directly via a simple factory class:
import { DocumentCollatorFactory } from '@backstage/plugin-search-backend-node';
import { createReadStream } from 'fs';
import { parse } from '@jsonlines/core';
export class YourCollatorFactory implements DocumentCollatorFactory {
public readonly type: string = 'your-type';
async getCollator() {
const parseStream = parse();
return createReadStream('./documents.ndjson').pipe(parseStream);
}
}
Rewriting alpha-style decorators for beta
In alpha versions of the Backstage Search Platform, decorators were classes that
implemented an execute
method which took an IndexableDocument
array as an
argument, and resolved a modified array of the same type.
In beta versions, the logic encapsulated by the aforementioned execute
method
is contained within an object-mode Transform
stream which reads objects of
type IndexableDocument
, and writes objects of a conforming type. Similar to
collators, instances of this stream are instantiated by a factory class
conforming to the DocumentDecoratorFactory
interface.
Although you can choose to implement a Transform
stream from scratch, the
@backstage/plugin-search-backend-node
package provides a DecoratorBase
class
in order to simplify the developer experience. With this base class, all that's
needed is to transfer your old decorator class logic into the base class' three
methods (initialize
, decorate
, and finalize
), and implement the factory
class that instantiates the stream:
import { DecoratorBase } from '@backstage/plugin-search-backend-node';
export class YourDecorator extends DecoratorBase {
async initialize() {
// Setup logic. Performed once before any documents are consumed.
}
async decorate(
document: YourIndexableDocument,
): Promise<YourIndexableDocument | YourIndexableDocument[] | undefined> {
// Perform transformation logic here.
return document;
}
async finalize() {
// Teardown logic. Performed once after all documents have been consumed.
}
}
export class YourDecoratorFactory implements DocumentDecoratorFactory {
async getDecorator() {
return new YourDecorator();
}
}
Note the return type of the decorate
method and how each can be used to
different effect.
By resolving a single
YourIndexableDocument
object, your decorator can be used to make simple transformations:class BooleanWidgetCoolnessDecorator extends DecoratorBase { async decorator(widget) { // Perform a simple, 1:1 transformation. widget.isCool = widget.isCool === 'true' ? true : false; return widget; } }
By resolving
undefined
, your decorator can filter out documents which shouldn't be in the index:class OnlyCoolWidgetsDecorator extends DecoratorBase { async decorator(widget) { // Perform a simple filter operation. return widget.isCool ? widget : undefined; } }
By resolving an array of
YourIndexableDocument
objects, you can generate multiple documents based on the content of one:class WidgetByVariantDecorator extends DecoratorBase { async decorator(widget) { // Generate one widget doc per widget variant. return widget.variants.map(variant => { // Each widget doc is the given widget plus a "variant" property // pulled from a widget.variants string array. return { ...widget, variant, }; }); } }
In alpha versions, a decorator had access to every IndexableDocument
simultaneously. This is no longer possible in beta versions (precisely to make
the indexing process more efficient and performant). You will need to modify
your decorator's logic so that it does not need access to every document at
once.
Rewriting alpha-style search engines for beta
Search Engines are responsible for both querying and indexing documents to an underlying search engine technology. While the search engine query interface didn't change between alpha and beta versions, the indexing half of the interface did change.
In alpha versions of the Backstage Search Platform, a search engine implemented
an index
method which took a type
and an IndexableDocument
array and was
responsible for writing these documents to the underlying search engine.
In beta versions, the logic encapsulated by the aforementioned index
method is
contained within an object-mode Writable
stream which expects objects of type
IndexableDocument
. On the search engine class itself, the index
method is
replaced with a getIndexer
factory method which still takes the type
, but
resolves an instance of the aforementioned Writable
stream.
Although you can choose to implement a Writable
stream from scratch, the
@backstage/plugin-search-backend-node
package provides a
BatchSearchEngineIndexer
class in order to simplify the developer experience.
With this base class, which collects documents in batches of a configurable size
on your behalf, all that's needed is to transfer your old index
method logic
into the base class' three methods (initialize
, index
, and finalize
), and
implement the factory method that instantiates the stream:
import { BatchSearchEngineIndexer } from '@backstage/plugin-search-backend-node';
import { SearchEngine } from '@backstage/plugin-search-common';
export class YourSearchEngineIndexer extends BatchSearchEngineIndexer {
constructor({ type }: { type: string }) {
// Customize the number of documents passed to the index method per batch.
super({ batchSize: 500 });
// An imaginary search engine indexing client.
this.index = new SomeSearchEngineIndex({ indexName: type });
}
async initialize() {
// Setup logic. Performed once before any documents are consumed.
}
async index(documents: IndexableDocument[]) {
await this.index.batchOf(documents);
}
async finalize() {
// Teardown logic. Performed once after all documents have been consumed.
}
}
export class YourSearchEngine implements SearchEngine {
async getIndexer(type: string) {
return new YourSearchEngineIndexer({ type });
}
}
How to customize search results highlighting styling
The default highlighting styling for matched terms in search results is your
browsers default styles for the <mark>
HTML tag. If you want to customize
how highlighted terms look you can follow Backstage's guide on how to
Customize the look-and-feel of your App
to create an override with your preferred styling.
For example, the following will result in highlighted terms to be bold & underlined:
const highlightOverride = {
BackstageHighlightedSearchResultText: {
highlight: {
color: 'inherit',
backgroundColor: 'inherit',
fontWeight: 'bold',
textDecoration: 'underline',
},
},
};