The AWS S3 integration has a special discovery processor for discovering catalog entities located in an S3 Bucket. If you have a bucket that contains multiple catalog-info files and want to automatically discover them, you can use this processor. The processor will crawl your S3 bucket and register entities matching the configured path. This can be useful as an alternative to static locations or manually adding things to the catalog.
To use the discovery processor, you'll need an AWS S3 integration
set up with an
roleArn. Then you can add a location target to the catalog
catalog: locations: - type: s3-discovery target: https://sample-bucket.s3.us-east-2.amazonaws.com/
s3-discovery type, as this is not a regular