Skip to Content
Sources and SinksOverview

Sources and sinks

Grepr supports a wide range of data sources and sinks, with new integrations continually being added. For a complete list of supported platforms, see the support matrix.

An integration provides connection details for a particular service. The actual data movement happens through Grepr jobs, which use these integrations as sources (data inputs) or sinks (data outputs). Sources and sinks function as specific operators within a Grepr job’s processing pipeline.

Integration types

Grepr integrations fall into two primary categories:

External platform integrations: These connect Grepr with the observability platforms that produce or consume your data. Examples include Datadog, Splunk, New Relic, and OpenTelemetry. These integrations enable bidirectional data flow between Grepr and your existing observability tools.

Storage integrations: These manage where Grepr stores your raw data and metadata. Options include Grepr-hosted storage (for simplicity) or customer-managed storage (for data sovereignty and compliance). All storage uses efficient formats like Apache Parquet and Apache Iceberg for optimal performance and cost.

External platform capabilities

For each external integration, Grepr supports several capabilities depending on the vendor:

  • Agent source: enables the vendor’s agents or collectors to send data to Grepr
  • Cloud source: enables Grepr to read data from the vendor service and process it
  • Query language: enables you to query the Grepr data lake using the vendor’s query language
  • Exception parsing: enables Grepr to parse alerts and dashboards configured in the vendor and automatically add exceptions for them in the Grepr log reducer so as not to impact them
  • Grepr drops incoming data from external integration sources with timestamps older than 48 hours.
  • Grepr may split log events over 1MB, causing them to lose their original structured format. Each split event includes a common grepr.splitOriginId if you need to reconstruct the original message.

Storage integration capabilities

Storage integrations allow Grepr to read from and write to various storage locations. Grepr can store data in either Grepr-hosted storage or in a customer-provided storage location.

For customer-provided storage, you need to give Grepr access permissions. Details are described in each integration’s detail page.

Available integrations

The following integrations are available for connecting Grepr to external systems and storage platforms.

See Integrations.

Last updated on