Skip to Content
What is Grepr?

What is Grepr?

Grepr is an enterprise-grade data storage and analytics platform for your observability data. Grepr collects, transforms, and analyzes your log data, and then sends the processed logs to your observability platform, such as Datadog, Splunk, New Relic, and Sumo Logic.

Before sending log data to the destination platform, Grepr uses machine learning to analyze and intelligently aggregate the data. During normal operations, Grepr reduces cost by reducing the volume of data sent to your observability tools, while retaining all of your raw data in low-cost cloud storage. If an incident or anomaly occurs, Grepr increases the granularity of the forwarded data to ensure there is sufficient data available for troubleshooting, and uses the raw data to backfill any historical data relevant to the incident.

Grepr internally uses a stateful stream processing engine that enables complex real-time processing and alerting on data streams.

Pipeline summary

Core capabilities

Several capabilities work together to ensure that Grepr delivers on this promise without impacting the developer experience and your Mean Time to Resolution (MTTR):

  • Dynamic aggregation: Grepr automatically understands the patterns in your logs by using unsupervised machine learning and aggregates similar messages together with zero configuration. This capability can reduce log volumes by 80-99% out of the box. Multiple configuration options are available to tune this behavior to your needs. This aggregation results in summaries in addition to samples being sent to your existing vendor. Summaries in logs mean that engineers can see exactly what is happening in their systems without having to sift through thousands of lines of logs. You can use the summaries to then drill down into the relevant logs in the Grepr UI.

  • Raw data storage: All the original raw logs are stored in low-cost object storage (S3) for later retrieval and debugging. No data is dropped unless you explicitly configure it to be. This store could be a bucket that Grepr hosts, or a bucket that you own.

  • Raw data query: Logs are stored efficiently using Apache Parquet files and the Apache Iceberg table format, which enables them to be queried efficiently using the Grepr system or any other standard query engine like Spark or Trino. The APIs and UI allow you to query using a Lucene-based language similar to Datadog’s query language.

  • Automated granularity adjustment: When an incident occurs or when there are alerts in your infrastructure, Grepr can automatically ensure that you have a complete set of logs to debug the issue. Grepr does this by (1) temporarily increasing the granularity of related logs passing through and (2) backfilling relevant logs from the raw store. This capability can either be triggered manually or automatically based on alerts from your monitoring system or on certain matches in the log data.

  • REST APIs and UI: Grepr provides a web-based user interface that allows you to create and manage pipelines and to search and manage log data. The same capabilities are available through REST APIs, which allow you to automate your observability pipelines, build more complex pipelines, and integrate with other systems.

  • Standard observability pipeline capabilities: Grepr is built on a general-purpose stream processing engine, which enables all the standard observability pipeline capabilities like filtering, parsing, remapping, sampling, routing, etc.

  • Security and scalability: Grepr is SOC2 Type 2 and HIPAA compliant, and is built with security and scalability as top concerns. Grepr automatically scales to handle any volume of logs. Performance and health of pipelines are monitored and managed by the Grepr team, so you can rest assured that you will always have the logs you need when you need them.

Deployment options

Grepr is available as a SaaS product. For larger deployments and for customers with strict compliance requirements, Grepr also offers a Private Cloud deployment model. For both of these models, Grepr runs in AWS.

Access methods

Grepr offers two ways to interact with the platform:

Web user interface

New users should start with the intuitive web UI. It provides:

  • A streamlined, task-oriented experience for creating and managing pipelines
  • Visual tools for searching and exploring log data
  • Dashboards that visualize pipeline performance and data flows
  • Guided workflows for common observability tasks

The UI abstracts away many underlying complexities, making it easy to accomplish common tasks without deep technical knowledge of the system internals.

RESTful API

For advanced users and automation scenarios, the comprehensive RESTful API provides:

  • Complete programmatic control over all Grepr capabilities
  • The ability to integrate Grepr into your existing workflows and tools
  • Customization options for complex observability pipelines
  • Support for infrastructure-as-code and GitOps approaches

As you become more familiar with Grepr, you can gradually transition to API usage for automating repetitive tasks and creating custom integrations.

Last updated on
Overview