The Log Event Model

Grepr's model for data passing through the pipeline is similar to many existing observability tools. Each log event has the following properties:

• id: A globally unique id that identifies the log event.
• receivedTimestamp: Timestamp when Grepr received the event.
• eventTimestamp: Timestamp when the event occurred, usually parsed from the event itself.
• tags: A set of key-value pairs that can be used to filter and route events. Some of the commonly used tags are host, service, environment, etc.
• attributes: Structured data and fields that are associated with the event. These could be sent with the event from the source or extracted by Grepr from the message.
• message: The log message itself.
• severity: The severity of the log message. This is an integer, following the OpenTelemetry convention of 1-4 for TRACE, 5-8 for DEBUG, 9-12 for INFO, 13-16 for WARN, 17-20 for ERROR, and 21-24 for FATAL. Severity is usually sent along with the message, but may be parsed from the event too.

An example log event in JSON would look like:

{
  "id": "0H19GZK97FTKS",
  "eventTimestamp": "2024-08-21T04:21:14.062Z",
  "receivedTimestamp": "2024-08-21T04:21:14.188Z",
  "severity": 9,
  "message": "State backend is set to heap memory",
  "tags": {
    "app": "greprdev-0gvrs39hhft9q",
    "kube_ownerref_kind": "deployment",
    "source": "grepr-query",
    "organizationId": "greprdev",
    "service": "grepr-query",
    "pod_phase": "running",
    "host": "ip-10-12-4-129.ec2.internal",
    "image_tag": "0.1.0-3421150",
    "container_id": "b743c4b78b8ec25c6c73ea03de443ebe19acf760d293dc4145bd31170768a216"
  },
  "attributes": {
    "process": {
      "thread": {
        "name": "thread-0"
      }
    },
    "@timestamp": "2024-08-21T04:21:14.062Z",
    "ecs": {
      "version": "1.2.0"
    },
    "timestamp": 1724214074114,
    "status": "info"
  }
}