Skip to Content
Process and transform dataStandardize messages with the remapper

Remap JSON attribute fields to top-level log event fields

The remapper transform moves or copies attributes to top-level log event fields (such as message or eventTimestamp) or to tags. The remapper processes a set of predefined parameters, but you can modify this list based on your requirements.

When you use the Grepr UI to create a pipeline, a remapper that transforms well-known fields is automatically added to the pipeline.

Example transformation

For example, this event:

{ "id": "ABCDEF", "eventTimestamp": 1765481731051, "message": "", "severity": "", "tags": {}, "attributes": { "syslog": { "severity": "10", "appname": "test name" }, "status": "5", "msg": "message 1", "timestamp": { "ms_since_epoch": 1765481730000 }, "eventTime": " " } }

Would be transformed using the default configuration to:

{ "id": "ABCDEF", "eventTimestamp": 1765481731051, "message": "message 1", "severity": "5", "tags": { "service": "test name" }, "attributes": { "syslog": { "severity": "10", "appname": "test name" }, "status": "5", "timestamp": { "ms_since_epoch": 9001 }, "eventTime": " ", "greprMeta": { "messageAttributePath": "msg" } } }
  • severity uses status: "5" instead of syslog.severity: "10" because status has a higher priority in the default configuration.
  • tags.service uses syslog.appname: "test name".
  • The attribute msg was removed because it was remapped to the top-level message field.
    • The greprMeta.messageAttributePath attribute was added to indicate which attribute was used for the remapped message field.
    • If the message attribute is a nested field (e.g., log.msg), only the msg field is removed. The parent log object remains, even if it is empty.
  • timestamp: {} and eventTime: " " are not used because they do not contain a valid timestamp value.

Default configuration

The following are the default settings:

AttributeRemovedDefault names
timestampfalsedate, @timestamp, T, syslog.timestamp, eventTime, _timestamp, Timestamp, published_date, timestamp
hostfalsehostname, sourceIPAddress, host, kubernetes.host, syslog.hostname
servicefalsedd.service, kubernetes.labels.app, service, eventSource, kubernetes.labels.k8s-app, syslog.appname
statusfalseseverity, level, syslog.severity, log.level, L, status
messagetruemsg, log, message, M
trace IDfalsedd.trace_id, contextMap.dd.trace_id, otelTraceId

For the full list of default configuration settings, see LogAttributesRemapper.

Last updated on