Remap JSON attribute fields to top-level log event fields

The remapper transform moves or copies attributes to top-level log event fields (such as message or eventTimestamp) or to tags. The remapper processes a set of predefined attributes, but you can modify this list based on your requirements.

When you use the Grepr UI to create a pipeline, a remapper that transforms well-known fields is automatically added to the pipeline.

Example transformation

For example, this event:


{
    "id": "ABCDEF",
    "timestamp": "",
    "message": "",
    "severity": "",
    "service": "",
    "attributes": {
        "syslog": {
            "severity": "10",
            "appname": "test name"
        },
        "status": "5",
        "message": "message 1",
        "timestamp": {
            "ms_since_epoch": 9001
        },
        "eventTime": " "
    }
}

Would be transformed using the default configuration to:


{
    "id": "ABCDEF",
    "timestamp": "",
    "message": "message 1",
    "severity": "5",
    "service": "test name",
    "attributes": {
        "syslog": {
            "severity": "10",
            "appname": "test name"
        },
        "status": "5",
        "timestamp": {
            "ms_since_epoch": 9001
        },
        "eventTime": " "
    }
}

severity uses status: "5" instead of syslog.severity: "10" because status has a higher priority in the default statusReservedAttributes.
Also note that syslog.appname: "test name" was still used, even though syslog.severity: "10" was skipped.
The attribute message was removed because it’s marked as removed once remapped.
If the message attribute was log.message, then message is removed, but its parent log would still exist, even if empty.
timestamp: {} and eventTime: " " are not used at all because they are not a non-blank string value.

Default configuration

The following are the default settings:

Attribute	Removed	Default names
timestamp	false	”@timestamp”, “timestamp”, “_timestamp”, “Timestamp”, “eventTime”, “date”, “published_date”, “syslog.timestamp”
host	false	”host”, “hostname”, “syslog.hostname”
service	false	”service”, “syslog.appname”, “dd.service”
status	false	”log.level”, “status”, “severity”, “level”, “syslog.severity”
message	true	”message”, “msg”, “log”

For the full list of default configuration settings, see LogAttributesRemapper.