telemetryapi receiver: function log records not fully parsed (severity, custom fields, and body extraction)

从 receiver.go 开始，跟踪纯文本、封装 JSON 和原始 JSON Telemetry API 记录如何转换为日志记录。复现四种 Node.js 日志记录情况，然后将 receiver 输出与这里描述的预期 body、severity、timestamp、invocation ID 和自定义属性进行比较；完成内容还包括提议的可配置 JSON 字段映射。

Describe the bug

The telemetryapireceiver does not fully parse function log records. Severity, timestamp, request ID, and application-defined custom fields are silently dropped depending on the log format and how the application emits logs. In the worst case (raw JSON written to stdout with process.stdout.write), the entire log body disappears — the record is emitted with no body and no attributes.

Steps to reproduce

Deploy a Node.js Lambda function with the following handler:

export const handler = async () => {
    console.log("Hello World")                                                                                       // case 1
    console.log(JSON.stringify({msg: "JSON stringified message", userId: "user-123", action: "login", durationMs: 42}))  // case 2
    process.stdout.write("Plain text written directly to stdout" + "\n")                                            // case 3
    process.stdout.write(JSON.stringify({msg: "JSON stringified text written directly to stdout", userId: "user-123", action: "login", durationMs: 42}) + "\n") // case 4
    await new Promise(r => setTimeout(r, 1000));
}

Invoke the function and observe the log records exported by the telemetry API receiver.

What did you expect to see?

All four cases should produce consistent, well-structured OTel log records — independent of how the application logged the data — with body containing the message text, severityNumber populated from the level, faas.invocation_id from the request ID, and any custom fields (e.g. userId, action, durationMs) preserved as log attributes.

What did you see instead?

Plain text log format (default) — all cases broken

#	body	severityNumber	faas.invocation_id
1	"2026-03-15T...\tREQ_ID\tINFO\tHello World\n" (raw prefix embedded)	0 ❌	absent ❌
2	"2026-03-15T...\tREQ_ID\tINFO\t{\"msg\":\"...\",\"userId\":\"user-123\",...}\n" (JSON unparsed)	0 ❌	absent ❌
3	"Plain text written directly to stdout\n"	0 ❌	absent ❌
4	"{\"msg\":\"...\",\"userId\":\"user-123\",...}\n" (JSON unparsed, custom fields unreachable)	0 ❌	absent ❌

JSON log format (AWS_LAMBDA_LOG_FORMAT=JSON) — case 4 is completely empty

#	body	severityNumber	faas.invocation_id
1	"Hello World" ✅	9 (Info) ✅	set ✅
2	"{\"msg\":\"...\",\"userId\":\"user-123\",...}" (JSON unparsed, custom fields dropped) ⚠️	9 (Info) ✅	set ✅
3	"Plain text written directly to stdout\n" ✅	9 (Info) ✅	set ✅
4	(no body at all — completely empty record) ❌❌	0 ❌	absent ❌

Case 4 with JSON log format is the worst case: Lambda detects the raw {...} stdout write as a structured record and skips wrapping. The receiver looks for a "message" key, finds none ("msg" is used instead), and emits a completely empty log record with all fields silently dropped.

Root cause (receiver.go):

// plain text: verbatim storage, Lambda prefix never parsed
if line, ok := el.Record.(string); ok {
    logRecord.Body().SetStr(line)  // timestamp+requestId+level+message all in one string
}

// JSON format: only hardcoded "message" key; "msg" or any other key → empty record
} else if msg, ok := record["message"]; ok {
    ...
}
// no fallback for other keys → completely empty log record for case 4

What version of collector/language SDK version did you use?

Version: latest (main)

What language layer did you use?

Node.js, but the issue exists at the collector layer and affects all runtimes.

Additional context

Key insight: the message is always a JSON-stringified string

The actual message content is always a JSON-stringified string in the Telemetry API payload. Lambda's record wrapper (JSON log format) is the only structure provided:

• Plain text format: record = "TIMESTAMP\tREQUEST_ID\tLEVEL\t<message as string>"
• JSON log format: record.message contains the JSON-stringified string (or plain string)
• Raw JSON stdout write: record is the JSON object directly — no wrapper, no message key

Expected output: plain text log format

#	body	severityNumber	faas.invocation_id	attributes
1	"Hello World"	9 (Info)	set	{}
2	"JSON stringified message"	9 (Info)	set	{userId, action, durationMs}
3	"Plain text written directly to stdout"	0 (no level)	set	{}
4	"JSON stringified text written directly to stdout"	0 (no level)	set	{userId, action, durationMs}

Case 2 expected OTel:

{
  "timeUnixNano": 1773606626605000000,
  "severityNumber": 9,
  "severityText": "Info",
  "body": "JSON stringified message",
  "attributes": {
    "type": "function",
    "faas.invocation_id": "6fed457f-f0d2-4c3e-b912-11e5820f74c5",
    "userId": "user-123",
    "action": "login",
    "durationMs": 42
  }
}

Case 4 expected OTel:

{
  "timeUnixNano": 1773606626606000000,
  "severityNumber": 0,
  "severityText": "",
  "body": "JSON stringified text written directly to stdout",
  "attributes": {
    "type": "function",
    "faas.invocation_id": "6fed457f-f0d2-4c3e-b912-11e5820f74c5",
    "userId": "user-123",
    "action": "login",
    "durationMs": 42
  }
}

Expected output: JSON log format (AWS_LAMBDA_LOG_FORMAT=JSON)

#	body	severityNumber	faas.invocation_id	attributes
1	"Hello World"	9 (Info)	set	{}
2	"JSON stringified message"	9 (Info)	set	{userId, action, durationMs}
3	"Plain text written directly to stdout"	9 (Info)	set	{}
4	"JSON stringified text written directly to stdout"	9 (Info)	set	{userId, action, durationMs}

Case 4 expected OTel (raw JSON record, no wrapper):

{
  "timeUnixNano": 1773606987430000000,
  "severityNumber": 9,
  "severityText": "Info",
  "body": "JSON stringified text written directly to stdout",
  "attributes": {
    "type": "function",
    "userId": "user-123",
    "action": "login",
    "durationMs": 42
  }
}

Proposed fix

1. Parse Lambda prefix for plain text records

For plain text records, parse the tab-delimited prefix:

"2026-03-15T20:30:26.603Z\t6fed457f-...\tINFO\tHello World\n"
 └─ timeUnixNano ──────────┘ └─ faas.invocation_id ─┘ └─ level ─┘ └─ message ─┘

1. Split on \t — if 4 parts match TIMESTAMP\tREQUEST_ID\tLEVEL\tMESSAGE, extract each field
2. TIMESTAMP → timeUnixNano; REQUEST_ID → faas.invocation_id; LEVEL → severityNumber
3. MESSAGE → attempt JSON parse, then apply field mapping (see below)

2. Add json_message_parsing config block

When the message (or the record itself) is a JSON object, apply configurable field mapping. Remaining fields become log attributes.

receivers:
  telemetryapi:
    json_message_parsing:
      body_fields:        [message, msg, text, content]    # first match → log body
      severity_fields:    [level, severity, lvl]           # first match → severityNumber
      timestamp_fields:   [timestamp, time, ts]            # first match → timeUnixNano
      trace_id_fields:    [traceId, trace_id]              # first match → traceId
      span_id_fields:     [spanId, span_id]                # first match → spanId
      trace_flags_fields: [traceFlags, trace_flags, flags] # first match → flags

3. Extraction algorithm

Plain text format:

1. Detect and parse Lambda prefix (TIMESTAMP\tREQUEST_ID\tLEVEL\tMESSAGE)
2. Extract timeUnixNano, faas.invocation_id, severityNumber from prefix fields
3. Treat everything after the third tab as the message string
4. If the message is valid JSON, parse and apply field mapping; otherwise use as-is as body

JSON log format:

1. Extract timestamp, level, requestId, message from the Lambda record wrapper
2. For the message value: if it's a string that parses as JSON, apply field mapping; otherwise use as-is
3. Special case — raw JSON record (no wrapper): apply field mapping directly to the top-level record object

Field mapping for JSON objects:

1. For each semantic field, iterate configured names in order and use first match
2. Convert values:
   • body: string as-is; non-string → marshal to JSON string
   • severity: parse to OTel severity number ("info" → 9, "warn" → 13, "error" → 17, etc.)
   • timestamp: parse ISO 8601 → nanoseconds; fall back to observed time
   • trace_id / span_id: use as-is (hex string)
   • trace_flags: parse hex byte → flags field
3. All remaining fields → log record attributes

_{Tip: React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}