LAL: introduce LogMetadata to decouple metadata from LogData in pipeline entry#13742
Merged
LAL: introduce LogMetadata to decouple metadata from LogData in pipeline entry#13742
Conversation
…ine entry Separate metadata (service, layer, timestamp, trace context) from input data (LogData body/tags, typed protos) in the LAL pipeline. Receivers now construct a LogMetadata POJO instead of relying on LogData for metadata fields, and Envoy ALS no longer needs a fake LogData. The ExecutionContext carries metadata via ctx.metadata() and the raw input via ctx.input(), removing extraLog() indirection and LogData type awareness from runtime. The test tool (LogTestQuery) is separated from production sink via dryRun flag instead of capture/logContainer mixing.
There was a problem hiding this comment.
Pull request overview
This PR refactors the LAL v2 log analysis entry contract by introducing LogMetadata as a uniform metadata carrier and decoupling it from the raw input object (LogData.Builder for standard logs vs typed protos like Envoy ALS). It updates receivers, runtime execution context, compiler/runtime helpers, and sink/listener plumbing to consume (LogMetadata, Object) rather than (LogData, Optional extraLog).
Changes:
- Introduce
LogMetadata+LogMetadataUtilsand route receivers/fetchers to pass metadata + input object into the LAL analyzer. - Update LAL v2 runtime APIs (
ExecutionContext, listeners, output builders, analyzer service) to usemetadata()+input()and adddryRunsupport for the log test tool. - Update Envoy ALS persistence/tests to stop fabricating
LogDataand pass typed ALS protos directly as the LAL input.
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/script-cases/script-runtime-with-groovy/mal-lal-v1-v2-checker/src/test/java/org/apache/skywalking/oap/server/checker/lal/LalComparisonTest.java | Adapts v2 test harness to init contexts with (LogMetadata, input) and compare against v1. |
| test/script-cases/script-runtime-with-groovy/mal-lal-v1-v2-checker/src/test/java/org/apache/skywalking/oap/server/checker/lal/LalBenchmark.java | Updates v2 benchmark execution to use (LogMetadata, input) context initialization. |
| oap-server/server-receiver-plugin/skywalking-log-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/log/provider/handler/rest/LogReportServiceHTTPHandler.java | HTTP log receiver now extracts metadata and passes LogData.Builder as input. |
| oap-server/server-receiver-plugin/skywalking-log-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/log/provider/handler/grpc/LogReportServiceGrpcHandler.java | gRPC log receiver now extracts metadata and passes LogData.Builder as input. |
| oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryLogHandler.java | OTLP log handler now builds LogData.Builder, extracts metadata, and calls analyzer with (metadata, builder). |
| oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/persistence/EnvoyAlsLalTest.java | Envoy ALS LAL tests now pass LogMetadata + ALS proto directly into the context. |
| oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/persistence/EnvoyAccessLogBuilderTest.java | Builder tests updated to init via (LogMetadata, HTTPAccessLogEntry) and execute LAL with typed input. |
| oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/persistence/TCPLogsPersistence.java | Envoy TCP persistence now creates LogMetadata and passes the ALS metric proto as input. |
| oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/persistence/LogsPersistence.java | Envoy HTTP persistence now creates LogMetadata and passes HTTPAccessLogEntry as input. |
| oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/persistence/EnvoyAccessLogBuilder.java | Output builder init updated to consume metadata + input and populate from metadata only. |
| oap-server/server-query-plugin/query-graphql-plugin/src/main/java/org/apache/skywalking/oap/query/graphql/resolver/LogTestQuery.java | Log test query now uses dryRun and post-evaluation init(metadata, input) to build a Log from output builder. |
| oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/handler/LogHandler.java | Kafka log fetcher now extracts metadata and passes LogData.Builder as input. |
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/LogMetadataUtils.java | Adds utility for extracting LogMetadata from LogData/LogData.Builder. |
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/LogMetadata.java | Adds the new LogMetadata POJO (including trace context). |
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/LogBuilder.java | Updates output builder initialization and makes LogData body/tags optional based on input type. |
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/LALOutputBuilder.java | Updates output builder contract to init(LogMetadata, Object, ModuleManager). |
| oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALExpressionExecutionTest.java | Updates compiler/runtime tests to init context via (metadata, input) and assert against ctx.metadata() / ctx.input(). |
| oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALClassGeneratorExtractorTest.java | Updates codegen expectations to use ctx.input() and metadata fallback. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/spi/LALSourceTypeProvider.java | Updates SPI docs to reference new input contract. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/TrafficSinkListener.java | Sink listener now consumes metadata rather than LogData.Builder. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/RecordSinkListener.java | Sink listener now initializes output builders via (metadata, input). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogSinkListener.java | Unifies sink listener signature to (LogMetadata, Object, ExecutionContext). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogFilterListener.java | Filter listener now creates contexts via init(metadata, input). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogAnalysisListener.java | Analysis listener contract now parses (LogMetadata, Object). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzerServiceImpl.java | Analyzer service now dispatches (LogMetadata, Object). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzer.java | Analyzer now validates/normalizes using metadata and passes (metadata, input) to listeners. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/ILogAnalyzerService.java | API updated to doAnalysis(LogMetadata, Object). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/TextParserSpec.java | Text parser reads body via (LogData.Builder) ctx.input(). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/filter/FilterSpec.java | JSON/YAML parse uses ctx.input() and sink path respects dryRun. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/ExecutionContext.java | Replaces log/extraLog with metadata/input and adds dryRun. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/DSL.java | Debug logging now reports metadata fields (no longer LogData body type). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/rt/LalRuntimeHelper.java | Tag access now reads from (LogData.Builder) ctx.input(). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALCodegenHelper.java | Splits metadata getters vs LogData-only getters for codegen. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALClassGenerator.java | Generated typed input var now casts from ctx.input(). |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALBlockCodegen.java | Codegen now routes log.* access to metadata vs LogData.Builder casts; parsed fallback becomes LogMetadata. |
| oap-server/analyzer/log-analyzer/CLAUDE.md | Updates internal docs for new metadata/input model (some examples still need adjustment). |
| oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/SampledTraceBuilder.java | Output builder init updated to consume LogMetadata. |
| oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/DatabaseSlowStatementBuilder.java | Output builder init updated to consume LogMetadata. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...lyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/spi/LALSourceTypeProvider.java
Show resolved
Hide resolved
...l-v1-v2-checker/src/test/java/org/apache/skywalking/oap/server/checker/lal/LalBenchmark.java
Show resolved
Hide resolved
...server-core/src/main/java/org/apache/skywalking/oap/server/core/source/LogMetadataUtils.java
Outdated
Show resolved
Hide resolved
…der extraction - LALSourceTypeProvider javadoc: fallback is LogMetadata, not LogData.Builder - CLAUDE.md: log.body codegen uses ((LogData.Builder) h.ctx().input()).getBody() - LalBenchmark: pass logData.toBuilder() for v2 (mutable Builder required) - LogMetadataUtils.fromLogData(Builder): extract fields directly without build()
LogMetadata.TraceContext now defaults to EMPTY (empty strings, spanId=0) matching protobuf's never-null getTraceContext() behavior. Generated code accessing h.ctx().metadata().getTraceContext().getTraceId() no longer throws NPE when the incoming log has no trace context.
tag("KEY") now only compiles for LogData.Builder input (inputType==null).
For typed proto inputs, it throws IllegalArgumentException at compile time
directing users to use parsed.* field chains instead. Unknown function
calls also fail at compile time rather than silently emitting null.
- LogMetadata.TraceContext defaults to EMPTY (empty strings, spanId=0)
matching protobuf's never-null getTraceContext() behavior, preventing
NPE on log.traceContext.* for untraced logs.
- LogMetadataUtils.fromLogData(Builder) extracts fields directly without
calling build().
- tag("KEY") codegen is now input-type-aware: only intercepts when
function name is "tag", single string literal arg, and input type is
LogData.Builder. Other cases fall through to normal type-oriented
method resolution.
…lasses
When a LAL rule has a parser (json/yaml/text), inputType from SPI is now
nulled out in GenCtx — it only applies to parser-less rules. This fixes
tag("KEY") generating null for network-profiling-slow-trace (MESH layer
with json{} parser), where SPI inputType was HTTPAccessLogEntry.
tag() now fails fast at compile time with clear error messages instead of
silently falling through. The generic fallback also throws instead of
generating null.
Split the 1600-line LALBlockCodegen into focused classes:
- LALBlockCodegen (725 lines): extractor/sink/sampler block scaffolding
- LALValueCodegen (903 lines): value access, conditions, data-source access
- LALDefCodegen (360 lines): def variable declarations and chain resolution
All methods documented with Javadoc and LAL script examples.
wankai123
approved these changes
Mar 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce LogMetadata POJO to separate metadata from input data in LAL pipeline
Cleanly decouple metadata (service, layer, timestamp, trace context) from input data
(LogData body/tags, typed protos like Envoy ALS) at the LAL pipeline entry point.
Key changes:
LogMetadataPOJO carries service/instance/endpoint/layer/timestamp/trace contextLogMetadataUtils.fromLogData()extracts metadata fromLogDataat receiver boundaryExecutionContextusesmetadata()+input()instead oflog()+extraLog()Envoy ALS receivers no longer build fake
LogData— pass typed proto asinputdirectlyLogSinkListener.parse()unified to single 3-parameter signature(LogMetadata, Object, ExecutionContext)LogTestQuery(test tool) separated from production sink viadryRunflagOnly compiler is LogData-aware (for
json{}/yaml{}fallback); runtime uses untypedctx.input()LalRuntimeHelper.tagValue()casts directly toLogData.Builder(no instanceof guard — only called by compiled code)If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the
CHANGESlog.