Skip to content

Add JFR scrubbing before profile upload#10577

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits intomasterfrom
jb/jfr_redacting
Feb 25, 2026
Merged

Add JFR scrubbing before profile upload#10577
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits intomasterfrom
jb/jfr_redacting

Conversation

@jbachorik
Copy link
Contributor

@jbachorik jbachorik commented Feb 12, 2026 •
edited
Loading

What Does This Do

Adds configurable scrubbing of sensitive fields in JFR profiling recordings before upload. Sensitive data (system properties, JVM arguments, environment variables, command lines) is replaced with redacted values.

Motivation

Prevents accidental exposure of sensitive information (credentials, API keys, internal paths) in profiling data uploaded to Datadog backend.

Additional Notes

This is a proof of concept. All string values from the defined attributes will be scrubbed, there is no support for regex and similar matching. It is disabled by default and present only to be able to infer the extra overhead it would expose on a real-life system after enabling.
If the concept is validated, we will add proper value matching and enable scrubbing by default in a later PR.

profiling-scrubber module (new)

  • JfrScrubber wraps jafar-tools Scrubber (Java 8 compatible)
  • DefaultScrubDefinition defines fields to scrub per event type: jdk.InitialSystemProperty/value, jdk.JVMInformation/jvmArguments, jdk.InitialEnvironmentVariable/value, jdk.SystemProcess/commandLine
  • Event types can be excluded from scrubbing via dd.profiling.scrub.event-type-excludes

Pipeline wiring

  • ScrubRecordingDataListener decorates RecordingDataListener, scrubbing JFR data before delegating to the upload listener
  • Uses RecordingData.getPath() (new) to avoid stream materialization for file-backed recordings (ddprof)
  • Fail-open mode (dd.profiling.scrub.fail-open) passes unscrubbed data on scrubber errors, with null-safety guard to prevent passing already-released data

Configuration

  • dd.profiling.scrub.enabled (default: false) — master switch
  • dd.profiling.scrub.fail-open (default: false) — upload unscrubbed data on scrubber failure
  • dd.profiling.scrub.event-type-excludes — comma-separated event types to skip

Native image support

  • Guard ThrowableInstanceAdvice during native-image build to prevent JFR event class initialization errors
  • Smoke test verifies profiling pipeline produces JFR files with system property events
  • Scrubbing assertion deferred (TODO) — jafar 0.14.0-SNAPSHOT does not yet handle SubstrateVM JFR chunk format

⚠️ HelperScanner change — reviewer attention requested

HelperScanner.visitField() and visitMethod() changed from REQUIRES to USES.

This fix was necessary to support the jafar classes injected transitively via VMRuntimeModule.injectHelperDependencies(). The previous behavior marked field types and method parameter/return types as load-time dependencies (REQUIRES), which created false dependency cycles. When removeCycles() broke these, subclasses could be positioned before their superclass in the topological sort, causing NoClassDefFoundError during batch class injection.

The fix is correct per the JVM spec: defineClass only eagerly resolves superclass and interfaces (handled by visit()). Field types, method parameter/return types, and declared exceptions are resolved lazily. However, this change affects all instrumentations that use injectHelperDependencies(), not just profiling. If any instrumentation relies on the old (incorrect) ordering where field/method types were treated as load-time dependencies, this change could surface latent issues.

Requesting review from someone familiar with HelperScanner / HelperInjector internals to verify no regressions.

Testing

  • Unit tests: JfrScrubberTest (scrubbing correctness, exclusion, no-op), ScrubRecordingDataListenerTest (decorator behavior, fail-open, error paths)
  • Integration test: JFRBasedProfilingIntegrationTest — verifies end-to-end scrubbing of jdk.InitialSystemProperty and jdk.JVMInformation in a real profiling session
  • Native image smoke test: SpringBootNativeInstrumentationTest — verifies native image builds and runs with scrubbing enabled

Contributor Checklist

Jira ticket: PROF-13548

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@jbachorik jbachorik added type: enhancement Enhancements and improvements comp: profiling Profiling AI labels Feb 12, 2026
@jbachorik jbachorik changed the title Add JFR scrubbing before profile upload [WIP] Add JFR scrubbing before profile upload Feb 12, 2026
@jbachorik jbachorik added tag: ai generated Largely based on code generated by an AI or LLM and removed AI labels Feb 12, 2026
@pr-commenter
Copy link

pr-commenter bot commented Feb 12, 2026 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/jfr_redacting
git_commit_date 1771927866 1771930633
git_commit_sha d77d632 3cbadcf
release_version 1.60.0-SNAPSHOT~d77d632808 1.60.0-SNAPSHOT~3cbadcf0a2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1771932309 1771932309
ci_job_id 1450865653 1450865653
ci_pipeline_id 98521275 98521275
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-vah1mvs5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-vah1mvs5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062111
Total [baseline] (8.723 s) : 0, 8723320
Agent [candidate] (1.065 s) : 0, 1065238
Total [candidate] (8.743 s) : 0, 8743248
section iast
Agent [baseline] (1.246 s) : 0, 1246275
Total [baseline] (9.416 s) : 0, 9416009
Agent [candidate] (1.231 s) : 0, 1231163
Total [candidate] (9.403 s) : 0, 9402636
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.246 s 184.164 ms (17.3%)
Total tracing 8.723 s -
Total iast 9.416 s 692.689 ms (7.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent iast 1.231 s 165.925 ms (15.6%)
Total tracing 8.743 s -
Total iast 9.403 s 659.388 ms (7.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.203 ms) : 0, 1203
BytebuddyAgent [baseline] (627.67 ms) : 0, 627670
BytebuddyAgent [candidate] (627.68 ms) : 0, 627680
AgentMeter [baseline] (29.003 ms) : 0, 29003
AgentMeter [candidate] (29.037 ms) : 0, 29037
GlobalTracer [baseline] (257.19 ms) : 0, 257190
GlobalTracer [candidate] (257.317 ms) : 0, 257317
AppSec [baseline] (33.149 ms) : 0, 33149
AppSec [candidate] (32.888 ms) : 0, 32888
Debugger [baseline] (63.518 ms) : 0, 63518
Debugger [candidate] (65.125 ms) : 0, 65125
Remote Config [baseline] (625.673 µs) : 0, 626
Remote Config [candidate] (599.139 µs) : 0, 599
Telemetry [baseline] (9.034 ms) : 0, 9034
Telemetry [candidate] (9.04 ms) : 0, 9040
Flare Poller [baseline] (4.548 ms) : 0, 4548
Flare Poller [candidate] (6.135 ms) : 0, 6135
section iast
crashtracking [baseline] (1.235 ms) : 0, 1235
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (807.38 ms) : 0, 807380
BytebuddyAgent [candidate] (794.925 ms) : 0, 794925
AgentMeter [baseline] (11.799 ms) : 0, 11799
AgentMeter [candidate] (11.277 ms) : 0, 11277
GlobalTracer [baseline] (250.154 ms) : 0, 250154
GlobalTracer [candidate] (248.085 ms) : 0, 248085
AppSec [baseline] (35.899 ms) : 0, 35899
AppSec [candidate] (34.773 ms) : 0, 34773
Debugger [baseline] (63.629 ms) : 0, 63629
Debugger [candidate] (65.058 ms) : 0, 65058
Remote Config [baseline] (558.703 µs) : 0, 559
Remote Config [candidate] (545.812 µs) : 0, 546
Telemetry [baseline] (8.62 ms) : 0, 8620
Telemetry [candidate] (8.715 ms) : 0, 8715
Flare Poller [baseline] (3.404 ms) : 0, 3404
Flare Poller [candidate] (3.533 ms) : 0, 3533
IAST [baseline] (27.32 ms) : 0, 27320
IAST [candidate] (26.994 ms) : 0, 26994
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064395
Total [baseline] (10.905 s) : 0, 10904843
Agent [candidate] (1.065 s) : 0, 1064583
Total [candidate] (10.846 s) : 0, 10846428
section appsec
Agent [baseline] (1.24 s) : 0, 1239968
Total [baseline] (10.997 s) : 0, 10997473
Agent [candidate] (1.247 s) : 0, 1246505
Total [candidate] (11.123 s) : 0, 11123492
section iast
Agent [baseline] (1.233 s) : 0, 1233342
Total [baseline] (11.203 s) : 0, 11202714
Agent [candidate] (1.231 s) : 0, 1231056
Total [candidate] (11.186 s) : 0, 11186319
section profiling
Agent [baseline] (1.189 s) : 0, 1189392
Total [baseline] (10.905 s) : 0, 10904699
Agent [candidate] (1.208 s) : 0, 1207830
Total [candidate] (11.017 s) : 0, 11017364
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.24 s 175.573 ms (16.5%)
Agent iast 1.233 s 168.946 ms (15.9%)
Agent profiling 1.189 s 124.996 ms (11.7%)
Total tracing 10.905 s -
Total appsec 10.997 s 92.63 ms (0.8%)
Total iast 11.203 s 297.87 ms (2.7%)
Total profiling 10.905 s -144.26 µs (-0.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.247 s 181.921 ms (17.1%)
Agent iast 1.231 s 166.473 ms (15.6%)
Agent profiling 1.208 s 143.247 ms (13.5%)
Total tracing 10.846 s -
Total appsec 11.123 s 277.064 ms (2.6%)
Total iast 11.186 s 339.891 ms (3.1%)
Total profiling 11.017 s 170.935 ms (1.6%) 
gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (626.622 ms) : 0, 626622
BytebuddyAgent [candidate] (627.215 ms) : 0, 627215
AgentMeter [baseline] (28.937 ms) : 0, 28937
AgentMeter [candidate] (29.117 ms) : 0, 29117
GlobalTracer [baseline] (257.489 ms) : 0, 257489
GlobalTracer [candidate] (257.546 ms) : 0, 257546
AppSec [baseline] (32.959 ms) : 0, 32959
AppSec [candidate] (32.887 ms) : 0, 32887
Debugger [baseline] (65.803 ms) : 0, 65803
Debugger [candidate] (62.345 ms) : 0, 62345
Remote Config [baseline] (633.85 µs) : 0, 634
Remote Config [candidate] (630.26 µs) : 0, 630
Telemetry [baseline] (9.318 ms) : 0, 9318
Telemetry [candidate] (13.134 ms) : 0, 13134
Flare Poller [baseline] (5.435 ms) : 0, 5435
Flare Poller [candidate] (4.505 ms) : 0, 4505
section appsec
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.2 ms) : 0, 1200
BytebuddyAgent [baseline] (658.14 ms) : 0, 658140
BytebuddyAgent [candidate] (661.704 ms) : 0, 661704
AgentMeter [baseline] (11.946 ms) : 0, 11946
AgentMeter [candidate] (11.992 ms) : 0, 11992
GlobalTracer [baseline] (258.657 ms) : 0, 258657
GlobalTracer [candidate] (259.893 ms) : 0, 259893
AppSec [baseline] (167.855 ms) : 0, 167855
AppSec [candidate] (168.965 ms) : 0, 168965
Debugger [baseline] (66.722 ms) : 0, 66722
Debugger [candidate] (67.022 ms) : 0, 67022
Remote Config [baseline] (650.472 µs) : 0, 650
Remote Config [candidate] (652.169 µs) : 0, 652
Telemetry [baseline] (9.541 ms) : 0, 9541
Telemetry [candidate] (9.48 ms) : 0, 9480
Flare Poller [baseline] (3.7 ms) : 0, 3700
Flare Poller [candidate] (3.689 ms) : 0, 3689
IAST [baseline] (25.444 ms) : 0, 25444
IAST [candidate] (25.671 ms) : 0, 25671
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (796.92 ms) : 0, 796920
BytebuddyAgent [candidate] (794.418 ms) : 0, 794418
AgentMeter [baseline] (11.331 ms) : 0, 11331
AgentMeter [candidate] (11.276 ms) : 0, 11276
GlobalTracer [baseline] (248.079 ms) : 0, 248079
GlobalTracer [candidate] (247.457 ms) : 0, 247457
AppSec [baseline] (34.713 ms) : 0, 34713
AppSec [candidate] (35.016 ms) : 0, 35016
Debugger [baseline] (65.691 ms) : 0, 65691
Debugger [candidate] (65.893 ms) : 0, 65893
Remote Config [baseline] (541.5 µs) : 0, 542
Remote Config [candidate] (536.12 µs) : 0, 536
Telemetry [baseline] (8.56 ms) : 0, 8560
Telemetry [candidate] (8.652 ms) : 0, 8652
Flare Poller [baseline] (3.448 ms) : 0, 3448
Flare Poller [candidate] (3.436 ms) : 0, 3436
IAST [baseline] (26.806 ms) : 0, 26806
IAST [candidate] (27.066 ms) : 0, 27066
section profiling
crashtracking [baseline] (1.179 ms) : 0, 1179
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (681.141 ms) : 0, 681141
BytebuddyAgent [candidate] (692.37 ms) : 0, 692370
AgentMeter [baseline] (8.539 ms) : 0, 8539
AgentMeter [candidate] (8.685 ms) : 0, 8685
GlobalTracer [baseline] (215.914 ms) : 0, 215914
GlobalTracer [candidate] (218.993 ms) : 0, 218993
AppSec [baseline] (32.321 ms) : 0, 32321
AppSec [candidate] (33.118 ms) : 0, 33118
Debugger [baseline] (67.03 ms) : 0, 67030
Debugger [candidate] (67.835 ms) : 0, 67835
Remote Config [baseline] (625.237 µs) : 0, 625
Remote Config [candidate] (643.064 µs) : 0, 643
Telemetry [baseline] (8.946 ms) : 0, 8946
Telemetry [candidate] (9.089 ms) : 0, 9089
Flare Poller [baseline] (3.706 ms) : 0, 3706
Flare Poller [candidate] (3.848 ms) : 0, 3848
ProfilingAgent [baseline] (99.463 ms) : 0, 99463
ProfilingAgent [candidate] (100.583 ms) : 0, 100583
Profiling [baseline] (100.032 ms) : 0, 100032
Profiling [candidate] (101.168 ms) : 0, 101168
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/jfr_redacting
git_commit_date 1771927866 1771930633
git_commit_sha d77d632 3cbadcf
release_version 1.60.0-SNAPSHOT~d77d632808 1.60.0-SNAPSHOT~3cbadcf0a2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1771932801 1771932801
ci_job_id 1450865655 1450865655
ci_pipeline_id 98521275 98521275
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-hdak81vg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-hdak81vg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 20 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:petclinic:no_agent:high_load better
[-1.770ms; -0.456ms] or [-9.935%; -2.561%]		 unsure
[-3.199ms; -0.469ms] or [-10.620%; -1.556%]		 unstable
[-12.459op/s; +47.459op/s] or [-4.903%; +18.675%]		 16.703ms 28.291ms 271.625op/s 17.816ms 30.125ms 254.125op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.359 ms) : 18171, 18547
.   : milestone, 18359,
appsec (18.572 ms) : 18380, 18764
.   : milestone, 18572,
code_origins (17.623 ms) : 17450, 17795
.   : milestone, 17623,
iast (17.733 ms) : 17556, 17909
.   : milestone, 17733,
profiling (18.264 ms) : 18077, 18451
.   : milestone, 18264,
tracing (18.612 ms) : 18426, 18797
.   : milestone, 18612,
section candidate
no_agent (17.175 ms) : 17004, 17346
.   : milestone, 17175,
appsec (18.316 ms) : 18132, 18500
.   : milestone, 18316,
code_origins (17.909 ms) : 17727, 18092
.   : milestone, 17909,
iast (17.378 ms) : 17207, 17549
.   : milestone, 17378,
profiling (18.646 ms) : 18464, 18828
.   : milestone, 18646,
tracing (18.464 ms) : 18278, 18650
.   : milestone, 18464,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.359 ms [18.171 ms, 18.547 ms] -
appsec 18.572 ms [18.38 ms, 18.764 ms] 212.474 µs (1.2%)
code_origins 17.623 ms [17.45 ms, 17.795 ms] -736.677 µs (-4.0%)
iast 17.733 ms [17.556 ms, 17.909 ms] -626.634 µs (-3.4%)
profiling 18.264 ms [18.077 ms, 18.451 ms] -95.433 µs (-0.5%)
tracing 18.612 ms [18.426 ms, 18.797 ms] 252.208 µs (1.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.175 ms [17.004 ms, 17.346 ms] -
appsec 18.316 ms [18.132 ms, 18.5 ms] 1.14 ms (6.6%)
code_origins 17.909 ms [17.727 ms, 18.092 ms] 734.023 µs (4.3%)
iast 17.378 ms [17.207 ms, 17.549 ms] 202.69 µs (1.2%)
profiling 18.646 ms [18.464 ms, 18.828 ms] 1.471 ms (8.6%)
tracing 18.464 ms [18.278 ms, 18.65 ms] 1.289 ms (7.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.174 ms) : 1162, 1185
.   : milestone, 1174,
iast (3.176 ms) : 3133, 3218
.   : milestone, 3176,
iast_FULL (5.685 ms) : 5628, 5742
.   : milestone, 5685,
iast_GLOBAL (3.637 ms) : 3577, 3698
.   : milestone, 3637,
profiling (2.083 ms) : 2064, 2102
.   : milestone, 2083,
tracing (1.769 ms) : 1756, 1783
.   : milestone, 1769,
section candidate
no_agent (1.175 ms) : 1164, 1187
.   : milestone, 1175,
iast (3.273 ms) : 3232, 3315
.   : milestone, 3273,
iast_FULL (5.842 ms) : 5784, 5901
.   : milestone, 5842,
iast_GLOBAL (3.589 ms) : 3528, 3649
.   : milestone, 3589,
profiling (1.96 ms) : 1941, 1979
.   : milestone, 1960,
tracing (1.76 ms) : 1746, 1774
.   : milestone, 1760,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.174 ms [1.162 ms, 1.185 ms] -
iast 3.176 ms [3.133 ms, 3.218 ms] 2.002 ms (170.5%)
iast_FULL 5.685 ms [5.628 ms, 5.742 ms] 4.511 ms (384.3%)
iast_GLOBAL 3.637 ms [3.577 ms, 3.698 ms] 2.463 ms (209.9%)
profiling 2.083 ms [2.064 ms, 2.102 ms] 908.868 µs (77.4%)
tracing 1.769 ms [1.756 ms, 1.783 ms] 595.568 µs (50.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.175 ms [1.164 ms, 1.187 ms] -
iast 3.273 ms [3.232 ms, 3.315 ms] 2.098 ms (178.5%)
iast_FULL 5.842 ms [5.784 ms, 5.901 ms] 4.667 ms (397.0%)
iast_GLOBAL 3.589 ms [3.528 ms, 3.649 ms] 2.413 ms (205.3%)
profiling 1.96 ms [1.941 ms, 1.979 ms] 784.469 µs (66.7%)
tracing 1.76 ms [1.746 ms, 1.774 ms] 584.448 µs (49.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/jfr_redacting
git_commit_date 1771927866 1771930633
git_commit_sha d77d632 3cbadcf
release_version 1.60.0-SNAPSHOT~d77d632808 1.60.0-SNAPSHOT~3cbadcf0a2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1771932659 1771932659
ci_job_id 1450865657 1450865657
ci_pipeline_id 98521275 98521275
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-o6rdfgju 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-o6rdfgju 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.458ms; -1.111ms] or [-38.420%; -29.276%]		 2.510ms 3.795ms
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (3.795 ms) : 3573, 4016
.   : milestone, 3795,
iast (2.244 ms) : 2175, 2312
.   : milestone, 2244,
iast_GLOBAL (2.29 ms) : 2220, 2359
.   : milestone, 2290,
profiling (2.099 ms) : 2043, 2155
.   : milestone, 2099,
tracing (2.051 ms) : 1998, 2104
.   : milestone, 2051,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.51 ms) : 2456, 2565
.   : milestone, 2510,
iast (2.252 ms) : 2183, 2321
.   : milestone, 2252,
iast_GLOBAL (2.283 ms) : 2214, 2351
.   : milestone, 2283,
profiling (2.07 ms) : 2016, 2124
.   : milestone, 2070,
tracing (2.05 ms) : 1997, 2103
.   : milestone, 2050,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 3.795 ms [3.573 ms, 4.016 ms] 2.32 ms (157.4%)
iast 2.244 ms [2.175 ms, 2.312 ms] 769.502 µs (52.2%)
iast_GLOBAL 2.29 ms [2.22 ms, 2.359 ms] 815.211 µs (55.3%)
profiling 2.099 ms [2.043 ms, 2.155 ms] 624.255 µs (42.3%)
tracing 2.051 ms [1.998 ms, 2.104 ms] 576.399 µs (39.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.51 ms [2.456 ms, 2.565 ms] 1.039 ms (70.6%)
iast 2.252 ms [2.183 ms, 2.321 ms] 780.612 µs (53.0%)
iast_GLOBAL 2.283 ms [2.214 ms, 2.351 ms] 810.89 µs (55.1%)
profiling 2.07 ms [2.016 ms, 2.124 ms] 598.195 µs (40.6%)
tracing 2.05 ms [1.997 ms, 2.103 ms] 578.543 µs (39.3%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~3cbadcf0a2, baseline=1.60.0-SNAPSHOT~d77d632808
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.896 s) : 14896000, 14896000
.   : milestone, 14896000,
appsec (15.057 s) : 15057000, 15057000
.   : milestone, 15057000,
iast (18.01 s) : 18010000, 18010000
.   : milestone, 18010000,
iast_GLOBAL (18.232 s) : 18232000, 18232000
.   : milestone, 18232000,
profiling (14.807 s) : 14807000, 14807000
.   : milestone, 14807000,
tracing (14.554 s) : 14554000, 14554000
.   : milestone, 14554000,
section candidate
no_agent (15.466 s) : 15466000, 15466000
.   : milestone, 15466000,
appsec (15.083 s) : 15083000, 15083000
.   : milestone, 15083000,
iast (18.116 s) : 18116000, 18116000
.   : milestone, 18116000,
iast_GLOBAL (17.795 s) : 17795000, 17795000
.   : milestone, 17795000,
profiling (15.579 s) : 15579000, 15579000
.   : milestone, 15579000,
tracing (14.502 s) : 14502000, 14502000
.   : milestone, 14502000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.896 s [14.896 s, 14.896 s] -
appsec 15.057 s [15.057 s, 15.057 s] 161.0 ms (1.1%)
iast 18.01 s [18.01 s, 18.01 s] 3.114 s (20.9%)
iast_GLOBAL 18.232 s [18.232 s, 18.232 s] 3.336 s (22.4%)
profiling 14.807 s [14.807 s, 14.807 s] -89.0 ms (-0.6%)
tracing 14.554 s [14.554 s, 14.554 s] -342.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.466 s [15.466 s, 15.466 s] -
appsec 15.083 s [15.083 s, 15.083 s] -383.0 ms (-2.5%)
iast 18.116 s [18.116 s, 18.116 s] 2.65 s (17.1%)
iast_GLOBAL 17.795 s [17.795 s, 17.795 s] 2.329 s (15.1%)
profiling 15.579 s [15.579 s, 15.579 s] 113.0 ms (0.7%)
tracing 14.502 s [14.502 s, 14.502 s] -964.0 ms (-6.2%)

@jbachorik jbachorik force-pushed the jb/jfr_redacting branch 10 times, most recently from 0efec0b to 6d2f36d Compare February 19, 2026 14:02
@jbachorik jbachorik force-pushed the jb/jfr_redacting branch 7 times, most recently from bd39939 to 46507cd Compare February 21, 2026 18:26
jbachorik and others added 2 commits February 23, 2026 13:19
@jbachorik @claude
Fix HelperScanner: treat field/method types as USES not REQUIRES
0b609a5 
The JVM only eagerly resolves superclass and interfaces during
defineClass. Field types, method parameter/return types, and declared
exceptions are resolved lazily. Marking them as REQUIRES created false
dependency cycles that broke topological sort ordering when injecting
large helper batches (2000+ classes).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jbachorik @claude
Add JFR scrubbing to profiling upload pipeline
1bb6683 
Scrub sensitive fields (system properties, JVM arguments, environment
variables, process command lines) from JFR recordings before upload.

- Add profiling-scrubber module wrapping jafar-tools Scrubber
- Wire ScrubRecordingDataListener decorator into ProfilingAgent
- Add RecordingData.getPath() to avoid stream materialization for
  file-backed recordings (ddprof)
- Add config: dd.profiling.scrub.enabled, dd.profiling.scrub.fail-open,
  dd.profiling.scrub.event-type-excludes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jbachorik @claude
Enable JFR scrubbing in native image smoke test
52dde93 
- Guard ThrowableInstanceAdvice during native-image build to prevent
  JFR event class initialization errors
- Enable profiling scrubber in native image build args
- Add smoke test verifying JFR files with system property events are
  produced (scrubbing assertion deferred until jafar handles
  SubstrateVM JFR chunk format)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jbachorik jbachorik changed the title [WIP] Add JFR scrubbing before profile upload Add JFR scrubbing before profile upload Feb 23, 2026
@jbachorik jbachorik changed the title Add JFR scrubbing before profile upload [WIP] Add JFR scrubbing before profile upload Feb 23, 2026
@jbachorik @claude
Update jafar to 0.14.1 and assert JFR scrubbing in native smoke test
01e01d4 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jbachorik jbachorik changed the title [WIP] Add JFR scrubbing before profile upload Add JFR scrubbing before profile upload Feb 23, 2026
@jbachorik jbachorik marked this pull request as ready for review February 23, 2026 21:47
@jbachorik jbachorik requested review from a team as code owners February 23, 2026 21:47
@jbachorik jbachorik requested review from mcculls and sarahchen6 and removed request for a team February 23, 2026 21:47
rkennke
rkennke approved these changes Feb 24, 2026
View reviewed changes
Copy link
Contributor

@rkennke rkennke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

mcculls
mcculls approved these changes Feb 25, 2026
View reviewed changes
Copy link
Contributor

@mcculls mcculls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the related cleanup

@mcculls
Copy link
Contributor

mcculls commented Feb 25, 2026

The HelperScanner changes look good - the only feature that uses that in-depth scanning atm is VMRuntimeModule, which is covered by the GraalVM native smoke tests.

@jbachorik
Copy link
Contributor Author

jbachorik commented Feb 25, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link

gh-worker-devflow-routing-ef8351 bot commented Feb 25, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-02-25 14:02:27 UTC ℹ️ Start processing command /merge

2026-02-25 14:02:32 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-02-25 14:03:12 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit d597de3 into master Feb 25, 2026
580 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the jb/jfr_redacting branch February 25, 2026 14:03
@github-actions github-actions bot added this to the 1.60.0 milestone Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

@rkennke rkennke rkennke approved these changes

@sarahchen6 sarahchen6 Awaiting requested review from sarahchen6 sarahchen6 is a code owner automatically assigned from DataDog/apm-lang-platform-java

Assignees

No one assigned

Labels

comp: profiling Profiling tag: ai generated Largely based on code generated by an AI or LLM type: enhancement Enhancements and improvements

Projects

None yet

Milestone

1.60.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jbachorik @mcculls @rkennke