[Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake #7388

dudikeleti · 2025-08-13T22:11:54Z

Summary of changes

Dynamic Instrumentation and Exception Replay are about to be enabled by default with strict redaction rules. To support that, we have to perform redaction in the tracer (already done) and in the backend. Up until now, we uploaded our snapshots through Logs Intake. In Logs Intake there is no built in support for redaction. In order to support redaction at ingestion time, we direct our snapshots through debugger's diagnostics endpoint instead of Logs Intake.

Reason for change

Enable DI and ER features by default.

Implementation details

Direct DI&ER snapshots through Debugger's intake.

dd-trace-dotnet-ci-bot · 2025-08-13T23:11:24Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7388) - mean (68ms)  : 67, 70
     .   : milestone, 68,
    master - mean (69ms)  : 64, 74
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (1,003ms)  : 978, 1029
     .   : milestone, 1003,
    master - mean (1,005ms)  : 975, 1035
     .   : milestone, 1005,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (107ms)  : 106, 108
     .   : milestone, 107,
    master - mean (107ms)  : 105, 108
     .   : milestone, 107,

    section Baseline
    This PR (7388) - mean (106ms)  : 104, 108
     .   : milestone, 106,
    master - mean (106ms)  : 104, 108
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (708ms)  : 691, 725
     .   : milestone, 708,
    master - mean (710ms)  : 688, 731
     .   : milestone, 710,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (94ms)  : 93, 95
     .   : milestone, 94,
    master - mean (94ms)  : 93, 95
     .   : milestone, 94,

    section Baseline
    This PR (7388) - mean (94ms)  : 92, 96
     .   : milestone, 94,
    master - mean (94ms)  : 92, 96
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (661ms)  : 643, 679
     .   : milestone, 661,
    master - mean (667ms)  : 645, 689
     .   : milestone, 667,

gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (93ms)  : 92, 95
     .   : milestone, 93,
    master - mean (94ms)  : 92, 95
     .   : milestone, 94,

    section Baseline
    This PR (7388) - mean (93ms)  : 91, 95
     .   : milestone, 93,
    master - mean (93ms)  : 90, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (596ms)  : 586, 606
     .   : milestone, 596,
    master - mean (603ms)  : 594, 613
     .   : milestone, 603,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (196ms)  : 193, 198
     .   : milestone, 196,
    master - mean (194ms)  : 188, 200
     .   : milestone, 194,

    section Baseline
    This PR (7388) - mean (193ms)  : 188, 198
     .   : milestone, 193,
    master - mean (192ms)  : 187, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (1,113ms)  : 1077, 1149
     .   : milestone, 1113,
    master - mean (1,096ms)  : 1071, 1121
     .   : milestone, 1096,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (278ms)  : 274, 281
     .   : milestone, 278,
    master - mean (274ms)  : 268, 280
     .   : milestone, 274,

    section Baseline
    This PR (7388) - mean (276ms)  : 270, 282
     .   : milestone, 276,
    master - mean (273ms)  : 266, 280
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (897ms)  : 870, 925
     .   : milestone, 897,
    master - mean (891ms)  : 851, 931
     .   : milestone, 891,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (270ms)  : 266, 273
     .   : milestone, 270,
    master - mean (266ms)  : 260, 273
     .   : milestone, 266,

    section Baseline
    This PR (7388) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (267ms)  : 262, 273
     .   : milestone, 267,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (877ms)  : 846, 909
     .   : milestone, 877,
    master - mean (871ms)  : 840, 901
     .   : milestone, 871,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (265ms)  : 259, 271
     .   : milestone, 265,

    section Baseline
    This PR (7388) - mean (268ms)  : 264, 272
     .   : milestone, 268,
    master - mean (264ms)  : 259, 270
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (790ms)  : 769, 812
     .   : milestone, 790,
    master - mean (788ms)  : 764, 812
     .   : milestone, 788,

pr-commenter · 2025-08-13T23:16:28Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7388 compared to master:

1 benchmarks are faster, with geometric mean 2.093
2 benchmarks are slower, with geometric mean 1.233
3 benchmarks have fewer allocations
9 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472	6.09 KB	5.99 KB	-96 B	-1.58%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`StartStopWithChild`	net6.0	10.5μs	57.5ns	350ns	0	0	5.5 KB
master	`StartStopWithChild`	netcoreapp3.1	13.2μs	64.6ns	266ns	0	0	5.7 KB
master	`StartStopWithChild`	net472	22.4μs	124ns	732ns	0.945	0.42	6.09 KB
#7388	`StartStopWithChild`	net6.0	10.8μs	59.3ns	335ns	0	0	5.5 KB
#7388	`StartStopWithChild`	netcoreapp3.1	14.1μs	72.8ns	334ns	0	0	5.7 KB
#7388	`StartStopWithChild`	net472	22.2μs	117ns	640ns	0.852	0.213	5.99 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	936μs	63.9ns	247ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.02ms	176ns	634ns	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.2ms	304ns	1.1μs	3.31 KB
#7388	`WriteAndFlushEnrichedTraces`	net6.0	934μs	232ns	899ns	2.7 KB
#7388	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.03ms	283ns	1.1μs	2.7 KB
#7388	`WriteAndFlushEnrichedTraces`	net472	1.19ms	82.7ns	286ns	3.31 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`AllCycleSimpleBody`	net6.0	322μs	1.32μs	5.1μs	0	172.07 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	462μs	972ns	3.77μs	0	174.18 KB
master	`AllCycleSimpleBody`	net472	438μs	315ns	1.22μs	30.2	194.25 KB
master	`AllCycleMoreComplexBody`	net6.0	332μs	1.56μs	6.05μs	0	175.58 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	507μs	513ns	1.99μs	0	177.6 KB
master	`AllCycleMoreComplexBody`	net472	437μs	174ns	674ns	30.2	197.76 KB
master	`ObjectExtractorSimpleBody`	net6.0	317ns	0.108ns	0.42ns	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	402ns	2.18ns	11.3ns	0	272 B
master	`ObjectExtractorSimpleBody`	net472	300ns	0.0488ns	0.189ns	0.0439	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.22μs	23.1ns	89.3ns	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.86μs	30.4ns	118ns	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.66μs	2.03ns	7.86ns	0.599	3.8 KB
#7388	`AllCycleSimpleBody`	net6.0	318μs	1.3μs	4.87μs	0	172.07 KB
#7388	`AllCycleSimpleBody`	netcoreapp3.1	497μs	2.54μs	11.9μs	0	174.19 KB
#7388	`AllCycleSimpleBody`	net472	428μs	140ns	542ns	29.2	194.24 KB
#7388	`AllCycleMoreComplexBody`	net6.0	333μs	233ns	903ns	0	175.58 KB
#7388	`AllCycleMoreComplexBody`	netcoreapp3.1	470μs	763ns	2.85μs	0	177.6 KB
#7388	`AllCycleMoreComplexBody`	net472	436μs	239ns	896ns	30.2	197.76 KB
#7388	`ObjectExtractorSimpleBody`	net6.0	324ns	0.135ns	0.506ns	0	280 B
#7388	`ObjectExtractorSimpleBody`	netcoreapp3.1	407ns	1.96ns	7.85ns	0	272 B
#7388	`ObjectExtractorSimpleBody`	net472	300ns	0.0253ns	0.0981ns	0.0439	281 B
#7388	`ObjectExtractorMoreComplexBody`	net6.0	6.34μs	7.36ns	28.5ns	0	3.78 KB
#7388	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.7μs	36.7ns	142ns	0	3.69 KB
#7388	`ObjectExtractorMoreComplexBody`	net472	6.65μs	2.84ns	11ns	0.599	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	76.8μs	220ns	823ns	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	98.5μs	245ns	948ns	0	32.4 KB
master	`EncodeArgs`	net472	109μs	48.6ns	188ns	4.92	32.51 KB
master	`EncodeLegacyArgs`	net6.0	141μs	30.7ns	119ns	0	2.15 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	199μs	221ns	854ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	262μs	40.4ns	156ns	0	2.16 KB
#7388	`EncodeArgs`	net6.0	76.3μs	318ns	1.23μs	0	32.4 KB
#7388	`EncodeArgs`	netcoreapp3.1	96.8μs	226ns	814ns	0	32.4 KB
#7388	`EncodeArgs`	net472	108μs	19ns	70.9ns	4.9	32.51 KB
#7388	`EncodeLegacyArgs`	net6.0	145μs	7.68ns	28.8ns	0	2.15 KB
#7388	`EncodeLegacyArgs`	netcoreapp3.1	199μs	139ns	538ns	0	2.14 KB
#7388	`EncodeLegacyArgs`	net472	262μs	25.9ns	100ns	0	2.16 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Fewer allocations 🎉

Faster 🎉 in #7388

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1	2.093	861,427.05	411,660.21

Fewer allocations 🎉 in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net472	2.3 KB	2.29 KB	-13 B	-0.56%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	396μs	122ns	458ns	4.56 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	792μs	14.9μs	148μs	4.48 KB
master	`RunWafRealisticBenchmark`	net472	434μs	48.4ns	188ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	292μs	50.3ns	195ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	724μs	3.14μs	11.3μs	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	313μs	70.9ns	275ns	2.3 KB
#7388	`RunWafRealisticBenchmark`	net6.0	396μs	144ns	557ns	4.55 KB
#7388	`RunWafRealisticBenchmark`	netcoreapp3.1	412μs	152ns	570ns	4.48 KB
#7388	`RunWafRealisticBenchmark`	net472	434μs	77.6ns	291ns	4.66 KB
#7388	`RunWafRealisticBenchmarkWithAttack`	net6.0	290μs	123ns	476ns	2.24 KB
#7388	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	720μs	5μs	50μs	2.22 KB
#7388	`RunWafRealisticBenchmarkWithAttack`	net472	314μs	46.8ns	181ns	2.29 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	61.3μs	41.5ns	161ns	14.52 KB
master	`SendRequest`	netcoreapp3.1	71.3μs	93.8ns	325ns	17.42 KB
master	`SendRequest`	net472	0.0117ns	0.00328ns	0.0127ns	0 b
#7388	`SendRequest`	net6.0	60.9μs	60.1ns	225ns	14.52 KB
#7388	`SendRequest`	netcoreapp3.1	71.4μs	110ns	427ns	17.42 KB
#7388	`SendRequest`	net472	0.00574ns	0.00211ns	0.00819ns	0 b

Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	2 B	4 B	2 B	100.00%

Fewer allocations 🎉 in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0	7 B	2 B	-5 B	-71.43%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`OriginalCharSlice`	net6.0	1.97ms	644ns	2.41μs	0	640.01 KB
master	`OriginalCharSlice`	netcoreapp3.1	2.13ms	11.2μs	57μs	0	640 KB
master	`OriginalCharSlice`	net472	2.73ms	174ns	650ns	100	641.95 KB
master	`OptimizedCharSlice`	net6.0	1.36ms	284ns	1.1μs	0	7 B
master	`OptimizedCharSlice`	netcoreapp3.1	1.72ms	295ns	1.14μs	0	1 B
master	`OptimizedCharSlice`	net472	1.95ms	444ns	1.72μs	0	0 b
master	`OptimizedCharSliceWithPool`	net6.0	916μs	50.4ns	195ns	0	2 B
master	`OptimizedCharSliceWithPool`	netcoreapp3.1	827μs	72.6ns	262ns	0	1 B
master	`OptimizedCharSliceWithPool`	net472	1.16ms	87.6ns	316ns	0	0 b
#7388	`OriginalCharSlice`	net6.0	1.91ms	2.59μs	10μs	0	640 KB
#7388	`OriginalCharSlice`	netcoreapp3.1	2.11ms	5.7μs	22.1μs	0	640 KB
#7388	`OriginalCharSlice`	net472	2.72ms	373ns	1.35μs	100	641.95 KB
#7388	`OptimizedCharSlice`	net6.0	1.39ms	4.48μs	17.4μs	0	2 B
#7388	`OptimizedCharSlice`	netcoreapp3.1	1.66ms	1.09μs	4.23μs	0	1 B
#7388	`OptimizedCharSlice`	net472	2.01ms	425ns	1.65μs	0	0 b
#7388	`OptimizedCharSliceWithPool`	net6.0	847μs	1.85μs	6.9μs	0	4 B
#7388	`OptimizedCharSliceWithPool`	netcoreapp3.1	843μs	99.4ns	372ns	0	1 B
#7388	`OptimizedCharSliceWithPool`	net472	1.13ms	45.8ns	165ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7388

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	1.203	903,090.21	1,086,322.40

More allocations ⚠️ in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.65 KB	42.61 KB	960 B	2.30%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.73 KB	42.03 KB	303 B	0.73%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	55.98 KB	56.28 KB	299 B	0.53%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	694μs	571ns	2.06μs	0	41.65 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	677μs	2.11μs	7.29μs	0	41.73 KB
master	`WriteAndFlushEnrichedTraces`	net472	905μs	4.57μs	20.4μs	8.33	55.98 KB
#7388	`WriteAndFlushEnrichedTraces`	net6.0	718μs	3.69μs	16.5μs	0	42.61 KB
#7388	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	714μs	2.2μs	8.52μs	0	42.03 KB
#7388	`WriteAndFlushEnrichedTraces`	net472	1.08ms	5.02μs	18.8μs	5.21	56.28 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.94μs	5.81ns	22.5ns	0	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.59μs	11.6ns	44.9ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.83μs	3.08ns	11.9ns	0.155	0.0141	987 B
#7388	`ExecuteNonQuery`	net6.0	1.88μs	7.61ns	29.5ns	0	0	1.02 KB
#7388	`ExecuteNonQuery`	netcoreapp3.1	2.48μs	7.93ns	30.7ns	0	0	1.02 KB
#7388	`ExecuteNonQuery`	net472	2.66μs	3.82ns	14.8ns	0.147	0.0133	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.71μs	8.15ns	33.6ns	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	2.35μs	6.04ns	23.4ns	0	1.03 KB
master	`CallElasticsearch`	net472	3.61μs	2.6ns	10.1ns	0.162	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.85μs	2.16ns	8.35ns	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.38μs	11.8ns	51.3ns	0	1.08 KB
master	`CallElasticsearchAsync`	net472	3.74μs	3.68ns	14.2ns	0.168	1.1 KB
#7388	`CallElasticsearch`	net6.0	1.73μs	9.24ns	48ns	0	1.03 KB
#7388	`CallElasticsearch`	netcoreapp3.1	2.23μs	7.47ns	27.9ns	0	1.03 KB
#7388	`CallElasticsearch`	net472	3.52μs	0.758ns	2.73ns	0.158	1.04 KB
#7388	`CallElasticsearchAsync`	net6.0	1.87μs	3.34ns	12.9ns	0	1.01 KB
#7388	`CallElasticsearchAsync`	netcoreapp3.1	2.47μs	9.1ns	32.8ns	0	1.08 KB
#7388	`CallElasticsearchAsync`	net472	3.72μs	1.31ns	5.08ns	0.167	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.87μs	2.26ns	8.47ns	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	2.33μs	11.5ns	47.2ns	0	952 B
master	`ExecuteAsync`	net472	2.58μs	4.02ns	15.6ns	0.14	915 B
#7388	`ExecuteAsync`	net6.0	1.75μs	0.954ns	3.69ns	0	952 B
#7388	`ExecuteAsync`	netcoreapp3.1	2.31μs	4.76ns	18.4ns	0	952 B
#7388	`ExecuteAsync`	net472	2.49μs	2.88ns	11.2ns	0.137	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	7.08μs	31.8ns	119ns	0	2.36 KB
master	`SendAsync`	netcoreapp3.1	8.35μs	26.9ns	104ns	0	2.9 KB
master	`SendAsync`	net472	12.2μs	8.46ns	31.6ns	0.486	3.18 KB
#7388	`SendAsync`	net6.0	7.1μs	6.71ns	25.1ns	0	2.36 KB
#7388	`SendAsync`	netcoreapp3.1	8.3μs	14.6ns	54.5ns	0	2.9 KB
#7388	`SendAsync`	net472	12.1μs	6.62ns	24.8ns	0.483	3.18 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7388

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	1.263	417,850.00	527,700.00

More allocations ⚠️ in #7388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	57.34 KB	65.54 KB	8.19 KB	14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	258.64 KB	278.03 KB	19.39 KB	7.50%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	44.33 KB	45.91 KB	1.58 KB	3.57%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	276.1 KB	278.79 KB	2.7 KB	0.98%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	42.64 KB	42.86 KB	216 B	0.51%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	49μs	387ns	3.67μs	44.33 KB
master	`StringConcatBenchmark`	netcoreapp3.1	49.7μs	236ns	1.9μs	42.64 KB
master	`StringConcatBenchmark`	net472	57.9μs	130ns	449ns	57.34 KB
master	`StringConcatAspectBenchmark`	net6.0	457μs	2.16μs	9.4μs	258.64 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	441μs	4.74μs	46.2μs	276.1 KB
master	`StringConcatAspectBenchmark`	net472	407μs	2.25μs	17μs	277.66 KB
#7388	`StringConcatBenchmark`	net6.0	43.3μs	253ns	2.09μs	45.91 KB
#7388	`StringConcatBenchmark`	netcoreapp3.1	52.3μs	604ns	5.94μs	42.86 KB
#7388	`StringConcatBenchmark`	net472	56.7μs	225ns	871ns	65.54 KB
#7388	`StringConcatAspectBenchmark`	net6.0	479μs	9.41μs	93.6μs	278.03 KB
#7388	`StringConcatAspectBenchmark`	netcoreapp3.1	530μs	2.63μs	11.2μs	278.79 KB
#7388	`StringConcatAspectBenchmark`	net472	405μs	2.19μs	16.4μs	278.53 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.52μs	13.6ns	70.5ns	0	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	3.47μs	12.3ns	47.8ns	0	1.7 KB
master	`EnrichedLog`	net472	3.91μs	4.06ns	15.7ns	0.254	1.64 KB
#7388	`EnrichedLog`	net6.0	2.55μs	0.319ns	1.24ns	0	1.7 KB
#7388	`EnrichedLog`	netcoreapp3.1	3.4μs	17.7ns	84.7ns	0	1.7 KB
#7388	`EnrichedLog`	net472	4.1μs	2.84ns	11ns	0.246	1.64 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	123μs	31.3ns	113ns	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	128μs	205ns	769ns	4.31 KB
master	`EnrichedLog`	net472	168μs	40.6ns	157ns	4.52 KB
#7388	`EnrichedLog`	net6.0	125μs	367ns	1.37μs	4.31 KB
#7388	`EnrichedLog`	netcoreapp3.1	128μs	267ns	1.03μs	4.31 KB
#7388	`EnrichedLog`	net472	167μs	90.1ns	325ns	4.52 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.96μs	16ns	57.8ns	0	2.26 KB
master	`EnrichedLog`	netcoreapp3.1	6.64μs	15.1ns	58.6ns	0	2.26 KB
master	`EnrichedLog`	net472	7.64μs	4.79ns	17.9ns	0.306	2.08 KB
#7388	`EnrichedLog`	net6.0	4.98μs	19.9ns	74.4ns	0	2.26 KB
#7388	`EnrichedLog`	netcoreapp3.1	6.77μs	18.4ns	69ns	0	2.26 KB
#7388	`EnrichedLog`	net472	7.41μs	5.3ns	20.5ns	0.294	2.08 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.96μs	8.88ns	35.5ns	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	2.65μs	8.52ns	30.7ns	0	1.2 KB
master	`SendReceive`	net472	3.11μs	2.93ns	11ns	0.185	1.2 KB
#7388	`SendReceive`	net6.0	1.96μs	9.3ns	38.4ns	0	1.2 KB
#7388	`SendReceive`	netcoreapp3.1	2.64μs	3.56ns	13.3ns	0	1.2 KB
#7388	`SendReceive`	net472	3.08μs	1.54ns	5.77ns	0.185	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.15μs	7.6ns	29.4ns	0	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	5.61μs	12ns	44.9ns	0	1.63 KB
master	`EnrichedLog`	net472	6.63μs	6.9ns	26.7ns	0.298	2.03 KB
#7388	`EnrichedLog`	net6.0	4.18μs	6.72ns	26ns	0	1.58 KB
#7388	`EnrichedLog`	netcoreapp3.1	5.55μs	9.56ns	35.8ns	0	1.63 KB
#7388	`EnrichedLog`	net472	6.43μs	7.91ns	30.6ns	0.29	2.03 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	733ns	3.49ns	13.5ns	0	576 B
master	`StartFinishSpan`	netcoreapp3.1	945ns	5ns	22.4ns	0	576 B
master	`StartFinishSpan`	net472	896ns	0.431ns	1.61ns	0.0904	578 B
master	`StartFinishScope`	net6.0	884ns	4.87ns	30.4ns	0	696 B
master	`StartFinishScope`	netcoreapp3.1	1.17μs	5.23ns	20.3ns	0	696 B
master	`StartFinishScope`	net472	1.1μs	0.952ns	3.56ns	0.0991	658 B
#7388	`StartFinishSpan`	net6.0	757ns	3.92ns	19.2ns	0	576 B
#7388	`StartFinishSpan`	netcoreapp3.1	954ns	3.26ns	12.6ns	0	576 B
#7388	`StartFinishSpan`	net472	912ns	0.47ns	1.76ns	0.0916	578 B
#7388	`StartFinishScope`	net6.0	891ns	4.72ns	24ns	0	696 B
#7388	`StartFinishScope`	netcoreapp3.1	1.14μs	0.239ns	0.862ns	0	696 B
#7388	`StartFinishScope`	net472	1.1μs	0.264ns	0.988ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.03μs	5.21ns	25ns	0	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.44μs	0.341ns	1.18ns	0	696 B
master	`RunOnMethodBegin`	net472	1.36μs	1.39ns	5.39ns	0.102	658 B
#7388	`RunOnMethodBegin`	net6.0	1.04μs	4.87ns	18.2ns	0	696 B
#7388	`RunOnMethodBegin`	netcoreapp3.1	1.35μs	6.81ns	32.7ns	0	696 B
#7388	`RunOnMethodBegin`	net472	1.37μs	0.147ns	0.528ns	0.103	658 B

datadog-official · 2025-08-27T09:23:25Z

⚠️ Tests

⚠️ Warnings

🧪 5 Tests failed

MethodProbeTest from Datadog.Trace.Debugger.IntegrationTests.ProbesTests (Datadog)


Verified path: D:\a\_work\1\s\tracer\test\Datadog.Trace.Debugger.IntegrationTests\Approvals\statuses\ProbeTests.SpanDecorationSameTagsFirstError.verified.txt0ea0",
        "probeVersion": 0,
        "runtimeId": "scrubbed",
        "status": "EMITTING"
      }
    },
    "message": "Emitted probe 3410cda1-5b13-a34e-6f84-a54adf7a0ea0.",
    "service": "probes"
  },
...

MethodProbeTest from Datadog.Trace.Debugger.IntegrationTests.ProbesTests (Datadog)

Snapshot count not found. Expected 1, actual 0

MethodProbeTest from Datadog.Trace.Debugger.IntegrationTests.ProbesTests (Datadog)


Verified path: D:\a\_work\1\s\tracer\test\Datadog.Trace.Debugger.IntegrationTests\Approvals\statuses\ProbeTests.SpanDecorationErrorInWhen.verified.txt54adf7a0ea0",
        "probeVersion": 0,
        "runtimeId": "scrubbed",
        "status": "EMITTING"
      }
    },
    "message": "Emitted probe 3410cda1-5b13-a34e-6f84-a54adf7a0ea0.",
    "service": "probes"
  },
...

View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8eaef3d | Docs | Was this helpful? Give us feedback!}

This reverts commit 3e1344c.

andrewlock

LGTM from the tracer side

tracer/src/Datadog.Trace/Debugger/Upload/SymbolUploadApi.cs

Co-authored-by: Andrew Lock <[email protected]>

GreenMatan

LGTM

GreenMatan · 2025-09-18T12:25:38Z

tracer/src/Datadog.Trace/Debugger/DynamicInstrumentation.cs

            _snapshotUploader.Add(probe.ProbeId, snapshot);
            SetProbeStatusToEmitting(probe);
        }

+        internal void AddLog(ProbeInfo probe, string log)


nit: we can mark it as private

Theoretically we can use it outside the class, like AddSnapshot so I'll keep it internal for now. We have to have a final decision about LogProbe, then we can update it accordingly.

GreenMatan · 2025-09-18T12:27:38Z

tracer/src/Datadog.Trace/Debugger/DynamicInstrumentation.cs

@@ -493,10 +497,27 @@ internal void AddSnapshot(ProbeInfo probe, string snapshot)
                return;
            }

+            if (!probe.IsFullSnapshot)


why do you direct only non full snapshot to the LogUploader?

For now we forwarding only snapshot probe

GreenMatan · 2025-09-18T12:29:43Z

tracer/src/Datadog.Trace/Debugger/Upload/LogUploadApi.cs

+            IGitMetadataTagsProvider gitMetadataTagsProvider)
+            : base(apiRequestFactory, gitMetadataTagsProvider)
+        {
+            discoveryService.SubscribeToChanges(c => Endpoint = c.DebuggerEndpoint);


Are you sure that we plan to use c.DebuggerEndpoint in here and not DiagnosticsEndpoint or what you're doing in SnapshotUploadApi;
Endpoint = c.DebuggerV2Endpoint ?? c.DiagnosticsEndpoint ?? c.DebuggerEndpoint;?

For now we forwarding only snapshot probe

github-actions bot added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Aug 13, 2025

dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 7c13a45 to 864a0f2 Compare August 27, 2025 08:41

dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 864a0f2 to cea6960 Compare September 2, 2025 15:59

dudikeleti changed the title ~~send snapshots to diagnostics endpoint~~ [Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake Sep 3, 2025

dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from cea6960 to f1940c3 Compare September 3, 2025 10:00

github-actions bot added the area:debugger label Sep 3, 2025

dudikeleti marked this pull request as ready for review September 3, 2025 10:13

dudikeleti requested review from a team as code owners September 3, 2025 10:13

dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from f1940c3 to 518ed8e Compare September 12, 2025 11:40

dudikeleti added 6 commits September 15, 2025 10:04

send snapshots to diagnostics endpoint

2ffee88

update mock agent

7a35a17

add debugger.type and debugger.product

814099e

Support debugger V2 and send logs to V1

a6f0d2d

update debugger type and forwarding based on new logic

3e1344c

Revert "update debugger type and forwarding based on new logic"

925d913

This reverts commit 3e1344c.

dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 518ed8e to 925d913 Compare September 15, 2025 09:14

andrewlock approved these changes Sep 15, 2025

View reviewed changes

tracer/src/Datadog.Trace/Debugger/Upload/SymbolUploadApi.cs Outdated Show resolved Hide resolved

Update tracer/src/Datadog.Trace/Debugger/Upload/SymbolUploadApi.cs

8eaef3d

Co-authored-by: Andrew Lock <[email protected]>

GreenMatan approved these changes Sep 18, 2025

View reviewed changes

[Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake #7388

Are you sure you want to change the base?

[Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake #7388

Uh oh!

Conversation

dudikeleti commented Aug 13, 2025 • edited by GreenMatan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Uh oh!

dd-trace-dotnet-ci-bot bot commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

pr-commenter bot commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

Fewer allocations 🎉 in #7388

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #7388

Fewer allocations 🎉 in #7388

Raw results

Raw results

More allocations ⚠️ in #7388

Fewer allocations 🎉 in #7388

Raw results

Slower ⚠️ in #7388

More allocations ⚠️ in #7388

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7388

More allocations ⚠️ in #7388

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

datadog-official bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GreenMatan left a comment

Choose a reason for hiding this comment

Uh oh!

GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

dudikeleti commented Aug 13, 2025 •

edited by GreenMatan

Loading

dd-trace-dotnet-ci-bot bot commented Aug 13, 2025 •

edited

Loading

pr-commenter bot commented Aug 13, 2025 •

edited

Loading

datadog-official bot commented Aug 27, 2025 •

edited

Loading