poc: App & API Protection #695

duncanista · 2025-06-03T17:58:03Z

No description provided.

main proxy code

its not used

also added some small configs like `appsec_rules` and `appsec_waf_timeout`

so it can use decide whether to act on appsec

…-api-protection

duncanista · 2025-07-16T12:32:25Z

bottlecap/src/appsec/payload/mod.rs

+    async fn extract(self) -> HttpData;
+}
+
+pub(super) async fn extract_request_address_data(body: &Bytes) -> Option<(HttpData, RequestType)> {


I remember you mentioned maybe we want to eventually consolidate this with the triggers – overall, great

Deserializing into type might be slow since payloads can technically be up to 6MB

Ah, next file is doing a similar check from triggers code

I see that specific logic, as you mentioned privately, was ported from Go, maybe for the ones that already exist here in the triggers, could we use that same logic and for the events that we don't cover for inferred spans, I.E. Kong, we maintain the Go port?

I think the current approach is more complex than necessary -- rust deserialization is more strict than Go's (i.e, unlike in Go, not everything is deemed optional)... Probably we can simply do the "try deserializing into the candidate types until one works".

The Kong payload we probably just need to carry a specific model struct for; but I am struggling to find documentation for what shape it's supposed to have from Kong... More research warranted.

duncanista · 2025-07-16T12:44:07Z

bottlecap/src/lifecycle/invocation/context.rs

+        for (key, value) in appsec_context.tags() {
+            self.invocation_span
+                .meta
+                .entry(key.clone())
+                .or_insert(value.clone());
+        }


Can't we use extend here?

Extend will replace existing keys with values found in the iterator, and I precisely want this NOT to happen.

duncanista · 2025-07-16T12:45:36Z

bottlecap/src/lifecycle/invocation/context.rs

+
+        // Note: We intentionally don't set `actor.ip` because we don't have a definitive signal here.
+
+        //TODO(romain.marcadier): Figure out whether we can set "_dd.runtime_family" here; and to what value.


if there is a set of valid values, we can talk about how and when to set it, we get the specific runtime at a later point of init, but we guarantee it will arrive

There is indeed... Valid values are:

nodejs

ruby

jvm

dotnet

go

php

python

I see, we can do this at a later stage, but it should be possible as we get the set of runtimes with specific keys, we can just map them to the family later

duncanista · 2025-07-16T12:47:46Z

bottlecap/src/lifecycle/invocation/context.rs

My main concern is if there's a better way to tie contexts, I won't know until I inspect the code thoroughly, but so far this looks good

I am definitely open to suggestions on that front... The cold start handling is the main reason it is done this way (because its derection relies on the buffer being empty).

bottlecap/LICENSE-3rdparty.yml

bottlecap/src/config/yaml.rs

duncanista · 2025-07-22T05:44:03Z

bottlecap/src/appsec/processor.rs

+        debug!("Starting ASM processor");
+
+        if is_standalone() {
+            info!(


Should this be a warn?

In principle this is the result of an intentional action by the user... so I doubt it's relevant to warn here? But I don't feel all too strongly about that.

to avoid failing checks on gitlab

also remove the manual check for building image since it was needed one

adds `clang-libclang` and static feature for musl targets in `cargo.toml`

…nd-api-protection

duncanista · 2025-07-23T17:29:11Z

bottlecap/src/appsec/payload/body.rs

+    let mime_type = match content_type
+        .unwrap_or("application/json")
+        .parse::<mime::Mime>()
+    {
+        Ok(mime) => mime,
+        Err(e) => return Err(e.into()),
+    };


Would this actually error at any moment since we're always setting the default as application/json which in theory should always parse successfully?

You can't assume the body is even valid JSON -- it comes from the client-side of the chain... Folks make mistakes!

…26.0

…port-application-and-api-protection

duncanista · 2025-07-25T05:40:46Z

bottlecap/src/lifecycle/invocation/processor.rs

@@ -390,7 +408,8 @@ impl Processor {
            _ = offsets.process_chan_tx.send(());
        }

-        // todo(duncanista): Add missing metric tags for ASM
+        context.absorb_appsec_tags();


I think we could, since this is important part, make this method receive a mutable span which it will add the tags to.

Maybe in the processor, or trace agent, whichever seems fit, we detect if a span is the Lambda span and we absorb the tags there. This would make it work for all runtimes, WDYT?

Yes I think this is the way forward, really. And this logic can can probably be moved to the appsec::Processor as well.

RomainMuller · 2025-08-05T07:54:26Z

Superceded by #755

duncanista force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from a93c4e8 to 2ec14e9 Compare June 3, 2025 18:35

duncanista added 9 commits June 6, 2025 13:35

add proxy interceptor

bb7d9f0

main proxy code

refactor proxy to accomodate LWA processing

7ee6fb7

remove config from proxy

a667b2e

its not used

add libddwaf crate

241fcec

remove fallback on appsec

6151c67

also added some small configs like `appsec_rules` and `appsec_waf_timeout`

add appsec config methods

2d10b46

add config to proxy interceptor

c7fb357

so it can use decide whether to act on appsec

add partial appsec processor

cbcc2c6

fix dependencies

ba7a1da

duncanista force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from 346a130 to ba7a1da Compare June 6, 2025 17:36

RomainMuller force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from 0e3ae0b to ba7a1da Compare July 7, 2025 16:11

RomainMuller added 2 commits July 7, 2025 18:28

Merge origin/main into jordan.gonzalez/appsec/support-application-and…

d229800

…-api-protection

progress: integrate latest state of DataDog/libddwaf-rust#5

e0902ba

RomainMuller force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from 97a4a9e to e0902ba Compare July 7, 2025 16:42

RomainMuller added 2 commits July 8, 2025 18:01

progress: partial request payload detection & parsing

1d0b374

Update to latest commit from DataDog/libddwaf-rust#5

cd8e2cf

RomainMuller force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from a177439 to cd8e2cf Compare July 9, 2025 10:39

RomainMuller added 8 commits July 9, 2025 15:31

Finish implementing body parsing hueristics for more mime types

da3d640

merge origin/main into jordan.gonzalez/appsec/support-application-and…

87837d1

…-api-protection

Process WAF result

40c0e81

Wire up response body handling

68c055f

Merge origin/main into jordan.gonzalez/appsec/support-application-and…

8989366

…-api-protection

Wire AppSec information through to the lifecycle/invocation processor

4eaa311

Add test for invalid rules configuration behavior

26cd814

AI CHECKPOINT - PLAN.md

910f542

duncanista commented Jul 16, 2025

View reviewed changes

RomainMuller force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from 639307e to cec9428 Compare July 21, 2025 14:56

duncanista commented Jul 22, 2025

View reviewed changes

bottlecap/LICENSE-3rdparty.yml Outdated Show resolved Hide resolved

duncanista commented Jul 22, 2025

View reviewed changes

bottlecap/src/config/yaml.rs Outdated Show resolved Hide resolved

duncanista commented Jul 22, 2025

View reviewed changes

duncanista added 11 commits July 22, 2025 09:32

add clang-dev to alpine compile

1671086

update alpine to 3.22

3eddce0

alpine typo

a2a1ccc

update config merging for yaml

e80a5f0

add llvm-dev for alpine builds

9b922a9

remove licenses in .yml

abfc396

manual build check for ci image

86c699b

to avoid failing checks on gitlab

update Dockerfile

6183ea2

also remove the manual check for building image since it was needed one

ensure build for musl targets can succeed

09b67f8

adds `clang-libclang` and static feature for musl targets in `cargo.toml`

eol

598cdeb

Merge branch 'main' into jordan.gonzalez/appsec/support-application-a…

61eff51

…nd-api-protection

duncanista commented Jul 23, 2025

View reviewed changes

RomainMuller and others added 6 commits July 24, 2025 09:56

Move musl RUSTFLAGS into .cargo/config.toml, upgrade to libddwaf@1.…

b32ac72

…26.0

Move to rust 2024 edition

8f9e0d6

edition2024 style fixes

9f4016e

Merge remote-tracking branch 'origin' into jordan.gonzalez/appsec/sup…

63a67cc

…port-application-and-api-protection

resolve merge conflicts

8c3de95

resolve issues in pipelines

b349d91

duncanista commented Jul 25, 2025

View reviewed changes

Make CI pass by tweaking linker setups

23d8fb0

RomainMuller force-pushed the jordan.gonzalez/appsec/support-application-and-api-protection branch from 3db3c34 to 23d8fb0 Compare July 25, 2025 13:30

update layer size for amd64

f4cea50

duncanista changed the title ~~feat: serverless App & API Protection~~ poc: App & API Protection Jul 25, 2025

RomainMuller closed this Aug 5, 2025

RomainMuller deleted the jordan.gonzalez/appsec/support-application-and-api-protection branch August 5, 2025 07:54


		// Note: We intentionally don't set `actor.ip` because we don't have a definitive signal here.

		//TODO(romain.marcadier): Figure out whether we can set "_dd.runtime_family" here; and to what value.

poc: App & API Protection #695

poc: App & API Protection #695

Uh oh!

Conversation

duncanista commented Jun 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RomainMuller commented Aug 5, 2025

Uh oh!

Uh oh!