Skip to content

New DNS probes#222

Merged
haesbaert merged 7 commits intomainfrom
new-dns
Apr 1, 2025
Merged

New DNS probes#222
haesbaert merged 7 commits intomainfrom
new-dns

Conversation

@haesbaert
Copy link
Contributor

@haesbaert haesbaert commented Feb 17, 2025

This implements new probes for DNS based on cgroup socket programs.

We use BPF_PROG_TYPE_CGROUP_SOCK (cgroup/sock_{create,release}) for populating the
sk_to_tgid map, these have the process context as it's basically the path of
socket(2) and close(2).

We use BPF_PROG_TYPE_CGROUP_SOCK_ADDR (cgroup/{sendmsg4,recvmsg4,connect4}) also
for populating sk_to_tgid, here we end up overwriting the value from
sock_create, this is wanted since the socket might have been created in a
different process, then passed down to the children, which then does
sendmsg/recvmsg/connect, meaning we always get the "correct" tgid for an event.

We use BPF_PROG_TYPE_CGROUP_SKB (cgroup/{egress,ingress}) for the actual tapping
of the packet.

  • cap_len is how much we captured (like pcap)
  • orig_len is how much we wanted to capture (like pcap)
  • direction is ingress/egress
  • tgid is tgid
  • data is the full packet, including ip/udp headers (like pcap)

Contrary to the other probes, we can't stash the full process context, as we
really only have tgid, it would make little sense to stash all the process info
in some map and then send it later. This is a departure from tracing and
"context full probes". We already have all the needed context in userland, both
in quark and $OTHER_PRODUCT.

Main motivation for this change is that the old probes would fail for non-linear
skbs.

Cgroup probes need the file descriptor of the root cgroup, and bluebox doesn't
mount it for us, so we mount it manually.

Regarding the weirdness in probes

  • Old verifiers are sensitive and not always update umin for a scalar value in the
    case of a JNZ, see inline comments.
  • Old verifiers also do not like spilled bpf-helpers calls (when the argument is
    in the stack), see inline comments.

Regarding tests

I've zapped the old udp_send.c program, as it makes little sense since we can do
it all in goland. We now also send and receive a packet and verify the contents,
the tests in quark are a bit more advanced and check the ip/udp header as well,
but this is good enough imho.

This implements new probes for DNS based on cgroup socket programs.

We use BPF_PROG_TYPE_CGROUP_SOCK (cgroup/sock_{create,release}) for populating the
sk_to_tgid map, these have the process context as it's basically the path of
socket(2) and close(2).

We use BPF_PROG_TYPE_CGROUP_SOCK_ADDR (cgroup/{sendmsg4,recvmsg4,connect4}) also
for populating sk_to_tgid, here we end up overwriting the value from
sock_create, this is wanted since the socket might have been created in a
different process, then passed down to the children, which then does
sendmsg/recvmsg/connect, meaning we always get the "correct" tgid for an event.

We use BPF_PROG_TYPE_CGROUP_SKB (cgroup/{egress,ingress}) for the actual tapping
of the packet.

 - cap_len is how much we captured (like pcap)
 - orig_len is how much we wanted to capture (like pcap)
 - direction is ingress/egress
 - tgid is tgid
 - data is the full packet, including ip/udp headers (like pcap)

Contrary to the other probes, we can't stash the full process context, as we
really only have tgid, it would make little sense to stash all the process info
in some map and then send it later. This is a departure from tracing and
"context full probes". We already have all the needed context in userland, both
in quark and $OTHER_PRODUCT.

Main motivation for this change is that the old probes would fail for non-linear
skbs.

Cgroup probes need the file descriptor of the root cgroup, and bluebox doesn't
mount it for us, so we mount it manually.

Regarding the weirdness in probes:

- Old verifiers are sensitive and not always update umin for a scalar value in the
  case of a JNZ, see inline comments.
- Old verifiers also do not like spilled bpf-helpers calls (when the argument is
  in the stack), see inline comments.

Regarding tests:

I've zapped the old udp_send.c program, as it makes little sense since we can do
it all in goland. We now also send and receive a packet and verify the contents,
the tests in quark are a bit more advanced and check the ip/udp header as well,
but this is good enough imho.
@haesbaert haesbaert marked this pull request as ready for review March 26, 2025 14:41
@haesbaert haesbaert requested a review from a team as a code owner March 26, 2025 14:41
@haesbaert haesbaert changed the title New dns - draft New dns Mar 26, 2025
@haesbaert haesbaert changed the title New dns New DNS probes Mar 26, 2025
haesbaert and others added 4 commits March 26, 2025 16:39
Co-authored-by: Nicholas Berlin <56366649+nicholasberlin@users.noreply.github.com>
Co-authored-by: Nicholas Berlin <56366649+nicholasberlin@users.noreply.github.com>
Co-authored-by: Nicholas Berlin <56366649+nicholasberlin@users.noreply.github.com>
@haesbaert
Copy link
Contributor Author

I think I've addressed all points, thanks for spotting them!

Copy link
Contributor

@nicholasberlin nicholasberlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM. Thanks.

Copy link
Contributor

@fearful-symmetry fearful-symmetry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly interesting in the documentation here, but I feel like the cgroup root thing is worth investigating.

@haesbaert haesbaert merged commit 981f5b7 into main Apr 1, 2025
26 checks passed
@haesbaert haesbaert deleted the new-dns branch April 1, 2025 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants