Lsof improvements (show deleted as in lsof output) + files_only argument #1827

SolitudePy · 2025-06-04T19:22:53Z

Improvements for Lsof to show deleted files as in lsof output on live system.
its probably shouldnt be in that function directly as its no longer mimic prepend_path kernel function
this PR breaks commit 68b51e8 but it can easily be fixed if desired.

(venv) ubuntu@ubuntuPC:~/Dev/volatility3$ vol -f ~/dumps/deleted_proc_fd_dump.raw -r json linux.lsof --pid 8321
Volatility 3 Framework 2.26.2
/home/ubuntu/Dev/volatility3/volatility3/framework/deprecation.py:105: FutureWarning: This plugin (PluginRequirement) has been renamed and will be removed in the first release after 2026-06-01. PluginRequirement is to be deprecated. Use VersionRequirement instead.
  warnings.warn(
Progress:  100.00               Stacking attempts finished           
[
...
...
  {
    "Accessed": "2025-06-04T18:18:04.438000+00:00",
    "Changed": "2025-06-04T18:15:11.438000+00:00",
    "Device": "0:23",
    "FD": 2,
    "Inode": 4,
    "Mode": "crw--w----",
    "Modified": "2025-06-04T18:18:04.438000+00:00",
    "PID": 8321,
    "Path": "/dev/pts/1",
    "Process": "copied_bash",
    "Size": 0,
    "TID": 8321,
    "Type": "CHR",
    "__children": []
  },
  {
    "Accessed": "2025-06-04T17:49:01.296000+00:00",
    "Changed": "2025-06-04T18:17:41.693000+00:00",
    "Device": "253:0",
    "FD": 255,
    "Inode": 201866930,
    "Mode": "-rw-r--r--",
    "Modified": "2025-06-04T17:48:56.399000+00:00",
    "PID": 8321,
    "Path": "/tmp/evil.sh (deleted)",
    "Process": "copied_bash",
    "Size": 19,
    "TID": 8321,
    "Type": "REG",
    "__children": []
  }
]

(venv) ubuntu@ubuntuPC:~/Dev/volatility3$ vol -f ~/dumps/deleted_proc_fd_dump.raw linux.lsof --files-only | grep deleted
/home/ubuntu/Dev/volatility3/volatility3/framework/deprecation.py:105: FutureWarning: This plugin (PluginRequirement) has been renamed and will be removed in the first release after 2026-06-01. PluginRequirement is to be deprecated. Use VersionRequirement instead.
  warnings.warn(
799gress799100.0systemd-udevd   8tacking/var/lib/sss/mc/group (deleted) 253:0   201339099       REG     -rw-rw-r--      2025-06-04 17:46:02.113000 UTC  2025-06-04 17:46:02.113000 UTC   2025-06-04 17:46:01.736000 UTC  6940392
799     799     systemd-udevd   9       /var/lib/sss/mc/passwd (deleted)        253:0   201866915       REG     -rw-rw-r--      2025-06-04 17:46:02.109000 UTC  2025-06-04 17:46:02.109000 UTC   2025-06-04 17:46:01.724000 UTC  9253600
906     906     auditd  4       /var/lib/sss/mc/group (deleted) 253:0   201339099       REG     -rw-rw-r--      2025-06-04 17:46:02.113000 UTC  2025-06-04 17:46:02.113000 UTC  2025-06-04 17:46:01.736000 UTC   6940392
906     907     auditd  4       /var/lib/sss/mc/group (deleted) 253:0   201339099       REG     -rw-rw-r--      2025-06-04 17:46:02.113000 UTC  2025-06-04 17:46:02.113000 UTC  2025-06-04 17:46:01.736000 UTC   6940392
...
960     1218    gmain   9       / (deleted)     0:1     26770   REG     -rwxrwxrwx      2025-06-04 17:46:02.397000 UTC  2025-06-04 17:46:02.397000 UTC  2025-06-04 17:46:02.397000 UTC   4096
961     961     sssd_nss        6       /var/lib/sss/mc/passwd (deleted)        253:0   201866915       REG     -rw-rw-r--      2025-06-04 17:46:02.109000 UTC  2025-06-04 17:46:02.109000 UTC   2025-06-04 17:46:01.724000 UTC  9253600

ikelos

Thanks for this, a couple of things fell out of it. Firstly, messing with the core linux extensions can have knock on effects, which is why we version everything. There is a way of updating it so that not everything breaks, but that one's a show stopper.

The other is just about appending to filenames to represent information about the file. If we're going to do that, it has to be unambiguously separable from the filename (ie, you could always tell what was the tag and what was the filename).

volatility3/framework/symbols/linux/__init__.py

ikelos · 2025-06-11T21:33:47Z

volatility3/framework/symbols/linux/__init__.py

            return f"<potentially smeared> {path}"
+
+        if inode and inode.is_readable() and inode.is_valid() and inode.i_nlink == 0:
+            path = f"{LinuxUtilities.deleted} {path}"


We tend to put constants into the constants.linux module, as all caps? Again, I'm not sure it's worthwhile for a tag like this?

what if the tag value will change in the future? I can see other modules rely on this tag to check specifics...

not sure if it fits there, I have only seen constants from linux kernel header files

I'm really not keen on tacking flags at the start of the path. It feels more and more like there should be a structure or a method that can determine this, that's separate to the filename. I don't really like it for smear, but at least the filename can be smeared. It's not the filename that's deleted, it's the file, this should be a property of the file object, or something you can pass the file to which tells you whether it's been deleted or not. That then also solves the need for string based flags to say meta things about the file that shouldn't actually be part of the filename.

how about the logic in 8f60299?

As a follow up to my own comment, we already show (deleted) correctly, which was fixed in the parity push, in our d_path replication, so whatever new PRs replace/enhance the existing code need to keep it. Link here:

volatility3/volatility3/framework/symbols/linux/__init__.py

Line 263 in 1eb8717

return "/" + pre_name + " (deleted)"

Ok, so my bad, apparently this is how the kernel does it (and it does it at the end). Sorry for messing you about @SolitudePy . I do still need @atcuno to do a review of it (because clearly I don't know linux that deeply), but he's very busy in the run up to blackhat/defcon so this may have to just wait a while I'm afraid. In the interim, feel free to move the string back to the end of the filename, sorry for the confusion I caused! 5:S

well I absolutely agree that it shouldnt be in the end for the problems you have mentioned, but thats convention. were there other issues I should fix except that?

@SolitudePy @ikelos what is the latest on this now that the "(deleted)" string is understood?

It's probably waiting on me. I'm still not happy about it, but apparently this is how the kernel does it. I absolutely wash my hands of resolving any difficulties that arise later from us doing things this way though. It seems ill thought out by the kernel to conflate the filename and its deleted state. I'll merge it though, thanks for your work on it (and trying your best to appease our differing opinions @SolitudePy )

volatility3/framework/symbols/linux/extensions/__init__.py

ikelos

Much improved, last little bits and then it should be good to go.

ikelos

Still not happy with tagging filesnames with a string to identify them. It feels like a quick solution that could be setting us up for a whole heap of trouble in the future. Hopefully @atcuno can give this a look and suggest a better means of handling this?

volatility3/framework/plugins/linux/lsof.py

ikelos · 2025-06-22T22:15:17Z

volatility3/framework/symbols/linux/__init__.py

            return f"<potentially smeared> {path}"
+
+        if inode and inode.is_readable() and inode.is_valid() and inode.i_nlink == 0:
+            path = f"{LinuxUtilities.deleted} {path}"


I'm really not keen on tacking flags at the start of the path. It feels more and more like there should be a structure or a method that can determine this, that's separate to the filename. I don't really like it for smear, but at least the filename can be smeared. It's not the filename that's deleted, it's the file, this should be a property of the file object, or something you can pass the file to which tells you whether it's been deleted or not. That then also solves the need for string based flags to say meta things about the file that shouldn't actually be part of the filename.

ikelos · 2025-06-22T22:20:44Z

volatility3/framework/plugins/linux/lsof.py


            fd_generator = linux.LinuxUtilities.files_descriptors_for_process(
-                context, linuxutils_symbol_table, task
+                context, linuxutils_symbol_table, task, files_only=include_files_only


Sure this can be determined from the file desriptor, and therefore doesn't require tampering with the files_descriptors_for_process method? I really can't believe that we get this structure back, but we can only separate out files from not files within that method? I'd also prefer that we just add it to the file descriptor to say "type is file" rather than this overly specific additional parameter added to this method...

you mean the method files_descriptors_for_process should return file type as well that will then be used in linux.lsof to filter out non-files? right now linux.lsof uses files_descriptors_for_process directly, which uses LinuxUtilities.path_for_file - and that's where the filtering happens via dname_is_valid, so both methods need to have this argument. or I can also reuse the dname_is_valid logic in linux.lsof which seems less elegant?

I meant it feels like it should be one of the fd_fields returned in the generator and then handled by FDInternal. It might be a lot of work to thread it in that way, and I haven't even considered the version bumps that might be required to achieve it, plus I've no idea whether @atcuno would be happy doing that, but there's no indication that there aren't more fields (requiring more boolean flags to be added to the method) still to come. As such I'd like to find an extensible way of adding those flags, and given the entire method is about returning a set of flags, it feels as though this should be added to that existing set of flags...

ikelos

Sorry, my view on these hasn't really changed. Rather than designing the data flow properly, we're trying to carry the information by cramming it into other fields which is then extremely difficult to separate at a later date. It's the wrong way of doing it, and though I don't know how much work the right way would be, this is not it.

ikelos · 2025-07-07T18:26:21Z

volatility3/framework/plugins/linux/lsof.py


            fd_generator = linux.LinuxUtilities.files_descriptors_for_process(
-                context, linuxutils_symbol_table, task
+                context, linuxutils_symbol_table, task, files_only=include_files_only


I meant it feels like it should be one of the fd_fields returned in the generator and then handled by FDInternal. It might be a lot of work to thread it in that way, and I haven't even considered the version bumps that might be required to achieve it, plus I've no idea whether @atcuno would be happy doing that, but there's no indication that there aren't more fields (requiring more boolean flags to be added to the method) still to come. As such I'd like to find an extensible way of adding those flags, and given the entire method is about returning a set of flags, it feels as though this should be added to that existing set of flags...

ikelos · 2025-07-07T18:28:47Z

volatility3/framework/symbols/linux/__init__.py

            return f"<potentially smeared> {path}"
+
+        if inode and inode.is_readable() and inode.is_valid() and inode.i_nlink == 0:
+            path = f"{LinuxUtilities.deleted} {path}"


That's still fundamentally trying to cram to pieces of information into a single field. Whether the file has been deleted or not is distinct from the filename it has. We should be using a separate piece of information to track that, not overloading the filename field with two pieces of information that we'll later have difficult separating.

ikelos requested changes Jun 11, 2025

View reviewed changes

volatility3/framework/symbols/linux/__init__.py Outdated Show resolved Hide resolved

volatility3/framework/symbols/linux/__init__.py Outdated Show resolved Hide resolved

volatility3/framework/symbols/linux/__init__.py Show resolved Hide resolved

SolitudePy requested a review from ikelos June 11, 2025 16:50

ikelos reviewed Jun 11, 2025

View reviewed changes

volatility3/framework/symbols/linux/extensions/__init__.py Outdated Show resolved Hide resolved

ikelos reviewed Jun 11, 2025

View reviewed changes

SolitudePy requested a review from ikelos June 12, 2025 18:28

ikelos reviewed Jun 22, 2025

View reviewed changes

ikelos requested a review from atcuno June 22, 2025 22:21

ikelos assigned atcuno Jun 22, 2025

SolitudePy requested a review from ikelos June 26, 2025 10:36

ikelos reviewed Jul 7, 2025

View reviewed changes

SolitudePy requested a review from ikelos July 11, 2025 12:29

SolitudePy added 12 commits July 11, 2025 15:39

linux_utilities show deleted fd + lsof files_only arg

2f8aecc

debug print remove

bf68bb6

black

e29a01e

declare default value :((

2efb7f5

black

e0dc649

prepend <deleted> for deleted sock & file instead of (deleted)

23e4d35

files_only default arg & minor version bump

0a8f2cd

move config arg to run()

c64c622

remove code for next PR

606323f

create potential smear tag

c471707

plugins: bump lsof minor version

7ed7d43

plugins: lsof change back to (deleted)

eb79008

SolitudePy force-pushed the lsof_deleted branch from f5981e6 to eb79008 Compare July 11, 2025 12:40

ikelos merged commit 0dd8d3a into volatilityfoundation:develop Sep 28, 2025
14 checks passed

SolitudePy deleted the lsof_deleted branch September 29, 2025 04:28

Lsof improvements (show deleted as in lsof output) + files_only argument #1827

Lsof improvements (show deleted as in lsof output) + files_only argument #1827

Uh oh!

Conversation

SolitudePy commented Jun 4, 2025

Uh oh!

ikelos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ikelos left a comment

Choose a reason for hiding this comment

Uh oh!

ikelos left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikelos left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants