Show HN: Watch what files any Linux process accesses (opens in new tab)

(github.com)

131 pointsspieglt6y ago37 comments

37 comments

32 comments · 12 top-level

hnlmorg6y ago· 7 in thread

Not taking anything away from the worth of this tool but if you do happen to find yourself needing to quickly inspect which files a process has open you can do so using the /proc file system:

    ls -l /proc/$PID/fd/

Additionally you can also use the /proc file system to display where the cursor is in those files by outputting the contents of

    /proc/$PID/fdinfo/$FD

which is handy if you have a long running process but forgot to pipe it into `pv` (or any other long running ingest that lacks a progress UI)

(Both tricks are Linux only)

simcop23876y ago

Pv supports this directly with -d pid, http://manpages.ubuntu.com/manpages/bionic/man1/pv.1.html

hnlmorg6y ago

I learn something new every day. Thank you

1 more reply

im3w1l6y ago

The main drawback of this is that the program may access a file too quickly for a polling strategy to notice.

Wowfunhappy6y ago

On Mac, there's also lsof. (Also on Linux? I always assumed it was just a standard Unix tool, but maybe not?)

    lsof -c [process name]
    lsof -p [pid]

hnlmorg6y ago

It is available for Linux however it's not always part of the base install of every distribution (but a worthwhile install none-the-less).

Not taking anything away from lsof as I do use it myself but I do also think there is also merit in learning the /proc file system too because it helps illuminate some of the not-so-hidden magic behind Linux. Even if you then still find lsof or similar become your preferred utility.

thebruce87m6y ago

Admittedly my Linux skills are basic so I’m not sure how it compares, but I like the utility “progress” for monitoring things: https://github.com/Xfennec/progress/blob/master/README.md

Edit: By monitoring things, I mean like finding how far through a dd is after you have started it already.

PuffinBlue6y ago

Similarly, and perhaps obviously to some, you can use that to check how many files that process has open in total:

ls -al /proc/$PID/fd/ | wc -l

Useful for actually knowing what that specific process has open and is respecting your Max Open Files settings.

I'm looking at you Influx.

st0le6y ago· 6 in thread

IMO ProcMon on Windows is its equivalent. Not Process Explorer.

pjmlp6y ago

Since Windows Vista there is a mini version of it via "Open Resource Monitor" on the process explorer.

https://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-12...

ToFab1236y ago

There was news 2 years ago that MS was porting the sysinternals tool to linux. Did that ever happen?

https://mspoweruser.com/microsoft-working-on-sysinternals-fo...

fractalf6y ago

Ah, the system internal tools is one thing I really miss after switching to Linux on all my pcs. This port would be so awesome

pjmlp6y ago

Azure Insights provides similar kind of information.

Maybe that is where the porting effort went.

1 more reply

spiegltOP6y ago

You're right, thanks. Fixed.

pjmlp6y ago

You can also get the same information via Resource Monitor, available by default since Vista.

https://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-12...

MCOfficer6y ago· 3 in thread

Just a heads up (read: shameless plug), there's an AUR package:

https://aur.archlinux.org/packages/whatfiles-git/

spiegltOP6y ago

Wow, thank you very much! I've thought about trying to get a few of my projects into distribution repos but was somewhat intimidated by the process.

MCOfficer6y ago

You're welcome - but please check out #1/#2 so i can remove that ugly patch ;)

1 more reply

bavell6y ago

This is one of the reasons I love Arch so much :) Thank you!

bostonsre6y ago· 1 in thread

biotop and biolatency surface similar info. they come with a ton of other ridiculously awesome tools in BCC tools. they are a set of python wrapper scripts that run eBPF programs. using eBPF generally has a really low impact on performance when compared with other tools that do similar work.

https://github.com/iovisor/bcc

microtonal6y ago

bcc also has opensnoop, which is really nice for seeing which files are being opened:

https://github.com/iovisor/bcc/blob/master/tools/opensnoop_e...

ravinder_sbu6y ago· 1 in thread

How is this different from using something like,

`strace -e trace=file`

I see that you are using ptrace to monitor a process. That is also used by strace. Is there something else your application does that strace does not (In relation to files)?

capableweb6y ago

From the README:

> Isn't this just a reimplementation of strace -fe trace=creat,open,openat,unlink,unlinkat ./program?

> Yes. Though it aims to be simpler and more user friendly.

dkdk82836y ago· 1 in thread

Any reason why this is better than audit? I read README but i’m still not clear.

tyingq6y ago

Auditd is system wide (as would be inotify or fanotify based solutions).

This traces file events of a single process. Strace can be coaxed into something similar.

amelius6y ago· 1 in thread

Can it be invoked recursively?

Because strace on Linux still fails with:

    strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted

in those cases :(

spiegltOP6y ago

I have strace'd whatfiles, in fact that was a very useful way to debug a couple things, so maybe? I have not been able to attach to the same process with both whatfiles and strace, however.

Doctor_Fegg6y ago

For macOS, fs_usage does the same job. I find it invaluable to find out what process is churning the disk (usually mds...).

atrudeau6y ago

For doing the opposite - what processes access a given file - I like to use Audit (https://wiki.archlinux.org/index.php/Audit_framework#Audit_f...).

em5006y ago

This looks very similar to fatrace, which is already in the standard ubuntu and fedora repos.

edit: fatrace is system-wide, whereas the current tools monitors a specific process

http://manpages.ubuntu.com/manpages/trusty/man1/fatrace.1.ht...

https://piware.de/2012/02/fatrace-report-system-wide-file-ac...

unhammer6y ago

Lots more such tools at https://jvns.ca/debugging-zine.pdf (opensnoop-bpfcc and strace would be the most like this one)

Erwin6y ago

BTW, if you are using strace for this, check out the -y option recently added to strace. It will print the filename next to each file descriptor like this:

     read(3</proc/filesystems>, "", 1024)    = 0

Another interesting new strace option is -k which does a stack dump after each syscall. this can be useful to find out what part of the application, like some obscure lib, does weird system calls in your app.

j / k navigate · click thread line to collapse

37 comments

32 comments · 12 top-level

hnlmorg6y ago· 7 in thread

Not taking anything away from the worth of this tool but if you do happen to find yourself needing to quickly inspect which files a process has open you can do so using the /proc file system:

    ls -l /proc/$PID/fd/

Additionally you can also use the /proc file system to display where the cursor is in those files by outputting the contents of

    /proc/$PID/fdinfo/$FD

which is handy if you have a long running process but forgot to pipe it into `pv` (or any other long running ingest that lacks a progress UI)

(Both tricks are Linux only)

simcop23876y ago

Pv supports this directly with -d pid, http://manpages.ubuntu.com/manpages/bionic/man1/pv.1.html

hnlmorg6y ago

I learn something new every day. Thank you

1 more reply

im3w1l6y ago

The main drawback of this is that the program may access a file too quickly for a polling strategy to notice.

Wowfunhappy6y ago

On Mac, there's also lsof. (Also on Linux? I always assumed it was just a standard Unix tool, but maybe not?)

    lsof -c [process name]
    lsof -p [pid]

hnlmorg6y ago

It is available for Linux however it's not always part of the base install of every distribution (but a worthwhile install none-the-less).

thebruce87m6y ago

Admittedly my Linux skills are basic so I’m not sure how it compares, but I like the utility “progress” for monitoring things: https://github.com/Xfennec/progress/blob/master/README.md

Edit: By monitoring things, I mean like finding how far through a dd is after you have started it already.

PuffinBlue6y ago

Similarly, and perhaps obviously to some, you can use that to check how many files that process has open in total:

ls -al /proc/$PID/fd/ | wc -l

Useful for actually knowing what that specific process has open and is respecting your Max Open Files settings.

I'm looking at you Influx.

st0le6y ago· 6 in thread

IMO ProcMon on Windows is its equivalent. Not Process Explorer.

pjmlp6y ago

Since Windows Vista there is a mini version of it via "Open Resource Monitor" on the process explorer.

https://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-12...

ToFab1236y ago

There was news 2 years ago that MS was porting the sysinternals tool to linux. Did that ever happen?

https://mspoweruser.com/microsoft-working-on-sysinternals-fo...

fractalf6y ago

Ah, the system internal tools is one thing I really miss after switching to Linux on all my pcs. This port would be so awesome

pjmlp6y ago

Azure Insights provides similar kind of information.

Maybe that is where the porting effort went.

1 more reply

spiegltOP6y ago

You're right, thanks. Fixed.

pjmlp6y ago

You can also get the same information via Resource Monitor, available by default since Vista.

https://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-12...

MCOfficer6y ago· 3 in thread

Just a heads up (read: shameless plug), there's an AUR package:

https://aur.archlinux.org/packages/whatfiles-git/

spiegltOP6y ago

Wow, thank you very much! I've thought about trying to get a few of my projects into distribution repos but was somewhat intimidated by the process.

MCOfficer6y ago

You're welcome - but please check out #1/#2 so i can remove that ugly patch ;)

1 more reply

bavell6y ago

This is one of the reasons I love Arch so much :) Thank you!

bostonsre6y ago· 1 in thread

https://github.com/iovisor/bcc

microtonal6y ago

bcc also has opensnoop, which is really nice for seeing which files are being opened:

https://github.com/iovisor/bcc/blob/master/tools/opensnoop_e...

ravinder_sbu6y ago· 1 in thread

How is this different from using something like,

`strace -e trace=file`

I see that you are using ptrace to monitor a process. That is also used by strace. Is there something else your application does that strace does not (In relation to files)?

capableweb6y ago

From the README:

> Isn't this just a reimplementation of strace -fe trace=creat,open,openat,unlink,unlinkat ./program?

> Yes. Though it aims to be simpler and more user friendly.

dkdk82836y ago· 1 in thread

Any reason why this is better than audit? I read README but i’m still not clear.

tyingq6y ago

Auditd is system wide (as would be inotify or fanotify based solutions).

This traces file events of a single process. Strace can be coaxed into something similar.

amelius6y ago· 1 in thread

Can it be invoked recursively?

Because strace on Linux still fails with:

    strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted

in those cases :(

spiegltOP6y ago

I have strace'd whatfiles, in fact that was a very useful way to debug a couple things, so maybe? I have not been able to attach to the same process with both whatfiles and strace, however.

Doctor_Fegg6y ago

For macOS, fs_usage does the same job. I find it invaluable to find out what process is churning the disk (usually mds...).

atrudeau6y ago

For doing the opposite - what processes access a given file - I like to use Audit (https://wiki.archlinux.org/index.php/Audit_framework#Audit_f...).

em5006y ago

This looks very similar to fatrace, which is already in the standard ubuntu and fedora repos.

edit: fatrace is system-wide, whereas the current tools monitors a specific process

http://manpages.ubuntu.com/manpages/trusty/man1/fatrace.1.ht...

https://piware.de/2012/02/fatrace-report-system-wide-file-ac...

unhammer6y ago

Lots more such tools at https://jvns.ca/debugging-zine.pdf (opensnoop-bpfcc and strace would be the most like this one)

Erwin6y ago

BTW, if you are using strace for this, check out the -y option recently added to strace. It will print the filename next to each file descriptor like this:

     read(3</proc/filesystems>, "", 1024)    = 0

j / k navigate · click thread line to collapse