Everything you never wanted to know about file locking (2010)
74 points
5 days ago
| 7 comments
| apenwarr.ca
| HN
chasil
6 hours ago
[-]
One sure way to get a lock is to make a directory.

  #!/bin/sh

  if mkdir /your/lockdir
  then trap "rmdir /your/lockdir" EXIT INT ABRT TERM
       ...code goes here...
  else echo somebody else has the lock
  fi
No matter how many processes attempt to make the directory, only one will succeed. That works for my scripting, but I have never used it in C.
reply
acuozzo
3 hours ago
[-]
Is this guaranteed to be atomic on all filesystems?
reply
chasil
2 hours ago
[-]
For POSIX, yes.

https://rcrowley.org/2010/01/06/things-unix-can-do-atomicall...

Windows has a deep well of POSIX in the kernel (plus hard file locks), and it appears to hold there.

https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem

reply
jofla_net
5 hours ago
[-]
this is great thanks,

was just wondering, could something else remove the dir in between the if and then, before trap?

Just wondering about the atomicity.

reply
chasil
4 hours ago
[-]
The permissions on the parent and lock directory could restrict the access to a specific user and group, but yes, other processes could interfere with this locking if directed to do so.

One condition where this interference is helpful is a crash, where a @reboot entry in the crontab could:

  [ -d /your/lockdir ] && rmdir /your/lockdir
You would also not want to place the lock directory in /tmp or otherwise where other users could manipulate (or see) it. In Red Hat, there is a /var/run/lock directory that might be appropriate.

My biggest use case for directory locking in scripts is handling inotify events.

reply
cryptonector
2 hours ago
[-]
The problem with lock files and lock directories is that if the lock holder dies without cleaning up you now need to do something to clean up.
reply
mscdex
1 hour ago
[-]
On Linux, this is why I always turn to using abstract sockets when I only need local locking. Only one process can bind and the kernel cleans up automatically on process exit.

You could do the same thing with TCP/UDP, but abstract sockets give you more flexibility in naming with 108 characters vs. being forced to use a 16-bit integer. Also it means you aren't using up a port that could otherwise be used for actual network communication.

Abstract sockets also make for a nice process existence monitoring mechanism since any processes connected to the bound socket are guaranteed to be immediately notified when the process dies.

reply
formerly_proven
5 hours ago
[-]
Yes, but that is not a weakness in the locking.
reply
Bratmon
6 hours ago
[-]
Usually when I read these writeups, I walk away thinking "Wow, $foo was a more complicated problem than I thought".

With this one, it was "Wow, $foo was a simpler problem than I thought and Unix (and thus Linux and OSX) just totally screwed it up for no reason"

reply
jabl
4 hours ago
[-]
As TFA mentions, Unix/POSIX locking is insane.

Note that this page is slightly outdated wrt. flock(). From the manpage (online at https://man7.org/linux/man-pages/man2/flock.2.html):

>

       Since Linux 2.0, flock() is implemented as a system call in its
       own right rather than being emulated in the GNU C library as a
       call to fcntl(2).  With this implementation, there is no
       interaction between the types of lock placed by flock() and
       fcntl(2), and flock() does not detect deadlock.  (Note, however,
       that on some systems, such as the modern BSDs, flock() and
       fcntl(2) locks do interact with one another.)

   CIFS details
       Up to Linux 5.4, flock() is not propagated over SMB.  A file with
       such locks will not appear locked for remote clients.

       Since Linux 5.5, flock() locks are emulated with SMB byte-range
       locks on the entire file.  Similarly to NFS, this means that
       fcntl(2) and flock() locks interact with one another.  Another
       important side-effect is that the locks are not advisory anymore:
       any IO on a locked file will always fail with EACCES when done
       from a separate file descriptor.  This difference originates from
       the design of locks in the SMB protocol, which provides mandatory
       locking semantics.

       Remote and mandatory locking semantics may vary with SMB protocol,
       mount options and server type.  See mount.cifs(8) for additional
       information.

   NFS details
       Up to Linux 2.6.11, flock() does not lock files over NFS (i.e.,
       the scope of locks was limited to the local system).  Instead, one
       could use fcntl(2) byte-range locking, which does work over NFS,
       given a sufficiently recent version of Linux and a server which
       supports locking.

       Since Linux 2.6.12, NFS clients support flock() locks by emulating
       them as fcntl(2) byte-range locks on the entire file.  This means
       that fcntl(2) and flock() locks do interact with one another over
       NFS.  It also means that in order to place an exclusive lock, the
       file must be opened for writing.

       Since Linux 2.6.37, the kernel supports a compatibility mode that
       allows flock() locks (and also fcntl(2) byte region locks) to be
       treated as local; see the discussion of the local_lock option in
       nfs(5).
reply
krautburglar
52 minutes ago
[-]
It would be nice to have a unix for the new millenium--one that discards everything the greybeards deem a mistake. The window of opportunity is closing. We won't have them much longer.
reply
pseudohadamard
3 days ago
[-]
Another good read is the SQLite locking module, https://www.sqlite.org/src/artifact/0240c5b547b4cf585c8cac35..., since these guys have to deal with the insanity of locking across different systems in real life.

You know things are bad when the least awful implementation of OS-level locking is the one from Microsoft.

reply
Number-Six
4 days ago
[-]
So good in depth post. THANK YOU.
reply
IshKebab
4 hours ago
[-]
Hmm I just ran into an issue with uv where it deadlocks because of something to do with file locking on NFS. This looks informative!
reply