FrançaisGet started

Locks and concurrent file access

Stopping two instances of a script from stepping on each other: the race, the test-then-act flaw, atomic operations, mkdir, set -C, flock, the lock left after a crash and writing through a temporary file then renaming.

  • 30 cards
  • 5 topics
  • Free
Study this deck

The outline, topic by topic

Each topic is weighted the way the exam weighs it.

  1. Concepts
  2. Questions
  3. Sort and link
  4. Scenarios
  5. Understand

A few cards from the deck

  • Race condition: what is it?

    Show the answer

    A flaw where the result depends on the unpredictable order in which two simultaneous runs interleave their instructions.

    It shows up rarely and at the worst moment, which makes it hard to reproduce. Two instances of a script that test then create a lock can both pass the test.

  • Why can two instances of this script run the processing at the same time?

    ```bash
    if [ ! -e verrou ]; then
    touch verrou
    traiter
    rm verrou
    fi
    `

    • The second can pass the test before the first has done touch
    • rm verrou runs before touch in the second instance
    • The -e test only looks at directories and never at ordinary files
    • touch only creates the file if traiter has finished without error
    Show the answer

    The second can pass the test before the first has done touch

    The test and the touch are two instructions: between them, the other instance also sees that verrou does not exist. A single operation that tests and creates is needed, like mkdir.

  • With bash, set … (noclobber) makes a > redirection fail on a file that already exists: creating the file becomes atomic.

    Show the answer

    With bash, set -C (noclobber) makes a > redirection fail on a file that already exists: creating the file becomes atomic.

    The message is “cannot overwrite existing file” and the code is 1. It is a lock without mkdir: set -C; echo $$ > verrou.

  • Test then act: what is it?

    Show the answer

    A flaw where the state checked (the file does not exist) has changed when you act on it, because the check and the action are two separate instructions.

    In [ -e verrou ] || touch verrou, another process can create verrou between the two. The cure is a single operation that tests and acts together: mkdir, flock.

  • The site sometimes reads a half-written configuration file. The deployment script regenerates it with generer > config.json. What is the cause, and the fix?

    • The site reads too fast just after the deployment: make it wait a few seconds before it reopens the file
    • The file is too big for a single write: split it into several smaller files that are assembled at read time
    • Writing in place truncates the file then fills it: write alongside in a temporary file in the same directory, then mv
    • The file system is corrupt: unmount it, repair it with fsck then run the whole site deployment script again from the start
    Show the answer

    Writing in place truncates the file then fills it: write alongside in a temporary file in the same directory, then mv

    As soon as > opens the file, its old content is gone and the new content arrives in pieces: a reader can land in the middle. The rename, by contrast, replaces the file in one go.

Study this deck