Semaphores


Process Synchronization With Semaphores

One of the problems when writing multi-process application is the need to synchronize various operations between the processes. Communicating requests using pipes, sockets and message queues is one way to do it. however, sometimes we need to synchronize operations amongst more then two processes, or to synchronize access to shmem data resources that might be accessed by several processes in parallel. Semaphores are a means supplied with SysV IPC that allow us to synchronize such operations.


What Is A Semaphore? What Is A Semaphore Set?

A semaphore is a resource that contains an integer value, and allows processes to synchronize by testing and setting this value in a single atomic operation. This means that the process that tests the value of a semaphore and sets it to a different value (based on the test), is guaranteed no other process will interfere with the operation in the middle.

Two types of operations can be carried on a semaphore: wait and signal. A set operation first checks if the semaphore's value equals some number. If it does, it decreases its value and returns. If it does not, the operation blocks the calling process until the semaphore's value reaches the desired value. A signal operation increments the value of the semaphore, possibly awakening one or more processes that are waiting on the semaphore. How this mechanism can be put to practical use will be exlained later.

A semaphore set is a structure that stores a group of semaphores together, and possibly allows the process to commit a transaction on part or all of the semaphores in the set together. Here, a transaction means that we are guaranteed that either all operations are done successfully, or none is done at all. Note that a semaphore set is not a general parallel programming concept, it's just an extra mechanism supplied by SysV IPC.


Creating A Semaphore Set - semget()

Creation of a semaphore set is done using the semget() system call. We supply some ID for the set, and some flags (used to define access permission mode and a few options). We also supply the number of semaphores we want to have in the given set. This number is limited to SEMMSL, as defined in file /usr/include/sys/sem.h. Lets see an example:


/* ID of the semaphore set.     */
int sem_set_id_1;
int sem_set_id_2;

/* create a private semaphore set with one semaphore in it, */
/* with access only to the owner.                           */
sem_set_id_1 = semget(IPC_PRIVATE, 1, IPC_CREAT | 0600);
if (sem_set_id_1 == -1) {
    perror("main: semget");
    exit(1);
}

/* create a semaphore set with ID 250, three semaphores */
/* in the set, with access only to the owner.           */
sem_set_id_2 = semget(250, 3, IPC_CREAT | 0600);
if (sem_set_id_2 == -1) {
    perror("main: semget");
    exit(1);
}

Note that in the second case, if a semaphore set with ID 250 already existed, we would get access to the existing set, rather then a new set be created. This works just like it worked with message queues.


Setting And Getting Semaphore Values With semctl()

After the semaphore set is created, we need to initialize the value of the semaphores in the set. We do that using the semctl() system call. Note that this system call has other uses, but they are not relevant to our needs right now. Lets assume we want to set the values of the three semaphores in our second set to values 3, 6 and 0, respectively. The ID of the first semaphore in the set is '0', the ID of the second semaphore is '1', and so on.



/* use this to store return values of system calls.   */
int rc;

/* initialize the first semaphore in our set to '3'.  */
rc = semctl(sem_set_id_2, 0, SETVAL, 3);
if (rc == -1) {
    perror("main: semctl");
    exit(1);
}

/* initialize the second semaphore in our set to '6'. */
rc = semctl(sem_set_id_2, 1, SETVAL, 6);
if (rc == -1) {
    perror("main: semctl");
    exit(1);
}

/* initialize the third semaphore in our set to '0'.  */
rc = semctl(sem_set_id_2, 2, SETVAL, 0);
if (rc == -1) {
    perror("main: semctl");
    exit(1);
}

There are one comment to be made about the way we used semctl() here. According to the manual, the last parameter for this system call should be a union of type union semun. However, since the SETVAL (set value) operation only uses the int val part of the union, we simply passed an integer to the function. The proper way to use this system call was to define a variable of this union type, and set its value appropriately, like this:


/* use this variable to pass the value to the semctl() call */
union semun sem_val;

/* initialize the first semaphore in our set to '3'. */
sem_val.val = 3;
rc = semctl(sem_set_id_2, 2, SETVAL, sem_val);
if (rc == -1) {
    perror("main: semctl");
    exit(1);
}

We used the first form just for simplicity. From now on, we will only use the second form.


Using Semaphores For Mutual Exclusion With semop()

Sometimes we have a resource that we want to allow only one process at a time to manipulate. For example, we have a file that we only want written into only by one process at a time, to avoid corrupting its contents. Of-course, we could use various file locking mechanisms to protect the file, but we will demonstrate the usage of semaphores for this purpose as an example. Later on we will see the real usage of semaphores, to protect access to shared memory segments. Anyway, here is a code snippest. It assumes the semaphore in our set whose id is "sem_set_id" was initialized to 1 initially:


/* this function updates the contents of the file with the given path name. */
void update_file(char* file_path, int number)
{
    /* structure for semaphore operations.   */
    struct sembuf sem_op;
    FILE* file;

    /* wait on the semaphore, unless it's value is non-negative. */
    sem_op.sem_num = 0;
    sem_op.sem_op = -1;   /* <-- Comment 1 */
    sem_op.sem_flg = 0;
    semop(sem_set_id, &sem_op, 1);

    /* Comment 2 */
    /* we "locked" the semaphore, and are assured exclusive access to file.  */
    /* manipulate the file in some way. for example, write a number into it. */
    file = fopen(file_path, "w");
    if (file) {
        fprintf(file, "%d\n", number);
        fclose(file);
    }

    /* finally, signal the semaphore - increase its value by one. */
    sem_op.sem_num = 0;
    sem_op.sem_op = 1;   /* <-- Comment 3 */
    sem_op.sem_flg = 0;
    semop(sem_set_id, &sem_op, 1);
}

This code needs some explanations, especially regarding the semantics of the semop() calls.

  1. Comment 1 - before we access the file, we use semop() to wait on the semaphore. Supplying '-1' in sem_op.sem_op means: If the value of the semaphore is greater then or equal to '1', decrease this value by one, and return to the caller. Otherwise (the value is 1 or less), block the calling process, until the value of the semaphore becomes '1', at which point we return to the caller.
  2. Comment 2 - The semantics of semop() assure us that when we return from this function, the value of the semaphore is 0. Why?
  3. Comment 3 - after we are done manipulating the file, we increase the value of the semaphore by 1, possibly waking up a process waiting on the semaphore. If several processes are waiting on the semaphore, the first that got blocked on it is wakened and continues its execution.
  4. The logic for three cases for the value of sem_op is
    1. sem_op < 0:
               if (semval >= ABS(sem_op))
               {
                set semval to semval - ABS(sem_op)
               }
               else
               {
                 if (sem_flg & IPC_NOWAIT)
                   return -1 immediately
                 else
                 {
                 wait until semval reaches or exceeds ABS(sem_op)
                 then subtract ABS(sem_op) as above
                 }
               }
              
    2. sem_op > 0:
      The value of sem_op is addto to the corresponding semval. Other processes waiting on the new value of the semaphore wil be woken up.
    3. sem_op = 0:
      Wait until semval becomes zero, but semval is not altered. If IPC_NOWAIT is set in sem_flg, and semval is not already zero, then semop returns an error immediately

    Now, lets assume that any process that tries to access the file, does it only via a call to our "update_file" function. As you can see, when it goes through the function, it always decrements the value of the semaphore by 1, and then increases it by 1. Thus, the semaphore's value can never go above its initial value, which is '1'. Now lets check two scenarios:

    1. No other process is executing the "update_file" concurrently. In this case, when we enter the function, the semaphore's value is '1'. after the first semop() call, the value of the semaphore is decremented to '0', and thus our process is not blocked. We continue to execute the file update, and with the second semop() call, we raise the value of the semaphore back to '1'.
    2. Another process is in the middle of the "update_file" function. If it already managed to pass the first call to semop(), the value of the semaphore is '0', and when we call semop(), our process is blocked. When the other process signals the semaphore with the second semop() call, it increases the value of the semaphore back to '0', and it wakes up the process blocked on the semaphore, which is our process. We now get into executing the file handling code, and finally we raise the semaphore's value back to '1' with our second call to semop().

    We have the source code for a program demonstrating the mutex concept, in the file named sem-mutex.c. The program launches several processes (5, as defined by the NUM_PROCS macro), each of which is executing the "update_file" function several times in a row, and then exits.


    Using Semaphores For Producer-Consumer Operations With semop()

    Using a semaphore as a mutex is not utilizing the full power of the semaphore. As we saw, a semaphore contains a counter, that may be used for more complex operations. Those operations often use a programming model called "producer-consumer". In this model, we have one or more processes that produce something, and one or more processes that consume that something. For example, one set of processes accept printing requests from clients and place them in a spool directory, and another set of processes take the files from the spool directory and actually print them using the printer.

    To control such a printing system, we need the producers to maintain a count of the number of files waiting in the spool directory and incrementing it for every new file placed there. The consumers check this counter, and whenever it gets above zero, one of them grabs a file from the spool, and sends it to the printer. If there are no files in the spool (i.e. the counter value is zero), all consumer processes get blocked.

    Lets see how we can use a semaphore as a counter. We still use the same two operations on the semaphore, namely "signal" and "wait".

    
    /* this variable will contain the semaphore set. */
    int sem_set_id;
    
    /* semaphore value, for semctl().                */
    union semun sem_val;
    
    /* structure for semaphore operations.           */
    struct sembuf sem_op;
    
    /* first we create a semaphore set with a single semaphore, */
    /* whose counter is initialized to '0'.                     */
    sem_set_id = semget(IPC_PRIVATE, 1, 0600);
    if (sem_set_id == -1) {
        perror("semget");
        exit(1);
    }
    sem_val.val = 0;
    semctl(sem_set_id, 0, SETVAL, sem_val);
    
    /* we now do some producing function, and then signal the   */
    /* semaphore, increasing its counter by one.                */
    .
    .
    sem_op.sem_num = 0;
    sem_op.sem_op = 1;
    sem_op.sem_flg = 0;
    semop(sem_set_id, &sem_op, 1);
    .
    .
    .
    /* meanwhile, in a different process, we try to consume the      */
    /* resource protected (and counter) by the semaphore.            */
    /* we block on the semaphore, unless it's value is non-negative. */
    sem_op.sem_num = 0;
    sem_op.sem_op = -1;
    sem_op.sem_flg = 0;
    semop(sem_set_id, &sem_op, 1);
    
    /* when we get here, it means that the semaphore's value is '0'  */
    /* or more, so there's something to consume.                     */
    .
    .
    

    Note that our "wait" and "signal" operations here are just like we did with when using the semaphore as a mutex. The only difference is in who is doing the "wait" and the "signal". With a mutex, the same process did both the "wait" and the "signal" (in that order). In the producer-consumer example, one process is doing the "signal" operation, while the other is doing the "wait" operation.

    The full source code for a simple program that implements a producer-consumer system with two processes, is found in the file sem-producer-consumer.c.


    Semaphores - A Complete Example

    As a complete example of using semaphores, we write a very simple print spool system. Two separate programs will be used. One runs as the printing command, and is found in the file tiny-lpr.c. It gets a file path on its command line, and copies this file into the spool area, increasing a global (on-private) semaphore by one. Another program runs as the printer daemon, and is found in the file tiny-lpd.c. It waits on the same global semaphore, and whenever its value is larger then one, it locates a file in the spool directory and sends it to the printer. In order to avoid race conditions when copying files into the directory and removing files from this directory, a second semaphore will be used as a mutex, to protect the spool directory. The complete tiny-spooler mini-project is found in the tiny-spool directory.

    One problem might be that copying a file takes a lot of time, and thus locking the spool directory for a long while. In order to avoid that, 3 directories will be used. One serves as a temporary place for tiny-lpr to copy files into. One will be used as the common spool directory, and one will be used as a temporary directory into which tiny-lpd will move the files before printing them. By putting all 3 directories on the same disk, we assure that files can be moved between them using the rename() system call, in one fast operation (regardless of the file size).