How to Use Semaphores
               by Gene Cooperman (gene@ccs.neu.edu)
	       Copyright (c) 2021 -- all rights reserved

Semaphores are the lowest-level thread synchronization construct in
common usage that is normally used between two threads.  One thread calls
sem_wait() and another thread calls sem_post() on the same semaphore.

Note that in contrast, a mutex lock/unlock pair is normally executed
by a _single_ thread.  if a thread calls pthread_mutex_lock(), that
same thread should call pthread_mutex_unlock().  The behavior would be
undefined if one thread calls pthread_mutex_lock(), and a second thread
then called pthread_mutex_unlock() on the same mutex.

The two most common uses of semaphores are:
  A. producer-consmuer (aka bounded buffer)
       This uses two sempahores, and also a mutex used to create
       a thread-safe circular buffer.
  B. Enforcing a certain order of operations
       This uses just a single sempahore.

Case B is discussed under "comments in depth", concerning order violation.

Case A uses a standard well-known pattern:
  Circular buffer:
    int buffer[N];
    int next_idx = 0;
    int last_idx = 0;
    pthread_mutex_t buffer_mutex;
    void put_in_buffer(int elt) {
      pthread_mutex_lock(&buffer_mutex);
      buffer[last_idx] = elt;
      last_idx = (last_idx + 1) % N;
      pthread_mutex_unlock(&buffer_mutex);
    }
    int get_from_buffer() {
      pthread_mutex_lock(&buffer_mutex);
      int elt = buffer[next_idx]; // Copy this now, while we are protected
                                  // by the guard mutex.
      next_idx = (next_idx + 1) % N;
      pthread_mutex_unlock(&buffer_mutex);
      return elt; // Return the copy here; Never return while holding lock.
    }
  Threads:
    sem_t sem_producer; // count is the number of empty slots
                        // producer waits when no empty slots
    sem_t sem_consumer; // count is the number of available slots
                        // consumer waits when no available slots
    void producer() { // start function of producer thread
      sem_wait(&sem_producer, ...);
      put_in_buffer(...);
      sem_post(&sem_consumer, ...);
    }
    void consumer() { // start function of consumer thread
      sem_wait(&sem_consumer, ...);
      get_from_buffer(...);
      sem_post(&sem_producer, ...);
    }

COMMENTS IN DEPTH:

In production code, users of sem_wait should check its return value.
A typical paradigm is:
    int rc = -1;
    do {
      rc = sem_wait(&semaphore, ...);
    } while (rc == -1 && errno == EINTR);
This is because sem_wait() can be interrupted by a signal (e.g.,
by SIGWINCH, if the terminal window is resized).

Order violation:
Three of the most common bugs for multi-threaded programs are deadlock,
atomicity, and order violation.  Order violation occurs when two
statements by two different threads are executed in the wrong order.
Semaphores are an excellent way to fix order violations:
 Thread 1: <Statement A>;
           sem_post(&sem, ...);
 Thread 2: sem_wait(sem, ...);
           <Statement B>;

Avoiding order violation bugs is required to ensure ordering for
startup of a newly created thread, or to wait while an existing thread
accomplishes a task for the requesting thread.
(It could also be used to force the parent thread to wait while the
child thread finishes, but in this case, the parent might raterh use
pthread_join(child_thread, NULL);)