Thread safety

Ring buffers are effectively used in embedded systems with or without operating systems. Common problem most of implementations have is linked to multi-thread environment (when using OS) or reading/writing from/to interrupts. This is linked to common question What happens if I write to buffer while another thread is reading from it?

One of the main requirements (beside being lightweight) of LwRB was to allow read-while-write or write-while-read operations. This is achieved only when there is single write entry point and single read exit point.

Write and read operation with single entry and exit points

Write and read operation with single entry and exit points

Often called and used as pipe to write (for example) raw data to the buffer allowing another task to process the data from another thread.

Note

No race-condition is introduced when application uses LwRB with single write entry and single read exit point. LwRB uses C11 standard stdatomic.h library to ensure read and write operations are race-free for any platform supporting C11 and its respected atomic library.

Thread (or interrupt) safety, with one entry and one exit points, is achieved by storing actual buffer read and write pointer variables to the local ones before performing any calculation. Therefore multiple conditional checks are guaranteed to be performed on the same local variables, even if actual buffer pointers get modified.

  • Read pointer could get changed by interrupt or another thread when application tries to write to buffer

  • Write pointer could get changed by interrupt or another thread when application ties to read from buffer

Note

Even single entry and single exit points may introduce race condition, especially on smaller system, such as 8-bit or 16-bit system, or in general, where arbitrary type (normaly size_t) is sizeof(type) < architecture_size. This is solved by C11 atomic library, that ensures atomic reads and writes to key structure members

Thread safety gets completely broken when application does one of the following:

  • Uses multiple write entry points to the single LwRB instance

  • Uses multiple read exit points to the single LwRB instance

  • Uses multiple read/write exit/entry points to the same LwRB instance

Write operation to same LwRB instance from 2 threads

Write operation to same LwRB instance from 2 threads. Write protection is necessary to ensure thread safety.

Write operation to same LwRB instance from main loop and interrupt context

Write operation to same LwRB instance from main loop and interrupt context. Write protection is necessary to ensure thread safety.

Read operation from same LwRB instance from 2 threads

Read operation from same LwRB instance from 2 threads. Read protection is necessary to ensure thread safety.

Read from and write to operations using LwRB instance from multiple threads

Read and write operations are executed from multiple threads. Both, read and write, operations require exclusive access.

Above use cases are examples when thread safety gets broken. Application must ensure exclusive access only to the part in dashed-red rectangle.

Thread safety example
 1/* Declare variables */
 2lwrb_t rb;
 3
 4/* 2 mutexes, one for write operations,
 5    one for read operations */
 6mutex_t m_w, m_r;
 7
 8/* 4 threads below, 2 for write, 2 for read */
 9void
10thread_write_1(void* arg) {
11    /* Use write mutex */
12    while (1) {
13        mutex_get(&m_w);
14        lwrb_write(&rb, ...);
15        mutex_give(&m_w);
16    }
17}
18
19void
20thread_write_2(void* arg) {
21    /* Use write mutex */
22    while (1) {
23        mutex_get(&m_w);
24        lwrb_write(&rb, ...);
25        mutex_give(&m_w);
26    }
27}
28
29void
30thread_read_1(void* arg) {
31    /* Use read mutex */
32    while (1) {
33        mutex_get(&m_r);
34        lwrb_read(&rb, ...);
35        mutex_give(&m_r);
36    }
37}
38
39void
40thread_read_2(void* arg) {
41    /* Use read mutex */
42    while (1) {
43        mutex_get(&m_r);
44        lwrb_read(&rb, ...);
45        mutex_give(&m_r);
46    }
47}

Read and write operations can be used simultaneously hence it is perfectly valid if access is granted to read operation while write operation from one thread takes place.

Note

2 different mutexes are used for read and write due to the implementation, allowing application to use buffer in read-while-write and write-while-read mode. Mutexes are used to prevent write-while-write and read-while-read operations respectively

Tip

For multi-entry-point-single-exit-point use case, read mutex is not necessary. For single-entry-point-multi-exit-point use case, write mutex is not necessary.

Tip

Functions considered as read operation are read, skip, peek and linear read. Functions considered as write operation are write, advance and linear write.