Packet buffer¶

Packet buffer (or pbuf) is buffer manager to handle received data from any connection. It is optimized to construct big buffer of smaller chunks of fragmented data as received bytes are not always coming as single packet.

Pbuf block diagram¶

Block diagram of pbuf chain¶

Image above shows structure of pbuf chain. Each pbuf consists of:

Pointer to next pbuf, or NULL when it is last in chain
Length of current packet length
Length of current packet and all next in chain
- If pbuf is last in chain, total length is the same as current packet length
Reference counter, indicating how many pointers point to current pbuf
Actual buffer data

Top image shows 3 pbufs connected to single chain. There are 2 custom pointer variables to point at different pbuf structures. Second pbuf has reference counter set to 2, as 2 variables point to it:

next of pbuf 1 is the first one
User variable 2 is the second one

Block structure¶
Block number	Next pbuf	Block size	Total size in chain	Reference counter
Block 1	Block 2	`150`	`550`	`1`
Block 2	Block 3	`130`	`400`	`2`
Block 3	`NULL`	`270`	`270`	`1`

Reference counter¶

Reference counter holds number of references (or variables) pointing to this block. It is used to properly handle memory free operation, especially when pbuf is used by lib core and application layer.

Note

If there would be no reference counter information and application would free memory while another part of library still uses its reference, application would invoke undefined behavior and system could crash instantly.

When application tries to free pbuf chain as on first image, it would normally call lwgsm_pbuf_free() function. That would:

Decrease reference counter by 1
If reference counter == 0, it removes it from chain list and frees packet buffer memory
If reference counter != 0 after decrease, it stops free procedure
Go to next pbuf in chain and repeat steps

As per first example, result of freeing from user variable 1 would look similar to image and table below. First block (blue) had reference counter set to 1 prior freeing operation. It was successfully removed as user variable 1 was the only one pointing to it, while second (green) block had reference counter set to 2, preventing free operation.

Block diagram of pbuf chain after free from user variable 1¶

Block diagram of pbuf chain after free from *user variable 1*¶
Block number	Next pbuf	Block size	Total size in chain	Reference counter
Block 2	Block 3	`130`	`400`	`1`
Block 3	`NULL`	`270`	`270`	`1`

Note

Block 1 has been successfully freed, but since block 2 had reference counter set to 2 before, it was only decreased by 1 to a new value 1 and free operation stopped instead. User variable 2 is still using pbuf starting at block 2 and must manually call lwgsm_pbuf_free() to free it.

Concatenating vs chaining¶

This section will explain difference between concat and chain operations. Both operations link 2 pbufs together in a chain of pbufs, difference is that chain operation increases reference counter to linked pbuf, while concat keeps reference counter at its current status.

Different pbufs, each pointed to by its own variable¶

Concat operation¶

Concat operation shall be used when 2 pbufs are linked together and reference to second is no longer used.

Structure after pbuf concat¶

After concating 2 pbufs together, reference counter of second is still set to 1, however we can see that 2 pointers point to second pbuf.

Note

After application calls lwgsm_pbuf_cat(), it must not use pointer which points to second pbuf. This would invoke undefined behavior if one pointer tries to free memory while second still points to it.

An example code showing proper usage of concat operation:

Packet buffer concat example¶

lwgsm_pbuf_p a, b;

/* Create 2 pbufs of different sizes */
a = lwgsm_pbuf_new(10);
b = lwgsm_pbuf_new(20);

/* Link them together with concat operation */
/* Reference on b will stay as is, won't be increased */
lwgsm_pbuf_cat(a, b);

/*
 * Operating with b variable has from now on undefined behavior,
 * application shall stop using variable b to access pbuf.
 *
 * The best way would be to set b reference to NULL
 */
b = NULL;

/*
 * When application doesn't need pbufs anymore,
 * free a and it will also free b
 */
lwgsm_pbuf_free(a);

Chain operation¶

Chain operation shall be used when 2 pbufs are linked together and reference to second is still required.

Structure after pbuf chain¶

After chainin 2 pbufs together, reference counter of second is increased by 1, which allows application to reference second pbuf separatelly.

Note

After application calls lwgsm_pbuf_chain(), it also has to manually free its reference using lwgsm_pbuf_free() function. Forgetting to free pbuf invokes memory leak

An example code showing proper usage of chain operation:

Packet buffer chain example¶

lwgsm_pbuf_p a, b;

/* Create 2 pbufs of different sizes */
a = lwgsm_pbuf_new(10);
b = lwgsm_pbuf_new(20);

/* Chain both pbufs together */
/* This will increase reference on b as 2 variables now point to it */
lwgsm_pbuf_chain(a, b);

/*
 * When application does not need a anymore, it may free it

 * This will free only pbuf a, as pbuf b has now 2 references:
 *  - one from pbuf a
 *  - one from variable b
 */

/* If application calls this, it will free only first pbuf */
/* As there is link to b pbuf somewhere */
lwgsm_pbuf_free(a);                               

/* Reset a variable, not used anymore */
a = NULL;

/*
 * At this point, b is still valid memory block,
 * but when application doesn't need it anymore,
 * it should free it, otherwise memory leak appears
 */
lwgsm_pbuf_free(b);

/* Reset b variable */
b = NULL;

Extract pbuf data¶

Each pbuf holds some amount of data bytes. When multiple pbufs are linked together (either chained or concated), blocks of raw data are not linked to contiguous memory block. It is necessary to process block by block manually.

An example code showing proper reading of any pbuf:

Packet buffer data extraction¶

const void* data;
size_t pos, len;
lwgsm_pbuf_p a, b, c;

const char str_a[] = "This is one long";
const char str_a[] = "string. We want to save";
const char str_a[] = "chain of pbufs to file";

/* Create pbufs to hold these strings */
a = lwgsm_pbuf_new(strlen(str_a));
b = lwgsm_pbuf_new(strlen(str_b));
c = lwgsm_pbuf_new(strlen(str_c));

/* Write data to pbufs */
lwgsm_pbuf_take(a, str_a, strlen(str_a), 0);
lwgsm_pbuf_take(b, str_b, strlen(str_b), 0);
lwgsm_pbuf_take(c, str_c, strlen(str_c), 0);

/* Connect pbufs together */
lwgsm_pbuf_chain(a, b);
lwgsm_pbuf_chain(a, c);

/*
 * pbuf a now contains chain of b and c together
 * and at this point application wants to print (or save) data from chained pbuf
 *
 * Process pbuf by pbuf with code below
 */

/*
 * Get linear address of current pbuf at specific offset
 * Function will return pointer to memory address at specific position
 * and `len` will hold length of data block
 */
pos = 0;
while ((data = lwgsm_pbuf_get_linear_addr(a, pos, &len)) != NULL) {
    /* Custom process function... */
    /* Process data with data pointer and block length */
    process_data(data, len);
    printf("Str: %.*s", len, data);

    /* Increase offset position for next block */
    pos += len;
}

/* Call free only on a pbuf. Since it is chained, b and c will be freed too */
lwgsm_pbuf_free(a);

group LWGSM_PBUF

Packet buffer manager.

Typedefs

typedef struct lwgsm_pbuf *lwgsm_pbuf_p¶: Pointer to lwgsm_pbuf_t structure.

Functions

lwgsm_pbuf_p lwgsm_pbuf_new(size_t len)¶

Allocate packet buffer for network data of specific size.

Return

Pointer to allocated memory, NULL otherwise

Parameters

[in] len: Length of payload memory to allocate

size_t lwgsm_pbuf_free(lwgsm_pbuf_p pbuf)¶

Free previously allocated packet buffer.

Return

Number of freed pbufs from head

Parameters

[in] pbuf: Packet buffer to free

void *lwgsm_pbuf_data(const lwgsm_pbuf_p pbuf)¶

Get data pointer from packet buffer.

Return

Pointer to data buffer on success, NULL otherwise

Parameters

[in] pbuf: Packet buffer

size_t lwgsm_pbuf_length(const lwgsm_pbuf_p pbuf, uint8_t tot)¶

Get length of packet buffer.

Return

Length of data in units of bytes

Parameters

[in] pbuf: Packet buffer to get length for
[in] tot: Set to 1 to return total packet chain length or 0 to get only first packet length

lwgsmr_t lwgsm_pbuf_take(lwgsm_pbuf_p pbuf, const void *data, size_t len, size_t offset)¶

Copy user data to chain of pbufs.

Return

lwgsmOK on success, member of lwgsmr_t enumeration otherwise

Parameters

[in] pbuf: First pbuf in chain to start copying to
[in] data: Input data to copy to pbuf memory
[in] len: Length of input data to copy
[in] offset: Start offset in pbuf where to start copying

size_t lwgsm_pbuf_copy(lwgsm_pbuf_p pbuf, void *data, size_t len, size_t offset)¶

Copy memory from pbuf to user linear memory.

Return

Number of bytes copied

Parameters

[in] pbuf: Pbuf to copy from
[out] data: User linear memory to copy to
[in] len: Length of data in units of bytes
[in] offset: Possible start offset in pbuf

lwgsmr_t lwgsm_pbuf_cat(lwgsm_pbuf_p head, const lwgsm_pbuf_p tail)¶

Concatenate 2 packet buffers together to one big packet.

Note

After tail pbuf has been added to head pbuf chain, it must not be referenced by user anymore as it is now completely controlled by head pbuf. In simple words, when user calls this function, it should not call lwgsm_pbuf_free function anymore, as it might make memory undefined for head pbuf.

Return

lwgsmOK on success, member of lwgsmr_t enumeration otherwise

See

lwgsm_pbuf_chain

Parameters

[in] head: Head packet buffer to append new pbuf to
[in] tail: Tail packet buffer to append to head pbuf

lwgsmr_t lwgsm_pbuf_chain(lwgsm_pbuf_p head, lwgsm_pbuf_p tail)¶

Chain 2 pbufs together. Similar to lwgsm_pbuf_cat but now new reference is done from head pbuf to tail pbuf.

Note

After this function call, user must call lwgsm_pbuf_free to remove its reference to tail pbuf and allow control to head pbuf: lwgsm_pbuf_free(tail)

Return

lwgsmOK on success, member of lwgsmr_t enumeration otherwise

See

lwgsm_pbuf_cat

Parameters

[in] head: Head packet buffer to append new pbuf to
[in] tail: Tail packet buffer to append to head pbuf

lwgsmr_t lwgsm_pbuf_ref(lwgsm_pbuf_p pbuf)¶

Increment reference count on pbuf.

Return

lwgsmOK on success, member of lwgsmr_t enumeration otherwise

Parameters

[in] pbuf: pbuf to increase reference

uint8_t lwgsm_pbuf_get_at(const lwgsm_pbuf_p pbuf, size_t pos, uint8_t *el)¶

Get value from pbuf at specific position.

Return

1 on success, 0 otherwise

Parameters

[in] pbuf: Pbuf used to get data from
[in] pos: Position at which to get element
[out] el: Output variable to save element value at desired position

size_t lwgsm_pbuf_memcmp(const lwgsm_pbuf_p pbuf, const void *data, size_t len, size_t offset)¶

Compare pbuf memory with memory from data.

Note

Compare is done on entire pbuf chain

Return

0 if equal, LWGSM_SIZET_MAX if memory/offset too big or anything between if not equal

See

lwgsm_pbuf_strcmp

Parameters

[in] pbuf: Pbuf used to compare with data memory
[in] data: Actual data to compare with
[in] len: Length of input data in units of bytes
[in] offset: Start offset to use when comparing data

size_t lwgsm_pbuf_strcmp(const lwgsm_pbuf_p pbuf, const char *str, size_t offset)¶

Compare pbuf memory with input string.

Note

Compare is done on entire pbuf chain

Return

0 if equal, LWGSM_SIZET_MAX if memory/offset too big or anything between if not equal

See

lwgsm_pbuf_memcmp

Parameters

[in] pbuf: Pbuf used to compare with data memory
[in] str: String to be compared with pbuf
[in] offset: Start memory offset in pbuf

size_t lwgsm_pbuf_memfind(const lwgsm_pbuf_p pbuf, const void *data, size_t len, size_t off)¶

Find desired needle in a haystack.

Return

LWGSM_SIZET_MAX if no match or position where in pbuf we have a match

See

lwgsm_pbuf_strfind

Parameters

[in] pbuf: Pbuf used as haystack
[in] needle: Data memory used as needle
[in] len: Length of needle memory
[in] off: Starting offset in pbuf memory

size_t lwgsm_pbuf_strfind(const lwgsm_pbuf_p pbuf, const char *str, size_t off)¶

Find desired needle (str) in a haystack (pbuf)

Return

LWGSM_SIZET_MAX if no match or position where in pbuf we have a match

See

lwgsm_pbuf_memfind

Parameters

[in] pbuf: Pbuf used as haystack
[in] str: String to search for in pbuf
[in] off: Starting offset in pbuf memory

uint8_t lwgsm_pbuf_advance(lwgsm_pbuf_p pbuf, int len)¶

Advance pbuf payload pointer by number of len bytes. It can only advance single pbuf in a chain.

Note

When other pbufs are referencing current one, they are not adjusted in length and total length

Return

1 on success, 0 otherwise

Parameters

[in] pbuf: Pbuf to advance
[in] len: Number of bytes to advance. when negative is used, buffer size is increased only if it was decreased before

lwgsm_pbuf_p lwgsm_pbuf_skip(lwgsm_pbuf_p pbuf, size_t offset, size_t *new_offset)¶

Skip a list of pbufs for desired offset.

Note

Reference is not changed after return and user must not free the memory of new pbuf directly

Return

New pbuf on success, NULL otherwise

Parameters

[in] pbuf: Start of pbuf chain
[in] offset: Offset in units of bytes to skip
[out] new_offset: Pointer to output variable to save new offset in returned pbuf

void *lwgsm_pbuf_get_linear_addr(const lwgsm_pbuf_p pbuf, size_t offset, size_t *new_len)¶

Get linear offset address for pbuf from specific offset.

Note

Since pbuf memory can be fragmented in chain, you may need to call function multiple times to get memory for entire pbuf chain

Return

Pointer to memory on success, NULL otherwise

Parameters

[in] pbuf: Pbuf to get linear address
[in] offset: Start offset from where to start
[out] new_len: Length of memory returned by function

void lwgsm_pbuf_set_ip(lwgsm_pbuf_p pbuf, const lwgsm_ip_t *ip, lwgsm_port_t port)¶

Set IP address and port number for received data.

Parameters

[in] pbuf: Packet buffer
[in] ip: IP to assing to packet buffer
[in] port: Port number to assign to packet buffer

struct lwgsm_pbuf_t¶

#include <lwgsm_private.h>

Packet buffer structure.

Public Members

struct lwgsm_pbuf *next¶: Next pbuf in chain list

size_t tot_len¶: Total length of pbuf chain

size_t len¶: Length of payload

size_t ref¶: Number of references to this structure

uint8_t *payload¶: Pointer to payload memory

lwgsm_ip_t ip¶: Remote address for received IPD data

lwgsm_port_t port¶: Remote port for received IPD data