Packet buffer¶
Packet buffer (or pbuf) is buffer manager to handle received data from any connection. It is optimized to construct big buffer of smaller chunks of fragmented data as received bytes are not always coming as single packet.
Pbuf block diagram¶
Image above shows structure of pbuf chain. Each pbuf consists of:
Pointer to next pbuf, or
NULL
when it is last in chainLength of current packet length
Length of current packet and all next in chain
If pbuf is last in chain, total length is the same as current packet length
Reference counter, indicating how many pointers point to current pbuf
Actual buffer data
Top image shows 3
pbufs connected to single chain.
There are 2
custom pointer variables to point at different pbuf structures. Second pbuf has reference counter set to 2
, as 2
variables point to it:
next of pbuf 1 is the first one
User variable 2 is the second one
Block number |
Next pbuf |
Block size |
Total size in chain |
Reference counter |
---|---|---|---|---|
Block 1 |
Block 2 |
|
|
|
Block 2 |
Block 3 |
|
|
|
Block 3 |
|
|
|
|
Reference counter¶
Reference counter holds number of references (or variables) pointing to this block. It is used to properly handle memory free operation, especially when pbuf is used by lib core and application layer.
Note
If there would be no reference counter information and application would free memory while another part of library still uses its reference, application would invoke undefined behavior and system could crash instantly.
When application tries to free pbuf chain as on first image, it would normally call lwesp_pbuf_free()
function. That would:
Decrease reference counter by
1
If reference counter
== 0
, it removes it from chain list and frees packet buffer memoryIf reference counter
!= 0
after decrease, it stops free procedureGo to next pbuf in chain and repeat steps
As per first example, result of freeing from user variable 1 would look similar to image and table below.
First block (blue) had reference counter set to 1
prior freeing operation.
It was successfully removed as user variable 1 was the only one pointing to it,
while second (green) block had reference counter set to 2
, preventing free operation.
Block number |
Next pbuf |
Block size |
Total size in chain |
Reference counter |
---|---|---|---|---|
Block 2 |
Block 3 |
|
|
|
Block 3 |
|
|
|
|
Note
Block 1 has been successfully freed, but since block 2 had reference counter set to 2
before, it was only decreased by 1
to a new value 1
and free operation stopped instead.
User variable 2 is still using pbuf starting at block 2 and must manually call lwesp_pbuf_free()
to free it.
Concatenating vs chaining¶
This section will explain difference between concat and chain operations.
Both operations link 2
pbufs together in a chain of pbufs,
difference is that chain operation increases reference counter to linked pbuf,
while concat keeps reference counter at its current status.
Concat operation¶
Concat operation shall be used when 2
pbufs are linked together and reference to second is no longer used.
After concating 2 pbufs together, reference counter of second is still set to 1
, however we can see that 2
pointers point to second pbuf.
Note
After application calls lwesp_pbuf_cat()
, it must not use pointer which points to second pbuf.
This would invoke undefined behavior if one pointer tries to free memory while second still points to it.
An example code showing proper usage of concat operation:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | lwesp_pbuf_p a, b;
/* Create 2 pbufs of different sizes */
a = lwesp_pbuf_new(10);
b = lwesp_pbuf_new(20);
/* Link them together with concat operation */
/* Reference on b will stay as is, won't be increased */
lwesp_pbuf_cat(a, b);
/*
* Operating with b variable has from now on undefined behavior,
* application shall stop using variable b to access pbuf.
*
* The best way would be to set b reference to NULL
*/
b = NULL;
/*
* When application doesn't need pbufs anymore,
* free a and it will also free b
*/
lwesp_pbuf_free(a);
|
Chain operation¶
Chain operation shall be used when 2
pbufs are linked together and reference to second is still required.
After chainin 2 pbufs together, reference counter of second is increased by 1
, which allows application to reference second pbuf separatelly.
Note
After application calls lwesp_pbuf_chain()
,
it also has to manually free its reference using lwesp_pbuf_free()
function.
Forgetting to free pbuf invokes memory leak
An example code showing proper usage of chain operation:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | lwesp_pbuf_p a, b;
/* Create 2 pbufs of different sizes */
a = lwesp_pbuf_new(10);
b = lwesp_pbuf_new(20);
/* Chain both pbufs together */
/* This will increase reference on b as 2 variables now point to it */
lwesp_pbuf_chain(a, b);
/*
* When application does not need a anymore, it may free it
* This will free only pbuf a, as pbuf b has now 2 references:
* - one from pbuf a
* - one from variable b
*/
/* If application calls this, it will free only first pbuf */
/* As there is link to b pbuf somewhere */
lwesp_pbuf_free(a);
/* Reset a variable, not used anymore */
a = NULL;
/*
* At this point, b is still valid memory block,
* but when application doesn't need it anymore,
* it should free it, otherwise memory leak appears
*/
lwesp_pbuf_free(b);
/* Reset b variable */
b = NULL;
|
Extract pbuf data¶
Each pbuf holds some amount of data bytes. When multiple pbufs are linked together (either chained or concated), blocks of raw data are not linked to contiguous memory block. It is necessary to process block by block manually.
An example code showing proper reading of any pbuf:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | const void* data;
size_t pos, len;
lwesp_pbuf_p a, b, c;
const char str_a[] = "This is one long";
const char str_a[] = "string. We want to save";
const char str_a[] = "chain of pbufs to file";
/* Create pbufs to hold these strings */
a = lwesp_pbuf_new(strlen(str_a));
b = lwesp_pbuf_new(strlen(str_b));
c = lwesp_pbuf_new(strlen(str_c));
/* Write data to pbufs */
lwesp_pbuf_take(a, str_a, strlen(str_a), 0);
lwesp_pbuf_take(b, str_b, strlen(str_b), 0);
lwesp_pbuf_take(c, str_c, strlen(str_c), 0);
/* Connect pbufs together */
lwesp_pbuf_chain(a, b);
lwesp_pbuf_chain(a, c);
/*
* pbuf a now contains chain of b and c together
* and at this point application wants to print (or save) data from chained pbuf
*
* Process pbuf by pbuf with code below
*/
/*
* Get linear address of current pbuf at specific offset
* Function will return pointer to memory address at specific position
* and `len` will hold length of data block
*/
pos = 0;
while ((data = lwesp_pbuf_get_linear_addr(a, pos, &len)) != NULL) {
/* Custom process function... */
/* Process data with data pointer and block length */
process_data(data, len);
printf("Str: %.*s", len, data);
/* Increase offset position for next block */
pos += len;
}
/* Call free only on a pbuf. Since it is chained, b and c will be freed too */
lwesp_pbuf_free(a);
|
-
group
LWESP_PBUF
Packet buffer manager.
Typedefs
-
typedef struct lwesp_pbuf *
lwesp_pbuf_p
¶ Pointer to lwesp_pbuf_t structure.
Functions
-
lwesp_pbuf_p
lwesp_pbuf_new
(size_t len)¶ Allocate packet buffer for network data of specific size.
- Return
Pointer to allocated memory,
NULL
otherwise- Parameters
[in] len
: Length of payload memory to allocate
-
size_t
lwesp_pbuf_free
(lwesp_pbuf_p pbuf)¶ Free previously allocated packet buffer.
- Return
Number of freed pbufs from head
- Parameters
[in] pbuf
: Packet buffer to free
-
void *
lwesp_pbuf_data
(const lwesp_pbuf_p pbuf)¶ Get data pointer from packet buffer.
- Return
Pointer to data buffer on success,
NULL
otherwise- Parameters
[in] pbuf
: Packet buffer
-
size_t
lwesp_pbuf_length
(const lwesp_pbuf_p pbuf, uint8_t tot)¶ Get length of packet buffer.
- Return
Length of data in units of bytes
- Parameters
[in] pbuf
: Packet buffer to get length for[in] tot
: Set to1
to return total packet chain length or0
to get only first packet length
-
uint8_t
lwesp_pbuf_set_length
(lwesp_pbuf_p pbuf, size_t new_len)¶ Set new length of pbuf.
- Note
New length can only be smaller than existing one. It has no effect when greater than existing one
- Note
This function can be used on single-chain pbufs only, without
next
pbuf in chain- Return
1
on success,0
otherwise- Parameters
[in] pbuf
: Pbuf to make it smaller[in] new_len
: New length in units of bytes
-
lwespr_t
lwesp_pbuf_take
(lwesp_pbuf_p pbuf, const void *data, size_t len, size_t offset)¶ Copy user data to chain of pbufs.
-
size_t
lwesp_pbuf_copy
(lwesp_pbuf_p pbuf, void *data, size_t len, size_t offset)¶ Copy memory from pbuf to user linear memory.
- Return
Number of bytes copied
- Parameters
[in] pbuf
: Pbuf to copy from[out] data
: User linear memory to copy to[in] len
: Length of data in units of bytes[in] offset
: Possible start offset in pbuf
-
lwespr_t
lwesp_pbuf_cat
(lwesp_pbuf_p head, const lwesp_pbuf_p tail)¶ Concatenate
2
packet buffers together to one big packet.- Note
After
tail
pbuf has been added tohead
pbuf chain, it must not be referenced by user anymore as it is now completely controlled byhead
pbuf. In simple words, when user calls this function, it should not call lwesp_pbuf_free function anymore, as it might make memory undefined forhead
pbuf.- Return
lwespOK on success, member of lwespr_t enumeration otherwise
- See
- Parameters
[in] head
: Head packet buffer to append new pbuf to[in] tail
: Tail packet buffer to append to head pbuf
-
lwespr_t
lwesp_pbuf_chain
(lwesp_pbuf_p head, lwesp_pbuf_p tail)¶ Chain 2 pbufs together. Similar to lwesp_pbuf_cat but now new reference is done from head pbuf to tail pbuf.
- Note
After this function call, user must call lwesp_pbuf_free to remove its reference to tail pbuf and allow control to head pbuf:
lwesp_pbuf_free(tail)
- Return
lwespOK on success, member of lwespr_t enumeration otherwise
- See
- Parameters
[in] head
: Head packet buffer to append new pbuf to[in] tail
: Tail packet buffer to append to head pbuf
-
lwesp_pbuf_p
lwesp_pbuf_unchain
(lwesp_pbuf_p head)¶ Unchain first pbuf from list and return second one.
tot_len
andlen
fields are adjusted to reflect new values and reference counter isas is
- Note
After unchain, user must take care of both pbufs (
head
andnew returned one
)- Return
Next pbuf after
head
- Parameters
[in] head
: First pbuf in chain to remove from chain
-
lwespr_t
lwesp_pbuf_ref
(lwesp_pbuf_p pbuf)¶ Increment reference count on pbuf.
-
uint8_t
lwesp_pbuf_get_at
(const lwesp_pbuf_p pbuf, size_t pos, uint8_t *el)¶ Get value from pbuf at specific position.
- Return
1
on success,0
otherwise- Parameters
[in] pbuf
: Pbuf used to get data from[in] pos
: Position at which to get element[out] el
: Output variable to save element value at desired position
-
size_t
lwesp_pbuf_memcmp
(const lwesp_pbuf_p pbuf, const void *data, size_t len, size_t offset)¶ Compare pbuf memory with memory from data.
- Note
Compare is done on entire pbuf chain
- Return
0
if equal,LWESP_SIZET_MAX
if memory/offset too big or anything between if not equal- See
- Parameters
[in] pbuf
: Pbuf used to compare with data memory[in] data
: Actual data to compare with[in] len
: Length of input data in units of bytes[in] offset
: Start offset to use when comparing data
-
size_t
lwesp_pbuf_strcmp
(const lwesp_pbuf_p pbuf, const char *str, size_t offset)¶ Compare pbuf memory with input string.
- Note
Compare is done on entire pbuf chain
- Return
0
if equal,LWESP_SIZET_MAX
if memory/offset too big or anything between if not equal- See
- Parameters
[in] pbuf
: Pbuf used to compare with data memory[in] str
: String to be compared with pbuf[in] offset
: Start memory offset in pbuf
-
size_t
lwesp_pbuf_memfind
(const lwesp_pbuf_p pbuf, const void *data, size_t len, size_t off)¶ Find desired needle in a haystack.
- Return
LWESP_SIZET_MAX
if no match or position where in pbuf we have a match- See
- Parameters
[in] pbuf
: Pbuf used as haystack[in] needle
: Data memory used as needle[in] len
: Length of needle memory[in] off
: Starting offset in pbuf memory
-
size_t
lwesp_pbuf_strfind
(const lwesp_pbuf_p pbuf, const char *str, size_t off)¶ Find desired needle (str) in a haystack (pbuf)
- Return
LWESP_SIZET_MAX
if no match or position where in pbuf we have a match- See
- Parameters
[in] pbuf
: Pbuf used as haystack[in] str
: String to search for in pbuf[in] off
: Starting offset in pbuf memory
-
uint8_t
lwesp_pbuf_advance
(lwesp_pbuf_p pbuf, int len)¶ Advance pbuf payload pointer by number of len bytes. It can only advance single pbuf in a chain.
- Note
When other pbufs are referencing current one, they are not adjusted in length and total length
- Return
1
on success,0
otherwise- Parameters
[in] pbuf
: Pbuf to advance[in] len
: Number of bytes to advance. when negative is used, buffer size is increased only if it was decreased before
-
lwesp_pbuf_p
lwesp_pbuf_skip
(lwesp_pbuf_p pbuf, size_t offset, size_t *new_offset)¶ Skip a list of pbufs for desired offset.
- Note
Reference is not changed after return and user must not free the memory of new pbuf directly
- Return
New pbuf on success,
NULL
otherwise- Parameters
[in] pbuf
: Start of pbuf chain[in] offset
: Offset in units of bytes to skip[out] new_offset
: Pointer to output variable to save new offset in returned pbuf
-
void *
lwesp_pbuf_get_linear_addr
(const lwesp_pbuf_p pbuf, size_t offset, size_t *new_len)¶ Get linear offset address for pbuf from specific offset.
- Note
Since pbuf memory can be fragmented in chain, you may need to call function multiple times to get memory for entire pbuf chain
- Return
Pointer to memory on success,
NULL
otherwise- Parameters
[in] pbuf
: Pbuf to get linear address[in] offset
: Start offset from where to start[out] new_len
: Length of memory returned by function
-
void
lwesp_pbuf_set_ip
(lwesp_pbuf_p pbuf, const lwesp_ip_t *ip, lwesp_port_t port)¶ Set IP address and port number for received data.
- Parameters
[in] pbuf
: Packet buffer[in] ip
: IP to assing to packet buffer[in] port
: Port number to assign to packet buffer
-
void
lwesp_pbuf_dump
(lwesp_pbuf_p p, uint8_t seq)¶ Dump and debug pbuf chain.
- Parameters
[in] p
: Head pbuf to dump[in] seq
: Set to1
to dump allpbufs
in linked list or0
to dump first one only
-
struct
lwesp_pbuf_t
¶ - #include <lwesp_private.h>
Packet buffer structure.
Public Members
-
struct lwesp_pbuf *
next
¶ Next pbuf in chain list
-
size_t
tot_len
¶ Total length of pbuf chain
-
size_t
len
¶ Length of payload
-
size_t
ref
¶ Number of references to this structure
-
uint8_t *
payload
¶ Pointer to payload memory
-
lwesp_ip_t
ip
¶ Remote address for received IPD data
-
lwesp_port_t
port
¶ Remote port for received IPD data
-
struct lwesp_pbuf *
-
typedef struct lwesp_pbuf *