Netconn API

Netconn API is addon on top of existing connection module and allows sending and receiving data with sequential API calls, similar to POSIX socket API.

It can operate in client or server mode and uses operating system features, such as message queues and semaphore to link non-blocking callback API for connections with sequential API for application thread.

Note

Connection API does not directly allow receiving data with sequential and linear code execution. All is based on connection event system. Netconn adds this functionality as it is implemented on top of regular connection API.

Warning

Netconn API are designed to be called from application threads ONLY. It is not allowed to call any of netconn API functions from within interrupt or callback event functions.

Netconn client

Netconn API client block diagram

Netconn API client block diagram

Above block diagram shows basic architecture of netconn client application. There is always one application thread (in green) which calls netconn API functions to interact with connection API in synchronous mode.

Every netconn connection uses dedicated structure to handle message queue for data received packet buffers. Each time new packet is received (red block, data received event), reference to it is written to message queue of netconn structure, while application thread reads new entries from the same queue to get packets.

Netconn client example
  1#include "netconn_client.h"
  2#include "lwesp/lwesp.h"
  3
  4/**
  5 * \brief           Host and port settings
  6 */
  7#define NETCONN_HOST        "example.com"
  8#define NETCONN_PORT        80
  9
 10/**
 11 * \brief           Request header to send on successful connection
 12 */
 13static const char
 14request_header[] = ""
 15                   "GET / HTTP/1.1\r\n"
 16                   "Host: " NETCONN_HOST "\r\n"
 17                   "Connection: close\r\n"
 18                   "\r\n";
 19
 20/**
 21 * \brief           Netconn client thread implementation
 22 * \param[in]       arg: User argument
 23 */
 24void
 25netconn_client_thread(void const* arg) {
 26    lwespr_t res;
 27    lwesp_pbuf_p pbuf;
 28    lwesp_netconn_p client;
 29    lwesp_sys_sem_t* sem = (void*)arg;
 30
 31    /*
 32     * First create a new instance of netconn
 33     * connection and initialize system message boxes
 34     * to accept received packet buffers
 35     */
 36    client = lwesp_netconn_new(LWESP_NETCONN_TYPE_TCP);
 37    if (client != NULL) {
 38        /*
 39         * Connect to external server as client
 40         * with custom NETCONN_CONN_HOST and CONN_PORT values
 41         *
 42         * Function will block thread until we are successfully connected (or not) to server
 43         */
 44        res = lwesp_netconn_connect(client, NETCONN_HOST, NETCONN_PORT);
 45        if (res == lwespOK) {                     /* Are we successfully connected? */
 46            printf("Connected to " NETCONN_HOST "\r\n");
 47            res = lwesp_netconn_write(client, request_header, sizeof(request_header) - 1);    /* Send data to server */
 48            if (res == lwespOK) {
 49                res = lwesp_netconn_flush(client);    /* Flush data to output */
 50            }
 51            if (res == lwespOK) {                 /* Were data sent? */
 52                printf("Data were successfully sent to server\r\n");
 53
 54                /*
 55                 * Since we sent HTTP request,
 56                 * we are expecting some data from server
 57                 * or at least forced connection close from remote side
 58                 */
 59                do {
 60                    /*
 61                     * Receive single packet of data
 62                     *
 63                     * Function will block thread until new packet
 64                     * is ready to be read from remote side
 65                     *
 66                     * After function returns, don't forgot the check value.
 67                     * Returned status will give you info in case connection
 68                     * was closed too early from remote side
 69                     */
 70                    res = lwesp_netconn_receive(client, &pbuf);
 71                    if (res == lwespCLOSED) {     /* Was the connection closed? This can be checked by return status of receive function */
 72                        printf("Connection closed by remote side...\r\n");
 73                        break;
 74                    } else if (res == lwespTIMEOUT) {
 75                        printf("Netconn timeout while receiving data. You may try multiple readings before deciding to close manually\r\n");
 76                    }
 77
 78                    if (res == lwespOK && pbuf != NULL) { /* Make sure we have valid packet buffer */
 79                        /*
 80                         * At this point read and manipulate
 81                         * with received buffer and check if you expect more data
 82                         *
 83                         * After you are done using it, it is important
 84                         * you free the memory otherwise memory leaks will appear
 85                         */
 86                        printf("Received new data packet of %d bytes\r\n", (int)lwesp_pbuf_length(pbuf, 1));
 87                        lwesp_pbuf_free(pbuf);    /* Free the memory after usage */
 88                        pbuf = NULL;
 89                    }
 90                } while (1);
 91            } else {
 92                printf("Error writing data to remote host!\r\n");
 93            }
 94
 95            /*
 96             * Check if connection was closed by remote server
 97             * and in case it wasn't, close it manually
 98             */
 99            if (res != lwespCLOSED) {
100                lwesp_netconn_close(client);
101            }
102        } else {
103            printf("Cannot connect to remote host %s:%d!\r\n", NETCONN_HOST, NETCONN_PORT);
104        }
105        lwesp_netconn_delete(client);             /* Delete netconn structure */
106    }
107
108    if (lwesp_sys_sem_isvalid(sem)) {
109        lwesp_sys_sem_release(sem);
110    }
111    lwesp_sys_thread_terminate(NULL);             /* Terminate current thread */
112}

Netconn server

Netconn API server block diagram

Netconn API server block diagram

When netconn is configured in server mode, it is possible to accept new clients from remote side. Application creates netconn server connection, which can only accept clients and cannot send/receive any data. It configures server on dedicated port (selected by application) and listens on it.

When new client connects, server callback function is called with new active connection event. Newly accepted connection is then written to server structure netconn which is later read by application thread. At the same time, netconn connection structure (blue) is created to allow standard send/receive operation on active connection.

Note

Each connected client has its own netconn connection structure. When multiple clients connect to server at the same time, multiple entries are written to connection accept message queue and are ready to be processed by application thread.

From this point, program flow is the same as in case of netconn client.

This is basic example for netconn thread. It waits for client and processes it in blocking mode.

Warning

When multiple clients connect at the same time to netconn server, they are processed one-by-one, sequentially. This may introduce delay in response for other clients. Check netconn concurrency option to process multiple clients at the same time

Netconn server with single processing thread
 1/*
 2 * Netconn server example is based on single thread
 3 * and it listens for single client only on port 23
 4 */
 5#include "netconn_server_1thread.h"
 6#include "lwesp/lwesp.h"
 7
 8/**
 9 * \brief           Basic thread for netconn server to test connections
10 * \param[in]       arg: User argument
11 */
12void
13netconn_server_1thread_thread(void* arg) {
14    lwespr_t res;
15    lwesp_netconn_p server, client;
16    lwesp_pbuf_p p;
17
18    /* Create netconn for server */
19    server = lwesp_netconn_new(LWESP_NETCONN_TYPE_TCP);
20    if (server == NULL) {
21        printf("Cannot create server netconn!\r\n");
22    }
23
24    /* Bind it to port 23 */
25    res = lwesp_netconn_bind(server, 23);
26    if (res != lwespOK) {
27        printf("Cannot bind server\r\n");
28        goto out;
29    }
30
31    /* Start listening for incoming connections with maximal 1 client */
32    res = lwesp_netconn_listen_with_max_conn(server, 1);
33    if (res != lwespOK) {
34        goto out;
35    }
36
37    /* Unlimited loop */
38    while (1) {
39        /* Accept new client */
40        res = lwesp_netconn_accept(server, &client);
41        if (res != lwespOK) {
42            break;
43        }
44        printf("New client accepted!\r\n");
45        while (1) {
46            /* Receive data */
47            res = lwesp_netconn_receive(client, &p);
48            if (res == lwespOK) {
49                printf("Data received!\r\n");
50                lwesp_pbuf_free(p);
51            } else {
52                printf("Netconn receive returned: %d\r\n", (int)res);
53                if (res == lwespCLOSED) {
54                    printf("Connection closed by client\r\n");
55                    break;
56                }
57            }
58        }
59        /* Delete client */
60        if (client != NULL) {
61            lwesp_netconn_delete(client);
62            client = NULL;
63        }
64    }
65    /* Delete client */
66    if (client != NULL) {
67        lwesp_netconn_delete(client);
68        client = NULL;
69    }
70
71out:
72    printf("Terminating netconn thread!\r\n");
73    if (server != NULL) {
74        lwesp_netconn_delete(server);
75    }
76    lwesp_sys_thread_terminate(NULL);
77}

Netconn server concurrency

Netconn API server concurrency block diagram

Netconn API server concurrency block diagram

When compared to classic netconn server, concurrent netconn server mode allows multiple clients to be processed at the same time. This can drastically improve performance and response time on clients side, especially when many clients are connected to server at the same time.

Every time server application thread (green block) gets new client to process, it starts a new processing thread instead of doing it in accept thread.

  • Server thread is only dedicated to accept clients and start threads

  • Multiple processing thread can run in parallel to send/receive data from multiple clients

  • No delay when multi clients are active at the same time

  • Higher memory footprint is necessary as there are multiple threads active

Netconn server with multiple processing threads
  1/*
  2 * Netconn server example is based on single "user" thread
  3 * which listens for new connections and accepts them.
  4 *
  5 * When a new client is accepted by server,
  6 * separate thread for client is created where
  7 * data is read, processed and send back to user
  8 */
  9#include "netconn_server.h"
 10#include "lwesp/lwesp.h"
 11
 12static void netconn_server_processing_thread(void* const arg);
 13
 14/**
 15 * \brief           Main page response file
 16 */
 17static const uint8_t
 18rlwesp_data_mainpage_top[] = ""
 19                           "HTTP/1.1 200 OK\r\n"
 20                           "Content-Type: text/html\r\n"
 21                           "\r\n"
 22                           "<html>"
 23                           "   <head>"
 24                           "       <link rel=\"stylesheet\" href=\"style.css\" type=\"text/css\" />"
 25                           "       <meta http-equiv=\"refresh\" content=\"1\" />"
 26                           "   </head>"
 27                           "   <body>"
 28                           "       <p>Netconn driven website!</p>"
 29                           "       <p>Total system up time: <b>";
 30
 31/**
 32 * \brief           Bottom part of main page
 33 */
 34static const uint8_t
 35rlwesp_data_mainpage_bottom[] = ""
 36                              "       </b></p>"
 37                              "   </body>"
 38                              "</html>";
 39
 40/**
 41 * \brief           Style file response
 42 */
 43static const uint8_t
 44rlwesp_data_style[] = ""
 45                    "HTTP/1.1 200 OK\r\n"
 46                    "Content-Type: text/css\r\n"
 47                    "\r\n"
 48                    "body { color: red; font-family: Tahoma, Arial; };";
 49
 50/**
 51 * \brief           404 error response
 52 */
 53static const uint8_t
 54rlwesp_error_404[] = ""
 55                   "HTTP/1.1 404 Not Found\r\n"
 56                   "\r\n"
 57                   "Error 404";
 58
 59/**
 60 * \brief           Netconn server thread implementation
 61 * \param[in]       arg: User argument
 62 */
 63void
 64netconn_server_thread(void const* arg) {
 65    lwespr_t res;
 66    lwesp_netconn_p server, client;
 67
 68    /*
 69     * First create a new instance of netconn
 70     * connection and initialize system message boxes
 71     * to accept clients and packet buffers
 72     */
 73    server = lwesp_netconn_new(LWESP_NETCONN_TYPE_TCP);
 74    if (server != NULL) {
 75        printf("Server netconn created\r\n");
 76
 77        /* Bind network connection to port 80 */
 78        res = lwesp_netconn_bind(server, 80);
 79        if (res == lwespOK) {
 80            printf("Server netconn listens on port 80\r\n");
 81            /*
 82             * Start listening for incoming connections
 83             * on previously binded port
 84             */
 85            res = lwesp_netconn_listen(server);
 86
 87            while (1) {
 88                /*
 89                 * Wait and accept new client connection
 90                 *
 91                 * Function will block thread until
 92                 * new client is connected to server
 93                 */
 94                res = lwesp_netconn_accept(server, &client);
 95                if (res == lwespOK) {
 96                    printf("Netconn new client connected. Starting new thread...\r\n");
 97                    /*
 98                     * Start new thread for this request.
 99                     *
100                     * Read and write back data to user in separated thread
101                     * to allow processing of multiple requests at the same time
102                     */
103                    if (lwesp_sys_thread_create(NULL, "client", (lwesp_sys_thread_fn)netconn_server_processing_thread, client, 512, LWESP_SYS_THREAD_PRIO)) {
104                        printf("Netconn client thread created\r\n");
105                    } else {
106                        printf("Netconn client thread creation failed!\r\n");
107
108                        /* Force close & delete */
109                        lwesp_netconn_close(client);
110                        lwesp_netconn_delete(client);
111                    }
112                } else {
113                    printf("Netconn connection accept error!\r\n");
114                    break;
115                }
116            }
117        } else {
118            printf("Netconn server cannot bind to port\r\n");
119        }
120    } else {
121        printf("Cannot create server netconn\r\n");
122    }
123
124    lwesp_netconn_delete(server);                 /* Delete netconn structure */
125    lwesp_sys_thread_terminate(NULL);             /* Terminate current thread */
126}
127
128/**
129 * \brief           Thread to process single active connection
130 * \param[in]       arg: Thread argument
131 */
132static void
133netconn_server_processing_thread(void* const arg) {
134    lwesp_netconn_p client;
135    lwesp_pbuf_p pbuf, p = NULL;
136    lwespr_t res;
137    char strt[20];
138
139    client = arg;                               /* Client handle is passed to argument */
140
141    printf("A new connection accepted!\r\n");   /* Print simple message */
142
143    do {
144        /*
145         * Client was accepted, we are now
146         * expecting client will send to us some data
147         *
148         * Wait for data and block thread for that time
149         */
150        res = lwesp_netconn_receive(client, &pbuf);
151
152        if (res == lwespOK) {
153            printf("Netconn data received, %d bytes\r\n", (int)lwesp_pbuf_length(pbuf, 1));
154            /* Check reception of all header bytes */
155            if (p == NULL) {
156                p = pbuf;                       /* Set as first buffer */
157            } else {
158                lwesp_pbuf_cat(p, pbuf);          /* Concatenate buffers together */
159            }
160            if (lwesp_pbuf_strfind(pbuf, "\r\n\r\n", 0) != LWESP_SIZET_MAX) {
161                if (lwesp_pbuf_strfind(pbuf, "GET / ", 0) != LWESP_SIZET_MAX) {
162                    uint32_t now;
163                    printf("Main page request\r\n");
164                    now = lwesp_sys_now();        /* Get current time */
165                    sprintf(strt, "%u ms; %d s", (unsigned)now, (unsigned)(now / 1000));
166                    lwesp_netconn_write(client, rlwesp_data_mainpage_top, sizeof(rlwesp_data_mainpage_top) - 1);
167                    lwesp_netconn_write(client, strt, strlen(strt));
168                    lwesp_netconn_write(client, rlwesp_data_mainpage_bottom, sizeof(rlwesp_data_mainpage_bottom) - 1);
169                } else if (lwesp_pbuf_strfind(pbuf, "GET /style.css ", 0) != LWESP_SIZET_MAX) {
170                    printf("Style page request\r\n");
171                    lwesp_netconn_write(client, rlwesp_data_style, sizeof(rlwesp_data_style) - 1);
172                } else {
173                    printf("404 error not found\r\n");
174                    lwesp_netconn_write(client, rlwesp_error_404, sizeof(rlwesp_error_404) - 1);
175                }
176                lwesp_netconn_close(client);      /* Close netconn connection */
177                lwesp_pbuf_free(p);               /* Do not forget to free memory after usage! */
178                p = NULL;
179                break;
180            }
181        }
182    } while (res == lwespOK);
183
184    if (p != NULL) {                            /* Free received data */
185        lwesp_pbuf_free(p);
186        p = NULL;
187    }
188    lwesp_netconn_delete(client);                 /* Destroy client memory */
189    lwesp_sys_thread_terminate(NULL);             /* Terminate this thread */
190}

Non-blocking receive

By default, netconn API is written to only work in separate application thread, dedicated for network connection processing. Because of that, by default every function is fully blocking. It will wait until result is ready to be used by application.

It is, however, possible to enable timeout feature for receiving data only. When this feature is enabled, lwesp_netconn_receive() will block for maximal timeout set with lwesp_netconn_set_receive_timeout() function.

When enabled, if there is no received data for timeout amount of time, function will return with timeout status and application needs to process it accordingly.

Tip

LWESP_CFG_NETCONN_RECEIVE_TIMEOUT must be set to 1 to use this feature.

group LWESP_NETCONN

Network connection.

Defines

LWESP_NETCONN_RECEIVE_NO_WAIT

Receive data with no timeout.

Note

Used with lwesp_netconn_set_receive_timeout function

Typedefs

typedef struct lwesp_netconn *lwesp_netconn_p

Netconn object structure.

Enums

enum lwesp_netconn_type_t

Netconn connection type.

Values:

enumerator LWESP_NETCONN_TYPE_TCP

TCP connection

enumerator LWESP_NETCONN_TYPE_SSL

SSL connection

enumerator LWESP_NETCONN_TYPE_UDP

UDP connection

enumerator LWESP_NETCONN_TYPE_TCPV6

TCP connection over IPv6

enumerator LWESP_NETCONN_TYPE_SSLV6

SSL connection over IPv6

Functions

lwesp_netconn_p lwesp_netconn_new(lwesp_netconn_type_t type)

Create new netconn connection.

Parameters

type[in] Netconn connection type

Returns

New netconn connection on success, NULL otherwise

lwespr_t lwesp_netconn_delete(lwesp_netconn_p nc)

Delete netconn connection.

Parameters

nc[in] Netconn handle

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_bind(lwesp_netconn_p nc, lwesp_port_t port)

Bind a connection to specific port, can be only used for server connections.

Parameters
  • nc[in] Netconn handle

  • port[in] Port used to bind a connection to

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_connect(lwesp_netconn_p nc, const char *host, lwesp_port_t port)

Connect to server as client.

Parameters
  • nc[in] Netconn handle

  • host[in] Pointer to host, such as domain name or IP address in string format

  • port[in] Target port to use

Returns

lwespOK if successfully connected, member of lwespr_t otherwise

lwespr_t lwesp_netconn_receive(lwesp_netconn_p nc, lwesp_pbuf_p *pbuf)

Receive data from connection.

Parameters
  • nc[in] Netconn handle used to receive from

  • pbuf[in] Pointer to pointer to save new receive buffer to. When function returns, user must check for valid pbuf value pbuf != NULL

Returns

lwespOK when new data ready

Returns

lwespCLOSED when connection closed by remote side

Returns

lwespTIMEOUT when receive timeout occurs

Returns

Any other member of lwespr_t otherwise

lwespr_t lwesp_netconn_close(lwesp_netconn_p nc)

Close a netconn connection.

Parameters

nc[in] Netconn handle to close

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

int8_t lwesp_netconn_get_connnum(lwesp_netconn_p nc)

Get connection number used for netconn.

Parameters

nc[in] Netconn handle

Returns

-1 on failure, connection number between 0 and LWESP_CFG_MAX_CONNS otherwise

lwesp_conn_p lwesp_netconn_get_conn(lwesp_netconn_p nc)

Get netconn connection handle.

Parameters

nc[in] Netconn handle

Returns

ESP connection handle

void lwesp_netconn_set_receive_timeout(lwesp_netconn_p nc, uint32_t timeout)

Set timeout value for receiving data.

When enabled, lwesp_netconn_receive will only block for up to timeout value and will return if no new data within this time

Parameters
  • nc[in] Netconn handle

  • timeout[in] Timeout in units of milliseconds. Set to 0 to disable timeout feature Set to > 0 to set maximum milliseconds to wait before timeout Set to LWESP_NETCONN_RECEIVE_NO_WAIT to enable non-blocking receive

uint32_t lwesp_netconn_get_receive_timeout(lwesp_netconn_p nc)

Get netconn receive timeout value.

Parameters

nc[in] Netconn handle

Returns

Timeout in units of milliseconds. If value is 0, timeout is disabled (wait forever)

lwespr_t lwesp_netconn_connect_ex(lwesp_netconn_p nc, const char *host, lwesp_port_t port, uint16_t keep_alive, const char *local_ip, lwesp_port_t local_port, uint8_t mode)

Connect to server as client, allow keep-alive option.

Parameters
  • nc[in] Netconn handle

  • host[in] Pointer to host, such as domain name or IP address in string format

  • port[in] Target port to use

  • keep_alive[in] Keep alive period seconds

  • local_ip[in] Local ip in connected command

  • local_port[in] Local port address

  • mode[in] UDP mode

Returns

lwespOK if successfully connected, member of lwespr_t otherwise

lwespr_t lwesp_netconn_listen(lwesp_netconn_p nc)

Listen on previously binded connection.

Parameters

nc[in] Netconn handle used to listen for new connections

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_listen_with_max_conn(lwesp_netconn_p nc, uint16_t max_connections)

Listen on previously binded connection with max allowed connections at a time.

Parameters
  • nc[in] Netconn handle used to listen for new connections

  • max_connections[in] Maximal number of connections server can accept at a time This parameter may not be larger than LWESP_CFG_MAX_CONNS

Returns

lwespOK on success, member of lwespr_t otherwise

lwespr_t lwesp_netconn_set_listen_conn_timeout(lwesp_netconn_p nc, uint16_t timeout)

Set timeout value in units of seconds when connection is in listening mode If new connection is accepted, it will be automatically closed after seconds elapsed without any data exchange.

Note

Call this function before you put connection to listen mode with lwesp_netconn_listen

Parameters
  • nc[in] Netconn handle used for listen mode

  • timeout[in] Time in units of seconds. Set to 0 to disable timeout feature

Returns

lwespOK on success, member of lwespr_t otherwise

lwespr_t lwesp_netconn_accept(lwesp_netconn_p nc, lwesp_netconn_p *client)

Accept a new connection.

Parameters
  • nc[in] Netconn handle used as base connection to accept new clients

  • client[out] Pointer to netconn handle to save new connection to

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_write(lwesp_netconn_p nc, const void *data, size_t btw)

Write data to connection output buffers.

Note

This function may only be used on TCP or SSL connections

Parameters
  • nc[in] Netconn handle used to write data to

  • data[in] Pointer to data to write

  • btw[in] Number of bytes to write

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_flush(lwesp_netconn_p nc)

Flush buffered data on netconn TCP/SSL connection.

Note

This function may only be used on TCP/SSL connection

Parameters

nc[in] Netconn handle to flush data

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_send(lwesp_netconn_p nc, const void *data, size_t btw)

Send data on UDP connection to default IP and port.

Parameters
  • nc[in] Netconn handle used to send

  • data[in] Pointer to data to write

  • btw[in] Number of bytes to write

Returns

lwespOK on success, member of lwespr_t enumeration otherwise

lwespr_t lwesp_netconn_sendto(lwesp_netconn_p nc, const lwesp_ip_t *ip, lwesp_port_t port, const void *data, size_t btw)

Send data on UDP connection to specific IP and port.

Note

Use this function in case of UDP type netconn

Parameters
  • nc[in] Netconn handle used to send

  • ip[in] Pointer to IP address

  • port[in] Port number used to send data

  • data[in] Pointer to data to write

  • btw[in] Number of bytes to write

Returns

lwespOK on success, member of lwespr_t enumeration otherwise