Netconn API

Netconn API is addon on top of existing connection module and allows sending and receiving data with sequential API calls, similar to POSIX socket API.

It can operate in client or server mode and uses operating system features, such as message queues and semaphore to link non-blocking callback API for connections with sequential API for application thread.

Note

Connection API does not directly allow receiving data with sequential and linear code execution. All is based on connection event system. Netconn adds this functionality as it is implemented on top of regular connection API.

Warning

Netconn API are designed to be called from application threads ONLY. It is not allowed to call any of netconn API functions from within interrupt or callback event functions.

Netconn client

Netconn API client block diagram

Netconn API client block diagram

Above block diagram shows basic architecture of netconn client application. There is always one application thread (in green) which calls netconn API functions to interact with connection API in synchronous mode.

Every netconn connection uses dedicated structure to handle message queue for data received packet buffers. Each time new packet is received (red block, data received event), reference to it is written to message queue of netconn structure, while application thread reads new entries from the same queue to get packets.

Netconn client example
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
#include "netconn_client.h"
#include "esp/esp.h"

/**
 * \brief           Host and port settings
 */
#define NETCONN_HOST        "example.com"
#define NETCONN_PORT        80

/**
 * \brief           Request header to send on successful connection
 */
static const char
request_header[] = ""
"GET / HTTP/1.1\r\n"
"Host: " NETCONN_HOST "\r\n"
"Connection: close\r\n"
"\r\n";

/**
 * \brief           Netconn client thread implementation
 * \param[in]       arg: User argument
 */
void
netconn_client_thread(void const* arg) {
    espr_t res;
    esp_pbuf_p pbuf;
    esp_netconn_p client;
    esp_sys_sem_t* sem = (void *)arg;

    /*
     * First create a new instance of netconn
     * connection and initialize system message boxes
     * to accept received packet buffers
     */
    client = esp_netconn_new(ESP_NETCONN_TYPE_TCP);
    if (client != NULL) {
        /*
         * Connect to external server as client
         * with custom NETCONN_CONN_HOST and CONN_PORT values
         *
         * Function will block thread until we are successfully connected (or not) to server
         */
        res = esp_netconn_connect(client, NETCONN_HOST, NETCONN_PORT);
        if (res == espOK) {                     /* Are we successfully connected? */
            printf("Connected to " NETCONN_HOST "\r\n");
            res = esp_netconn_write(client, request_header, sizeof(request_header) - 1);    /* Send data to server */
            if (res == espOK) {
                res = esp_netconn_flush(client);    /* Flush data to output */
            }
            if (res == espOK) {                 /* Were data sent? */
                printf("Data were successfully sent to server\r\n");

                /*
                 * Since we sent HTTP request,
                 * we are expecting some data from server
                 * or at least forced connection close from remote side
                 */
                do {
                    /*
                     * Receive single packet of data
                     *
                     * Function will block thread until new packet
                     * is ready to be read from remote side
                     *
                     * After function returns, don't forgot the check value.
                     * Returned status will give you info in case connection
                     * was closed too early from remote side
                     */
                    res = esp_netconn_receive(client, &pbuf);
                    if (res == espCLOSED) {     /* Was the connection closed? This can be checked by return status of receive function */
                        printf("Connection closed by remote side...\r\n");
                        break;
                    } else if (res == espTIMEOUT) {
                        printf("Netconn timeout while receiving data. You may try multiple readings before deciding to close manually\r\n");
                    }

                    if (res == espOK && pbuf != NULL) { /* Make sure we have valid packet buffer */
                        /*
                         * At this point read and manipulate
                         * with received buffer and check if you expect more data
                         *
                         * After you are done using it, it is important
                         * you free the memory otherwise memory leaks will appear
                         */
                        printf("Received new data packet of %d bytes\r\n", (int)esp_pbuf_length(pbuf, 1));
                        esp_pbuf_free(pbuf);    /* Free the memory after usage */
                        pbuf = NULL;
                    }
                } while (1);
            } else {
                printf("Error writing data to remote host!\r\n");
            }

            /*
             * Check if connection was closed by remote server
             * and in case it wasn't, close it manually
             */
            if (res != espCLOSED) {
                esp_netconn_close(client);
            }
        } else {
            printf("Cannot connect to remote host %s:%d!\r\n", NETCONN_HOST, NETCONN_PORT);
        }
        esp_netconn_delete(client);             /* Delete netconn structure */
    }

    if (esp_sys_sem_isvalid(sem)) {
        esp_sys_sem_release(sem);
    }
    esp_sys_thread_terminate(NULL);             /* Terminate current thread */
}

Netconn server

Netconn API server block diagram

Netconn API server block diagram

When netconn is configured in server mode, it is possible to accept new clients from remote side. Application creates netconn server connection, which can only accept clients and cannot send/receive any data. It configures server on dedicated port (selected by application) and listens on it.

When new client connects, server callback function is called with new active connection event. Newly accepted connection is then written to server structure netconn which is later read by application thread. At the same time, netconn connection structure (blue) is created to allow standard send/receive operation on active connection.

Note

Each connected client has its own netconn connection structure. When multiple clients connect to server at the same time, multiple entries are written to connection accept message queue and are ready to be processed by application thread.

From this point, program flow is the same as in case of netconn client.

This is basic example for netconn thread. It waits for client and processes it in blocking mode.

Warning

When multiple clients connect at the same time to netconn server, they are processed one-by-one, sequentially. This may introduce delay in response for other clients. Check netconn concurrency option to process multiple clients at the same time

Netconn server with single processing thread
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
/*
 * Netconn server example is based on single thread
 * and it listens for single client only on port 23
 */
#include "netconn_server_1thread.h"
#include "esp/esp.h"

/**
 * \brief           Basic thread for netconn server to test connections
 * \param[in]       arg: User argument
 */
void
netconn_server_1thread_thread(void* arg) {
    espr_t res;
    esp_netconn_p server, client;
    esp_pbuf_p p;

    /* Create netconn for server */
    server = esp_netconn_new(ESP_NETCONN_TYPE_TCP);
    if (server == NULL) {
        printf("Cannot create server netconn!\r\n");
    }

    /* Bind it to port 23 */
    res = esp_netconn_bind(server, 23);
    if (res != espOK) {
        printf("Cannot bind server\r\n");
        goto out;
    }

    /* Start listening for incoming connections with maximal 1 client */
    res = esp_netconn_listen_with_max_conn(server, 1);
    if (res != espOK) {
        goto out;
    }

    /* Unlimited loop */
    while (1) {
        /* Accept new client */
        res = esp_netconn_accept(server, &client);
        if (res != espOK) {
            break;
        }
        printf("New client accepted!\r\n");
        while (1) {
            /* Receive data */
            res = esp_netconn_receive(client, &p);
            if (res == espOK) {
                printf("Data received!\r\n");
                esp_pbuf_free(p);
            } else {
                printf("Netconn receive returned: %d\r\n", (int)res);
                if (res == espCLOSED) {
                    printf("Connection closed by client\r\n");
                    break;
                }
            }
        }
        /* Delete client */
        if (client != NULL) {
            esp_netconn_delete(client);
            client = NULL;
        }
    }
    /* Delete client */
    if (client != NULL) {
        esp_netconn_delete(client);
        client = NULL;
    }

out:
    printf("Terminating netconn thread!\r\n");
    if (server != NULL) {
        esp_netconn_delete(server);
    }
    esp_sys_thread_terminate(NULL);
}

Netconn server concurrency

Netconn API server concurrency block diagram

Netconn API server concurrency block diagram

When compared to classic netconn server, concurrent netconn server mode allows multiple clients to be processed at the same time. This can drastically improve performance and response time on clients side, especially when many clients are connected to server at the same time.

Every time server application thread (green block) gets new client to process, it starts a new processing thread instead of doing it in accept thread.

  • Server thread is only dedicated to accept clients and start threads

  • Multiple processing thread can run in parallel to send/receive data from multiple clients

  • No delay when multi clients are active at the same time

  • Higher memory footprint is necessary as there are multiple threads active

Netconn server with multiple processing threads
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
/*
 * Netconn server example is based on single "user" thread
 * which listens for new connections and accepts them.
 *
 * When a new client is accepted by server,
 * separate thread for client is created where 
 * data is read, processed and send back to user
 */
#include "netconn_server.h"
#include "esp/esp.h"

static void netconn_server_processing_thread(void* const arg);

/**
 * \brief           Main page response file
 */
static const uint8_t
resp_data_mainpage_top[] = ""
"HTTP/1.1 200 OK\r\n"
"Content-Type: text/html\r\n"
"\r\n"
"<html>"
"   <head>"
"       <link rel=\"stylesheet\" href=\"style.css\" type=\"text/css\" />"
"       <meta http-equiv=\"refresh\" content=\"1\" />"
"   </head>"
"   <body>"
"       <p>Netconn driven website!</p>"
"       <p>Total system up time: <b>";

/**
 * \brief           Bottom part of main page
 */
static const uint8_t
resp_data_mainpage_bottom[] = ""
"       </b></p>"
"   </body>"
"</html>";

/**
 * \brief           Style file response
 */
static const uint8_t
resp_data_style[] = ""
"HTTP/1.1 200 OK\r\n"
"Content-Type: text/css\r\n"
"\r\n"
"body { color: red; font-family: Tahoma, Arial; };";

/**
 * \brief           404 error response
 */
static const uint8_t
resp_error_404[] = ""
"HTTP/1.1 404 Not Found\r\n"
"\r\n"
"Error 404";

/**
 * \brief           Netconn server thread implementation
 * \param[in]       arg: User argument
 */
void
netconn_server_thread(void const* arg) {
    espr_t res;
    esp_netconn_p server, client;
                                                
    /*
     * First create a new instance of netconn 
     * connection and initialize system message boxes
     * to accept clients and packet buffers
     */
    server = esp_netconn_new(ESP_NETCONN_TYPE_TCP);
    if (server != NULL) {
        printf("Server netconn created\r\n");

        /* Bind network connection to port 80 */
        res = esp_netconn_bind(server, 80);
        if (res == espOK) {
            printf("Server netconn listens on port 80\r\n");
            /*
             * Start listening for incoming connections
             * on previously binded port
             */
            res = esp_netconn_listen(server);
            
            while (1) {
                /*
                 * Wait and accept new client connection
                 * 
                 * Function will block thread until 
                 * new client is connected to server
                 */
                res = esp_netconn_accept(server, &client);
                if (res == espOK) {
                    printf("Netconn new client connected. Starting new thread...\r\n");
                    /*
                     * Start new thread for this request.
                     *
                     * Read and write back data to user in separated thread
                     * to allow processing of multiple requests at the same time
                     */
                    if (esp_sys_thread_create(NULL, "client", (esp_sys_thread_fn)netconn_server_processing_thread, client, 512, ESP_SYS_THREAD_PRIO)) {
                        printf("Netconn client thread created\r\n");
                    } else {
                        printf("Netconn client thread creation failed!\r\n");

                        /* Force close & delete */
                        esp_netconn_close(client);
                        esp_netconn_delete(client);
                    }
                } else {
                    printf("Netconn connection accept error!\r\n");
                    break;
                }
            }
        } else {
            printf("Netconn server cannot bind to port\r\n");
        }
    } else {
        printf("Cannot create server netconn\r\n");
    }
    
    esp_netconn_delete(server);                 /* Delete netconn structure */
    esp_sys_thread_terminate(NULL);             /* Terminate current thread */
}

/**
 * \brief           Thread to process single active connection
 * \param[in]       arg: Thread argument
 */
static void
netconn_server_processing_thread(void* const arg) {
    esp_netconn_p client;
    esp_pbuf_p pbuf, p = NULL;
    espr_t res;
    char strt[20];

    client = arg;                               /* Client handle is passed to argument */
                                                
    printf("A new connection accepted!\r\n");   /* Print simple message */
                    
    do {
        /*
         * Client was accepted, we are now
         * expecting client will send to us some data
         *
         * Wait for data and block thread for that time
         */
        res = esp_netconn_receive(client, &pbuf);

        if (res == espOK) {
            printf("Netconn data received, %d bytes\r\n", (int)esp_pbuf_length(pbuf, 1));
            /* Check reception of all header bytes */
            if (p == NULL) {
                p = pbuf;                       /* Set as first buffer */
            } else {
                esp_pbuf_cat(p, pbuf);          /* Concatenate buffers together */
            }
            if (esp_pbuf_strfind(pbuf, "\r\n\r\n", 0) != ESP_SIZET_MAX) {
                if (esp_pbuf_strfind(pbuf, "GET / ", 0) != ESP_SIZET_MAX) {
                    uint32_t now;
                    printf("Main page request\r\n");
                    now = esp_sys_now();        /* Get current time */
                    sprintf(strt, "%u ms; %d s", (unsigned)now, (unsigned)(now / 1000));
                    esp_netconn_write(client, resp_data_mainpage_top, sizeof(resp_data_mainpage_top) - 1);
                    esp_netconn_write(client, strt, strlen(strt));
                    esp_netconn_write(client, resp_data_mainpage_bottom, sizeof(resp_data_mainpage_bottom) - 1);
                } else if (esp_pbuf_strfind(pbuf, "GET /style.css ", 0) != ESP_SIZET_MAX) {
                    printf("Style page request\r\n");
                    esp_netconn_write(client, resp_data_style, sizeof(resp_data_style) - 1);
                } else {
                    printf("404 error not found\r\n");
                    esp_netconn_write(client, resp_error_404, sizeof(resp_error_404) - 1);
                }
                esp_netconn_close(client);      /* Close netconn connection */
                esp_pbuf_free(p);               /* Do not forget to free memory after usage! */
                p = NULL;
                break;
            }
        }
    } while (res == espOK);

    if (p != NULL) {                            /* Free received data */
        esp_pbuf_free(p);
        p = NULL;
    }
    esp_netconn_delete(client);                 /* Destroy client memory */
    esp_sys_thread_terminate(NULL);             /* Terminate this thread */
}

Non-blocking receive

By default, netconn API is written to only work in separate application thread, dedicated for network connection processing. Because of that, by default every function is fully blocking. It will wait until result is ready to be used by application.

It is, however, possible to enable timeout feature for receiving data only. When this feature is enabled, esp_netconn_receive() will block for maximal timeout set with esp_netconn_set_receive_timeout() function.

When enabled, if there is no received data for timeout amount of time, function will return with timeout status and application needs to process it accordingly.

Tip

ESP_CFG_NETCONN_RECEIVE_TIMEOUT must be set to 1 to use this feature.

group ESP_NETCONN

Network connection.

Defines

ESP_NETCONN_RECEIVE_NO_WAIT

Receive data with no timeout.

Note

Used with esp_netconn_set_receive_timeout function

Typedefs

typedef struct esp_netconn *esp_netconn_p

Netconn object structure.

Enums

enum esp_netconn_type_t

Netconn connection type.

Values:

ESP_NETCONN_TYPE_TCP = ESP_CONN_TYPE_TCP

TCP connection

ESP_NETCONN_TYPE_SSL = ESP_CONN_TYPE_SSL

SSL connection

ESP_NETCONN_TYPE_UDP = ESP_CONN_TYPE_UDP

UDP connection

Functions

esp_netconn_p esp_netconn_new(esp_netconn_type_t type)

Create new netconn connection.

Return

New netconn connection on success, NULL otherwise

Parameters
  • [in] type: Netconn connection type

espr_t esp_netconn_delete(esp_netconn_p nc)

Delete netconn connection.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle

espr_t esp_netconn_bind(esp_netconn_p nc, esp_port_t port)

Bind a connection to specific port, can be only used for server connections.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle

  • [in] port: Port used to bind a connection to

espr_t esp_netconn_connect(esp_netconn_p nc, const char *host, esp_port_t port)

Connect to server as client.

Return

espOK if successfully connected, member of espr_t otherwise

Parameters
  • [in] nc: Netconn handle

  • [in] host: Pointer to host, such as domain name or IP address in string format

  • [in] port: Target port to use

espr_t esp_netconn_receive(esp_netconn_p nc, esp_pbuf_p *pbuf)

Receive data from connection.

Return

espOK when new data ready

Return

espCLOSED when connection closed by remote side

Return

espTIMEOUT when receive timeout occurs

Return

Any other member of espr_t otherwise

Parameters
  • [in] nc: Netconn handle used to receive from

  • [in] pbuf: Pointer to pointer to save new receive buffer to. When function returns, user must check for valid pbuf value pbuf != NULL

espr_t esp_netconn_close(esp_netconn_p nc)

Close a netconn connection.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle to close

int8_t esp_netconn_get_connnum(esp_netconn_p nc)

Get connection number used for netconn.

Return

-1 on failure, connection number between 0 and ESP_CFG_MAX_CONNS otherwise

Parameters
  • [in] nc: Netconn handle

esp_conn_p esp_netconn_get_conn(esp_netconn_p nc)

Get netconn connection handle.

Return

ESP connection handle

Parameters
  • [in] nc: Netconn handle

void esp_netconn_set_receive_timeout(esp_netconn_p nc, uint32_t timeout)

Set timeout value for receiving data.

When enabled, esp_netconn_receive will only block for up to timeout value and will return if no new data within this time

Parameters
  • [in] nc: Netconn handle

  • [in] timeout: Timeout in units of milliseconds. Set to 0 to disable timeout feature Set to > 0 to set maximum milliseconds to wait before timeout Set to ESP_NETCONN_RECEIVE_NO_WAIT to enable non-blocking receive

uint32_t esp_netconn_get_receive_timeout(esp_netconn_p nc)

Get netconn receive timeout value.

Return

Timeout in units of milliseconds. If value is 0, timeout is disabled (wait forever)

Parameters
  • [in] nc: Netconn handle

espr_t esp_netconn_connect_ex(esp_netconn_p nc, const char *host, esp_port_t port, uint16_t keep_alive, const char *local_ip, esp_port_t local_port, uint8_t mode)

Connect to server as client, allow keep-alive option.

Return

espOK if successfully connected, member of espr_t otherwise

Parameters
  • [in] nc: Netconn handle

  • [in] host: Pointer to host, such as domain name or IP address in string format

  • [in] port: Target port to use

  • [in] keep_alive: Keep alive period seconds

  • [in] local_ip: Local ip in connected command

  • [in] local_port: Local port address

  • [in] mode: UDP mode

espr_t esp_netconn_listen(esp_netconn_p nc)

Listen on previously binded connection.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle used to listen for new connections

espr_t esp_netconn_listen_with_max_conn(esp_netconn_p nc, uint16_t max_connections)

Listen on previously binded connection with max allowed connections at a time.

Return

espOK on success, member of espr_t otherwise

Parameters
  • [in] nc: Netconn handle used to listen for new connections

  • [in] max_connections: Maximal number of connections server can accept at a time This parameter may not be larger than ESP_CFG_MAX_CONNS

espr_t esp_netconn_set_listen_conn_timeout(esp_netconn_p nc, uint16_t timeout)

Set timeout value in units of seconds when connection is in listening mode If new connection is accepted, it will be automatically closed after seconds elapsed without any data exchange.

Note

Call this function before you put connection to listen mode with esp_netconn_listen

Return

espOK on success, member of espr_t otherwise

Parameters
  • [in] nc: Netconn handle used for listen mode

  • [in] timeout: Time in units of seconds. Set to 0 to disable timeout feature

espr_t esp_netconn_accept(esp_netconn_p nc, esp_netconn_p *client)

Accept a new connection.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle used as base connection to accept new clients

  • [out] client: Pointer to netconn handle to save new connection to

espr_t esp_netconn_write(esp_netconn_p nc, const void *data, size_t btw)

Write data to connection output buffers.

Note

This function may only be used on TCP or SSL connections

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle used to write data to

  • [in] data: Pointer to data to write

  • [in] btw: Number of bytes to write

espr_t esp_netconn_flush(esp_netconn_p nc)

Flush buffered data on netconn TCP/SSL connection.

Note

This function may only be used on TCP/SSL connection

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle to flush data

espr_t esp_netconn_send(esp_netconn_p nc, const void *data, size_t btw)

Send data on UDP connection to default IP and port.

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle used to send

  • [in] data: Pointer to data to write

  • [in] btw: Number of bytes to write

espr_t esp_netconn_sendto(esp_netconn_p nc, const esp_ip_t *ip, esp_port_t port, const void *data, size_t btw)

Send data on UDP connection to specific IP and port.

Note

Use this function in case of UDP type netconn

Return

espOK on success, member of espr_t enumeration otherwise

Parameters
  • [in] nc: Netconn handle used to send

  • [in] ip: Pointer to IP address

  • [in] port: Port number used to send data

  • [in] data: Pointer to data to write

  • [in] btw: Number of bytes to write