WiFi SSE: Server Sent Events

Source Code

Abstract/Summary: This project introduces WiFi SSE. (Server Sent Events). In normal HTML operation the client, for example a smartphone will need to request an update from the server (access point) in order to follow activity at the server. With SSE the server will be programmed to update the client when there are any changes.

My ultimate target is to add WiFi SSE capability to a model train monitor/controller. This project is being undertaken using a divide and conquer process: (i) Have the WiFi SSE up and running (this project), and (ii) integrate the train controller.

Possible Audience: This project will be for software programmers interested in WiFi applications with the ESP8266 as it is almost exclusively about using SSE software libraries.

Keywords: ESP8266, SSE Server Sent Events, Asynchronous callbacks, Placeholders, Events, Server, Client, Access Point.

Hardware Components Required: The design was tested using an ESP8266 Mini and smartphone to observe the response.

Libraries: ESPAsyncWebServer.h, ESPAsyncTCP.h and ESP8266WiFi.h

Background/Introduction

I have a model railway layout that consists of multiple blocks of track. One such block is illustrated:

The following circuit monitors the sensors and controls the signals:

Ultimately I wish to use the WiFi capability of the ESP8266 to transmit the train status. Rather than go to the complete final circuit this project will concentrate on implementing the WiFi aspects by simulating the LDR sensors. It will ignore all the hardware attached to the ESP8266.

The simulation will be for a train moving across the 4 LDR sensors and then back again. A library/class has been developed for the hardware. It includes a method monitor_train( ) that generates an enum type where train_loc is an enum type where_is_train which is defined**:

enum where_is_train {waiting, coming0, approaching0, arriving0, station0, departing0, leaving0, going0, coming1, approaching1, arriving1, station1, departing1, leaving1, going1};

enum where_is_train train_loc; //location of train

The simulator must generate the same signal train_loc of type where_is_train.

** Definitions assume the train can enter the block of track from either direction - hence coming0 and coming1 etc

The Development Environment

The project will be developed using the Arduino IDE with ESP8266 Add On extension. If this is not available see appendix.

The following libraries are required: ESPAsyncWebServer, ESPAsyncTCP and ESP8266WiFi

If there are not available download the *.zip files at https://github.com/me-no-dev/ESPAsyncWebServer and https://github.com/me-no-dev/ESPAsyncTCP

In the Arduino IDE at sketch -->Include Library -->Add *zip library and select the downloaded *.zip files.

The files ESPAsyncWebServer, ESPAsyncTCP, ESP8266WiFi and others will now be part of the contributed libraries. To observe choose Sketch-->Include Library and note contributed libraries.

The Starting Code

The program will be developed in two steps:

1. Have the client (eg smartphone) communicate with the ESP8266. The client will request a response from the ESP8266. In the final product this will be done once but there can be multiple requests.

2. Have the ESP8266 (Access Point) update the client display whenever new data occurs. This will imply that the HTML response above must include the capability to receive the unsolicted data from the ESP8266.

The first step will be in the Arduino IDE to create a new file. I have given it the name WiFi_SSE to match the title of this page. The Arduino IDE will give the file the extension *.ino.

Possible starting code will be:

//WiFi_SSE.ino
const char* ssid = "WiFi_SSE";const char* password = "12345678";
enum where_is_train {waiting, coming0, approaching0, arriving0, station0, departing0, leaving0, going0, coming1, approaching1, arriving1, station1, departing1, leaving1, going1 };enum where_is_train train_loc; //location of train
#include <ESPAsyncWebServer.h>
void setup() {Serial.begin(115200);pinMode(D8,OUTPUT);}
void loop() {digitalWrite(D8,HIGH);digitalWrite(D8,LOW);}

Notes:

1. For later in the project the service set identifier (ssid) has been defined as "WiFi_SSE" and the password "12345678". Note the password must either be empty for an open network or 8 or more characters.

2. An enum type where_is_train is defined using the anticipated states with the model train.

3. In the setup( ) routine the serial port is set to 115200 to allow for debugging messages to the Arduino IDE serial monitor.

4. To also assist with any debugging and performance evaluation the pin D8 is set as an output and set to toggle in the program loop( ).

The HTML File

Upon the initial contact/request from the client the server (ie the ESP8266 Access Point) will respond with a HTML file that the client will display.

//index.h const char WiFi_Mess [] PROGMEM = R"(<!DOCTYPE html><HTML><HEAD><TITLE>WiFi SSE</TITLE><meta name="viewport" content="width=device-width, initial-scale=1"><link rel="icon" href="data:,"><style>html {font-family: Arial;text-align: center;}body {max-width: 400px;margin:0px auto;}</style></HEAD><BODY><H2>WiFi SSE</H2>Response to Client Request</BODY></HTML>)";

Notes:

1. I have chosen to create a file I have called index.h. In the Arduino IDE click on the inverted triangle on the upper right and select New_Tab. Give it the label index.h and copy paste the above code.

2. The HTML code is an array of characters known as WiFi_Mess [ ].

3. The directive PROGMEM tells the compiler to generate code that will read Flash ROM rather than dynamic RAM. Further at start up a copy is not placed in RAM.

4. The R" ( )" prefix tells the compiler what you see is what we want. That is there is no need for new line characters for example.

5. The line <meta name="viewport" content="width=device-width, initial-scale=1"> effectively allows the HTML display to adjust for different width devices.(Smartphone vs PC)

6. The line <link rel="icon" href="data:,"> tells the client not to request the favicon a small image that would be displayed in the tool bar.

7. The information between the <style> ...</style> indicates how the text will be displayed.

8. The text in the <BODY> section is what is displayed with "WiFi SSE" in the heading 2 format.

The WiFi Initialisation Code

The code to initialise the Wifi will be:

//WiFi_SSE.ino
...........
#include <ESPAsyncWebServer.h>
// Create AsyncWebServer object on port 80AsyncWebServer server(80);
void setup() {...........WiFi.mode(WIFI_AP); //Access point onlyWiFi.softAP(ssid, password); // Handle Web Serverserver.on("/", HTTP_GET, [](AsyncWebServerRequest *request){request->send_P(200, "text/html", WiFi_Mess);});server.begin();}
void loop(){....... }

Notes**:

1. The first step is to create an object server of class/type AsyncWebServer on port 80. Port 80 is the port typically used for passing HTML.

2. In the setup( ) function the ESP6266 is set up as an Access Point with the ssid and password defined earlier.

3. The server.on ( ) operation sets up the ESP8266 to respond to a WiFi HTTP_GET request for a "/" with the message WiFi_Mess of type "text/html" that includes the status 200 (transfer OK).

4. server.begin( ) will enable the operation.

5. The loop( ) method is empty. The server.on( ) has set up the interrupt/asynchronous service or callback routine that sends the html message in response to a client request.

The code may be tested using a smartphone/PC by selecting WiFi_SSE with password 12345678 and noting the address (usually 192.168.4.1) and selecting that address in the browser. The response at the client will be:

The next step is to add the code to update the client display when a new event occurs at the server (ESP8266).

** Some of the operations are discussed in a little more detail in the page WiFi_123. In that page see design option 2.

Handling Events: Source Code.

The code to this point handles requests from the client only. In the model train example it would be nice if the client display was updated every time the train moved to a new state. This can be achieved using the events source. The additional set code will be:

// Create an Event Source on /eventsAsyncEventSource events("/events");
void setup() {................... // Handle Web Server Eventsevents.onConnect([](AsyncEventSourceClient *client){// send event with message "hello!", id and current millis// and set reconnect delay to 1 secondsclient->send("hello!", NULL, millis(), 1000);});server.addHandler(&events);server.begin();}

The HTML code and the program loop( ) must both be modified to handle the events. The above code will only handle the set up.

Handling Events: HTML Code

To handle the events the following Java script is added to the HTML code between the <BODY>..</BODY> tags

<BODY> ................ <script>if (!!window.EventSource) {var source = new EventSource('/events'); source.addEventListener('open', function(e) {console.log("Events Connected");}, false);source.addEventListener('error', function(e) {if (e.target.readyState != EventSource.OPEN) {console.log("Events Disconnected");}}, false); source.addEventListener('train_state', function(e) {console.log("train_state", e.data);document.getElementById("train").innerHTML = e.data;}, false);}</script>
</BODY>

Notes:

1. The (!!window.EventSource) checks that the browser supports EventSource. Internet Explorer apparently does not.

2. var source = new EventSource('/events'); instantiates the EventSource object where /events is the url that generates the events.

3. The three handlers open, error and train_state bind those operations to source to deal with those message situations.

4. In the case of the 'train_state' the existing message "Response to Client Request" will need to be modified to display the new message for example: "No trains present."

5.The script will replace the text in the HTML document enclosed by the "train" tag with the new message. The data portion of e.data where e is the short var reference for event object passed to the event handlers.

The final modification is to add the "train" tag to the default HTML message <span id="train">Response to Client Request</span>

Events.send

To test the Java Script added to the HTML code the following code can be included inside the program loop( );

delay(10000);events.send("No trains present.","train_state");

The client display will now change from "Response to Client Request" to "No trains present".

The next step is to include the full suite of messages. Also at this point an evaluation of the performance was undertaken

The Simulation and Multiple Messages.

The final step in the project is to include the full code for the simulation. In this example the code is relatively simple as the train moves from one state to the next. There is no "if then else" situation that is typical of most state machines.

long save_time; //last updatelong sample_time = 1000; //time between updates ............
where_is_train simulate( ){if ((millis( ) - save_time)>sample_time) {save_time = millis( );switch (train_loc){case waiting: train_loc = coming0; break;case coming0: train_loc = approaching0; break;case approaching0: train_loc = arriving0; break;case arriving0: train_loc = station0; break;case station0: train_loc = departing0; break;case departing0: train_loc = leaving0; break;case leaving0: train_loc = going0; break;case going0: train_loc = coming1; break;case coming1: train_loc = approaching1; break;case approaching1: train_loc = arriving1; break;case arriving1: train_loc = station1; break;case station1: train_loc = departing1; break;case departing1: train_loc = leaving1; break;case leaving1: train_loc = going1; break;case going1: train_loc = waiting; break;default : train_loc = waiting; }} return train_loc;}

As given the simulation returns the new train state train_loc that is the same as returned by the routine monitor_train( ) in what will be the actual hardware.

The program loop( ) will become:

void loop() {static where_is_train save_loc;digitalWrite(D8,HIGH);train_loc = simulate( );if (train_loc != save_loc) {save_loc = train_loc;switch (train_loc){case waiting : events.send("No trains present.","train_state"); break;case coming0 : events.send("Train coming from North.","train_state"); break;case approaching0 : events.send("Train approaching from North.","train_state"); break;case arriving0 : events.send("Train arriving from North.","train_state"); break;case station0 : events.send("Train from North in station.","train_state"); break;case departing0 : events.send("Train departing to South.","train_state"); break;case leaving0 : events.send("Train leaving to South.","train_state"); break;case going0 : events.send("Train going to South.","train_state"); break;case coming1 : events.send("Train coming from South.","train_state"); break;case approaching1 : events.send("Train approaching from South.","train_state"); break;case arriving1 : events.send("Train arriving from South.","train_state"); break;case station1 : events.send("Train from South in station.","train_state"); break;case departing1 : events.send("Train departing to North.","train_state"); break;case leaving1 : events.send("Train leaving to North.","train_state"); break;case going1 : events.send("Train going to North.","train_state"); break;}} digitalWrite(D8,LOW);}

Note as given simulate will be called on each loop( ) but the events.send( ) will only be called if there is a change in train status. Since the simulate( ) program only updates every second this implies a new "events.send( )" every second.

Testing verified the smartphone display updating every second. Some tests were then performed to obtain a feel for the system capability.

A separate project will replace the simulate routine with real hardware.

Performance Testing.

The following tests were taken by setting pin 8 of the ESP8266 high at the start of the program loop( ) and clearing pin 8 at the end. The results were then observed using a mixed signal oscilloscope. Rather than trying to evaluate the system sending text messages it was easier to read integers. The test program was:

int n = 0;void loop() {static where_is_train save_loc;n += 1;digitalWrite(D8,HIGH);delay(100);events.send(String(n).c_str(),"train_state"); digitalWrite(D8,LOW);}

With the code as given the smartphone should display a number being incremented. This was the case for delays of 100mSec or greater. With smaller delays the smartphone did not display every number (many numbers were missed).

With a delay of 50mSec the display seemed to increment most of the time but it frequently stalled and skipped numbers.

With no delay the display seemed to go "bang" "bang" and then pause for over 1 second before displaying two more numbers. The numbers displayed appeared to be correct suggesting that the program was operational but the WiFi sub-system was becoming overloaded and only responding occasionally. The following display was obtained after leaving the system running for approximately 8 hours.

Looking at the waveforms without any delay most of the time the waveforms were as follows:

With out any delay I believe that this would have been when the ESP8266 was ignoring many events.send( ); statements. As shown above the program loop() took 135uSec and the time out of the loop (background) was of the order of 10uSec.

However there were times when all "hell broke loose".

Over a period of 8.1 mSec the firmware oscillated between foreground and background activity. On each foreground cycle the events.send( ) code was "executed" and I'm guessing in the background the ESP8266 WiFi sub system was sending its message.

Different delays were tried. It was found that with a delay of 2 mS the waveforms seemed "perfect". -See trace below. (A delay of 1mSec was still in the "all hell broke loose" category)

In the captured trace the program was cycling in and out of the loop approximately every 2mSec. Then the program suddenly was in the loop for 4-5 mSec and then in the background for greater than 100uSec** (Pulse in centre). My assumption is that at this point the ESP8266 WiFi subsystem has accepted that events.send( ) request and in the background started to take the required action. This action was completed during the low time.

There were other responses with the high time not so long but the low time significantly longer than 100uSec. See trace below where the signal is high for over 1 division (5mSec) and then later it is low for greater than 1 mSec. (With the low time of 1mS possibly explains why delays up to 1mSec gave an "all hell broke loose" response).

These traces with a 2mSec delay correspond to the smartphone display skipping many updates.

Conclusion

My conclusion was that with a delay of 100mSec the events.send( ) requests were at a rate the ESP8266 WiFi sub-system could follow without becoming saturated/confused. That is the update rate was 10 updates/second. My observation was also that when the ESP8266 became saturated/confused the update rate dropped well below 10 updates/second.

** The trigger point was for the trace was a low signal with a pulse width greater than 100uSec. When the smartphone was not connected this never occurred.

Range

While not a consideration for this project the range or distance over which the ESP8266/Smartphone could operate was of interest to a subsequent project. In an open space it was found that the reception dropped out after about 30 metres separation. Since the new project was looking at distances up to 200metres it was concluded that using WiFi with a simple ESP8266 was not the solution.

Appendix: Adding the ESP8266 to the Development Environment

This project will use the Arduino IDE with the ESP8266 Add-ons.

This will allow the Arduino IDE editor to be used but the results are then compiled for the ESP8266. The advantage is that applications can be developed for the ESP8266 without the developer having to learn more complex and/or different tools. The disadvantage is that some of the features/attributes of theESP8266 may be hidden in the simplification. It may also lead to the impression that the ESP8266 is similar to the ATMega828 in the Arduino when it is not.

If required Install the drivers: (I was using a WiFi Mini product code XC3802 from Jaycar) and found my PC recognized the XC3802 I was using so this step was not necessary)

If the library is not found the WiFi Mini uses a CH340G USB-Serial IC. The drivers for this can be downloaded from the IC manufacturer’s website: http://www.wch.cn/download/CH341SER_ZIP.html

To add board support for tjhe ESP8266 it is recommended to use Arduino IDE version 1.6.4 or later so that the Boards Manager can handle the installation.

1. To install board support for ESP8266, in File>Preferences>Additional Board Manager URLS add: http://arduino.esp8266.com/stable/package_esp8266com_index.json separating from existing entries with a comma.

2: Go to Tools>Boards>Boards Manager and type 'esp' in the search box

3. Install ESP8266 by ESP8266 Community. (Button on lower right) This is about 150MB download and can take a while.

4. Select the desired ESP6266 board. My board was identical to the 'WeMos D1 Mini Lite’

The Arduino IDE will now start creating, compiling and running programs with the ESP8266.

______________

Page updated

Google Sites

Report abuse