Arduino Software: Software Libraries

The Arduino IDE

The Arduino IDE is the entry point for many hobbyists into the micro-controller environment. The Arduino IDE is relatively simple allowing projects to be easily and quickly developed. It is free to download from the Internet.

A simple program in the Arduino IDE is shown below :

To create a new program click on the centre button (the page symbol with the corner folded) and a program template will be generated. The Arduino calls a program a "sketch". The user then adds their code. (By chance in the example all lines starting in red are user code)

One of the attributes of the Arduino platform is that there are plenty of C libraries that hide many of the implementation details that would be a challenge for first time uses. In the above screen example there is no need for the developer to know anything of the micro-controller registers and how to program them. For example the supplied function pinMode( ) sets the register that is wired to the pin as an output, the digitalWrite( ) sets/clears the pin without impacting on other pins while the delay( ) uses the timing registers to determine the delay time. In the Arduino IDE select Help-->Reference for a list of available functions.

For more advance projects there are many libraries developed by the Arduino Community that may be used to perform all the "grunt" work and simplify the developer's task..

The Arduino File Structure

The Arduino IDE (Integrated Development Environment) has a large file structure. A partial file structure is shown below. It is not complete and there could be changes with later versions of the IDE.

hardware Arduino AVR cores Arduino 44 Files

There will be additional folders for user sketches and contributed libraries. Further there is a hidden structure including the file Arduino15

Some interpretation of the table will be necessary. In the examples directory under the directory C://Program Files (x86)/Arduino there are 11 directories/folders titled 01.Basics through 11.ArduinoISP. Under 01.Basics there are a further 6 folders for 6 projects. The first of these is in the folder AnalogReadSerial and includes a range of files.

By contrast with 11.ArduinoISP there is only one project/folder ArduinoISP containing the file/sketch ArduinoISP.ino

In the directory C://Program Files (x86)/Arduino/hardware/Arduino/avr/cores/Arduino/ there are 44 files two of which are main.cpp and Arduino.h which will be discussed in coming sections.

One of the reasons for such a complex structure is that the files cover a range of devices. If the universe just consisted of the Arduino UNO with the ATmega328 micro-controller there would only be the need for a limited number of files. However the files cover the family of Arduino devices along with the families of the AVR micro-controllers.

The main program: main.cpp

Creating a new sketch/file/program in the Arduino IDE gives the sketch shown in blue below.

In the background supporting this sketch is the program main.cpp. (See file structure in previous section). With traditional programs upon power-up or reset the micro-controller will end up at the routine main( ).

For an embedded system within main( ) there are basically two routines:

The initialisation or setup( ) routine. A simple example of the setup routine is to initialise selected pins as digital outputs. This is only ever done once.

The main program operation or loop( ) that runs for ever. This is enclosed in either a for(;;){ } or a while(1){ } statement. An example here may be continuously toggling the output pins between high and low.

The operation of the main.cpp code is transparent to the end user. But it is the end users responsibility to implement the code for the routines setup( ) and loop( )

For many micro-controller operations it is the initialisation/setup that is the most difficult. This is where library routines can offer the most assistance

A simplified version of the main.cpp code is shown below. For the complete code see C:\Program Files (x86)\Arduino\hardware\arduino\avr\cores\arduino. Refer to diagram of links

The above listing introduces three new items

1. Arduino.h: The Arduino header file which is the subject of discussion in the following sections. See Arduino.h

2. init( ): An initialisation routine. This will be initialising something in the Arduino Environment and will be completely transparent to the end user.

3. serialEventRun: These are background or house keeping events that must operate in the background. These will generally be transparent to the end user until they are blocked by any user-code in the program loop( ). For example, code waiting for an infrequent event. In the case of WiFi it is in this time slot that the actual WiFi messages are transmitted and received.

The Arduino Header File: Arduino.h

The Arduino.h file contains 193 lines of links, includes, definitions and function prototypes. The above listing gives a selection.

1.The first line #ifndef Arduino_h says don't include this file in the project if it has already been included via some other source. That is ignore multiple definitions.

2.The file includes a number of standard libraries and then an include file #include "binary.h" that lists all combinations of the binary numbers B0 through B11111111. ( 0..255 decimal)

3.There is a section of the file bounded by the test extern "C" that tells a C++ compiler to compile that section as "C"code. That is no classes and no overloading of functions.

4.There are a number of links to Arduino specific hardware.

<avr/io.h>

#include "pins_arduino.h"

These files are discussed in later sections.

In the header file there are 156 lines included in the "#ifdef __cplusplus" test. The extern "C" is to notify the C++ compiler to compile included operations as C code. Since C++ is an object-orientated language one of its features is polymorphism. That is many different forms. A proceedural language, such as "C" upon which C++ is built does not have multiple forms.

As an example assume in "C" there are several routines to display different variables on an LCD. For example

void LcdString(char *characters);

void LcdCharacter(char character);

void LcdInt(unsigned int i);

In developing a C++ library all three methods/functions need to be combined (overloaded) into one LcdPrint( ) where the C++ compiler knows what form of LcdPrint( ) (or just print( )) to use by the type of parameter. However, the C++ compiler needs different names for the same method but with different parameters. Therefore the C++ compiler implicitly renames (mangles) functions based on their parameters. Extern says "don't mangle function names"

If there are legacy C files that include function names from other legacy C files all of these files would need to be mangled which would become very messy. So it is convenient to provide a keyword which can tell the C++ compiler not to mangle a function name. That is the "extern" keyword which tells the Arduino C++ compiler to compile as "C" code and not to mangle (rename) function names.

One easy to read reference is https://stackoverflow.com/questions/1041866/what-is-the-effect-of-extern-c-in-c

When developing library files the Arduino header file is not explicitly included. For that reason when developing libraries it is necessary to include the Arduino header file to include the definition of for example a byte. That is #include <Arduino.h>

Definitions of Binary Bit Patterns: binary.h

Note with binary numbers there are two possibilities

1.Use substitution as illustrated. This is fast but as shown requires many lines of defines.

2.Have the compiler calculate the result.

It is also possible to have the micro-controller perform the calculation. This has two defects:

1. It will take time.

2.The time taken will vary with different combinations of bit patterns. For time sensitive applications this will be an issue. For time sensitive applications all routines must take the same time.

Input-Output Header File: io.h

The "avr/sfr_defs.h" file is included to define macros that make the special function register definitions of the ATMega328 microcontroller look like C variables or simple constants

The avr/io.h file directs the compiler to the appropriate AVR device. That is iom328P.h for the Arduino ATMega328. Users are warned not to include avr/iom8.h directly in their code. The iom8.h file actually checks for _AVR_IO_H.

Special Function Registers: sfr_defs.h

This header file looks at the need to program input output registers as opposed to generic memory. The basic data memory map consists of four areas

0000-001F: Micro-controller registers. Some of these registers will have special functions such as the stack pointers. Others will be used for manipulating data.

0020-005F: Micro-controller input-output registers. These registers will be used in controlling or monitoring input-output pins on the micro-controllers.

Note down at the assembly level these registers can be addressed either using the Load/Store or the In/Out instructions.

There is one complication - With the In/Out instructions the address range is 0x0000 through 0x001F where as the Load/Store instructions the registers are in the address range 0x20 through 0x3F. That is an offset of 0x20..

0060-00FF: Extended I/O registers

0100-08FF: 2048 bytes of internal static ram. Note in the Arduino IDE this is known as dynamic memory.

The Arduino programs or sketches will be stored in a separate memory (ROM or Flash). The variable data will be placed in the static RAM (SRAM).

In general it will not matter where variables are located in RAM. The compiler will start at location 0x0100 and work upwards as required. However, the input-output registers are a different situation. These must be at a specified location. The sfr_defs.h file defines macros that may be used to make the special function register definitions look like C variables or simple constants.

The above listing is only a small snippet off the full file. The full file accounts for legacy tools and assembly language programs where the offset __SRF_OFFSET might not be defined. If it is not defined it is set to 0x00 otherwise it is set to 0x20 to account for the offset illustrated in the previous diagram. As illustrated 0x20 is added to the input/output address to give the memory address.

The sfr_defs.h file is included in the header files for all micro-controllers. For the Arduino iom328P.h provides examples of its application.

The ATMega Port Pins Header File: portpins.h

This file is concerned with the various labels that might be given to the input-output pins and registers

This header file ensures PA0 and PORTA for example are both defined. They may be interchanged in any code.

Input Output header file for the AVR ATmega328P: iom328P.h

This header file is concerned with the hardware and software interface for the ATmega328P microcontroller as used in the Arduino UNO. Basically the software needs to control input/output pins so it needs to known their addresses. The following table gives the memory map for the PORTB through PORTD registers (There is no PortA with the ATmega328).

The ATmega328P does not have a PORTA register so there are no offset addresses in the range 0x20 through 0x22. Further PORTC does not have a bit 7.

The header file 10m328P.h defines the definitions of the input-output registers for use in the Cpp programs/sketches. This file should only be included from <avr/io.h>, never directly.

io.h includes the definitions from avr\sfr_defs.h which are used in this header file. Note there will be an error if avr/io.h has not been included.

In the snippet shown

1. #define PIND _SFR_IO8(0x09) uses the macro of sfr_defs.h to define PIND as the io address 0x09 or the memory address 0x29.

2. There are a number of constant definitions related to the ATmega328P. For example RAM is in the range 0x100 through 0x8FF - Flash memory will be in the range 0x0000 through 0x7FFF.

3. There are a number of definitions related for example to the Vectors and Sleep modes. - Vectors are related to when the normal program mode is interrupted. The processor will vector to an interrupt service routine.

Arduino Pins Header File: pins_arduino.h

Previous sections have focused on the header files for the ATmega328P. This section concentrates on the header file relevant to the Arduino pins. The simplified diagram shows the Arduino digital pins. These are the pins that will be seen by the end user. On the Arduino UNO board there is a Led wired to pin 13. As shown below this is actually pin 19 or PB5 on the ATmega328P chip. While the developer writes to Arduino pin 13 to control the built-in led the underlying code will actually program bit 5 of the ATMega328's PB register.

The above header file uses the addresses of the input-output special function registers. These were defined in the section iom328P header files. (Note the memory or offset address is offset by 0x20 from the input-output address. Thus for example "#define PORTD _SFR_IO8(0x0B)" defines PORTD as the memory address 0x0B + 0x20 = 0x2B.)

Continuing the example of the built in LED.

1. The above header file includes "#define LED_BUILTIN 13"

2. From the array "digital_pin_to_port_PGM[]" element 13 is given as PB.

From this the software now knows the LED is wired to one of the pins on the ATMega's port B.

Further from the 13th element of the array "digital_pin_to_bit_mask_PGM[]" it knows it is bit 5. The mask will be (1 << 5 = B00100000). This will be calculated by the compiler following the definition of __BV(bit) in header file sfr_defs.h not shown in sfr_defs.h snippet)

3. To configure the pin it will be necessary to program the ATMega328 MODE register. Its address is given in the "array port_to_mode_PGM[]".

While not shown in the Arduino.h snippet PA is defined as 1, PB as 2, PC will be 3 etc.

Since PB==2 the address is the second element.

4. Finally to actually set the output high the bit mask B00100000 will be written to Port B who's address is given by the third element (element 2) in the array "digital_pin_to_port_PGM[]"

Note all this work is done by the compiler.

Arduino.h also defines NOT_A_PORT as 0. In the array there is no port corresponding to element 0 and for the ATmega328 there is no port A hence there is no element 1 either.

By changing the contents of the two functions "digital_pin_to_port_PGM[]" and "digital_pin_to_bit_mask_PGM[]" it should be possible to use the Arduino software with other user defined boards. The board numbering would need to start at "0".

Both the arrays contain constants of type uint8_t. There is also the directive PROGMEM. This tells the compiler

1. to generate code that uses the copy of these arrays contained in program/flash memory, and

2. not to include these arrays in the image of the variables copied into RAM from flash memory at run time.

The program space header file: pgmspace.h

This section is presented for completeness. The header file contains much deeper detail than the information presented in the previous sections about the other header files. The contents of pgmspace are about accessing data in the micro-controller program memory space. These facilities might be used in the LCD examples where the character patterns are stored in program memory to free up the scarce RAM resources.

Generating Libraries

The Arduino has many built in libraries and functions. It also has the capability to add additional libraries. This section looks at generating libraries by hand.

Before creating a class or library some revision of the Arduino file structure is useful. In the diagram below the Arduino is located in a path Users/John/Downloads/.... One of the folders in that path contains the supplied libraries. The left hand path in the diagram. More see section on files.

The example that follows is for a project involving a LCD part XC4616 from Jaycar.

My programs/sketchbooks are placed in the directory 00rmit/Arduino/Projects. In the Arduino IDE the location can be found (and changed) using File -> Preferences.

To test my library I created a new program/sketch "LCD_TEST". The Arduino gave the sketch the extension "*ino" and saved it in a directory/folder also called "LCD_TEST". This is illustrated in the file structure shown below (The centre column):

Also in the directory/folder 00rmit/Arduino/Projects there will be a folder for user supplied/generated libraries. The files must conform with the structure illustrated in the right column above.

1. a directory/folder with the label libraries.

2. a directory/folder with the label that corresponds to the name of the library. In the above example LCD_XC4616.

3. two files: the implementation file with extension "*.cpp" and a header file with extension "*.h"

Note the directory and the two files must all have the same name/label. In my example LCD_XC4614

The library folder and files may be generated by hand. Care must be taken that the files have extensions ".cpp" and ".h". Trying to create and save a file with the Arduino Editor will add an extension ".ino"

A better way is to create a project and create a *.zip file. (see next section)

Each time the Arduino IDE is opened it will load any libraries. This does imply that if any changes are made to either file the Ardunio IDE must be closed and reopened.

Generating Libraries

If a library is to be created as part of a project this may be done by including the *.cpp and *.h files as part of the project.

In the example below I am developing a project MESS_2_ESP_AP.

The Arduino IDE will give the extension "*.ino" to the main code. The other two files are created using the inverted triangle on the upper right. Both these files must have the same name MESS_2_ESP_AP (in my example) but with the extensions *.cpp" and "*.h".

Note the main code includes the statement "#include "MESS_2_ESP_AP.h" so the header file (and implemntation) file will be included in any build/compile.

1. Once satisfied that the code operates as expected the two file "*.cpp" "*.h" should be zipped into a file "MESS_2_ESP_AP.zip"

2. In the Arduino IDE select Sketch-->IncludeLibrary-->Add.Zip Library

There will now be a library MESS_2_ESP_AP in the contributed libraries.

Note if changes are necessary it is a bit messy since "Add .ZIP Library" will not allow you to overwrite an existing library. You need to use Windows Explorer for example and delete the folder ....../ MESS_2_ESP_AP.

Importing Libraries

The previous sections looked at the Ardunio folder structure with an emphasis on user libraries. To import a library from the internet the procedure will be:

1. Download the "*.zip" file. I choose to place all my program downloads in a folder programsJK.

2. In the Arduino IDE select Sketch-->IncludeLibrary-->Add.Zip Library (See above screen shot)

In my example I loaded a file TinyGPSPlus.zip. Then in my list of available contributed libraries it gave me the library TinyGPS++.

The Arduino15 File.

The folder " AppData\Local\arduino15" is a hidden folder. To make it visible select "Hidden items" in Windows Explorer.

From https://forum.arduino.cc/index.php?topic=668104.0 .arduino15 is where board packages and stuff get installed to (and which one should generally never touch, except maybe to delete the contents when board manager packages get screwed up). See next section

A Problem: The Arduino IDE suddenly would not start up.

I had been using the Arduino several times a day for many years.The Arduino IDE suddenly would not start up.It would reach the screen saying it is "Initialising Packages" and then exit.

The solution was to delete the file "arduino15".

For reference a search of the Arduino Forum provided the following suggestions. (Some of the discussion appeared to apply to problems with Arduino updates but I didn't believe this was the problem in my case).

Suggested solutions that applied if the final screen was "hidden" were:

1. Check if screen going to second monitor.

2. Go to Task Manager -> Application->Select App->Bring to front

3. Hover over active Arduino Task Bar Icon -> Right Click -> Maximise ->Resize IDE Window ->exit Arduino ->Restart Arduino

4. Delete c:\user\john\AppData\Local\arduino15. (See close to bottom in preferences screen shot.)

5. Make sure Documents\Arduino\libraries was present.

6. In the command file run Arduino_debug. In my case I had 3 issues related to Java

The folder " AppData\Local\arduino15" was not visible until I selected "Hidden items" on Windows Explorer. See screen shot above.

Deleting the folder "AppData\Local\arduino15" did solve the restart problem. However I then had to reload all of my contributed libraries.