SONiC

Software for Open Networking in the Cloud (SONiC) is the open networking solution used in Microsoft's Azure Cloud. The System is used in the lower layer of their Datacenter fabric (TOR and Aggregation).

We installed SONiC on the Mellanox Spectrum 2700 platform.

We downloaded the software from the public SONiC supported devices page which you can find at https://github.com/Azure/SONiC/wiki/Supported-Devices-and-Platforms

To be able to install SONiC we first had to update the switch line-card firmware, as the version we were running was too old. After this, installation via ONIE went fine, and we were able to look around the system. It is very important at this point to know that installing SONiC will set the console baudrate to 9600. Be prepared to loose console connection, and that you need to set your console to 9600 to reconnect.

After logging into the switch we can check that all the docker containers are running:

Sonic-1

We are now on the base host that holds the configuration for the whole system, the containers, and the connections between them.

The configuration is stored in a file named /etc/sonic/minigraph.xml. You will find an example below:

Untitled document

The running containers have this file mapped into their file system, parse it and extract the information needed for the specific container. The container then creates the configuration files needed by the application running on the container, for example for quagga.

Containerization of the components provides multiple advantages. First of all, is defines a very clear interface between the different components, and allows modification and replacement of one component without the danger of breaking other components because of the good isolation containers provide. The interface is the only point that needs to stay unchanged, but besides the interface to the other components a container can be seen as a black box. This allows for a complete replacement of, for example a routing implementation, without changing any of the other components.

The container approach also allows for very good isolation on the OS level, in particular for hardware resources and libraries used by the systems.

We are able to connect to each of the the different docker containers, and take a look at the configuration files we find there.

If we, for example, take a look at the bgp container, we can see that we find the config files as expected for quagga, there is no "special" adaption, except parsing the config from the minigraph.xml.

This changes a bit if we start looking behind the curtains. syncd is the container, and connectin there we can find several scripts that allow for configuring functionallity closer to the hardware, written againgst SAI. This again shows that sai is the narrow waist in this ecosystem.

Connecting to syncd is done this way:

sudo docker exec -it syncd bash

And looking into /usr/local/bin shows the scripts available on this machine:

root@sonic:/usr/local/bin# ls

1012_eth_start.sh

1016_eth_start.sh

1024_eth_start.sh

1036_eth_start.sh

1400_eth_start.sh

1410_eth_start.sh

1700_eth_start.sh

1710_eth_start.sh

2100_eth_start.sh

2410_eth_start.sh

2700_eth_start.sh

7400_eth_start.sh

chassis_manager

dvk_eth_start.sh

dvs_manager

dvs_start.sh

dvs_stop.sh

evb_start.sh

get_system_type.sh

pld_4_ports_spectrum_start.sh

pld_spectrum_start.sh

resources_manager

rpp_4_ports_spectrum_start.sh

run_chassis.sh

run_dvs_manager.sh

run_resources.sh

run_resources_sgmii.sh

sdk_api_cli

sdk_api_cli.py

sgmii_manager

sx_api_bridge_vport.py

sx_api_cos_ets.py

sx_api_cos_redecn.py

sx_api_cos_sb.py

sx_api_cos_sb_buff_stat.py

sx_api_cos_sb_delete_all.py

sx_api_cos_sb_get0.py

sx_api_cos_sb_pool_list_get.py

sx_api_cos_sb_statistics.py

sx_api_cos_shared_buffers_dump.py

sx_api_cos_trust_port.py

sx_api_cos_trust_port_both.py

sx_api_cos_trust_port_l2.py

sx_api_cos_trust_port_l3.py

sx_api_dbg_generate_dump.py

sx_api_fdb.py

sx_api_fdb_add_nve_entry.py

sx_api_fdb_dump.py

sx_api_fdb_mc_set.py

sx_api_flex_acl.py

sx_api_flex_acl_key_attr_get.py

sx_api_flow_counter.py

sx_api_host_ifc_dump.py

sx_api_host_ifc_policer.py

sx_api_host_ifc_trap.py

sx_api_host_ifc_trap_recv.py

sx_api_lag.py

sx_api_lag_mstp.py

sx_api_mc_router.py

sx_api_mstp.py

sx_api_port.py

sx_api_port_counter_dump.py

sx_api_port_counter_dump_all.py

sx_api_port_speed_get.py

sx_api_port_speed_set_100G.py

sx_api_port_speed_set_10G.py

sx_api_port_speed_set_25G.py

sx_api_port_speed_set_40G.py

sx_api_port_speed_set_50G.py

sx_api_port_speed_set_56G.py

sx_api_port_storm_control.py

sx_api_ports_dump.py

sx_api_router.py

sx_api_router_ecmp_dump.py

sx_api_router_flow_counter.py

sx_api_router_interface_dump_all.py

sx_api_router_neigh_dump.py

sx_api_router_uc_routes_dump_all.py

sx_api_router_virtual_routers_dump_all.py

sx_api_sflow_lag_port.py

sx_api_sflow_operations.py

sx_api_sflow_port.py

sx_api_span.py

sx_api_span_egress_mirror.py

sx_api_transaction_mode_set.py

sx_api_tunneling.py

sx_api_tunneling_vxlan.py

sx_api_vlag_mstp.py

sx_api_vlan.py

sx_api_vlan_get_all_vlans_ports.py

sx_api_vlan_multi_set.py

sx_api_vlan_qinq_dump.py

sx_player_start.sh

sx_tags

sxd_api_get_cable_info.py

sxd_api_get_rpm.py

sxd_api_get_sbib.py

sxd_api_pmcr.py

sxd_sgmii_start.sh

sxd_sgmii_stop.sh

sxd_tags

test_infra_common.py

test_infra_common.pyc

So here we see the functions that really control the switching hardware. These are also some very good examples for writing our own code agains SAI.

We will have some more posts in regards to SONiC, one looking at the database itself, and one running SONiC on different hardware platforms, and exchanging BGP information between the,