Step 1: Create two docker containers named as "left" and "right" with network "none":

docker run --name left --network none -itd ramansingh1984/dweb-base:dweb-base-ubuntu

docker run --name right --network none -itd ramansingh1984/dweb-base:dweb-base-ubuntu

I have used my own repository image of Ubuntu from docker hub. This image has basic utilities installed which we will use later like sudo, ping, ifconfig etc. You re free to use any Ubuntu image and can install packages like sudo, iputils-ping and net-tools in it. Verify the container by docker ps -a. You should get output something like it:

If you get into these container by running command docker attach left or docker attach right, and see the Ethernet details, you will find there is no Eth attached and no IP is assigned except loopback address. This is because you have selected network "none". You can learn more about docker networking by reading this informative article:

https://runnable.com/docker/basic-docker-networking

The output will be like this:

Step 2: Find out the Process ID (PID) of containers. These PIDs will be used later to identify the individual containers. It is also possible to use names of containers in place of PIDs.

pid_left=$(docker inspect --format '{{ .State.Pid }}' left)

pid_right=$(docker inspect --format '{{ .State.Pid }}' right)

pid_left is PID of left container whereas pid_right is PID of right container. The output will goes like this:

Step 3: Creation of Bridges (br-left and br-right)

We should create two bridges now. These bridges will connect docker container and NS-3 nodes. At one end, docker bridge is connected by container through its Eth-Veth pair (Ethernet inside container- Virtual Ethernet with outside world) while on the other end, it is connected to NS-3 nodes through tap interface.

sudo brctl addbr br-left

sudo brctl addbr br-right

then you can view created bridges by running sudo brctl show command as given in image:

Step 4: Creation of Tap Devices (tap-left, and tap-right)

Now, we should create tap devices. These tap devices will be connected with bridges and tap-bridge interface of NS-3 Nodes. The tap device can be created by this command:

sudo tunctl -t tap-left

sudo tunctl -t tap-right

Now, switched up the devices and set 0.0.0.0 IP address to these tap devices. These tap devices are like of tunnel and will just forward frames through it from one end to other end , no addressing requires and hence assigned 0.0.0.0 address.

sudo ifconfig tap-left 0.0.0.0 promisc up

sudo ifconfig tap-right 0.0.0.0 promisc up

We can see the created tap devices by command sudo ip link show, see the output:

Step 5: Connecting tap devices with bridges

Now, we should connect tap devices with bridges. This can be done by adding tap devices to the respective bridges:

sudo brctl addif br-left tap-left

sudo ifconfig br-left up

sudo brctl addif br-right tap-right

sudo ifconfig br-right up

Now we can see the connection interface by command sudo brctl show:

Step 6: Writing 0 to each bridge files to allow everything through all bridges

This step need root authentication and I suggest to become super user first by command su and then run this small script.

pushd /proc/sys/net/bridge

for f in bridge-nf-*; do echo 0 > $f; done

popd

For this step either directly copy these script instruction and paste on terminal or write a script file and run that file. both ways will work as long as you have root permissions.

Step 7: Creating namespace and adding new Ethernet interface for first container i.e. left

NOTE: Before moving to this step, make sure you have PID in pid_left variable by running.

echo $pid_left

If it is 0 or empty assign PID in the variable again as shown in step 2.

This is typical but important step which needs reasonable understanding about namespace concept in docker. You can learn about namespace by reading this topic:

https://platform9.com/blog/container-namespaces-deep-dive-container-networking/

To sum up, All the devices, interfaces should be in same namespace if you want them to recognize each other. In this example, our container's internal Ethernet interface should be in the same namespace as the outside bridge interface, in order to attach outside bridge with inside Ethernet interface.

Now we will create a namespace region by entering namespace entry for left container:

sudo mkdir -p /var/run/netns

sudo ln -s /proc/$pid_left/ns/net /var/run/netns/$pid_left

Now, We need to create "peer" interfaces: internal-left and external-left. The internal-left will be attached with the bridge.

sudo ip link add internal-left type veth peer name external-left

sudo brctl addif br-left internal-left

sudo ip link set internal-left up

Now, the other end named as external-left will be renamed as Eth0. This will act as Ethernet Interface of left container. Also, we will assign IP address and random MAC address to it. This is possible because br-left bridge, external-left interface and container left are in the same namespace, otherwise they could not have recognized each other.

sudo ip link set external-left netns $pid_left

sudo ip netns exec $pid_left ip link set dev external-left name eth0

sudo ip netns exec $pid_left ip link set eth0 address 12:34:88:5D:61:BD

sudo ip netns exec $pid_left ip link set eth0 up

sudo ip netns exec $pid_left ip addr add 10.0.0.1/16 dev eth0

If you see bridge details by brctl show, you will see the entry of internal-left in interface of br-left bridge:

The external-left is renamed as eth0 and IP i assigned. You can verify it by accessing bash terminal of left container and viewing network details like:

The left container now has one Ethernet with IP 10.0.01/26, connected to bridge br-left through virtual interface internal-left. Please remember, I run these commands inside super user and hence sudo is not used.

Step 8: Creating namespace and adding new Ethernet interface for first container i.e. right

Now the step 7, will be repeated for right container as well. Explanation is exactly same as previous step. Even you can create a script file for each container.

Check pid_right by running echo $pid_right, it it show 0 or empty assign PID to pid_right as shown in step 2.

sudo ln -s /proc/$pid_right/ns/net /var/run/netns/$pid_right

Now, We need to create "peer" interfaces: internal-right and external-right. The internal-right will be attached with the bridge.

sudo ip link add internal-right type veth peer name external-right

sudo brctl addif br-right internal-right

sudo ip link set internal-right up

Now, the other end named as external-right will be renamed as Eth0. This will act as Ethernet Interface of right container. Also, we will assign IP address and random MAC address to it. This is possible because br-right bridge, external-right interface and container right are in the same namespace, otherwise they could not have recognized each other.

sudo ip link set external-right netns $pid_right

sudo ip netns exec $pid_right ip link set dev external-right name eth0

sudo ip netns exec $pid_right ip link set eth0 address 5A:34:88:5D:61:BD

sudo ip netns exec $pid_right ip link set eth0 up

sudo ip netns exec $pid_right ip addr add 10.0.0.2/16 dev eth0

If you see bridge details by brctl show, you will see the entry of internal-right in interface of br-right bridge:

Note: Please remember IP address and MAC address can be changed as per user's requirement.

Step 9: Testing Connection using NS-3 simulated CSMA network

Now every configuration is in place and time has come to check the connection between Docker container and NS-3 nodes. For this example we have used one NS-3 script "tap-csma-virtual-machine.cc" available in $NS3-HOME/src/tap-bridge/examples/ folder. My NS3-HOME folder is /home/raman/repos/ns-3-allinone/ns-3.30 but please verify your NS-3 home folder. Also I assume that you have already enabled examples by running commands in NS3-HOME folder:

$./waf configure --enable-examples --enable-tests --enable-sudo

$ ./waf build

TESTING CONNECTIVITY:

Now, cd to NS3-HOME folder and you can test connectivity by running script tap-csma-virtual-machine.cc like this:

$cd /repos/ns-3-allinone/ns-3.30

$./waf --run tap-csma-virtual-machine

This simulation will run for 10 minutes, and you can play by testing connectivity between left and right containers. The simplest way to test it is pinging each other like this:

From left container:

$docker attach left

#ping 10.0.0.2

From right container

$docker attach right

#ping 10.0.0.1

If everything goes well, both should ping each other. The output can be seen in this quick video:

Step 10: After you finish successful testing - Destroy Everything

# Down the bridges

ip link set br-left down

ip link set br-right down

#Remove the taps from the bridges

$ sudo brctl delif br-left tap-left

$ sudo brctl delif br-right tap-right

# Delete the bridges

brctl delbr br-left

brctl delbr br-right

#Delete the taps

sudo tunctl -d tap-left

sudo tunctl -d tap-right

# Stop all running container

docker container stop $(docker container ls -aq)

# Destroy all stopped container

docker container rm $(docker container ls -aq)

# Verify if still any container existed

docker container ls -a

The output of this command should be empty. It means you have deleted all containers.

Common Issues:

There are common issues I encountered while trying this connectivity experiment.

1. The Eth and Veth are virtual and not persistent, so every time docker container stops, PID and other thins get reset. Hence, Need to repeat that portion again.

2. There are instances when docker stop automatically after times, hence needs to keep an eye on docker if running or not.

For further readings, other blogs related to Docker container communication and other issues can be found at my HOME PAGE..

Thank you very much........