StepByStep: Sending data to/from docker container from ns-3 Node
Date: 23-05-2020
My other blog StepByStep: Establishing virtual network between Docker container and NS-3 nodes discusses the concept of establishing communication with docker container and ns-3 nodes. It uses two ns-3 nodes connecting through CSMA link (no IP addressing assigned) and both container on same IP network. However in complex network scenarios, users may needs communication through scenario where different networks are connected using different subnets/ topology. In this blog, we try to send some data (not just ping) from ns-3 node which resides on different network than docker container.
A Request: To thoroughly understand this experiment. Please complete Blog 1 first given at my HOME PAGE...
Step 1: Understand the underlying Network
In the below figure, We can see that Node 1 and Node 2 is connected through P2P link and has network 10.1.2.0/24. Node 1 is assigned IP address as 10.1.2.1. whereas IP address node 2 is assigned as 10.1.2.2.
The second network is 10.1.3.0/24 and 4 nodes are connected through CSMA link having different IP addresses as shown in figure. Please note that Node 2 has two IP addresses as this node has two interfaces: one connected to Node 1 (p2p link) , other interface connected to node 3 (csma link). Node 2 will also behave as router node in ns-3 in order to route packets through both networks.
Now we wants to connect docker container to Node 1 and make it able to communicate from all other nodes ( even if node 5 is lying in another network).
Figure 1 Underlying Network
Step 2: Create a Docker Container
For this experiment, I created alpine container using following command:
A. Download the Alpine image
B. Build own image using dockerfile
docker build -t my-image-alpine .
This will create my own alpine image with many essential utilities installed as given in docker file. Some utilities I installed are: wget, curl, nmap, netcat, net-tools. These utilities will help later while experimenting communications. The dockerfile I used can be found here.
C. Create docker container:
docker run --name testvm123 --network none --publish-all=true -itd my-image-alpine
Here I created container with "--publish-all=true" option which means all ports are exposed for external connection. Please note that this may be severe security concern if you wants to use this container in production environment. I have exposed all ports to eliminate one step to find required ports and publish them manually. You need to publish only those ports which you will use in your application if security is important concern in you working environment.
Step 3: Connect the Docker Container to Node 1
We can connect docker container through tap-bridge arrangement (as shown in my other blog). The tap-bridge can not be connected to a node having P2P link, Hence we need to add one ghost node which will be connected to node 1 through csma link (alternatively wifi node can be used as ghost node). The ghost node will connect to docker container through tap-bridge and Linux bridge arrangement. The logical network will then look like as shown in below figure:
Figure 2 Logical Network with Docker Container Attached
Docker Container will be connected to Node 0 using tap bridge and Linux bridge as discussed in other blog but have some additional steps to make it communicable with all networks. The following steps can be considered to successfully attach docker container to ghost node (The script file to complete these steps can be found here) :
# Create the Linux Bridge
sudo brctl addbr br-test
# Create tap devices, counting equal to number of containers
sudo tunctl -t tap-test1
sudo ifconfig tap-test1 0.0.0.0 promisc up
# Connecting tap devices and bridges
sudo brctl addif br-test tap-test1
# Assign the IP address to the linux bridge, this will insure Linux bridge will be on same network as docker container and will transfer data through.
sudo ifconfig br-test 10.1.1.5 netmask 255.255.255.0 up
sudo ifconfig br-test up
# Allow traffic across bridges (run these three lines simultaneously as it contains loops)
pushd /proc/sys/net/bridge
for f in bridge-nf-*; do echo 0 > $f; done
popd
#Find the pid of containers
pid_multi1=$(docker inspect --format '{{ .State.Pid }}' testvm123)
# Setting namespaces and interfaces with MAC and IP address for Other blocks
sudo mkdir -p /var/run/netns
ln -s /proc/$pid_multi1/ns/net /var/run/netns/$pid_multi1
#Now, We need to create "peer" interfaces: internal-left and external-left. The internal-left will be attached with the bridge.
sudo ip link add internal-test type veth peer name external-test
sudo brctl addif br-test internal-test
sudo ip link set internal-test up
# Random MAC address (run these three lines simultaneously as it contains loops)
hexchars="0123456789ABCDEF"
end=$( for i in {1..8} ; do echo -n ${hexchars:$(( $RANDOM % 16 )):1} ; done | sed -e 's/\(..\)/:\1/g' )
MAC_ADDR="12:34"$end
#Connect external tap to docker container and rename it as eth0. this will work as Ethernet interface of docker container. Then assign one random mac address to eth0 and up the interface
sudo ip link set external-test netns $pid_multi1
sudo ip netns exec $pid_multi1 ip link set dev external-test name eth0
sudo ip netns exec $pid_multi1 ip link set eth0 address $MAC_ADDR
sudo ip netns exec $pid_multi1 ip link set eth0 up
#Assign IP address 10.1.1.6 to eth0 of docker container interface. This IP address should be from same network as of ghost node of ns-3.
sudo ip netns exec $pid_multi1 ip addr add 10.1.1.6/24 dev eth0
# Add the routing information in docker container. Without this step you will not be able to transfer data to network other than home network. This step insure to send all default packets (i.e. packets destined for not home network) to gateway node which is node 1 of ns-3 network. Note that gateway IP address is of node 1 of network shown in figure 2.
sudo ip netns exec $pid_multi1 ip route add default via 10.1.1.2 dev eth0
Now the docker container should have connected to ns-3 network through ghost nodes.
Step 4: ns-3 Code to send data
I have modified tap-wifi-dubmbell.cc (https://www.nsnam.org/doxygen/tap-wifi-dumbbell_8cc_source.html ) to execute this experiment.
In my experiment, node 5 (see figure 2), having IP 10.1.3.4 sends text stream data to docker container having IP 10.1.1.6 using UDP and port 2399. Please note that node 5 and docker container are on different networks and we are trying to send text data and not just trying to ping.
The source code creates the ns-3 nodes as shown in figure 1 and 2. This code also create text stream and sends them to remote computer with IP 10.1.1.6 using UDP on port 2399.
The source code is attached here. Please study the source code carefully.
Step 5: Crosschecking IP address and routing information of Docker container
A. Open the terminal and attach the docker container to this terminal by running:
docker attach testvm123
B. You can check the IP address of this docker container.
ifconfig
The output will be something like this:
C. Check the routing information. This is very important as this will help to route packets from/to distant networks.
route
The output should be something like this:
Note that in routing information, for destination default all packets will be send to gateway 10.1.1.2 using interface eth0 of docker container. This will insure that, this machine will send all the packets having destination outside range IP address (i.e. not from 10.1.1.0/24 network) to node 10.1.1.2.
Step 6: Testing the communication with all nodes and Sending data from node to docker container
Now run the attached source file tap-wifie-dumbell3.cc and establish communication:
./waf --run tap-wifi-dumbbell3
Now, this source file will run for 600 seconds and you will have enough time to test communication.
A. Check the communication to Linux bridge (IP 10.1.1.5), Node 2 (IP 10.1.1.2), Node 3(IP: 10.1.3.2), Node 4 (IP: 10.1.3.3) and Node 5(IP: 10.1.3.4)
ping 10.1.1.5
ping 10.1.1.2
ping 10.1.3.2
ping 10.1.3.3
ping 10.1.3.4
If you have stitched everything perfectly, all pings should run successfully showing perfect connection.
B. Send the text stream from node 5 to docker container.
Since the node 5 will send text stream by UDP on port 2399, there should be some utility/ script/ program which should listen on this port. For this experiment we will use netcat which can send/receive/listen network connections using TCP or UDP.
Run netcat on docker container terminal to listen UDP data on port 2399
nc -lkvu 2399
Now since we are listening on docker container we should start sending data from node 5.
on host linux run
./waf --run tap-wifi-dumbell3
Now you should see the text stream on docker container terminal as shown in video. I have captured the video of output to showcase it.
new_video23052020.mp4Step 7: Sending data from Docker container to ns-3 node through UDP
Now as we have tested communication from ns-3 node to Docker container, we can test the communication from Docker container to ns-3. Here we will send data from docker container having IP address 10.1.1.6 to ns-3 node 5 having IP 10.1.3.4. To send data, we will again use netcat utility at Docker container however we can use C/C++ or any other programming language to write code for UDP client. If user wants to used TCP rather than UDP, I believe procedures will be similar.
The major issue will be to receive packets at ns-3 node and print the content. For this, following code snippet will help us to do so:
// Receive some data at ns-3 node
TypeId tid = TypeId::LookupByName ("ns3::UdpSocketFactory");
Ptr<Socket> recvSink = Socket::CreateSocket (nodesRight.Get (3), tid);
InetSocketAddress local = InetSocketAddress (Ipv4Address::GetAny (),2399);
recvSink->Bind (local);
recvSink->SetRecvCallback (MakeCallback (&ReceivePacket));
Here we created a socket with type UDP and port 2399 and then we bind it and received packet at this port using callback. The received packet is passed to function ReceivePacket. The functionality of fucntion is defined as below:
void ReceivePacket (Ptr<Socket> socket)
{
NS_LOG_UNCOND ("Received one packet!");
Ptr<Packet> packet = socket->Recv (1472,0);
std::cout<<"Packet Size:"<<packet->GetSize()<<std::endl;
uint8_t *buffer = new uint8_t[packet->GetSize ()-1];
packet->CopyData(buffer, packet->GetSize ());
std::string s = std::string((char*)buffer);
std::cout<<"Data Received:"<<s<<std::endl;
}
Here, we printed three things once packet is received and recognized through callback: that a packet is received, the size of the packet i.e. numbers of characters + 1 (null character) and the content of packets i.e. string message. The full code of this is attached here.
The UDP client can be invoked at Docker container using netcat as:
nc -u 10.1.3.4 2399
where, -u means connection is UDP, 10.1.3.4 is remote IP and 2399 is port at which data should be sent.
Step 8: The 0uput example of data communication from Docker container to ns-3 node
The output should be something like this:
new1.mov