Valid reason why TCP/IP stack will receive more TCP data than advertised MSS

Introduction

Laymen explanation

If you have used MSS, you know that it is described in rfc and sender should abide by MSS size. However, you might have got surprised by packet capture and seen that the amount of data received is more than advertised MSS(1410 bytes in below example). Is the sender buggy or is there any valid reason for this? This article tries to explain valid condition.

Technical explanation

Linux network driver faciliatate optimization of network data at the driver level. They use various approaches for the same. Assembiling incoming consecutive TCP segments is one of such activity. To do this, it will use IP header of first segment and combine consecutive TCP segments. In this way, it will save cost of extra headers. And you will see packet size bigger than advertised MSS.

Why this optimization is needed

MSS is controlled by MTU (Maximum Transmission Unit). Note that if an IP packet size is within MTU limit, it avoids IP fragmentation of packets. So, advertised MSS should also be honoring MTU. Due to this restriction, advertised MSS is usually lesser than the actual capability of processing unit.

This driver level optimization is done in order to handle this situation.

Caution

Due to this optimization, received packet may become Jumbo packet (>1500 bytes).

OS network stack should accept this happily. However, if you are relying on custom TCP/IP stack which doesn't support Jumbo packet, then this optimization will result in packet processing issue.

Method to disable this optimization

In Ubuntu Linux, this optimization is enabled by default. ethtool utility can be used to disable it

ethtool to disable tso

>> To disable assembly of tcp segments

root@ubuntu:~/personal# ethtool -K docker0 tx off rx off tso off

Cannot change rx-checksumming

Actual changes:

tx-checksumming: off

tx-checksum-ip-generic: off

tcp-segmentation-offload: off

tx-tcp-segmentation: off

tx-tcp-ecn-segmentation: off

tx-tcp6-segmentation: off

udp-fragmentation-offload: off [requested on]

>> To enable assembly of tcp segments

root@ubuntu:~/personal# ethtool -K docker0 tx on rx on tso on

Cannot change rx-checksumming

Actual changes:

tx-checksumming: on

tx-checksum-ip-generic: on

tcp-segmentation-offload: on

tx-tcp-segmentation: on

tx-tcp-ecn-segmentation: on

tx-tcp6-segmentation: on

udp-fragmentation-offload: on

Reference

https://sites.google.com/site/jbsakabffoi12449ujkn/home/networking/purpose-of-mss-in-tcp

https://tools.ietf.org/html/rfc879

https://en.wikipedia.org/wiki/Jumbo_frame

https://en.wikipedia.org/wiki/Large_receive_offload