Apple opens Multipath TCP in iOS11

Apple opens Multipath TCP in iOS11

During WDC, Apple engineers present new innovations that will appear in future releases of iOS. During the Advances in Networking session, Stuart Cheshire, David Schinazi and Christoph Paasch presented the networking innovations in the forthcoming iOS 11.
Stuart Cheshire starts the introduction with “very exiting news about multipath protocols. The iPhones that we all have are inherently multipath devices. They have multiple radios in them. For the most part, today we only use one radio at a time. It’s time we start at having multipath protocols for multipath devices. »
Then he focuses on ECN, Explicit Congestion Notification. Apple has been pushing this technique that allows end hosts to better react to congestion if routers mark packets instead of dropping them when they are congested. This technique has been initially proposed almost 30 years ago for the Decbit protocol and then suggested almost 20 years ago for the Internet in RFC2481.
There are three types of participants that have to cooperate to make ECN deployable: clients, routers and servers. On the server side, almost 3/4 of the Alexa top 1 million support ECN basically because it’s part of the default Linux kernel so that they get it for free. On the client side, iOS uses it for 50% of the connections since IOS 10.3 without any significant problem.
On the network side, routers need to mark packets instead of dropping them when they start to be congested. Apple’s measurements indicate that this feature has started to be deployed in some networks. Note that not all routers need to support ECN. ECN must be supported on the bottleneck, which is typically the access router. Apple’s measurements indicate that the fraction of networks that use the ECN bit to mark packets when they are congested is (slowly) growing.
Then David Schinazi provided a status update about IPv6. IPv6 is now a reality and several networks have started to only use IPv6 and access IPv4 servers through NAT64 services.
Another important announcement in David’s presentation is the move to « User space networking« . Traditionally, transport protocols have been implemented in the operating system kernel to be useable by all applications. This model makes it difficult to deploy new transport protocols quickly. Apple moves the transport layer to the user space inside libraries and will probably deprecate the BSD Unix sockets in the future. They indicate that this enables them to provide CPU usage reduction and thus better battery life.
The third and longest part of the presentation was delivered by Christoph Paasch who described the support of Multipath Protocols for Mobile Devices.
Together with other Ph.D. students at Université catholique de Louvain, Christoph was one of the early developers of Multipath TCP. Shortly after presenting his Ph.D thesis, he moved to Apple when they started to use Multipath TCP for the Siri application in September 2013. After almost four years of restricted usage, Multipath TCP becomes open and is useable through a public API in iOS 11. He starts by explaining the classical scenario of a user leaving home while interacting with a server on the Internet through an App. This use case was first covered in a scientific article published in 2012 that he wrote during his Ph.D.
WiFi Assist controls how connections are established through WiFi and cellular by creating contests between the two networks. If WiFi is poor, then the cellular flow will win and will be used to carry the data, which improves the user experience. But WiFi Assist does this at flow creation time. With regular TCP, if WiFi fails, all the connections need to be reestablished.
Users would expect that data transfer could continue seamlessly when WiFi assists switches from WiFi to cellular. With regular TCP, this is not possible for technical reasons. Fortunately, Multipath TCP provides the same service as regular TCP. It is also specified by the IETF and benefits from three decades of experience that we have with TCP. Christoph provides a brief presentation of Multipath TCP. A more detailed and readable introduction to the protocol may be found in an article that he wrote for ACM Queue.
The URLSession API makes it very simple for the application developers to use Multipath TCP in their applications. It does this by managing subflows. The application is not aware of the existence of those subflows and does not need to be modified to benefit from Multipath TCP.
The Siri application uses Multipath TCP since iOS 7 and its requirements are a perfect match for Multipath TCP that is able to seamlessly recover from failures of the cellular or WiFi interfaces that occur often when users are walking, a frequent scenario for Siri. Siri opens both WiFi and cellular links and prefers WiFi to send data but measures the latency. If the latency of the WiFi interface increases, Multipath TCP automatically switches to the cellular link to preserve a good user experience. Measurements show that the 95th percentile time to first word, i.e. the time that the user waits before seeing a first word from Siri on her screen, has been significantly reduced. It has also reduced the number of network failures by 5 times. Many developers have asked to open the API for Multipath TCP.
Three steps are required for Multipath TCP :
  • server support
  • selection of the connections that will use Multipath TCP
  • update the application code
Server Support. Today regular servers rarely support Multipath TCP by default. Many load balancers already support Multipath TCP and can be enabled on the load balancers. On Linux, the mainstream Linux kernel does not yet support Multipath TCP, but Christoph also maintains the Multipath TCP implementation in the Linux kernel. He contributed a lot to this implementation while finalising his Ph.D. thesis. The implementation is publicly available from This implementation is available as patches to major Linux kernel releases but is not yet included in the mainstream kernel. During the talk, Christoph announced that « We are working with the open-source community of this kernel to maintain it and make it available in the mainstream kernel so that all Linux distributions are getting it by default ».  This is a very important announcement that will bring Multipath TCP on all Linux devices in the coming years.
Besides upgrading the load balancer, Christoph also suggests the possibility of deploying a Linux server with the Multipath TCP patch that uses an HAProxy or an nginx proxy to serve regular servers that have not yet been updated with the Multipath TCP patch.
Finally, Christoph encourages application developers to « go to your server team and tell them to get the infrastructure ready for Multipath TCP. »
Once the server has been upgraded, application developers need to decide which connections will use Multipath TCP. Apple’s API exposes different service types:
  • Handover: high reliability for long-lived connections
  • Interactive: mode used for Siri. Provides low latency response for interactive sessions.

Handover mode

Connection starts over the WiFi link and no packet is sent over the cellular interface. If the signal gets worse, a new TCP subflow will be created on the cellular interface automatically. The cellular subflow will be removed once the user is back in a WiFi network. Apple encourages the utilisation of this mode for persistent and long-lived connections. WiFi Assist limits the amount of data sent over the cellular link and the connections that send too much data over cellular will be stopped. Applications should not use the handover mode for bulk data transfers.

The interactive mode establishes both WiFi and cellular subflows for each Multipath TCP connection, even if the WiFi network appears to be working well. The objective of this mode is to reduce latency. The Multipath TCP scheduler will select the flow that provides the lowest latency. It should be used for connections where latency is really key. This will be available in an upcoming beta release.
The last point to use Multipath TCP is to enable it for the selected connections. Apple provides a simple and clean API that encourages developers to use Multipath TCP with a minimum change to the code of their applications. Developing a Multipath TCP enabled app on iOS could not be simpler than that.
And as with many Apple presentations, Christoph finishes with one last thing : the aggregation mode. This was one of the initial use cases for Multipath TCP: aggregate low bandwidth links together to create a virtual path that has higher capacity. During his Ph.D. thesis, Christoph even showed that Multipath TCP could reach 50 Gbps by aggregating 6 x 10 Gbps Ethernet links on a single server. On the iPhone, the aggregation feature is currently experimental because the application will use cellular data and Apple wants to get feedback from developers on which applications would benefit from this kind of aggregation. This mode needs to be enabled with a specific setting on the iPhone. It will not be available on consumer devices.
The final slide of the presentation summarises well the main improvements brought by iOS 11

User experience is a core value at Apple. The network innovations announced for iOS11 will enable developers and users to benefits from Multipath TCP. This is only one of the many use cases for Multipath TCP. Another important use case are the hybrid access networks that are pioneered by Tessares.

Want to contribute to the development of new Multipath TCP use cases with Tessares ? We’d love for you to join us. Apply here !