Welcome to this third issue of Multipath News.

Feel free to send suggestions or any feedback to multipath-news@tessares.net or through our online form.

The last two months have been pretty busy for Multipath TCP. In June, Apple announced during WWDC2017 that they would enable Multipath TCP on iOS11 for any application. We have already discussed this announcement in more details in a previous blog post.

Internet Engineering Task Force

The IETF Multipath TCP Working Group met twice during IETF99 in Prague. The main objective of these two meetings were to progress on RFC6824bis so that the specification can be finalised and Multipath TCP can be submitted for last call and later on for Proposed Standard. The two IETF meetings have reflected a strong progress towards that goal.

During the first meeting, Christoph explained the latest news about the two main Multipath TCP implementations on iOS and Linux. It is unusual to see an Apple employee detailing both an iOS and a Linux implementation.

Christoph started with a description of the lessons that they’ve learned after almost four years of experience with Multipath TCP for the Siri application. Siri is a voice recognition application in which the smartphone sends voice samples to cloud services that perform the voice recognition and return responses which could be text, URLs, images, … The key metric for Siri is the time to first word, i.e. the time between the transmission of a voice sample and the reception of the beginning of the response on the smartphone. Apple analysed this metric and measurements show that the 95% percentile is 20% faster with Multipath TCP. Furthermore, they observe a five times reduction of the network failures. These two elements contribute to improving the user experience on iOS devices and Apple has decided to open it to any app developper in iOS that will ship in autumn.

In iOS11, Multipath TCP is closely coupled with WiFi Assist, a software that monitors the quality of the WiFi and LTE interfaces and can decide to switch from one to another or use both for Multipath TCP enabled applications.

Since the first announcement in June, Apple has released several versions of iOS11 to developers, each version adding new capabilities for Multipath TCP.

iOS11 provides a very simply API to developers via the URLSession API. To use Multipath TCP, the developer simply needs to set a special property to select the mode of operation of Multipath TCP. Three modes are supported :

• Handover mode: connections start on the WiFi interface and WiFi Assist will enable them to move to the cellular interface if the WiFi network breaks
• Interactive mode: connections use both WiFi and cellular but the Multipath TCP subflow over the cellular interface is set as a backup subflow. The smartphone uses the WiFi interface unless its delay increases too much. If this happens, WiFi Assist triggers the utilisation of the cellular interface to preserve the user experience
• Aggregate mode: connections use both WiFi and cellular simultaneously. In this mode, applications can use both cellular and WiFi to boost throughput and reduce download time. Unfortunately, this mode is currently experimental and only useable on developer phones.

iOS11 betas are available for any registered Apple developer and can be used on iPhones starting from the SE model. It is likely that other beta versions will be released by Apple before the official release of iOS11.

The Linux implementation has improved as well. A new official release (0.92) has been published in June. This release includes several new features:

• applications can now select the underlying scheduler and path manager by using socket options. Until then, these algorithms were configured system wide by using sysctl values.
• in the previous versions of Multipath TCP, the Linux kernel implementation did not automatically reestablish subflows that were reset because of excessive retransmission timeouts. Version 0.92 automatically reestablishes these subflows which is important for long-lived flows in networks were losses are frequent.

Implementation in the Linux kernel work continued during the IETF99 Hackathon. Quentin De Coninck summarised the main results of this code sprint in his presentation. He first worked on adding support for the MPTCP_RST option in the Linux kernel. This option was added in RFC6824bis to provide addition information on the reason for the transmission of the TCP RST on a given subflow. This MPTCP_RST option was motivated by the need to enable a host to determine whether it should attempt to re-establish a subflow once the other end has reset it. There are situations, e.g. excessive timeouts, where a server could decide to reset a subflow while still expecting the client to reestablish the flow afterwards. Benjamin Hesmans and Quentin De Coninck worked on implementing the experimental Multipath TCP option. These options are intended to be used to perform experiments with new options that could later on be included in the Multipath TCP standard. Fabien Duchene worked on implementing support for the reliable transmission of ADD_ADDR option that is described in RFC6824bis. One of the use cases sending reliable address advertisements is the load balancer that he presented at IETF99. Benjamin Hesmans and Olivier Bonaventure worked on respectively adding new features to the Multipath TCP socket API and on documenting its usage. Finally, Christoph Paasch worked on implementing the new Multipath TCP handshake that uses a 32 bits MP_CAPABLE option in the SYN. All the code developed during the hackathon will be stabilised, documented and merged later in the public release.

During his presentation, Fabien Duchene proposed a better solution to load balance Multipath TCP connections. The benefits of the proposed solution can be better understood by considering the left part of the figure below. A simple load balancer would typically hash the (sourceIP, destIP, sourcePort, destPort) tuple of a connection created by a client host. This stateless load balancing works well with regular TCP and if the hash function ignores the sourcePort, all the connection from a given client will reach the same server. Unfortunately, it does not work with Multipath TCP. Consider the same client that creates an additional subflow from its second network interface. The (sourceIP, destIP, sourcePort, destPort) tuple of this second subflow will be hashed by the load balancer towards a different server than the initial subflow. Since this server does not have state for this connection, it will refused the subflow.

Fabien’s solution to this problem is to add one bit in the MP_CAPABLE option, assign a public IP address to each server residing behind the load balancer and using reliable ADD_ADDR. The NoJoin bit in the MP_CAPABLE option is set by the server to indicate to the client that it should not try to establish a subflow towards the destination address of the initial SYN (1.2.3.4 in the example below).  Once a Multipath TCP connection has been initiated, the server immediately announces its unique address (5.6.7.9 in the example below) to enable the client to establish additional subflows if needed. This is already part of RFC6824bis.

Fabien has implemented this approach in the Linux kernel and describes it in more details in a paper that will be presented at ICNP2017. It is interesting to compare the approach proposed by Fabien with Direct Server Return. DSR is frequently used on load balancers in front of web servers. With DSR, the HTTP requests pass through the load balancer, but the responses are forwarded directly by the servers to the client without passing through the load balancers. Given the imbalance between requests and responses, this helps to scale the load balancer. Multipath TCP goes one step further since the servers can set the initial subflow as a backup subflow to force all the client traffic, both requests and responses, to bypass the load balancer. This would be very useful for applications where requests can be larger than responses such as uploads to storage services.

Alan Ford presented the status of the update of RFC6824bis. His presentation was very short with only three modifications :

• A clarification of the processing of the FAST_CLOSE option. This option is used to abruptly terminate a Multipath TCP connection. To abruptly terminate a connection, a host sends the FAST_CLOSE option over one subflow and  RST on the other subflows. If the FAST_CLOSE option is lost, it needs to be retransmitted and the subflow is closed if no answer has been received after several retransmissions. Unfortunately, terminating a subflow without sending a RST caused problems with some middleboxes (or the local firewall). The new text covers this case by transmitting a RST packet when closing the subflow in this case.
• The second modification is to replace SHA-1 by SHA-256 in Multipath TCP for the generation of the tokens, the initial DSS values and the authentication of the MP_JOIN and ADD_ADDR option. This change was motivated by the deprecation of SHA-1 in several IETF protocols, even if no specific attack was found on Multipath TCP.
• The third modification is a clarification of the utilisation of the DSS mapping after the handshake.

During the second meeting of the working group, other modifications were brought up.

Zhiyun Qian, Ali Munir, Zubair Safiq, Alex Liu and Franck Le sent an email to the IETF mailing list describing a new attack on Multipath TCP that will be presented at ICNP2017. The attack affects the MP_PRIO option. This option was designed to enable a host to dynamically change the backup status of a specific subflow. It is typically used by sending the MP_PRIO option on the subflow whose priority needs to change. However, the MP_PRIO option defined in RFC6824 also includes an address identifier. This implies that the official semantics of this option is to change the backup status of all subflows that are attached to the specified address identifier. This creates a new possibility for on-path attackers. Consider an attacker that resides on the path of one subflow. A typical example is a Multipath TCP connection passing through a rogue  WiFi access point and a cellular network. Thanks to the address identifier of the MP_PRIO option, the attacker could send MP_PRIO options to set the cellular subflow as a backup subflow and thus force all data to pass through the rogue WiFi access point. This issue was discussed on the list and during the meeting and the working group has finally decided to remove the address identifier from the MP_PRIO option to prevent this attack.

Another point was brought by Christoph Paasch based on his experience with the iOS implementation. When TFO and Multipath TCP are used together, there are two problems that need to be taken into account as described in an expired draft. First, with RFC6824, the TFO cookie must have a reduced length to ensure that both the TFO cookie and the MP_CAPABLE option can be placed in the extended TCP header. Second, if the SYN contains data, this data cannot consume Data Sequence Numbers since the initial DSS value has not yet been negotiated. The size of the TFO cookie is not a problem in RFC6824bis since MP_CAPABLE only consumes 32 bits. However, the second problem remains and text inspired by  expired draft will be added to RFC6824bis.

During the last year, the working group has discussed a lot on using proxies to enable Multipath TCP hosts to benefits from Multipath TCP in the access network even if the final server does not support Multipath TCP. Several solutions have been discussed during the previous meetings. After the March meeting, the working group chairs solicited new proposals. During IETF99, Olivier Bonaventure proposed the utilisation of Transport Converters. These Transport Converters are illustrated in the figure below.

The figure above considers a smartphone equipped with a Multipath TCP stack that uses a Transport Converter (shown as a box with C in the figure) to connect to a server. The converter is reachable via one IP address and uses a well known port. To open a connection via the converter, the client sends a SYN packet which contains both the TFO and MP_CAPABLE options and a Connect message inside its payload. The Connect message indicates the address and ports of the destination server. Upon reception of this packet, the converter sends a SYN packet towards the server with the MP_CAPABLE option. The server replies to the converter with a SYN+ACK. The converter confirms the establishment of the connection and returns in the payload of the SYN+ACK returned to the smartphone a copy of the MP_CAPABLE option received from the server. This enables the client to detect that the server supports Multipath TCP. It will then be able to bypass the converter for future Multipath TCP connections towards this server. The protocol is described in more details in an IETF draft.

Vladimir Olteanu presented a new version of the SOCKS protocol : SOCKSv6. SOCKS is an application layer protocol that allows clients to open TCP connections through a proxy. It is often supported by firewalls and used in several deployments of Multipath TCP.  The standard version of SOCKS, specified in RFC1928 suffers from two important limitations: several round-trip-times may be required to establish a connection through a SOCKS proxy. This is illustrated in the left part of figure below extracted from Vladimir’s presentation. A second limitation is that when a client opens a connection through a SOCKS proxy, the connection succeeds even if the server is not reachable, which causes problems to support solutions such as HappyEyeBalls where clients try to open several connections simultaneously the test the correct operation of a server.

The proposed SOCKSv6 protocol leverages TCP Fast Open like to pack all the SOCKS commands inside the SYN packet to minimise the round-trip-time overhead of establishing a TCP connection through the SOCKS proxy.

Two extensions and one experiment were discussed during the second meeting. Quentin De Coninck has announced on the IETF mailing list an article that describes how Multipath TCP can be tuned on smartphones. Many smartphone applications do not need to aggregate bandwidth. Many of these applications exchange small amounts of data. For these applications, always enabling the WiFi and cellular  interfaces is a waste of energy. A better approach is to only use one interface at a time but quickly switch to the other when the primary interface fails. In his presentation, Quentin explained how to enhance Multipath TCP to support the transmission of data in the SYN or SYN+ACK packets to provide faster handovers.

In his presentation, Markus Amend discussed several options to enhance the Multipath TCP handshake to support faster connection setup times under bad network conditions. His use case is a client that is connected to two different lossy networks. To open a Multipath TCP connection, a host usually sends a SYN over the interface having the best route towards the destination. If this interface performers badly, the SYN could need to be retransmitted several times. He discussed several options to reduce this connection establishment delay. Some involve changes to the kernel and the Multipath TCP protocol, others can be implemented in the application layer. The feedback from the working group has been mainly to focus on the application layer solutions to keep the protocol unmodified.

The last point that was discussed during IETF99 were the interactions between NAT64 and Multipath TCP. In his presentation, Quentin De Coninck explained the results of his experiments in the ietf-nat64 network and the importance of using the well known prefix when deploying NAT64 in IPv6 only networks.

All the slides presented during IETF99 are available from https://datatracker.ietf.org/meeting/99/session/mptcp/

Scientific papers

The last two months have been prolific in the scientific literature. The following papers have been published or announced :

• Y. Lim et al. describe in ECF: An MPTCP Path Scheduler to Manage Heterogeneous Paths a new type of packet scheduler that uses more information that the round-trip-time to schedule packets.
• J. Mena et al describe in Multipath TCP on a VANET: A Performance Study measurements with Multipath TCP in vehicular networks.
• A. Rao et al. propose in  Reliable Robotic Communication using Multi-Path TCP to use Multipath TCP to improve the interactions with robots by using multiple network interfaces.
• K. Yedugundla et al. propose and evaluate in Probe or Wait: Handling tail losses using Multipath TCP improvements to the loss recovery schemes used by Multipath TCP and Tail Loss Probe in particular. These improvements are evaluated by simulations with deterministic losses.
• S. Shailendra et al. propose in Multipath TCP Path Scheduler for Drones: A Segregation of Control and User Data to use Multipath TCP control drones.
• J. Wu et al. analyse in Quality-Aware Energy Optimization in Wireless Video Communication With Multipath TCP how to tune Multipath TCP to minimise energy consumption and maximise the quality of video delivery.
• Q. De Coninck et al. propose in Every Millisecond Counts: Tuning Multipath TCP for Interactive Applications on Smartphones several improvements to tune Multipath TCP on smartphones.
• A. Mondal et al. propose in  Enhanced Socket API for MPTCP-Controlling Sub-flow Priority an extension to the Multipath TCP socket API that enables hosts to control subflow priorities with the MP_PRIO option.

Another interesting news for Multipath TCP researchers is that the article Multipath TCP Algorithms: Theory, Design and Implementation written by Q. Peng and his colleagues that describes the BALIA congestion control algorithm received the IEEE Communications Society William R. Bennett Prize.