IPC Streaming Media Transport Protocol (Part 2) - SRT

1. The past and present of SRT

SRT is the acronym for Security (Security) and Reliable (Reliable) Transport (Transport). In simple terms, SRT is a secure and reliable transmission protocol.

SRT is derived from the UDT protocol. UDT was created in 2001 with the goal of transmitting files in the shortest possible time over the public network. UDT developers submitted four drafts to the IETF, and the final version was released in 2013. Also in 2013, Haivision (an encoder vendor) expanded UDT's real-time audio and video transmission capabilities, and four years later (2017) it was open sourced and named SRT.

2. Choose TCP or UDP?

SRT is a kind of streaming media transmission protocol. Before explaining it in detail, we need to first clarify the difference between the streaming media transmission protocol and the transport layer protocols TCP and UDP that we usually call?

The streaming media transmission protocol we usually refer to belongs to the application layer protocol, while TCP and UDP belong to the transport layer protocol, which means that the streaming media transmission protocol actually depends on the transport layer protocol to work, as shown in Figure 1. Therefore, the underlying layers of all streaming media protocols use TCP or UDP to achieve network transmission.

Figure 1

Should TCP or UDP be chosen as the underlying streaming protocol?

Mainstream streaming protocols use both TCP and UDP protocols. For example, the RTMP protocol uses TCP as the underlying layer, while QUIC and SRT protocols use UDP. However, judging from the trend, most new streaming protocols use UDP as the underlying transmission protocol. The main reason for this is related to the characteristics of the streaming service itself and TCP.

From the perspective of live streaming, the most common streaming service, users require fast live streaming, low latency, and no lag. In the case of a weak network, they can accept the loss of some of the images, but hope to recover quickly.

However, this requirement is difficult to achieve for the TCP protocol.

First of all, the TCP protocol needs three handshakes to establish a link. If TLS encryption is required, a fourth handshake is required. If there is a streaming media protocol on the upper layer, more interactive processes are required. Therefore, in terms of establishing a link, the TCP protocol has too many interactive processes and is not suitable for fast streaming.

Secondly, because the TCP protocol is a highly reliable and ordered protocol, if a data packet with a lower sequence number is lost, even if the data packet with a higher sequence number has been received, it must wait until the retransmission of the data packet with a lower sequence number is received before it can be used, causing the current window to be blocked in place (TCP head of line blocking). If the retransmitted data packet is also lost, the waiting time to trigger another retransmission will double (the retransmission strategy is mild), and this mechanism will affect the data transmission efficiency, which is unacceptable for streaming media services with high real-time requirements.

Finally, when encountering a weak network, the slow start and congestion avoidance algorithms of the TCP protocol will significantly reduce their own bandwidth occupancy, thereby protecting the stability of the entire communication link (TCP is a selfless protocol). Therefore, under this strategy, once a weak network is encountered, the live broadcast screen will be stuck, and there is basically no ability to resist a weak network.

In summary, based on the above points, the TCP protocol is not suitable as the underlying protocol of the streaming media transmission protocol . The protocol was not designed with real-time requirements in mind. Let's take a look at how the SRT protocol solves the above problems.

3. Detailed explanation of SRT protocol

The SRT protocol solves the problems of the TCP protocol in streaming media services through simple handshake, fixed delay, ARQ (automatic retransmission request), FEC (forward error correction), ACK, NACK and other strategies.

3.1. How does SRT solve the problem of long handshake time?

The SRT handshake is shown in Figure 2. Through two handshakes and parameter exchanges, an SRT link can be quickly established (total time: 2RTT). Compared with the handshake time of RTMP 3RTT, the TCP-based streaming protocol, the total time is reduced by 1 RTT.

Figure 2

3.2. How does SRT achieve reliable transmission?

From the flowchart in FIG2 , it can be seen that after the SRT handshake is completed, media data and control instructions can be sent. The media data structure is shown in FIG3 .

Figure 3

• Data packet sequence number: The initial value of the data packet sequence number is determined during the handshake, and the data packet sequence number will increase by 1 each time a data packet is sent;
• Message sequence number: The message sequence number starts from 0, and then increases by 1 for each data packet sent. The functions of the first 4 flag bits of the message sequence number are shown in Figure 3;
• Timestamp: based on the creation time, in microseconds;
• Destination port socket ID: can be used to distinguish different SRT streams during multiplexing.

The ACK control instruction is shown in Figure 4.

Figure 4

SRT can identify which data packets have been sent through the data packet sequence number and the message sequence number, and can know which data have been successfully received through the ACK control command sent by the receiving end. If packet loss occurs, the receiving end notifies the sending end to retransmit the data through the NAK control command (as shown in Figure 5). In this way, SRT achieves reliable transmission.

Figure 5

3.3. How does SRT solve the head-of-line blocking problem?

The above describes how SRT achieves reliable transmission. From the implementation point of view, the sender must have a send buffer and the receiver also needs a receive buffer to implement the mechanism of data retransmission after packet loss (this is similar to the implementation principle of TCP). The SRT protocol uniformly specifies the size of the send buffer and the receive buffer by setting a fixed delay.

The sending buffer is used to store data packets that may need to be resent. Once a data packet receives an ACK, it will be cleared from the sending buffer (this is similar to TCP). Once a data packet in the sending buffer has not received an ACK, the data packet will also be cleared from the sending buffer after exceeding the fixed delay time. SRT solves the TCP head-of-line blocking problem in this way. At this point, we need to clarify a fact, that is, although SRT is a reliable transmission protocol, this reliability is conditional. Only under a fixed delay amount can data transmission be reliable. Once the fixed delay time is exceeded, the data will still be discarded. In fact, this is also in line with the characteristics of media services (under abnormal circumstances, some data loss is allowed, but it must be able to recover quickly).

3.4. How to implement SRT adaptive problem

From the ACK content in Figure 4, we can see that the ACK contains many network parameters, such as RTT time. Through parameters such as RTT time and link bandwidth, SRT can estimate the entire network status and achieve adaptive adjustment. It should be noted that the sending strategy of SRT is relatively aggressive, unlike the TCP protocol, which is a selfless protocol. SRT will still maintain a large sending rate when the network is poor. It is precisely because of this feature that when the network condition is poor, SRT can occupy a larger network bandwidth to send media data and ensure smoothness.

However, if the network is full of protocols with aggressive sending strategies such as SRT, no one will be able to send data. Therefore, when selecting services, you can consider the characteristics of SRT and TCP protocols and give priority to ensuring that important services occupy network bandwidth.

3.5. Pros and Cons of SRT

In addition to the above introduction to SRT, SRT can also realize the multiplexing of media data. The principle is to use the destination port socket field in the SRT data packet (as shown in Figure 3). Multiple SRT port sockets are bound to the same UDP port to realize SRT multiplexing.

The SRT protocol also performs better than other streaming protocols in lossy network conditions. In addition to the mechanism described above, the SRT protocol uses precise timestamps (as shown in Figure 3) to ensure that the data received by the receiver can accurately restore the bit rate characteristics and fixed frame interval of the sender.

Although the SRT protocol has many advantages, it also has some disadvantages that cannot be ignored. First, the SRT protocol occupies a large amount of bandwidth; second, the SRT protocol has an aggressive transmission strategy, which will affect other users on the same network; finally, since the underlying layer of SRT uses UDP, and firewalls are not friendly to UDP, it will cause handshake failure.

4. How to evaluate a new streaming protocol?

Through the analysis of the SRT protocol, we know that an excellent streaming protocol should first meet the following requirements👇

1. Ability to quickly establish a connection with the other party

2. Know whether the data has been delivered

3. Can detect whether the data is lost and retransmit it in time after confirming the loss

4. Ability to send data as quickly as possible

5. Able to adapt to changes in the receiving ability of the other party. When the receiving ability is strong, the sending speed is accelerated, and when the receiving ability is weak, the sending speed is reduced.

6. It can automatically adapt to network changes, reduce transmission when the network is poor, increase transmission when the network is good, and occupy a reasonable bandwidth

To summarize the above points, the evaluation directions of a good streaming protocol are:

• Fast connection speeds
• Good confirmation mechanism
• Fast delivery mechanism
• Reasonable retransmission strategy
• Excellent adaptive ability

With the above evaluation criteria, we can quickly evaluate whether a streaming media protocol we have never come into contact with is excellent, and compare the advantages and disadvantages of common streaming media protocols on the market in this way. The five-dimensional evaluation diagram is shown in Figure 6.

Figure 6

5. Summary

Currently, there is no streaming protocol that leads other protocols in all aspects of design. The selection of streaming protocols needs to be combined with specific services. Only after the specific services are determined can the pros and cons of protocol design be distinguished. At the same time, we must also be clear that the leading protocol design is not the only criterion for selecting this protocol. The current status of the industry and the status of service providers must also be considered. For example, although the RTMP protocol has been discontinued for nearly 10 years, it is still used by a large number of service providers, so its compatibility is particularly good and it will not disappear in a short time.