TelePresence Packet Rates

A single TelePresence camera configured for 1080p best resolution can send up to approximately 4 Mbps of video traffic at peak rate. With an average of 1100 bytes per packet, this yields approximately 455 packets per second. However, with normal motion, each camera typically generates somewhere between 3 Mbps to 3.5 Mbps of video; yielding a packet rate between 340 packets to 400 packets per second. Likewise a single TelePresence microphone generates a voice packet of roughly 220 bytes every 20 msec. Thisyields a voice packet rate of approximately 50 packets per second. The audio rate is fixed regardless of the amount of speaking going on within the meeting. Therefore under normal motion, a TelePresence endpoint with one cameras and one microphone (such as the CTS-1000 or CTS-500) typically generates approximately 390 packets to 450 packets per second. Likewise, under normal motion, a TelePresence endpoint with three cameras and three microphones (such as the CTS-3000) typically generates approximately 1170 packets to 1350 packets per second. Note that this does not include the use of the auxiliary video and audio inputs, voice and video interoperability with legacy systems, and any signaling and management traffic; all of which increase the packet rate.

Multiplexing

The underlying transport protocol for RTP is not specified within RFC 3550. However, RTP and RTCP are typically implemented over UDP. When implemented over UDP, the port range of 16384 to 32767 is often used. RTP streams often use the even-numbered ports, with their accompanying RTCP streams using the next higher odd-numbered port. In this case, RTP sessions are identified by a unique destination address pair that consists of a network address plus a pair of ports for RTP and RTCP. Cisco TelePresence endpoints are capable of both sending and receiving multiple audio and video streams. The primary codec multiplexes these streams together into a single audio RTP stream and a single video RTP stream. Multiplexing is accomplished through the use of different Synchronization Source Identifiers (SSRC), for each video camera and each audio microphone, including the auxiliary inputs. In the case of Cisco TelePresence, SSRCs indicate not only the source position of the audio or video media, but also the destination position of the display or speaker to which the media is intended. Therefore, the receiving primary codec can demultiplex the RTP session and send the individual RTP streams to the appropriate display or speaker. This is how spatial audio is achieved with TelePresence and how video display positions are maintained throughout a TelePresence meeting.

When endpoints join a multipoint call, they first attempt to exchange RTCP packets. Successful exchange of these packets indicates the opposite endpoint is a Cisco TelePresence device, capable of supporting various Cisco extensions. Among other things, these extensions determine the number of audio and video channels each TelePresence endpoint can send and receive and their positions. Audio and video streams are sent and received based on their position within the CTS endpoint, which is then mapped to a corresponding SSRC. Figure 1 shows an example for a three-screen endpoint, such as a CTS-3000, communicating with a CTMS.

Figure 1: Audio and video positions for three-screen TelePresence endpoints

 

Each CTS-3000 or CTS-3200 can transmit (and receive) up to four audio streams and four video streams from the left, center, right, and auxiliary positions. These correspond to the left, center, and right cameras, microphones, and the auxiliary input. Therefore, in a point-to-point meeting between CTS-3000s or CTS-3200s, there can be as many as four audio SSRCs multiplexed together in a single audio RTP stream and four video SSRCs multiplexed together in a single video RTP stream sent from each endpoint.

As Figure 1 illustrates, the CTMS can transmit up to four video streams, corresponding to the left, center, and right displays of the CTS-3000 or CTS-3200, and either a projector or monitor connected to the auxiliary video output. However, the CTMS transmits only up to three audio streams, corresponding to the left, center, and right speaker positions of the CTS-3000 or CTS-3200. Audio sent by an originating CTS-3000 or CTS-3200 toward the auxiliary position is redirected to one of the three speaker positions of the destination CTS-3000 or CTS-3200 by the CTMS. The CTMS chooses the three loudest audio streams to send to the remote CTS-3000 or CTS-3200 when there are more than three streams with audio energy.

Figure 2 shows the audio and video positions for a multipoint call consisting of CTS-1000s or CTS-500s.

Figure 2: Audio and video positions for single-screen TelePresence endpoints

 

Each CTS-1000 or CTS-500 can transmit up to two audio streams and two video streams from the center and auxiliary positions. These correspond to the single camera and microphone of the CTS-1000 or CTS-500, and the auxiliary input. Therefore, in a point-to-point meeting between CTS-1000s or CTS-500s there can be as many as two audio SSRCs multiplexed together in a single audio RTP stream and two video SSRCs multiplexed together in a single video RTP stream sent from each endpoint.

As Figure 2 illustrates, the CTMS can still transmit up to three audio streams, corresponding to the left, center, and right microphone positions of the CTS-3000 or CTS-3200, even though the CTS-1000 or CTS-500 have only a single speaker. The CTS-1000 or CTS-500 mixes the audio from each of the three positions to play out on its single speaker. The CTS-1000 or CTS-500 can receive only up to two video streams, corresponding to the center display and picture-in-picture auxiliary video output.

RTP Control Protocol

The RTP Control Protocol (RTCP) provides four main functions for RTP sessions:

• Provides feedback on the quality of the distribution of RTP packets from the source.
• Carries a persistent transport-level identifier called the canonical name (CNAME), which associates multiple data streams from a given participant in a set of RTP sessions.
• Provides a feedback mechanism to scale the actual use of RTCP itself. Because all participants send RTCP packets, the rate of arrival of RTCP packets can determine the number of participants and the rate at which RTCP packets should be sent so that the network is not overwhelmed by RTCP packets.
• Can be optionally used to convey minimal session control information between participants.

Several different RTCP packets packet types convey the preceding information. These include a Sender Report (SR), Receiver Report (RR), Session Descriptor (SDES), and Application Specific (APP) packets; among others. Typically RTCP sends compound packets containing combinations of these reports in a single packet.

As mentioned previously, each RTP stream has an accompanying RTCP stream. These RTCP streams can be sent using the next higher odd-numbered port or optionally multiplexed in with the RTP streams themselves. Initial TelePresence software versions multiplexed in the RTCP streams within the RTP streams. For interoperability, current TelePresence software versions support both options. RTCP signals packet loss within TelePresence deployments, causing the sender to send a new reference frame (IDR) to resynchronize the video transmission. Additionally, RTCP is used between TelePresence endpoints to inform each other of the number of audio and video channels they are capable of supporting. The number of channels corresponds to the number of displays, cameras, microphones, and speakers supported by the particular endpoint. Finally, the Cisco TelePresence Multipoint Switch (CTMS) uses RTCP packets to perform session control within multipoint calls. The CTMS informs TelePresence endpoints that do not have active speakers to stop transmitting video. When a particular table segment or room has an active speaker, the CTMS detects this through the audio energy and informs the table segment or room, through RTCP packets, to send video.

Real-Time Transport Protocol | Encoding and Packetization

The preceding content covered the basic concepts of video and audio inputs and encoding processes. The video is encoded using H.264, and the audio is encoded using AAC-LD. Now those video and audio samples must be turned into IP packets and sent onto the network to be transported to the other end. This is done using the Real-Time Transport Protocol (RTP).

RTP is a network protocol specifically designed to provide transport services for real-time applications such as interactive voice and video. The services it provides include identification of payload type, sequence numbering, timestamps, and monitoring of the delivery of RTP packets through the RTP control protocol (RTCP). RTP and RTCP are both specified in IETF RFC 3550.

RTP Packet Format

RTP defines a standard packet format for delivering the media, as shown in Figure 1.

 Figure 1: RTP packet format

The following are the fields within the RTP packet:

• Version (V): A 2-bit field indicating the protocol version. The current version is 2.
• Padding (P): A 1-bit field indicating padding at the end of the RTP packet.
• Extension Header (X): A 1-bit field indicating the presence of an optional extension header.
• CRSC Count (CC): A 4-bit field indicating the number of Contributing Source (CSRC) identifiers that follow the fixed header. It is presented only when inserted by an RTP mixer such as a conference bridge or transcoder.
• Marker (M): A 1-bit marker bit that identifies events such as frame boundaries.
• Payload Type (PT): A 7-bit field that identifies the format of the RTP payload.
• Sequence Number: A 16-bit field that increments by one for each RTP packet sent. The receiver uses this field to identify lost packets.
• Timestamp: A 32-bit timestamp field that reflects the sampling instant of the first octet of the RTP packet.
• Synchronization Source Identifier (SSRC): A 32-bit field that uniquely identifies the source of a stream of RTP packets.
• Contributing Source Identifiers (CSRC): Variable length field that contains a list of sources of streams of RTP packets that have contributed to a combined stream produced by an RTP mixer. You can use this to identify the individual speakers when a mixer combines streams in an audio or video conference.
• RTP Extension (Optional): Variable length field that contains a 16-bit profile specific identifier and a 16-bit length identifier, followed by variable length extension data. Intended for limited use.
• RTP Payload: Variable-length field that holds the real-time application data (voice, video, and so on.).

Frames Versus Packets

To understand the role of RTP within TelePresence, you need to understand the behavior of voice and video over a network infrastructure. Figure 2 shows a sample comparison of voice and video traffic as it appears on a network.

Figure 2: Comparison of voice and video on the network

 

As you can see in Figure 2, voice appears as a series of audio samples, spaced at regular intervals. In the case of Cisco TelePresence, voice packets are typically sent every 20 msec. Each packet contains one or more encoded samples of the audio, depending on the encoding algorithm used and how many samples per packet it is configured to use. G.711 and G.729, two of the most common VoIP encoding algorithms, typically include two voice samples per packet. The sizes of the voice packets are consistent, averaging slightly over 220 bytes for a 2-sample G.711 packet; therefore, the overall characteristic of voice is a constant bit-rate stream.

Video traffic appears as a series of video frames spaced at regular intervals. In the case of Cisco TelePresence, video frames are sent approximately every 33 msec. The size of each frame varies based on the amount of changes since the previous frame. Therefore, the overall characteristic of TelePresence video is a relatively bursty, variable bit-rate stream.

A video frame can also be referred to as an Access Unit in H.264 terminology. The H.264 standard defines two layers, a Video Coding Layer (VCL) and a Network Abstraction Layer (NAL). Figure 3 shows a simplified example.

 

Figure 3: Mapping TelePresence video into RTP packets

The VCL is responsible for encoding the video. Its output is a string of bits representing the encoded video. The function of the NAL is to map the string of bits into units that can then be transported across a network infrastructure. IETF RFC 3984 defines the format for H.264 video carried within the payload of the RTP packets. Each video frame consists of multiple RTP packets spaced out over the frame interval. The boundary of each video frame is indicated through the use of the marker bit, as shown in Figure 1. Each RTP packet contains one or more NAL Units (NALU), depending upon the packet type: single NAL unit packet, single-time or multi-time aggregation packet, or fragmentation unit (part of a NALU). Each NALU consists of an integer number of bytes of coded video.

Note

RTP Packets within a single video frame and across multiple frames are not necessarily independent of each other. In other words, if one packet within a video frame is discarded, it affects the quality of the entire video frame and might possibly affect the quality of other video frames.

TelePresence Video Packet and Frame Sizes

The sizes of individual RTP packets within frames vary, depending upon the number of NALUs they carry and the sizes of the NALUs. Overall, packet sizes average 1100 bytes for Cisco TelePresence video. The number of packets per frame also varies considerably based upon how much information is contained within the video frame. This is partially determined by how the video is encoded; either reference frames or inter-reference frames.

Note

Coding is actually done at the macroblock layer. An integer number of macroblocks then form a slice, and multiple slices form a frame. Therefore, technically slices are intrapredicted (I-slices) or interpredicted (P-slices).

Compression of reference frames is typically only moderate because only spatial redundancy within the frame is eliminated. Therefore, reference frames tend to be much larger in size than inter-reference frames. Reference frame sizes up to 64 KB to 65 KB (and approximately 60 individual packets) have been observed with TelePresence endpoints. Inter-reference frames have much higher compression because only the difference between the frame and the reference frame is sent. This information is typically sent in the form of motion vectors indicating the relative motion of objects from the reference frame. The size of TelePresence inter-reference frames is dependent upon the amount of motion within the conference call. Under normal motion, TelePresence inter-reference frames tend to average 13 KB in size and typically consist of approximately 12 individual packets. Under high motion they can be approximately 19 KB in size and consist of approximately 17 individual packets. From a bandwidth utilization standpoint, much better performance can be achieved by sending reference frames infrequently. During normal operation, Cisco TelePresence codecs send reference frames (IDRs) only once every several minutes in point-to-point calls to reduce the burstiness of the video and lower the overall bit rate.

Audio Encoding | Cisco TelePresence

The microphones used in Cisco TelePresence are purposefully designed to capture the sounds emanating from a human subject sitting within a few feet of the microphone, along with the regular background noises that accompany that person within the room he or she sits in, while filtering out certain unwanted frequency ranges (such as the high-frequency whirrs of spinning fans in laptop computers or the low-frequency hums of heating and ventilation systems) and electrostatic interference (such as GSM/GPRS cellular signals).

The center (primary) Cisco TelePresence codec has four microphone input ports: three for the Cisco TelePresence microphones and one auxiliary audio input. The Cisco TelePresence microphones use a proprietary 6-pin Mini-XLR connector. The auxiliary audio input is a standard 3.5 mm (1/8-inch) mini-stereo connector, which enables the users to connect the audio sound card of their PC along with the VGA video input discussed in the previous sections.

On single-screen systems such as the CTS-1000 and CTS-500, only the center microphone input and the auxiliary audio input are used. On multiscreen systems, such as the CTS-3000 and CTS-3200, the left and right inputs are also used.

Each audio input is encoded autonomously, resulting in up to four discrete channels of audio. This is superior to most other systems on the market that mix all the microphone inputs into a single outgoing channel. By maintaining the channels separately, Cisco TelePresence can maintain the directionality and spatiality of the sound. If the sound emanates from the left, it will be captured by the left microphone and reproduced by the left speaker on the other end. If it emanates from the right, it will be captured by the right microphone and reproduced by the right speaker on the other end.

AAC-LD Compression Algorithm

Cisco TelePresence uses the latest audio encoding technology known as Advanced Audio Coding–Low Delay (AAC-LD). AAC is a wideband audio coding algorithm designed to be the successor of the MP3 format and is standardized by the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group (MPEG). It is specified both as Part 7 of the MPEG-2 standard and Part 3 of the MPEG-4 standard. As such, it can be referred to as MPEG-2 Part 7 and MPEG-4 Part 3, depending on its implementation; however, it is most often referred to as MPEG-4 AAC, or AAC for short.

AAC-LD (Low Delay) bridges the gap between the AAC codec, which is designed for high-fidelity applications such as music, and International Telecommunication Union (ITU) speech encoders such as G.711 and G.722, which are designed for speech. AAC-LD combines the advantages of high-fidelity encoding with the low delay necessary for real-time, bidirectional communications.

Sampling Frequency and Compression Ratio

The AAC-LD standard allows for a wide range of sample frequencies (8 kHz to 96 kHz). Cisco TelePresence implements AAC-LD at 48 kHz sampling frequency. This means that the audio is sampled 48,000 times per second, per channel. These samples are then encoded and compressed to 64 kbps, per channel, resulting in a total bandwidth of 128 kbps for single-screen systems (two channels) and 256 kbps for multiscreen systems (four channels).

Automatic Gain Control and Microphone Calibration

Automatic Gain Control (AGC) is an adaptive algorithm to dynamically adjust the input gain of the microphones to adapt to varying input signal levels. Whether the people are sitting close to the microphones or far away, speaking in a soft voices or yelling, or any combination in between, the microphones have to continuously adapt to keep the audio sounding lifelike and at the correct decibel levels to reproduce the sense of distance and directionality at the far end.

Keeping multiple discrete microphones autonomous and yet collectively synchronized so that the entire room is calibrated is no small task. Cisco TelePresence uses advanced, proprietary techniques to dynamically calibrate the microphones to the room and relative to each other. It is more complex for Cisco TelePresence than other implementations because the microphones need to be kept discrete and autonomous from each other. This preserves the notion of location, which is critical to the proper operation of multipoint switching in which the active speaker switches in on the appropriate screen. For example, if a person is sitting in the center segment of the room but facing the left wall when she talks, the speech emanating from her hits both the left and center microphones. The system must be smart enough to detect which microphone is closest to the source and switch to the correct camera (in this case the center camera), while playing the sound out both speakers on the other end to retain the sense of distance and directionality of the audio. It does this by assigning a 0 to 100 scale for each channel. In this scenario, the speech emanating from the person might be ranked an 80 at the center microphone and a 45 at the left microphone. These two microphone inputs are independently encoded and transported to the other end where they are played out both the center and right speakers at the appropriate decibel levels so that the people on the other end get the sense of distance and directionality. However, because the center microphone was a higher rank than the left microphone, the correct camera would be triggered (in this case, the center camera).