x-fabric home

Data and TV quality measurement

Customers expect the bandwidth they buy to deliver pages that load quickly and downloads that finish on time. Viewers are just as quick to notice a frozen picture or a slow channel change.

On a busy evening at home, phone, TV and downloads share one line, and mosprobe measures them all at once.

Web and data performance

Why measure web and data performance?

Peak-hour congestion, peering bottlenecks, slow name servers, overloaded servers, and TCP behaviour on long or lossy paths all slow a connection down, and link utilisation counters do not show it.

Measuring from the customer's point of view shows whether the advertised speed is delivered and provides figures for comparing regions, access technologies and providers, as well as for SLAs and regulatory reporting.

What the user waits for

A single web or file transfer passes through several phases. Each one can be measured on its own, and each points to a different part of the chain.

Phases of a transfer
PhaseWhat it shows
DNS resolutionWhether name servers are reachable and fast
Connection setupRound-trip time to the server and server responsiveness
Time to first byteHow quickly the service starts delivering
Download and upload throughputThe capacity available to an application
Transfer successReliability: completed, failed, and aborted transfers, with the reason

From network tests to service tests

Across the industry, several complementary layers are used to test data performance.

  1. Network benchmarking

    IETF RFC 2544 defines throughput, latency, frame loss, and back-to-back tests, originally for benchmarking network devices in the lab. ITU-T Y.1564 is its multiservice successor for service activation: it verifies several traffic classes at once against their committed rates, delay, delay variation, and loss. x-fabric's Coast Guard benchmarks IP networks to both.

  2. TCP throughput

    Most traffic runs over TCP, and TCP throughput depends on round-trip time, window size, and loss, not just on link capacity. IETF RFC 6349 describes a repeatable method: determine path MTU, baseline round-trip time, and bottleneck bandwidth first, then measure TCP throughput and compare it with what the path should achieve. Metrics such as TCP efficiency and buffer delay show why a line underperforms. Coast Guard measures TCP throughput this way.

  3. Application transfers

    HTTP and FTP downloads and uploads against a test server measure what a customer application gets. Broadband Forum TR-143 standardises such throughput tests (HTTP and FTP transfers, UDP echo) for customer premises equipment. ETSI TS 102 250-2 defines QoS parameters for popular services, including FTP and HTTP, with exact trigger points for setup time, session time, mean data rate, and failure ratios, so that results from different tools and networks are comparable.

  4. User perception

    ITU-T G.1030 provides a framework for estimating how IP network performance translates into the perceived performance of data applications such as web browsing.

One connection or many?

A single TCP connection shows what one application gets, and it is sensitive to latency and loss. Several parallel connections fill high-capacity lines and show the total capacity of the access line.

One connection. What a single application gets.
Parallel connections. The total capacity of the access line.

Running uploads and downloads at the same time reveals effects that sequential tests miss, such as growing latency when buffers fill under load.

What a good measurement needs

The customer's vantage point
Measure from real access lines, not from the data centre.
The right targets
A reference server close to the network edge isolates the access network; real internet targets bring peering, content delivery networks and DNS into the measurement.
Both directions and both IP versions
Upload and download, IPv4 and IPv6 measured separately.
Long enough transfers
Short transfers mostly measure TCP slow start, not the capacity of the line.
A test head that is not the bottleneck
Gigabit lines need test equipment that reliably sustains gigabit rates.
Around the clock
Performance at 3 a.m. says little about performance at peak hours.
Context
DNS time, connection setup and time to first byte, together with round-trip time and loss to reference targets, explain why a transfer was slow.

What mosprobe measures

Transfers
HTTP and FTP downloads and uploads, as single transfers or as scenarios with many uploads and downloads running in parallel, over IPv4 and IPv6, at up to gigabit speed.
Every transfer
Time for name resolution, connection setup, the server's first response and the first byte of data, the total duration, and the data rate in each direction.
Every test
Transfers started, completed and failed, each failure with its reason.

IPTV

Why measure IPTV quality?

Compressed video is fragile because one lost packet can damage several frames that depend on each other. Managed IPTV is usually delivered as multicast, so a fault at one replication point affects every viewer behind it.

Network counters show whether links are up. An IPTV quality measurement shows whether a given channel reaches the living room intact, channel by channel, from the subscriber's point of view.

What shapes the viewing experience

Viewing experience
AspectTypical cause of problems
Picture and sound qualityBitrate, codec, and encoder settings at the head end
Transmission impairments (blocking, freezes)Packet loss, loss bursts, jitter
Channel change time (zapping)Multicast join delay, key frame interval, set-top box buffering
Channel availabilityMissing or interrupted multicast streams, head end outages

Channel change and IGMP

When a viewer switches channels, the set-top box leaves one multicast group and joins another, using IGMP (IGMPv3, IETF RFC 3376; MLD in IPv6 networks). The zapping time the viewer perceives is the sum of several steps.

  1. Join request

    The set-top box joins the new multicast group.

  2. Stream starts

    The network starts to deliver the new stream.

  3. Next key frame

    The picture can only start at the next key frame, a complete image in the stream.

  4. Decoder buffering

    The set-top box fills its buffer before it shows the picture.

A test head that joins channels the way a set-top box does measures the network's share of this delay and detects channels that do not arrive at all. ITU-T G.1080 lists channel zapping time among the quality of experience requirements for IPTV.

Transport stream and network metrics

Across the industry, transport streams and IP delivery are checked against guidelines and metrics such as these.

ETSI TR 101 290
Measurement guidelines for MPEG-2 transport streams, grouped by priority: priority 1 checks whether a stream can be decoded at all (synchronisation, continuity, essential tables), priority 2 adds recommended continuous checks such as timing and CRC errors, priority 3 covers application-dependent checks. Written for DVB broadcasting, it is widely applied to transport streams carried over IP.
Media Delivery Index (MDI), IETF RFC 4445
Condenses IP delivery into two figures: the Delay Factor, which reflects jitter and the buffering a receiver needs, and the Media Loss Rate, which counts lost or out-of-order media packets.
IP metrics
Packet loss, loss bursts, and jitter, measured per channel, help locate a problem in the head end, the core, or the access network.

From network metrics to perceived quality

Requirements and quality models used across the industry relate network behaviour to what the viewer experiences.

ITU-T G.1080
Defines user requirements for IPTV quality of experience, independent of network architecture and transport protocol.
Broadband Forum TR-126
Describes quality of experience requirements for triple-play services and maps viewer expectations for video, voice, and data to engineering targets.
ITU-T P.1201
A parametric, non-intrusive model (P.1201.2 for higher-resolution services such as IPTV) that estimates audio, video, and audiovisual quality as a MOS from packet header information.
ITU-T P.1203
Addresses adaptive streaming over HTTP, as used by TV apps and OTT services, including the effect of quality switches and stalling.

What a good measurement needs

The subscriber's vantage point
Measure behind the same access as the customer, joining channels like a set-top box.
The whole channel line-up
Rotate through all channels, since a channel missing in one region is invisible from the head end.
Continuity
Around the clock, including peak viewing hours and major live events.
Precise timing
Join and zapping delays are short, so timestamps must be accurate.
Context
Transport metrics locate a fault, quality models show what it means for the viewer, and a comparison with the head end separates source problems from transport problems.

What mosprobe measures

mosprobe® test heads join a channel like a set-top box, over IPv4 and IPv6, receive its stream for a set time and leave again. The channels come from one centrally maintained line-up.

Reception
Whether the channel arrives at all.
Channel change
The time from joining the channel to its first packet: the network's share of the zapping time.
Delivery
Packets received and data rate; packet loss, reordering and duplicates; jitter as minimum, mean and maximum.