Latency and throughput – the two measures of system performance

Performance of the system is one of the most stringent criteria for its success. While performance increases the desirability among customers, cost is what makes it affordable. This is the reason why system designers aim for maximum performance with available resources such as power and area constraints. There are two related parameters that determine the performance output of a system –

Throughput - Throughput is a measure of the productivity of the system. In electronic/communication systems, throughput refers to rate at which output data is produced. Higher the throughput, more productive is the system. In most of the cases, it is measured as time difference between two consecutive outputs (nth and n+1th). Throughput also refers to the rate at which input data can be applied to system.

Let us discuss with the help of an example:

Above figure depicts the throughput of 3 number adder. Result of input set applied at 1st clock cycle appears at output at 3rd clock cycle and in 4th clock cycle next input set is applied and output comes in 6th clock cycle.  Hence, throughput of above design is ⅓ per clock cycle. As we can see from diagram, first input is applied in first clock cycle and 2nd input is applied in 4th clock cycle. Hence we can also say that throughput is rate at which input data can be applied to system.

Latency- Latency is the time taken by a system to produce output after input is applied. It is a measure of delay response of a design. Higher the latency value, slower is the system. in synchronous designs, it is measured in terms of number of clock cycles. In combinational designs, latency is basically propagation delay of circuit. In non pipelined designs, latency improvement is major area of concern. In more general terms, it is time difference between output and input time.

Relationship between throughput and latency: Both latency and throughput are inter-related. It is desired to have maximum throughput and minimum latency. Increasing latency and/or throughput might make the system costly. Let us take an example. Consider a park with 3 rides and it takes 5 minutes for a ride.  A child can take sequentially these rides; i.e, ride 1, ride 2 and then ride 3. Firstly, let us assume that only one child at a time is allowed to enter park at a time. While he is taking a ride, no one is allowed to enter the park. Thus, the throughput of the park is 15 minutes per child and latency is 15 minutes. Now, let us assume that while a child has finished taking ride1, another child is allowed to enter park. Thus, in this case, throughput will be 5 minutes per child whereas latency is still 15 minutes. Thus, we have increased the throughput of the system without affecting latency and at the same cost.

Clock latency

Definition of clock latency (clock insertion delay): In sequential designs, each timing path is triggered by a clock signal that originates from a source. The flops being triggered by the clock signal are known as sinks for the clock. In general, clock latency (or clock insertion delay) is defined as the amount of time taken by the clock signal in traveling from its source to the sinks. Clock latency comprises of two components - clock source latency and clock network latency.

• Source latency of clock (Source insertion delay): Source latency is defined as the time taken by the clock signal in traversing from clock source (may be PLL, oscillator or some other source) to the clock definition point. It is also known as source insertion delay. It can be used to model off-chip clock latency when clock source is not part of the chip itself. 

• Network latency of clock (Network insertion delay): Network latency is defined as the time taken by the clock signal in traversing from clock definition point to the sinks of the clock. Thus, each sink of the clock has a different network latency. If we talk about the clock, it will have:
• Maximum network latency: Maximum of all the network latencies
• Minimum network latency: Minimum of all the network latencies
• Average network latency: Average of all the network latencies

Total clock latency is given as the sum of source latency and network latency. In other words, total clock latency at a point is given as follows:

       Clock latency = Source latency + Network latency

It is generally stated that for a robust clock tree, ‘sum of source latency and network latency for all sinks of a clock should be equal’. If that is the case, the clock tree is said to be balanced as this means that all the registers are getting clock at the same time; i.e., clock skew is zero.

Figure 1 : Figure showing source latency and network latency components of clock latency

Figure 1 above shows the two components of clock latency, i.e. source latency and network latency. Each flop (sink, in general) has its own latency since the path traced by clock from source to it may be different. The above case may be found in block level constraints in case of hierarchical designs wherein clock source is sitting outside the block and clock signal enters the block through a block port. It may also represent a case of a chip in which the clock source is sitting outside; e.g. some external device is controlling the chip. In that case, clock source will be sitting inside that device.

How to specify clock latency: In EDA tools, we can model clock latency using SDC command ‘set_clock_latency’ to imitate the behavior there will be after clock tree will be built. Using this command, we can specify both the source latency for a clock as well as the network latency. After clock tree has been built, the latency for the sinks is calculated by the tool itself from the delays of various elements. However, in case the clock source is sitting outside, it still needs to be modeled by source latency even after the clock tree synthesis. To specify clock latency for clock signal named ‘CLK’, we may use SDC command set_clock_latency:

                set_clock_latency <value> CLK

                set_clock_latency <value> CLK –source

First command will specify the network latency whereas the second command will specify the source latency for CLK.

Also read:

• Clock - the incharge of synchronous designs
• Timing corners - dimensions in timing signoff
• Ensuring glitch-free propagation of clock
• Clock switching and clock gating checks
• Can hold be frequency dependent

Hope you liked this post. Let us know of your views through comments.