Knowledge Base & Community Wiki
Universal Scalability Law – Tutorial I
Introduction: The original derivation of the Universal Scalability Law, or USL, was presented at the 1993 CMG conference by Dr. Neil Gunter. The Universal Scalability Law or USL is used to quantify the scalability of hardware or software systems. USL was designed to be able to take sparse (few or limited) measurements from an existing system (in production or test) and predict the behavior of the system (throughput, etc.) at different loads.
A brief account of its application to parallel processing performance (under the name super-serial model) was presented in Chaps. 6 and 14 of Dr. Neil Gunther’s first book titled, “Practical Performance Analyst”. A more complete derivation with example applications is presented in Chaps. 4-6 of Dr. Neil Gunther’s book titled, “Guerrilla Capacity Planning.
According to Dr. Neil Gunther, some reasons why one should understand the USL include:
- General misuse of the term “scalability”. According to Dr. Gunther we all use the term “scalability” without clearly defining it, let alone defining it quantitatively. Computer system scalability according to Dr. Gunther must be quantified. If one can’t quantify it, you obviously isn’t able to deliver against it. Universal Scalability Law provides Performance Engineers the ability to measure and deliver against that definition.
- The other challenge is around the usage of Service Time to mode behavior or a given system. According to dr. Gunther one of the biggest challenges to application of QT (Queuing Theory) models (whether analytic or simulation) is the inscrutability of service times within an application. Every Queuing facility in a performance model requires a service time as an input parameter. Without the appropriate queues in the model, system performance metrics like throughput and response time, cannot be predicted. With USL models this challenge no longer exists since we completely work around the need for having Service Times.
USL decomposed: Dr. Gunther showed us how the scalability of every computer system can be described using a common rational function. The USL function is universal because it doesn’t assume any specific type of software, hardware or system architecture and can be applied to systems of any type (software and hardware).
Equation 1 has the Universal Scalability Law where C (N ) = X (N )=X (1) is the relative capacity given by the ratio of the measured throughput X (N ) for load N to the throughput X (1) for load 1.
C (N ) = N / [ 1+δ(N – 1)+K N (N -1) ] …………… (1)
The denominator of the above equation consists of three terms that all have a specific physical interpretation:
- Concurrency: The first term models linear scalability that would exist if the different parts of the system (processors, threads . . . ) could work without any interference due to their interaction.
- Contention: The second term (δ or Gamma) of the denominator models the contention between different parts of the system. Most common are issues caused by serialization or queueing effects.
- Coherence: The last term (K or Kappa) models the delay induced by keeping the system in a coherent and consistent state. Coherency is caused when writable data is shared in different parts of the system. Some of the factors for such a delay are caches implemented in software and hardware.
To read more about Universal Scalability Law please check Link.
Let’s look at an example: Let’s look at the following example and perform a prediction using the USL models implemented at Visualize-IT.
To be able to use the Universal Scalability Law modelling functionality you should be able to perform measurements on a system. This could be a system in production or a system in a test environment. Here’s the input required for USL :
- N : Concurrency, User Load, etc.
- X(N): TPS or Transactional Throughout
Let’s assume for purposes of this example we have a set of measured values obtained from a SUT (System Under Test). These values are provided below and are also available as a text (CSV or Comma Separated Values) file at the bottom of the Universal Scalability Law model form:
load,throughput 1,64.9 18,995.9 36,1652.4 72,1853.2 108,1828.9 144,1775.0 216,1702.2
Enter these values into the form and hit execute to see the prediction provided by Dr. Gunther’s Universal Scalability Law. Use the “Add” button to create as many rows as required and enter the values into the form. You can use the “Del” button to delete any rows you do not require.
The example provided has a set of 7 values measured from N (User Load) equals 1 to N equals 216. We recommend a minimum of at-least 6 data points events spread out. See choice of input values below to understand how you should obtain the relevant measurements for USL to provide a useful prediction.
Interpreting Results: Key in the values into the model and hit the execute button. Give the model 5-7 seconds to perform the computation and the results will be displayed on the screen. Please do not hit the execute button multiple times after you have submitted the data or you’ll end up with weird results. If the input values don’t lend to a forecast the model will complain and come back with possible suggestions for you to try out.
The model prediction for the sample dataset is provided in the above graph. The graph consists of three different sets of values.
- Red data points : This is your original dataset
- Blue data points with a line : USL’s prediction of your system performance.
- Shaded blue region : This includes the lower and upper confidence intervals for the prediction provided.
The Universal Scalability Law graphs provide an easy way to forecast system performance without having to dabble with Service Time parameters and build your own complex R models. A detailed table provides a listing of the information provided by USL for the given dataset.
Choice Of Input Values: For Universal Scalability Law to be useful you should be aware of the following:
- Enter a minimum of 6 values i.e. 6 values of N, X(N)
- The first measured value of N should be at N=1
- To be able to obtain sensible forecasts we would recommend obtaining measurements to at-least 60-70% system utilization.
- Try to obtain as many data points as practically feasible. E.g. For a system that runs at 60% utilization at 100 users try to obtain measurements at N =1, N=5, N=10, N=15………N=100.
- If you can’t do as many as we’ve suggested that’s fine, but remember the lower the number of data points and the lower the peak values you measure it at (as compared to what your known current peak is) the higher the possibility of a larger variance in the predicted performance.
Large number of data points, equally spread across the region of measurement and measured right upto the peak system throughput i.e. 60-70% utilization will potentially reduce USL forecast error. However, it’s highly likely that the reason you are using USL is because you do not have access to those number of data points and only know system scalability at low levels i.e. 50% (since your test environment is half of prod). You’ll have to make do with the insight USL is able to provide.
Here’s a summary of the values provided by the USL model for the given set of input values.
• Scale factor for normalization: The results include the scale factor used for normalization. The scale factor is used internally
to adjust the measured values to a common scale. It is equal to the value X (1) of the measurements.
• Efficiency: The model efficiency tells us something about the ratio of useful work that is performed per processor. It is obvious that two processors might be able to handle twice the work of one processor but not more. As we add more processors to the system the efficiency of the system tends to decrease. Calculating the ratio of the workload per processor should therefore always be less or equal to 1. To be able to verify this, we use the distribution of the efficiency values shown above. Here’s the actual unsorted efficiency values extracted from the above model.
Processors: 1 4 8 12 16 20 24 28 32 48 64
Efficiency: 1.000 0.975 0.812 0.708 0.594 0.500 0.438 0.411 0.406 0.292 0.242
• Residual values: We are performing a regression on the data to calculate the coefficients and therefore we determine the residuals for the fitted values. The distribution of the residuals is also given as part of the results above.
• Contention and Coherency Coefficients: The coefficients and are the result that we are essentially interested in. They tell us the magnitude of the contention and coherency effects within the system. Higher the percentage values higher the amount of contention and coherency.
• R Squared: Finally R(Square) estimates how well the model fits the data.
Gotchas: Keep in mind when using Universal Scalability Law that the predicted performance for a given system is specific to a given configuration (software, hardware, network, etc.). If any of those elements change you have invalidated the forecast and will need to re-obtain measurements on the new configuration (software, hardware, network, etc.) to re-do your prediction.
Analytical Modelling Solution: VisualizeIT offers access to a bunch of Analytical Models, Statistical Models and Simulation Models. Access to all the Analytical (Mathematical) models is free. We recommend you try out the Universal Scalability Law models at VisualizeIT and drop us a note with your suggestions, input and comments.
Conclusion: Universal Scalability Law by Dr. Neil Gunther is a great way to obtain insight into system performance without having to delve into component level service times. Universal Scalability Law can also be used to validate the system scalability thresholds for systems in test and production without having to push the system over the limits and interrupting business as usual. Thanks to Dr. Neil Gunther for the power of Universal Scalability Law and to Amdahl for providing the foundation on which Dr. Gunther built his theory.