Putting Linux Reliability To The Test Linux reliability measurement

Putting Linux reliability to the test

By Li Ge & Linda Scott & Mark VanderWiele - 2004-01-19 Page: 1 2 3 4 5 6

Linux reliability measurement

Test results at a glance
The following summary is based on the results of the tests and observations on the duration of the runs:

The Linux kernel and other core OS components -- including libraries, device drivers, file systems, networking, IPC, and memory management -- operated consistently and completed all the expected durations of runs with zero critical system failures.
Every run generated a high success rate (over 95%), with a very small number of expected intermittent failures that were the result of the concurrent executions of tests that are designed to overload resources.
Linux system performance was not degraded during the long duration of the run.
The Linux kernel properly scaled to use hardware resources (CPU, memory, disk) on SMP systems.
The Linux system handled continuous full CPU load (over 99%) and high memory stress well.
The Linux system handled overloaded circumstances correctly.

The tests demonstrate that the Linux kernel and other core OS components are reliable and stable over 30, 60, and 90 days, and can provide a robust, enterprise-level environment for customers over long periods of time.

Objectives

The objective of the Linux reliability effort at the IBM Linux Technology Center is to measure the Linux operating system's stability and reliability over long periods of time with an emphasis on workloads relevant to Linux customer environments using the LTP test suite (see Resources for more on the LTP). Identification of defects was not the primary focus.

Test environment overview

This article describes the test results and analysis of 30- and 60-day Linux reliability measure tests using the LTP test suite. The tests used SuSE Linux Enterprise Server v8 (SLES 8) as the testing kernel and IBM pSeries servers as testing hardware. A specially designed stress-test scenario of LTP was used to exercise a wide range of kernel components in parallel with networking and memory management, and to create a high stress workload on the testing system. The Linux kernel, TCP, NFS, and I/O test components were targeted with a heavy-stress workload.

The tests

At 30 days

30-day LTP stress execution results for pSeries

Machine: p650 LPAR
CPU: (2) Power4- 1.2 GHz
Kernel: Linux 2.4.19-ull-ppc64-SMP (SLES 8 SP 1)
LTP version: 20030514
99.00 percent Average CPU utilization (User: 48.65 percent, System: 50.35 percent)
80.09 percent Average memory utilization (8GB)

Observations:

SLES 8 PPC64 30-day stress run successfully completed on p650 LPAR
LTPstress was the test tool. Test cases were executed both in parallel and in sequence
Kernel, TCP, NFS, and I/O test components were targeted with heavy stress workloads
Success rate: 97.88 percent
Zero critical system failures

Figure 1. 30-day LTP stress execution results

At 60 days
60-day LTP stress execution results: pSeries

Machine: B80
CPU: (2) Power3- 375 MHz
Kernel: Linux 2.4.19-ull-ppc64-SMP (SLES 8 SP 1)
LTP version: 20030514
99.96 percent average CPU utilization (User: 75.02 percent, System: 24.94 percent)
61.69 percent average memory utilization (8GB)
3.86 percent average swap utilization (1GB)

Observations:

SLES 8 PPC64 60-day stress run successfully completed on pSeries B80
LTPstress was the test tool. Test cases were executed both in parallel and in sequence
Kernel, TCP, NFS, and I/O tests components were targeted with heavy stress workloads
Success rate: 95.12 percent
Zero critical system failures

Figure 2. 60-day LTP stress execution results

View Putting Linux reliability to the test Discussion

Page: 1 2 3 4 5 6 Next Page: Test infrastructure

First published by IBM developerWorks