1. Introduction

0-Day CI is an automated Linux kernel test service that provides comprehensive test coverage of the Linux kernel. It covers kernel build, static analysis, boot, functional, performance and power tests. This report shows the recent observations of kernel performance status on IA platform based on the test results from 0-Day CI service. It is structured in the following manner:

• Section 2, merged regressions and improvements in v4.17 release candidates

• Section 3, captured regressions and improvements by shift-left testing during developers’ and maintainers’ tree during v4.17 release cycle

• Section 4, test machine list

 

2. Linux Kernel v4.17 Release

The v4.17 release of the Linux kernel was on June 3rd. Headline features in this release include improved load estimation in the CPU scheduler, raw BPF tracepoints, lazytime support in the XFS filesystem, full in-kernel TLS protocol support, histogram triggers for tracing, and the removal of support for eight processor architectures. 0-Day CI monitored the release closely to trace down the performance status on IA platform. 0-Day observed 5 regressions and 5 improvements during feature development phase for v4.17. We will share more detailed information together with correlated patches that led to the results. Note that the assessment is limited by the test coverage we have now. The list is summarized in the observation summary section.

1. will-it-scale.per_process_ops

will-it-scale runs a test case from 1 to n parallel copies to see if the test case will scale. It builds both a process and threads based test in order to see any differences between the two.

1. scenario: test page fault in process mode

 

Commit e27be240df was reported to have -27.2% regression of will-it-scale.per_process_ops when comparing to v4.16." It was merged to mainline at v4.17-rc1." Aaron Lu worked on this and found that the commit itself looks innocent enough as it merely changed some event

counting mechanism and this test didn't trigger those events at all, it's likely due to the changed layout of 'struct mem_cgroup' that either make stat_cpu falling into a constantly modifying cache line or some hot fields stop being in the same cache line. Aaron has submitted a fix patch to community which can restore the performance.

 

Correlated commits

 

2. netperf.Throughput_Mbps

Netperf is a benchmark that can be used to measure the performance of many different types of networking. It provides tests for both unidirectional throughput, and end-to-end latency.

 

1. scenario: ipv4 SCTP STREAM test in localhost

 

Commit a37b969a61 was reported to have -25.3% regression of netperf.Throughput_Mbps when comparing to v4.16-rc4." It was merged to mainline at v4.17-rc1." and performance was recovered after v4.17-rc2.

 

Correlated commits

 

3. aim7.jobs-per-min

aim7 is a traditional UNIX system level benchmark suite that is used to test and measure the performance of a multiuser system.

1. Scenario: f2fs-sync_disk_rw

 

Commit 84b89e5d94 was reported to have 91.4% improvement of aim7.jobs-per-min when comparing to v4.16-rc2." It was merged to mainline at v4.17-rc1."

 

Correlated commits

 

4. unixbench.score

UnixBench is a system benchmark to provide a basic indicator of the performance of a Unix-like system.

1. scenario: execl test

 

Commit d519329f72 was reported to have -9.9% regression of unixbench.score when comparing to v4.16-rc6." It was merged to mainline at v4.17-rc1." The author is addressing the regression.

 

Correlated commits

 

5. Observation Summary

0-Day CI observed 5 regressions and 5 improvements during feature development phase for v4.17, which is in the time frame from v4.17-rc1 to v4.17 release.

 

3. Shift-Left Testing

Beyond testing trees in the upstream kernel, 0-Day CI also tests developers’ and maintainers’ trees, which can catch issues earlier and reduce wider impact. We call it “shift-left” testing. During the v4.17 release cycle, 0-Day CI had reported 10 major performance regressions and 5 major improvements by doing shift-left testing. We will share more detailed information together with possible code changes that led to this result for some of these, though the assessment is limited by the test coverage we have now. The whole list is summarized at report summary section.

1. aim7.jobs-per-min

aim7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of a multiuser system.

 

1. scenario: creat-clo test on btrfs

 

Commit 7d3c8e53f5 was reported to have 57.9% improvement of aim7.jobs-per-min when comparing to v4.17-rc4."

 

Correlated commits

 

2. fsmark.files_per_sec

The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload.

1. scenario: no synchronous write on xfs

 

Commit 499c0a9aec was reported to have +6.6% improvement of fsmark.files_per_sec when comparing to v4.17-rc4."

 

Correlated commits

 

3. stress-ng.copy-file.ops_per_sec

stress-ng will stress test a computer system in various selectable ways. It was designed to exercise various physical subsystems of a computer as well as the various operating system kernel interfaces.

1. scenario: 100%-1s-filesystem-performance

Commit 9cb0698bda was reported to have -97.9% regression of stress-ng.copy-file.ops_per_sec when comparing to v4.17-rc3."

 

Correlated commits

 

4. unixbench.score

UnixBench is a system benchmark to provide a basic indicator of the performance of a Unix-like system.

1. scenario: execl test

Commit f6f47c66af was reported to have -14.5% regression of unixbench.score when comparing to v4.17-rc2."

 

Correlated commits

 

8. Report Summary

0-Day CI had reported 8 performance regressions and 6 improvements by doing shift-left testing on developer and maintainer repos.

 

 

4. Test Machines

1. IVB Desktop

 

 

2. BDW EP

 

3. HSW EP

 

4. IVB EP

 

 

5. HSX EX