ColinIanKing/stress-ng V0.21.00

stress-ng V0.21.00 "systemic system smasher"

on GitHub

New stressors

• filehole - exercise randomized file holes/extents
• crc - cyclic redundancy check using gcc builtins and non-gcc CRC builtin alternatives

New options

• --schedpolicy-cpumix - move shedpolicy stressors to a randomly chosen CPUs
• --filehole-defrag - reduce file hole fragmentation
• --cachehammer-method - select between permuted or randomized cache operations

New build options

• BUILD_SMALL=1 option to build small unoptimized executable

Notable bugfixes

• lsm stressor now works on S390x systems
• fix builds where termio.h does not exist
• fix fsize stressor false positives on 32 bit systems (don't set _LARGEFILE_SOURCE)
• lockmix: add timer to interrupt any blocked file locks
• pthread: fix racy SIGSEGV during interrupted stressor termination
• stress-af-alg: handle ENOSYS errno on setsockopt ALG_SET_KEY

Notable improvements

• timerfd: also use CLOCK_MONOTONIC and CLOCK_BOOTTIME
• clock: use CLOCK_AUX
• sockabuse: add more exercising of socket and getsockopt calls
• workload: add vecint workload method (for x86 CPU frequency changing)
• loop: use random enabled/disabled O_DIRECT on backing file
• easy-opcode: add some more x86 simple ops
• tree: rework tree nodes, speed improvement of ~10-17%
• tree: add treap (tree/heap) method
• vm: make walk-flush stride through memory with cacheline sized gaps

Sponsors

Many thanks to the following sponsors of the project (in alphabetical order):

• Bruce Underhill (Codethink)
• Chris Wayne
• Gilbert Morgan
• kobataiwan
• nouraellm
• Paul Larson

Sponsor stress-ng via: https://github.com/sponsors/ColinIanKing

Detailed Changelog

Ethan Hsieh

• stress-af-alg: handle ENOSYS errno on setsockopt ALG_SET_KEY

Thomas Weißschuh

• stress-clock: run clock_nanosleep() for CLOCK_AUX
• core-clocks: add fallback definitions for CLOCK_AUX
• stress-clock: gracefully handle errors from CLOCK_AUX
• stress-clock: correctly check the return value of clock_nanosleep()
• stress-clock: do not test clock_nanosleep(CLOCK_THREAD_CPUTIME_ID)
• stress-clock: add missing commas at the end of all clock array elements

Colin Ian King

• debian: update changelog
• Makefile: bump version
• stress-lsm: fix size type, should be uint32_t
• stress-cache: only permute when --cache-permute is enabled
• stress-time-warp: silently ignore ENODEV on timers that are not supported
• README.md: update kernel bug list
• stress-sigxcpu: enable sanity checking with --verify option
• stress-branch: reduce optimization level to O2
• stress-sigsegv: reduce MAX_MASK_SHIFT to 49 for risc-v kernels
• README.md: add sponsor link
• stress-ng: constify us_total
• stress-ng: reduce scope of int e
• stress-sigxcpu: add RLIMIT_RTTIME for SIGXCPU signals, add verification
• stress-mmaptorture: make variable offset non-clobberable
• core-filesystem: add befs and virtualbox file system magic
• core-filesystem: add ufs magic
• kernel-coverage: add nfs
• stress-schedmix: add null check on cpus
• stress-schedmix: force more cpu affinity changes with --schedmix-cpumix option
• stress-schedpolicy: ignore E2BIG errno on sched_setscheduler
• stress-ng: don't set _LARGEFILE_SOURCE
• stress-workload: ensure each pthread has it's own buffer
• stress-fsize: ensure max_offset is not larger than rlim_max
• stress-list: cast &head to void *
• stress-filehole: assign zero_buf as 64 bit offset to buf
• stress-cache: make permute generate all permutations
• stress-timerfd: also use CLOCK_MONOTONIC and CLOCK_BOOTTIME
• stress-lockmix: voidify function argument args
• stress-lockmix: voidify millisec arg
• CITATIOBS.md: add another research paper reference
• core-lockmix: add timer to interrupt any blocked file locks
• stress-sockabuse: enable SOCK_RDM for OS X
• stress-sockabuse: use SOCK_RDM for specific systems
• stress-sockabuse: don't use SOCK_RDM for Haiku
• stress-timermix: cater for zero sized arrays for itimers and timers
• stress-sockabuse: use SOCK_DCCP and SOCK_PACKET for Linux only
• stress-sleep: remove unncessary setting/checking of eintr
• stress-schedpolicy: remove redundant initialization
• stress-close: ensure all fds are initialized
• stress-bind-mount: stash and restore errno for move_mount error value
• stress-ng: remove redundant initialization
• stress-time-warp: silently ignore ENOSYS or EINVAL on gettime calls
• stress-pty: fix builds where termio.h does not exist
• core-cachehammer: fix regression on memory increment
• core-sort: add shim_qsort for qsort
• core-sort: move qsort_bm to core-sort
• stress-vnni: only define shim_m256i_u if it is required
• README.md: update contributor list
• Makefile: add core-clock.h to Makefile HEADERS
• stress-sockabuse: add more exercising of socket and getsockopt calls
• core-sync: add missing default in switch statement
• core-hash: clean up some clang pedantic implicit conversion signness warnings
• core-cpu-cache: clean up some clang pedantic implicit conversion signness warnings
• stress-strnum: rename 'this' to 'method'
• core-bitops: cast return from builtins to expected return type
• core-cpu: prefix temp variables with tmp
• core-asm-ret: use fixed array instead of gcc flexible arrays
• core-affinity: fix clean-up of integer casts
• Makefile: add extern to core-config.h correctly
• core-affinity: use correct integer types
• stress-bad-altstack: replace function cast with void * pointer
• stress-bigheap: add void * cast for %p format specifier
• stress-bad-ioctl: cast -1 to size_t to clean up clang pedantic warning
• stress-bad-altstack: cast function pointer to void *
• stress-af-alg: use correct format specifier for size_t value
• core-workload: clean up a handful of pedantic warnings
• core-shim: clean up clang pedantic warnings
• core-parse-opts: clean up pedantic clang warning
• stress-ng: declare per stressor stessor_info_t
• Makefile: make stress_config extern
• Makefile: Add BUILD_SMALL=1 option to build small unoptimized executable
• stress-ng: add more comments
• stress-ng: add block comment for stress_stressor_names_fixup
• stress-ng: add block comment for stress_description_yamlify
• stress-ng: inline stress_verifiable
• stress-ng: get output fd using pr_fd()
• stress-ng: remove legacy proc naming from stressor items
• stress-ng: remove need to pass number of items to stress_shared_mmap
• stress-ng: remove need for stress_nth_stressor_get
• Manual: add notes about swap header warnings
• CITATIONS.md: add another research paper reference
• stress-cgroup: remove S_IRGRP | S_IWGRP mkdir flags
• stress-umount: remove S_IRGRP | S_IWGRP mkdir flags
• stress-ramfs: remove S_IRGRP | S_IWGRP mkdir flags
• stress-bind-mount: remove S_IRGRP | S_IWGRP mkdir flags
• stress-syscall: remove GRP and OTH open flags
• stress-uprobe: remove GRP and OTH open flags
• stress-zombie: remove S_IRGRP | S_IWGRP mkdir flags
• stress-dirdeep: remove S_IRGRP | S_IWGRP creat flags
• core-signal: remove blank line
• core-nt-store.h: remove blank line
• core-asm-riscv.h: remove empty line
• core-mwc.h: remove a couple of blank lines
• stress-schedmix: clean up built in help options order
• stress-schedpolicy: add --schedpolicy-cpumix
• stress-schedpolicy: where possible exercise shim_sched_setattr more
• stress-schedpolicy: make rate const
• stress-schedpolicy: set more DEADLINE attributes, add metric counters
• stress-schedmix: add cpus array free error exit path handling
• stress-fault: remove from scheduler class
• stress-eventfd: remove from scheduler class
• stress-dnotify: remove from scheduler class
• stress-netlink-task: remove from scheduler class
• stress-netlink-proc: remove from scheduler class
• Manual: add , in crypt stressor list
• stress-schedmix: replace spaces with tab in indentation
• stress-schedmix: add checks for sched_setaffinity being available
• stress-schedpolicy: remove blank line
• stress-schedmix: add --schedmix-cpumix option
• CITATIONS.md: add another paper reference
• stress-ng: remove redundant check on num_perms
• core-vmstat: rename stress_get_iostat to stess_iostat_get
• core-vmstat: rename stress_get_tz_info to stress_tz_info_get
• core-vmstat: rename netbsd_get_cpu_time to netbsd_cpu_time_get
• core-vmstat: rename freebsd_get_cpu_time to freebsd_cpu_time_get
• core-vmstat: rename stress_get_vmstat to stress_vmstat_get
• core-thermal-zone: rename stress_tz_get_temperatures to stress_tz_temperatures_get
• core-stack: rename stress_get_min_aux_sig_stack_size to stress_min_aux_sig_stack_size_get
• core-rapl: rename stress_rapl_get_power to stress_rapl_power_get
• core-net: rename stress_net_get_local_bind_ports to stress_net_local_bind_ports_get
• core-mwc: rename stress_mwc_default_seed to stress_mwc_seed_default
• core-mwc: rename stress_mwc_set_seed to stess_mwc_seed_set
• core-mwc: rename stress_mwc_get_seed to stress_mwc_seed_get
• core-helper: rename stress_get_exec_text_addr to stress_exec_text_addr
• core-helper: rename stress_get_libc_version to stress_libc_version_get
• core-ftrace: rename stress_ftrace_get_debugfs_path to stress_ftrace_debugfs_path_get
• core-affinity: rename stress_get_topology_set to stress_topology_set_get
• stress-cache: rename stress_get_cache_flags to stress_cache_flags_get
• stress-ng: inline some helper functions
• stress-ng: make metrics_lock into a metrics.lock struct/field
• stress-ng: re-order struct fields based on size
• stress-ng: rename stress_counter_info_t to stress_counter_t
• README.md: add another Linux kernel bug reference
• stress-ng: rename stress_stressors_enable_all to stress_stressors_enable
• stress-ng: rename limit_set to set_limit
• stress-ng: rename set_default to default_set
• stress-ng: rename stress_exclude_pathological to stress_pathological_set
• stress-ng: rename stressor_set_defaults to stressor_set_defaults
• stress-ng: rename stress_append_stressor to stress_list_item_append
• stress-ng: rename stress_set_proc_limits to stress_max_processes_limit_set
• stress-ng: rename stress_exclude_unimplemented to stress_unimplemented_exclude
• stress-ng: fix block comment for stress_shared_readonly
• stress-ng: rename stress_shared_map to stress_shared_mmap
• stress-ng: rename stress_map_page to stress_page_mmap
• stress-ng: rename stress_log_system_info to sytess_system_info_log
• stress-ng: rename stress_log_system_mem_info to stress_system_mem_info_log
• stress-ng: rename stress_log_args to stress_args_log
• stress-ng: rename stress_show_stressors to stress_stressors_show
• stress-ng: rename stress_run_child to stress_child_run
• stress-ng: rename stress_get_usage_stats to stress_rusage_stats_get
• stress-ng: rename stress_getrusage to stress_rusage_get
• stress-ng: rename stress_get_total_instances to stress_total_instances_get
• stress-ng: rename stress_get_num_stressors to stress_stressors_number_get
• stress-ng: rename stress_get_nth_stressor to stress_nth_stressor_get
• stress-ng: rename stress_handle_terminate to stress_terminate_handle
• stress-ng: rename stress_wait_stressors to stress_stressors_wait
• stress-ng: rename stress_wait_aggressive to stress_aggressive_wait
• stress-ng: rename stress_get_processors to stress_processors_get
• stress-ng: rename stress_opt_name to stress_opt_name_find
• stress-ng: rename stress_show_stressor_names to stress_stressor_names_show
• stress-ng: rename stress_usage_help_stressors to stress_stressors_usage_help
• stress-ng: rename stress_set_handler to stress_handler_set
• stress-ng: rename stress_block_signals to stress_signals_block
• stress-ng: rename stress_kill_stressors to stress_stressors_kill
• stress-ng: rename stress_zero_bogo_max_ops to stress_bogo_max_ops_zero
• stress-ng: rename stress_get_class to stress_class_get
• stress-ng: rename stress_get_class_id to stress_class_id_get
• stress-ng: rename stress_fixup_stressor_names to stress_stressor_name_fixup
• stress-ng: rename stress_mlock_executable to stress_executable_mlock
• stress-ng: rename stress_run_permute to stress_permute_run
• stress-ng: rename stress_run_parallel to stress_parallel_run
• stress-ng: rename stress_run_sequential to stress_sequential_run
• stress-ng: rename stress_setup_parallel to stress_parallel_setup
• stress-ng: rename stress_setup_sequential to stress_sequential_setup
• stress-ng: rename stress_set_default_timeout to stress_default_timeout_set
• stress-ng: rename stress_parse_opts to stress_opts_parse
• stress-ng: rename stress_parse_limit to stress_limit_parse
• stress-ng: rename stress_enable_classes to stress_classes_enable
• stress-ng: rename stress_enable_all_stressors to stress_stressors_enable_all
• stress-ng: rename stress_set_random_stressors to stress_random_stressors_set
• stress-ng: rename stress_setup_stats_buffers to stress_stats_buffers_setup
• stress-ng: rename stress_ignore_stressor to stress_stressor_ignore
• stress-ng: remove redundant struct stress_signal_map name
• stress-ng: re-order put elements to pack more efficiently in memory
• stress-ng: rename stress_alloc_proc_stats to stress_stats_alloc
• stress-ng: rename stress_find_proc_info to stress_list_item_find
• stress-ng: rename stress_t to stress_stressor_t
• stress-ng: rename stress_stressor_t to stress_item_list_t
• stress-ng: rename stress_metics_info to stress_metrics_info
• stress-ng: rename stress_stressor_t related struct name and fields
• stress-pthread: bail out of thread creation loop before creating next pthread
• stress-ng: write unexpected error messages to stderr/stdout
• stress-sendfile: bail out if fallocate returns EINTR
• stress-fcntl: check if parent directory has been removed
• stress-sigpipe: relax sanity check a small amount
• stress-goto: only sanity check counters after 1 bogo op
• stress-ng: remove debug total
• stress-open: don't close fds if min/max not set
• stress-ng: restructure metrics data and descriptions
• CITAIONS.md: add another reference
• stress-ng: remove debug for total metrics count
• stress-strnum: constify pointer ptr
• core-workload: fix missing L for long double constant value
• stress-crc: add pointer constifications
• strress-cachehammer: no need to pass global ctxt to stress_cachehammer_exercise
• stress-ng: re-work metrics array allocation and sizing
• stress-ng: remove redundant pointer metrics from args struct
• stress-mmaptorture: mmap onto same offset or random offset
• stress-filehole: add --filehole-defrag option to reduce extents
• stress-cached-hammer: add a random or permute cache method
• core-workload: add vecint workload method
• Manual: sort varyload methods alphabetically
• core-workload: sort workload methods alphabetically
• CITATIONS.md: add some more paper links
• stress-ng: make stress_stressor_t field run bool type
• stress-ng: add padding to make all fields in stress_checksum_t 64 byte padded
• core-shim: use uintptr_t casts on void * pointers on void * pointer arithmetic
• stress-crc: pass crc data to crc functions
• stress-crc: make polynomial const
• stress-crc: implement CRCs without __builtin_crc helpers
• stress-crc: add CRC bit rate to metrics
• stress-af-alg: turn bunch of if statements into switch statements
• Manual: explain how the 1024 bits of data are processed
• README.md: add reference to another kernel issue found with stress-ng
• stress-crc: add some crc function comment blocks
• stress-crc: use crc_metrics_t for cache aligned compact metrics
• stress-crc: add register hint, remove empty line
• stress-crc: ensure data is 64 byte aligned
• stress-crc: rename loop counter to loop
• stress-crc: add checks for __builtin_crc functions
• stress-crc: add gcc CRC builtin stressor
• stress-loop: use random enabled/disabled O_DIRECT on backing file
• core-cpu: add OPTIMIZE3 annotation
• core-cpu: cache cpuid flag checks using a macro
• core-shim: shim_cacheflush: enable more arch specific cache flushing
• core-shim: shim_flush_icache: use shim_cacheflush as a fallback
• stress-sockabuse: use shim_posix_fadvise
• stress-revio: use shim_posix_fadvise
• stress-readahead: use shim_posix_fadvise
• stress-iomix: use shim_posix_fadvise
• stress-hdd: use shim_posix_fadvise
• stress-filerace: use shim_posix_fadvise
• stress-fd-race: use shim_posix_fadvise
• stress-fd-abuse: use shim_posix_fadvise
• stress-filehole: use SHIM_* posix_fadvise advise only if it is defined
• stress-filehole: use shim_posix_fadvise hints
• core-shim: add shim for posix_fadvise
• stress-fileholes: add termination check in stress_filehole_non_zeros_to_holes
• stress-filehole: clean up cast
• stress-filehole: punch holes in non-zero data
• stress-filehole: only report extents metrics when they are non-zero
• stress-filehole: add mmap'd read error check, report errors with pr_fail
• stress-filehole: add missing comment block
• stress-filehole: perform non-pread read after lseek
• stress-filehole: add fallocate specific flag checks
• stress-filehole: use shim_msync, move msync flags into function
• stress-filehole: fix fsync counter, make less frequent too
• stress-filehole: fix up copyright
• stress-filehole: add random lseeks+reads with SEEK_SET, SEEK_HOLE and SEEK_DATA
• stress-filehole: a new file hole/extent stressor
• stress-str: use shim_strnlen instead of strlen
• CITATIONS.md: Add another reference link
• CITATIONS.md: Add another reference link
• stress-vm: make walk-flush stride through memory with cacheline sized gaps
• core-parse-opts: add integer to numeric error message
• stress-dev: use shim_strnlen instead of strlen
• stress-set: use shim_strnlen instead of strlen
• stress-af-alg: use shim_strnlen instead of strlen
• core-helper: replace strlen with the size of the str
• core-helper: refactor strlen of parent_ld_path
• core-filesystem: remove duplicated strlen call
• core-filesystem: use shim_strnlen instead of strlen
• core-interrupts: use shim_strnlen instead of strlen
• core-shim: constify str ptr arg in shim_strnlen
• stress-rdrand: stress-ng: report message without capitalization
• stress-ng: report errors without capitalization
• core-parse-opts: report errors without capitalization
• core-job: report errors without capitalization
• core-helper: report build info without capitalization
• stress-cpu-sched: use shim_strnlen instead of strlen
• stress-dir: use shim_strnlen instead of strlen
• stress-dirdeep: use shim_strnlen instead of strlen
• stress-dirmany: use shim_strnlen instead of strlen
• stress-filename: use shim_strnlen instead of strlen
• stress-fstat: use shim_strnlen instead of strlen
• stress-hsearch: use shim_strnlen instead of strlen
• stress-key: use shim_strnlen instead of strlen
• stress-led: use shim_strnlen instead of strlen
• stress-link: use shim_strnlen instead of strlen
• stress-module: use shim_strnlen instead of strlen
• stress-ng: use shim_strnlen instead of strlen
• stress-procfs: use shim_strnlen instead of strlen
• stress-syscall: use shim_strnlen instead of strlen
• stress-tmpfs: use shim_strnlen instead of strlen
• stress-workload: use shim_strnlen instead of strlen
• stress-xattr: use shim_strnlen instead of strlen
• core-shim: use shim_strnlen
• core-resctrl: use shim_strnlen
• core-numa: use shim_strnlen
• core-log: use shim_strnlen
• core-ignite-cpu: use shim_strnlen
• core-ftrace: use shim_strnlen
• core-filesystem: use shim_strnlen
• core-shim: add shim for strnlen, a "safer" strlen
• core-log: use use strscpy instead of strncpy
• core-ftrace: use strscpy instead of strcpy
• test/test-mq-sysv: use strncpy instead of strcpy
• stress-ng: fix incorrect type setting for sched-reclaim
• stress-ng: show global settings with -v verbose debug option
• stress-tree: minor code clean-ups
• stress-tree: add treap tree method
• stress-tree: name node structs with _node tag
• CITATIONS.md: add another research paper reference
• stress-tree: remove more unncessary pragmas
• stress-tree: adjust avl tree balance values
• stress-tree: remove temp pointer res
• stress-tree: improve inlining of btree_insert_node
• stress-tree: minor optimization of btree
• stress-tree: minor clean up of btree search
• stress-tree: remove find
• stress-tree: add UNLIKELY hint on binary tree comparison
• stress-tree: remove struct packing and warning pragmas
• stress-tree: unroll btree tree forward find
• stress-tree: unroll binary tree forward find
• stress-tree: unroll splay tree forward find
• stress-tree: add unlikely hint in btree find, ~0.2% speedup
• stress-tree: update q->bf and root before updating balance factors
• sterss-tree: add minor hints to improve avl insertion
• stress-tree: use per tree sized nodes rather than union of tree nodes
• stress-tree: replace cache/memory heavy shuffle with random value setting
• stress-monte-carlo: Use ULL for 64 bit constant
• test-asm-riscv-cbom.c: use explicit memset
• test-asm-riscv-cbo_zero.c: use explicit memset, ensure sched_getaffinity is defined
• stress-udp: remove unused variable j
• core-workload: ensure val is put so that it's not optimized out
• stress-easy-opcode: add some more x86 simple ops
• stress-sockfd: check for null return from CMSG_FIRSTHDR
• stress-sock: add null check on ctrls, keeps infer happy
• stress-resources: explicity free resources before _exit
• stress-fd-race: check for null return from CMSG_FIRSTHDR
• stress-ng: fetch cpu count as late as possible
• stress-ng: ensure stress_stats_hash_table_alloc failure is reported as a failure
• core-mmap: zero info to silence infer static analysis warning