36

I'm trying to use iotop to determine whether the CPUs are waiting on I/O at any point for a specific program called bwa (it's a next-generation sequence alignment program). If I start iotop without specifying a specific process (-p pid), I get all of the expected information, including SWAPIN and IO>, but the process I'm interested in doesn't show up on the list. If I then specify the process I'm interested in, iotop complains that "CONFIG_TASK_DELAY_ACCT not enabled in kernel". For sanity, I verified in my kernel config file (/boot/config-3.7.10-1.11-desktop) that CONFIG_TASK_DELAY_ACCT is enabled (CONFIG_TASK_DELAY_ACCT=y).

The questions:

  1. What could make iotop think CONFIG_TASK_DELAY_ACCT is not enabled for a specific process when I know it is?
  2. Can I fix it?
  3. Have I overlooked something silly?
Mark Ebbert
  • 521
  • 1
  • 5
  • 11

3 Answers3

27

Brock's Blog describes how you can do that with Ubuntu:

  1. Edit /etc/default/grub, adding “delayacct” as an option to the GRUB_CMDLINE_LINUX_DEFAULT entry. If you hadn’t already modified that line, it would go from

    GRUB_CMDLINE_LINUX_DEFAULT=""
    

    to

    GRUB_CMDLINE_LINUX_DEFAULT="delayacct"
    
  2. Run “sudo update-grub”

  3. Reboot, and you should be good to go
eSzeL
  • 381
  • 3
  • 3
  • This was necessary on my computer running Ubuntu with mainline kernel to get iotop working. Thanks! – Martin Pecka Dec 14 '21 at 16:19
  • 17
    FYI, here's the [kernel commit](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e4042ad492357fa995921376462b04a025dd53b6) that changed the default, which says "Assuming this stuff isn't actually used much; disable it by default and avoid allocating and tracking the task_delay_info structure. taskstats is changed to still report the regular sched and sched_info and only skip the missing task_delay_info fields instead of not reporting anything." So maybe iotop needs to use taskstats now? – Adam Katz Dec 30 '21 at 22:15
  • I second @MartinPecka's thanks! – Peter Berbec Feb 10 '22 at 21:43
19

An update:

From delay accounting kernel documentation:

Delay accounting is disabled by default at boot up. To enable, add:

delayacct

to the kernel boot options. Alternatively, use sysctl kernel.task_delayacct to switch the state at runtime. Note however that only tasks started after enabling it will have delayacct information.

Form iotop-c man page:

Starting with Linux kernel 5.14.x task_delayacct is configurable at runtime and set to off by default. This setting can be changed in interactive mode by the Ctrl-T shortcut. In batch mode a warning is printed when the setting is OFF. From the command line this can be enabled by:

$ sudo sysctl kernel.task_delayacct=1

and disabled again by:

$ sudo sysctl kernel.task_delayacct=0

It is advisable to keep this option off when not using this or another monitoring program because when enabled it has some effect on system performance.

Irfan Latif
  • 461
  • 5
  • 11
2

Expanding on the fantastic answer and explanation by @Irfan Latif, I use this as a zsh alias:

alias iotopd='bash -c "sudo sysctl kernel.task_delayacct=1 && sudo iotop ; sudo sysctl kernel.task_delayacct=0"'

Note the ; here after iotop, as it will run on every exit, instead of just successful ones.

This alias will set the kernel parameter, run the command, then set it back to 0. Useful if you run it only in a single session, but will be a painful with multiple users.

You can also run this as a shell script, which is useful as it traps on exit.

#!/bin/bash

# This script enables task delay accounting in the Linux kernel, runs iotop,
# then on exit sets task_delayacct back to 0.

# Disable task delay accounting when the script exits
disable_task_delayacct() {
  sudo sysctl kernel.task_delayacct=0
}

# Trap the EXIT signal to call the disable_task_delayacct function
trap 'disable_task_delayacct' EXIT

# Enable task delay accounting
sudo sysctl kernel.task_delayacct=1

# Run iotop with administrative privileges
sudo iotop
Josh
  • 121
  • 2