33

I am just wondering how I can find out in bash how many CPU cores a user is now using on a Linux Server?

I am submitting a fair amount of background jobs to the server, so I'd like to write a bash script to check before I submit a job if the running jobs (processes) submitted by me are too many, because I have to let other users have enough cores to use for their jobs. I'd like to know what bash command can tell me how many cores my jobs are using now.

Thanks and regards!

alex
  • 17,684
  • 7
  • 54
  • 77
Tim
  • 16,963
  • 69
  • 181
  • 260

6 Answers6

41

To get the number of CPU cores per CPU:

grep "^core id" /proc/cpuinfo | sort -u | wc -l

Or to get the number of physical CPUs:

grep "^physical id" /proc/cpuinfo | sort -u | wc -l
nathan
  • 955
  • 7
  • 6
Alix Axel
  • 1,202
  • 3
  • 15
  • 33
26

I don't know if it helps, but you can use the mpstat utility to get a breakdown of CPU usage by individual processor (or core). For example:

$ mpstat -P ALL 1

12:49:59 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
12:50:00 PM  all    7.89    0.00    1.25    0.88    0.00    0.00    0.00   89.97   1359.00
12:50:00 PM    0   14.00    0.00    0.00    0.00    0.00    0.00    0.00   86.00   1043.00
12:50:00 PM    1   15.84    0.00    7.92    3.96    0.00    0.99    0.00   71.29    297.00
12:50:00 PM    2    3.96    0.00    0.00    1.98    0.00    0.99    0.00   93.07      0.00
12:50:00 PM    3    3.96    0.00    0.99    2.97    0.00    0.00    0.00   92.08      0.00
12:50:00 PM    4    4.00    0.00    0.00    0.00    0.00    0.00    0.00   96.00      0.00
12:50:00 PM    5    4.95    0.00    0.99    0.00    0.00    0.00    0.00   94.06     18.00
12:50:00 PM    6   10.89    0.00    0.99    0.00    0.00    0.00    0.00   88.12      0.00
12:50:00 PM    7    5.05    0.00    0.00    0.00    0.00    0.00    0.00   94.95      0.00

In this example, you can see that CPU's 0, 1, and 6 are doing more work than the rest of them. Sometimes you will see that a single CPU is near (or at) 100% while others are at zero. This can be an indicator of program (or portion of a program) that is single-threaded and only able to use a single CPU at a time.

To install mpstat on a Fedora, RHEL, or CentOS system, use yum install sysstat.

Matt Solnit
  • 1,590
  • 12
  • 12
  • Thanks a million for providing this answer! My colleagues were using the "top" command and making all the wrong conclusions about processor utilization until I showed them the mpstat command. – Nav Mar 11 '11 at 05:07
  • Not available on Ubuntu: `# aptitude search mpstat #`. – Alix Axel Dec 02 '12 at 00:16
  • 2
    @AlixAxel try `aptitude search systat` instead. – Matt Solnit Dec 02 '12 at 05:07
  • @MattSolnit: Still no luck. – Alix Axel Dec 02 '12 at 10:08
  • mpstat is provided by the package `sysstat` [on Debian](https://packages.debian.org/jessie/sysstat) and [on Ubuntu](http://packages.ubuntu.com/xenial/sysstat). I would expect most Debian-derived distributions use the same package name. On Debian-derived distributions, start by `apt-cache search --full mpstat`. – user Jun 22 '16 at 08:46
  • This is the best answer - as it shows what each core is doing. It works right out of the box on a Centos 6 server. – itoctopus Jun 27 '16 at 22:46
2

Unless explicitly configured not to (ie. pinning a process to a specific CPU), all cores can be assumed to be in use at all times. The scheduler will allocate processes the next available core. Case in point, "System Monitor" (part of GNOME) is showing my load almost the same across all 4 cores of my machine.

EmmEff
  • 1,295
  • 6
  • 10
  • I am submitting a fair amount of background jobs to the server, so I like to write a bash script to check before I submit a job if the running jobs(processes) submit by me are too many, because I have to let other users have enough cores to use for their jobs. So I like to know what bash command can tell me how many cores my jobs are using now. – Tim Oct 01 '09 at 18:09
  • 3
    I don't think you understood my answer. The number of cores in use is ALWAYS the total number of cores available. Use the overall system load as a factor to determine whether or not your job submission should be throttled. – EmmEff Oct 01 '09 at 18:45
  • Thank you! I do check the overall system load. I have been told by one of my colleagues that I can not use up all the cores because who knows others will run their jobs soon and my jobs will last for quite a while. Do you also think I have to limit the cores I am using? – Tim Oct 01 '09 at 19:00
2

So you'll see there are responses here that will tell you how your cores are being utilized.

HOWEVER - this isn't really doing you a service. You've made a basic assumption that just doesn't hold - that your jobs will tend to group themselves onto some subset of the cores.

Instead, your jobs will be spread out across all of the cores, unless you implement something that keeps them "penned in" somehow. (Note: I'm not recommending that; just saying, "unless")

Here's an alternative strategy: Identify for your particular system what the LOAD level is when you feel it's "acceptable" for other users to be adding more jobs. Then, build something that only submits a new background job when the load level is down below that limit.

That way, the solution will be core-count-independent, more portable, more flexible, and easier to "tweak" as well.

pbr
  • 1,335
  • 1
  • 7
  • 14
  • I just now noticed a comment you added to another poster's answer, indicating this is because of something a colleague told you. So, indeed, it appears that the misconception is his, not yours. Sorry about that! – pbr Oct 01 '09 at 21:22
  • Thank you! As to "Identify for your particular system what the LOAD level is when you feel it's "acceptable" for other users to be adding more jobs", if the current load is not mainly caused by my running jobs then I would feel more free to use the rest CPUs, but if in the current load my running jobs take a fair amount then I probably have to give up using the rest CPUs. So do you still think I don't have to estimate the CPU usage of my total running jobs? – Tim Oct 01 '09 at 22:54
  • No, I really don't think it makes sense to distinguish between how much of the current load is from your jobs vs. other jobs. Consider the simplicity: your job tries to be a good citizen, and only submits itself if the load is below a level which you consider acceptable. The idea is NEVER to utilize all the remaining capacity; that leaves other future users in a bind. Ideally, all of the users would be using the same logic for this. – pbr Oct 02 '09 at 16:08
2

If you want to do this so it works on linux and OS X, you can do:

CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu)
tim_yates
  • 121
  • 3
1

You can get a rough estimate by running top, hitting 'U' and specifying your username, and then adding up the CPU utilization of your top few processes.

dmckee --- ex-moderator kitten
  • 7,817
  • 1
  • 29
  • 43
  • I like to know a bash command that can tell me the number, since I want to use it in a bash script. – Tim Oct 01 '09 at 18:10
  • @Tim `top` can run in batch mode with the -b switch. Combine it with the -n switch to get the number of iterations you need. –  Oct 01 '09 at 18:15
  • 1
    can you give an example of code for obtaining the number of cores used by all the jobs (processes) submit by a user? – Tim Oct 01 '09 at 18:20
  • @Tim Not off the bat. I'd either have to write such a script by parsing the `top` output or google to see if someone's already done it. However, I'm not convinced its worth the effort because I think I'll get different outputs for every run given the speed at which context switching occurs. Isn't the system load a good enough indicator? It effectively tells you how many virtual cores are required to handle the process queue (processes waiting on the CPU). –  Oct 01 '09 at 18:41
  • @ nagul: Yes system load is good and I also check that too. However some people keep telling me that I can not use up all the cores because who knows others will run their jobs soon. So I guess I have to limit the cores I am using. By the way, have you seen somewhere online scripts that check the CPU and memory load and dynamically submit jobs based on the usage? I have been looking them for so long. Thanks! – Tim Oct 01 '09 at 18:48
  • Trying to count cores is the *wrong* way to allocate CPU. That is the kernel's job. To tweak it, use `[re]nice`, or if that isn't enough get a queue manager like `pbs`. – dmckee --- ex-moderator kitten Oct 01 '09 at 19:21
  • Relevant link: http://superuser.com/questions/49020/linux-queuing-system – dmckee --- ex-moderator kitten Oct 01 '09 at 19:22
  • @Tim I'd suggest a different approach. Like EmmEff says, both cores *will* be used as long as system load is above 1.0. Have a script that parses `top` output to get %CPU utilization for your processes, and if the cumulative usage goes above a specified threshold, lowers their priority using `nice`. Or better still, if you absolutely have to make sure you don't hog all cores, bind *all* processes to one core: http://www.cyberciti.biz/tips/setting-processor-affinity-certain-task-or-process.html –  Oct 01 '09 at 19:27
  • @ nagul: Thanks! I finally get it done by parsing the top – Tim Oct 01 '09 at 20:19