Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Nordlöw]

How can I, from within Emacs in a platform independent way, read out
the number of CPUs/cores my local machine has? I then believe it
should be convenient to give this argument to make (through the -j,--
jobs flag). Thus we will always get full utilization when compiling on
multi-CPU/core machines.

Thanks,
Nordlöw

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Peter Dyballa]

Am 11.09.2007 um 16:53 schrieb Nordlöw:

> How can I, from within Emacs in a platform independent way, read out
> the number of CPUs/cores my local machine has?

Apropos cpu? And then checking the manual pages? Reading a file in / 
proc file system might work, too.

--
Greetings

   Pete

You can learn many things from children.  How much patience you have,
for instance.
                 -- Franklin P. Jones

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Dieter Wilhelm]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

> Am 11.09.2007 um 16:53 schrieb Nordlöw:
>
>> How can I, from within Emacs in a platform independent way, read out
>> the number of CPUs/cores my local machine has?
>
> Apropos cpu? And then checking the manual pages? Reading a file in /
> proc file system might work, too.

I checked the following:

dieter@debby:~$ cat /proc/cpuinfo|grep "proc\|cores\|model"
processor	: 0
model		: 15
model name	: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz
cpu cores	: 2
processor	: 1
model		: 15
model name	: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz
cpu cores	: 2

That's a bit confusing, I thought I had *one* processor with *two*
cores and the content in /proc/cpuinfo claims two processor, 0 and 1
with two cores, respectively, where am I wrong?

-- 
    Best wishes

    H. Dieter Wilhelm
    Darmstadt, Germany

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Peter Dyballa]

Am 11.09.2007 um 22:20 schrieb Dieter Wilhelm:

> That's a bit confusing, I thought I had *one* processor with *two*
> cores and the content in /proc/cpuinfo claims two processor, 0 and 1
> with two cores, respectively, where am I wrong?

HT – hyper-threading (parallel pipelines and parallel memory buses).  
Many modern intel CPUs claim they are two (Xeons are four, or even  
eight, I think). They are right, somehow: mostly you can assume that  
in your case four commands are (can be) executed at the same time.  
For your PC 'make -j 4' will improve compilation time. So doing in bash

	cores=`grep cores /proc/cpuinfo | wc -l`
	if [ $cores -eq 0 ]; then cores=1; fi
	procs=`grep processor /proc/cpuinfo | wc -l`
	thrds=`expr $cores \* $procs`

or in (t)csh

	set cores=`grep cores /proc/cpuinfo | wc -l`
	if ($cores == 0) set cores=1
	set procs=`grep processor /proc/cpuinfo | wc -l`
	set thrds=`expr $cores \* $procs`

would determine how many compilation threads can be executed in  
$thrds. And it should also work when the CPU has no core ... Solaris,  
AIX, HP-UX, IRIX ... have their own commands.


Anyway, a modern GNU make is able to determine by itself what's best.  
This is done via 'make -j' without a limiting number.

--
Mit friedvollen Grüßen

   Pete

Der Bezug einer Zeitung oder der Kauf eines Buches gibt niemand das  
Recht, an die Verfasserinnen von Artikeln oder Büchern dreiste Briefe  
zu schreiben.
                                             (Friedrich Sieburg)

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Dieter Wilhelm]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

> Am 11.09.2007 um 22:20 schrieb Dieter Wilhelm:
>
>> That's a bit confusing, I thought I had *one* processor with *two*
>> cores and the content in /proc/cpuinfo claims two processor, 0 and 1
>> with two cores, respectively, where am I wrong?
>
> HT – hyper-threading (parallel pipelines and parallel memory buses).
> Many modern intel CPUs claim they are two (Xeons are four, or even
> eight, I think). They are right, somehow: mostly you can assume that
> in your case four commands are (can be) executed at the same time.

I see, thanks for the explanation. 

> For your PC 'make -j 4' will improve compilation time. So doing in
> bash
>
> 	cores=`grep cores /proc/cpuinfo | wc -l`
> 	if [ $cores -eq 0 ]; then cores=1; fi
> 	procs=`grep processor /proc/cpuinfo | wc -l`
> 	thrds=`expr $cores \* $procs`

Wait!  I'm not the guy which asked for this make related problem.  I'm
just a miscellaneous ignorant who sneaked into this threat, I'm sorry
for that.

>
> or in (t)csh
>
> 	set cores=`grep cores /proc/cpuinfo | wc -l`
> 	if ($cores == 0) set cores=1
> 	set procs=`grep processor /proc/cpuinfo | wc -l`
> 	set thrds=`expr $cores \* $procs`
>
> would determine how many compilation threads can be executed in
> $thrds. And it should also work when the CPU has no core ... Solaris,
> AIX, HP-UX, IRIX ... have their own commands.
>
>
> Anyway, a modern GNU make is able to determine by itself what's best.
> This is done via 'make -j' without a limiting number.

-- 
    Best wishes

    H. Dieter Wilhelm
    Darmstadt, Germany

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Peter Dyballa]

Am 12.09.2007 um 00:11 schrieb Dieter Wilhelm:

> Wait!  I'm not the guy which asked for this make related problem.  I'm
> just a miscellaneous ignorant who sneaked into this threat

It's just a problem with my 'respond all' button.


BTW, I remember that some Linux 'wc -l' was returning something like  
'<some SPCes><number>'. Such "numbers" can't be multiplied or added  
or subtracted or divided ... but they can be purified!

--
Mit friedvollen Grüßen

   Pete

Upgraded:  Didn't work the first time.

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Unknown]

Dieter Wilhelm, on 09/11/2007 04:20 PM said:
> 
> That's a bit confusing, I thought I had *one* processor with *two*
> cores and the content in /proc/cpuinfo claims two processor, 0 and 1
> with two cores, respectively, where am I wrong?
> 

That's the way it shows up - it's a dual core processor, and the
information is the same for them except for the core number.  If you had
two dual core CPUs in your system, you would see four entires in
/proc/cpuinfo.  This is normal, because a dual core system (mostly)
performs just as a dual processor system would.  It's more or less like
having two CPUs.

In fact, many utilities and programming languages will even say that you
have two CPUs when you really only have one dual-core CPU.

That having been said, the only cross-platform method I know of for
finding the number of cores (processors) on a system would be in
.NET/Mono, since that works whereever there is an implmementation of
.NET.  Other than that, the only way that I know is to look in the
/proc/cpuinfo file on Linux.

	-- Mike

-- 
Michael B. Trausch                                http://www.trausch.us/
Pidgin 2.1.1 and plugins for Ubuntu Feisty!
(And Thunderbird 2.0.0.6, too!)             http://www.trausch.us/pidgin

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Nordlöw]

On 12 Sep, 08:30, Michael Trausch <"mike|s/\\x40/\\./g;s/|.*|/\\x40/g;|
trausch"@us> wrote:
> Dieter Wilhelm, on 09/11/2007 04:20 PM said:
>
>
>
> > That's a bit confusing, I thought I had *one* processor with *two*
> > cores and the content in /proc/cpuinfo claims two processor, 0 and 1
> > with two cores, respectively, where am I wrong?
>
> That's the way it shows up - it's a dual core processor, and the
> information is the same for them except for the core number.  If you had
> two dual core CPUs in your system, you would see four entires in
> /proc/cpuinfo.  This is normal, because a dual core system (mostly)
> performs just as a dual processor system would.  It's more or less like
> having two CPUs.
>
> In fact, many utilities and programming languages will even say that you
> have two CPUs when you really only have one dual-core CPU.
>
> That having been said, the only cross-platform method I know of for
> finding the number of cores (processors) on a system would be in
> .NET/Mono, since that works whereever there is an implmementation of
> .NET.  Other than that, the only way that I know is to look in the
> /proc/cpuinfo file on Linux.

In C on Linux and Mac OS X the following detects number of CPUs:

#include <unistd.h>
#ifdef __APPLE__
#include <sys/sysctl.h>
#endif

/***
 * Detect the number of CPUs/Processors/Cores Online.
 * @return number of CPUs found.
 */
static inline int get_CPUmult(void)
{
  int num = 1;			/* default to one CPU */
#if defined(__APPLE__)
  size_t len = sizeof(num);
  int mib[2];
  mib[0] = CTL_HW;
  mib[1] = HW_NCPU;
  if (sysctl(mib, 2, &num, &len, 0, 0) < 0 ||
      len != sizeof(num)) {
    num = 1;
  }
#elif defined(_SC_NPROCESSORS_ONLN)
  num = sysconf(_SC_NPROCESSORS_ONLN);
#endif
  return num;
}


/Nordlöw

Re: Getting Number of CPU(-core)s and giving it as the --jobs argument to GNU Make

[Peter Dyballa]

Am 12.09.2007 um 10:20 schrieb Nordlöw:

> In C on Linux and Mac OS X the following detects number of CPUs:

Wouldn't

	sysctl -n hw.logicalcpu_max

in Mac OS X or Darwin be a bit simpler? (There are also \x18 
hw.activecpu, hw.physicalcpu, hw.physicalcpu_max, and hw.logicalcpu.)


--
Greetings

   Pete

A child of five could understand this!  Fetch me a child of five.