Re: [PATCH] support for accessing CPU/core count (processor-count)

[Arthur Miller <arthur.miller@live.com>]

Omar Polo <op@omarpolo.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> Omar Polo <op@omarpolo.com> writes:
>>
>>> Arthur Miller <arthur.miller@live.com> writes:
>>>
>>>> [...]
>>>> And that is the beauty of having it as a Lisp function. You can just tweak it,
>>>> don't need to recompile entire Emacs :).
>>>
>>> I know I'm getting off-topic, but I just don't understand your point.  I
>>> don't see how spawning a bunch of commands, checking their return code
>>> and parsing their output is better than a couple of lines of C that do
>>> the right thing depending on the platform (decided at compile time!) and
>>> get directly an int.
>> I don't undestand what you don't udnerstand :-)
>>
>> I don't know my man; what do you mean with "bunch of commands" and how you would
>> achieve this for all platforms with "couple of lines of C".
>>
>> Here you have it; based on Andreas code from comp.el. I have just chagned part
>> shell command on gnu/linux since it can fail dependning on flags. Of course you
>> get an int back, "directly" :).
>>
>> #+begin_src emacs-lisp
>> (declare-function w32-get-nproc "w32.c")
>>
>> (defun processor-count ()
>>   (cond ((executable-find "nproc")
>>          (with-temp-buffer
>>            (call-process (executable-find "nproc") nil t nil)
>>            (string-to-number (buffer-string))))
>>         ((eq 'windows-nt system-type)
>>          (w32-get-nproc))
>>         ((eq 'berkeley-unix system-type)
>>          (string-to-number
>>           (shell-command-to-string "sysctl -n hw.ncpu")))
>>         (t 1)))
>> #+end_src
>>
>>
>> Compare to original patch in C, and tell me how is doing same in C better than
>> doing it in Lisp? Your Lisp routine should 
>> return an int directly. I don't see what is different there and what advantage C
>> will give you here; more than extra work to implement it and maintain it later on.
>>
>> To note here is that 'shell-command-to-string' is not recommended since it can
>> return "more", than what expected, depending on what flags are used to pass to
>> bash. I am not sure if it can also differ if user uses some other
>> shell. call-process should be fine. I don't have a bsd system to test though.
>>
>> I haven't used /proc/cpuinfo. It is a bit dependning on what is goal here: is it
>> to get number of "usable" cpus for spawning threads, or is it to get real
>> hardware number of cpus. The reason is that Emacs can run in a "restricted"
>> system such as a Docker environement where number of CPUs available can be
>> limited. /proc/cpuinfo (on linux kernel) records hardware number of cores but
>> nproc return "available" number. So you could have something like this:
>>
>> #+begin_src emacs-lisp
>> (declare-function w32-get-nproc "w32.c")
>>
>> (defun processor-hardware-count ()
>>   (cond ((eq 'gnu/linux system-type)
>>          (with-current-buffer (find-file-noselect "/proc/cpuinfo")
>>            (if (re-search-forward "cpu cores.*: " nil t)
>>                (string-to-number (current-word))
>>              1)))
>>         ((eq 'windows-nt system-type)
>>          (w32-get-nproc))
>>         ((eq 'berkeley-unix system-type)
>>          (string-to-number
>>           (shell-command-to-string "sysctl -n hw.ncpu")))
>>         (t 1)))
>> #+end_src
>>
>> Could be done with "-all" flag to nproc too, but I think reading /proc/cpuinfo
>> is faster.
>>
>>> I love lisp, don't get me wrong, and I actually prefer writing elisp
>>> rather than following the GNU C coding style (I love C too but GNU style
>>> hurts my eyes.)
>>
>> Trust me; if anyone I always vote for doing it in C; but this one is probably
>> not worth doing in C. I have no idea how suggested posix sysconf deals with
>> restricted environements either.
>>
>>> Sure, checking the number of cpus is not something that is done a lot,
>>> and I can't imagine a situation where it would be a bottleneck, but on
>>> the other hand, for the same argument, it's not something that needs to
>>> be tweaked often
>>
>> Do you want hardware count; logical cores (think hyperthreading); should it work
>> in restricted environments? Quite a few things to take into consideration, isn't
>> it?
>>
>> Hope you understand what I mean better after examples. Something tells me you
>> won't agree :-), but that is OK. I just present my opinion.
>
> I don't really want to start a pointless thread, so I hope I didn't
> sound annoying.  If that's the case, I'm sorry.
Neither do I :). I was just trying to explain why I reason the way I do.

> I kind of get your point, and as I sad before, I don't have opinions on
> this particular case.
>
> I'm still not sure how C can be more difficult to maintain than an
> elisp, as to my eyes they're equal.

You can just open a lisp file in Emacs, hack it and eval the function; no need
to recompile and restart Emacs. Also the code is easier to read, type etc. In my
eyes it is a winner for the maintance.

> in most cases, I won't be ever writing a major mode in C for example,
> but this is not one of those IMHO).  But I've never really contributed
> something significant to Emacs, and I spend almost all my free time
> hacking in C, so I'm kinda biased ;-)

I had fingers in one patch to dired.c and in some smaller lisp patches, but
nothing of sifnicance either. I really contribute mostly discussions and rarely
code. I hope to change it in the future .. :)

> But I'd like to add a small correction to your example.  The sysctl is
> not correct on OpenBSD (and maybe NetBSD too?  I can't check.)  It
> should read
>
> (shell-command-to-string "sysctl -n hw.ncpuonline || sysctl -n hw.ncp")
It wasn't my; I copy that from comp.el; please report it as a bug to Andrea so
he can correct it. I don't have access to neither *bsd nor mac, so I can't tell.

I also think it should be wrapped into call-process rather than
shell-command-to-string, since shell-command-to-string lumps stdout and stderr
into same buffer, which might result in erronous output. call-process seems to
avoid that.

> or something equivalent, please refer to my reply to the OP for the
> HW_NCPUONLINE vs HW_NCPU on OpenBSD.
>
> Cheers :)

Cheers!