all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Problems with GNU Parallel and ulimit
@ 2017-08-14  2:31 Arun Isaac
  2017-08-14  2:46 ` Ben Woodcroft
  0 siblings, 1 reply; 11+ messages in thread
From: Arun Isaac @ 2017-08-14  2:31 UTC (permalink / raw)
  To: help-guix@gnu.org


Does `parallel' work for anybody? When I run

$ ls | parallel echo

I get the following error message, and nothing happens.

parallel: Warning: No more processes: Decreasing number of running jobs to 1.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Error: No more processes: cannot run a single job. Something is wrong at file-name.

Any idea about what's happening?

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-14  2:31 Arun Isaac
@ 2017-08-14  2:46 ` Ben Woodcroft
  2017-08-14 16:14   ` Leo Famulari
  0 siblings, 1 reply; 11+ messages in thread
From: Ben Woodcroft @ 2017-08-14  2:46 UTC (permalink / raw)
  To: Arun Isaac, help-guix@gnu.org

On 14/08/17 12:31, Arun Isaac wrote:
> Does `parallel' work for anybody? When I run
>
> $ ls | parallel echo
>
> I get the following error message, and nothing happens.
It works fine for me using Guix on a foreign OS, and has since I can 
remember.
ben

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-14  2:46 ` Ben Woodcroft
@ 2017-08-14 16:14   ` Leo Famulari
  2017-08-14 18:44     ` Arun Isaac
  0 siblings, 1 reply; 11+ messages in thread
From: Leo Famulari @ 2017-08-14 16:14 UTC (permalink / raw)
  To: Ben Woodcroft; +Cc: help-guix@gnu.org

[-- Attachment #1: Type: text/plain, Size: 415 bytes --]

On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
> On 14/08/17 12:31, Arun Isaac wrote:
> > Does `parallel' work for anybody? When I run
> > 
> > $ ls | parallel echo
> > 
> > I get the following error message, and nothing happens.
> It works fine for me using Guix on a foreign OS, and has since I can
> remember.
> ben

Same here. Arun, can you give some more details about your system?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-14 16:14   ` Leo Famulari
@ 2017-08-14 18:44     ` Arun Isaac
  2017-08-14 19:16       ` Marius Bakke
  2017-08-14 20:07       ` myglc2
  0 siblings, 2 replies; 11+ messages in thread
From: Arun Isaac @ 2017-08-14 18:44 UTC (permalink / raw)
  To: Leo Famulari; +Cc: help-guix@gnu.org


Leo Famulari writes:

> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
>> On 14/08/17 12:31, Arun Isaac wrote:
>> > Does `parallel' work for anybody? When I run
>> > 
>> > $ ls | parallel echo
>> > 
>> > I get the following error message, and nothing happens.
>> It works fine for me using Guix on a foreign OS, and has since I can
>> remember.
>> ben
>
> Same here. Arun, can you give some more details about your system?

This is probably a problem only on GuixSD systems. Could you try on
GuixSD instead of using Guix on a foreign distro?

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-14 18:44     ` Arun Isaac
@ 2017-08-14 19:16       ` Marius Bakke
  2017-08-14 20:07       ` myglc2
  1 sibling, 0 replies; 11+ messages in thread
From: Marius Bakke @ 2017-08-14 19:16 UTC (permalink / raw)
  To: Arun Isaac, Leo Famulari; +Cc: help-guix@gnu.org

[-- Attachment #1: Type: text/plain, Size: 780 bytes --]

Arun Isaac <arunisaac@systemreboot.net> writes:

> Leo Famulari writes:
>
>> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
>>> On 14/08/17 12:31, Arun Isaac wrote:
>>> > Does `parallel' work for anybody? When I run
>>> > 
>>> > $ ls | parallel echo
>>> > 
>>> > I get the following error message, and nothing happens.
>>> It works fine for me using Guix on a foreign OS, and has since I can
>>> remember.
>>> ben
>>
>> Same here. Arun, can you give some more details about your system?
>
> This is probably a problem only on GuixSD systems. Could you try on
> GuixSD instead of using Guix on a foreign distro?

I tested it on GuixSD, works fine. Are you trying the command in a
folder with > 1024 files perhaps? What does `ulimit -n` print?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-14 18:44     ` Arun Isaac
  2017-08-14 19:16       ` Marius Bakke
@ 2017-08-14 20:07       ` myglc2
  1 sibling, 0 replies; 11+ messages in thread
From: myglc2 @ 2017-08-14 20:07 UTC (permalink / raw)
  To: Arun Isaac; +Cc: help-guix@gnu.org

On 08/15/2017 at 00:14 Arun Isaac writes:

> Leo Famulari writes:
>
>> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
>>> On 14/08/17 12:31, Arun Isaac wrote:
>>> > Does `parallel' work for anybody? When I run
>>> > 
>>> > $ ls | parallel echo
>>> > 
>>> > I get the following error message, and nothing happens.
>>> It works fine for me using Guix on a foreign OS, and has since I can
>>> remember.
>>> ben
>>
>> Same here. Arun, can you give some more details about your system?
>
> This is probably a problem only on GuixSD systems. Could you try on
> GuixSD instead of using Guix on a foreign distro?

I think you are correct. I run GuixSD and, after installing parallel I
get ...

***
g1@g1 ~$ ls | parallel echo
Academic tradition requires you to cite works you base your article on.
If you use programs that use GNU Parallel to process data for an article in a
scientific publication, please cite:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
  ;login: The USENIX Magazine, February 2011:42-47.

This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.

To silence this citation notice: run 'parallel --citation'.

parallel: Warning: No more processes: Decreasing number of running jobs to 7.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 6.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 5.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 4.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 3.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 2.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Warning: No more processes: Decreasing number of running jobs to 1.
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Error: No more processes: cannot run a single job. Something is wrong at bin.
g1@g1 ~$ guix --version
guix (GNU Guix) 0.13.0.1765-9ef59
Copyright (C) 2017 the Guix authors
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
g1@g1 ~$ 
***

I am not a parallel user so I can't comment further.

HTH - George

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
@ 2017-08-15  4:41 ericbavier
  0 siblings, 0 replies; 11+ messages in thread
From: ericbavier @ 2017-08-15  4:41 UTC (permalink / raw)
  To: myglc2

> On 08/15/2017 at 00:14 Arun Isaac writes:
> 
>> Leo Famulari writes:
>>
>>> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
>>>> On 14/08/17 12:31, Arun Isaac wrote:
>>>> > Does `parallel' work for anybody? When I run
>>>> > 
>>>> > $ ls | parallel echo
>>>> > 
>>>> > I get the following error message, and nothing happens.
>>>> It works fine for me using Guix on a foreign OS, and has since I can
>>>> remember.
>>>> ben
>>>
>>> Same here. Arun, can you give some more details about your system?
>>
>> This is probably a problem only on GuixSD systems. Could you try on
>> GuixSD instead of using Guix on a foreign distro?
> 
> I think you are correct. I run GuixSD and, after installing parallel I
> get ...
> 
> ***
> g1@g1 ~$ ls | parallel echo
> ...
> parallel: Warning: No more processes: Decreasing number of running jobs to 7.
> parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.

The problem appears if you do not have 'perl' available in $PATH.  GNU parallel calls it recursively for a few tasks:

$ guix package -I | grep "perl"
$ echo $?
1
$ parallel echo ::: 1 2 3
...
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Error: No more processes: cannot run a single job. Something is wrong at 1.
$ guix environment --ad-hoc perl
[dev]$ parallel echo ::: 1 2 3
1
2
3

There is some path patching that goes on in the package recipe, but it must have gone out of sync in latest versions.  The 'post-install-test' phase wouldn't catch this particular error without '(setenv "PATH" "")'.

I wouldn't feel quite comfortable doing a "blind" update of the regex, since some mentions of "perl" I think are meant to be executed by remote hosts and so shouldn't be patched to absolute store references.

It might be useful to get Ole's opinion on this.

`~Eric

> I am not a parallel user so I can't comment further.
> 
> HTH - George

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
@ 2017-08-15  4:58 Eric Bavier
  2017-08-17 15:00 ` Arun Isaac
  2018-06-21 22:55 ` Clément Lassieur
  0 siblings, 2 replies; 11+ messages in thread
From: Eric Bavier @ 2017-08-15  4:58 UTC (permalink / raw)
  To: help-guix

Sending response to the list this time...

> Arun Isaac <address@hidden> writes:
> 
> > Leo Famulari writes:
> >
> >> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
> >>> On 14/08/17 12:31, Arun Isaac wrote:
> >>> > Does `parallel' work for anybody? When I run
> >>> > 
> >>> > $ ls | parallel echo
> >>> > 
> >>> > I get the following error message, and nothing happens.
> >>> It works fine for me using Guix on a foreign OS, and has since I
> >>> can remember.
> >>> ben
> >>
> >> Same here. Arun, can you give some more details about your system?
> >
> > This is probably a problem only on GuixSD systems. Could you try on
> > GuixSD instead of using Guix on a foreign distro?
> 
> I tested it on GuixSD, works fine. Are you trying the command in a
> folder with > 1024 files perhaps? What does `ulimit -n` print?

The problem appears if you do not have 'perl' available in $PATH. GNU
parallel calls it recursively for a few tasks:

$ guix package -I | grep "perl"
$ echo $?
1
$ parallel echo ::: 1 2 3
...
parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
parallel: Error: No more processes: cannot run a single job. Something is wrong at 1.
$ guix environment --ad-hoc perl
[dev]$ parallel echo ::: 1 2 3
1
2
3

There is some path patching that goes on in the package recipe, but it
must have gone out of sync in latest versions.  The 'post-install-test'
phase wouldn't catch this particular error without '(setenv "PATH" "")'.

I wouldn't feel quite comfortable doing a "blind" update of the regex,
since some mentions of "perl" I think are meant to be executed by
remote hosts and so shouldn't be patched to absolute store references.

It might be useful to get Ole's opinion on this.

`~Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-15  4:58 Problems with GNU Parallel and ulimit Eric Bavier
@ 2017-08-17 15:00 ` Arun Isaac
  2018-06-21 22:55 ` Clément Lassieur
  1 sibling, 0 replies; 11+ messages in thread
From: Arun Isaac @ 2017-08-17 15:00 UTC (permalink / raw)
  To: Eric Bavier; +Cc: help-guix


Eric Bavier writes:

> The problem appears if you do not have 'perl' available in $PATH. GNU
> parallel calls it recursively for a few tasks:

This is correct. I have verified it. When I have perl in my PATH, GNU
parallel works fine.

> There is some path patching that goes on in the package recipe, but it
> must have gone out of sync in latest versions.  The 'post-install-test'
> phase wouldn't catch this particular error without '(setenv "PATH" "")'.
>
> I wouldn't feel quite comfortable doing a "blind" update of the regex,
> since some mentions of "perl" I think are meant to be executed by
> remote hosts and so shouldn't be patched to absolute store references.

Why does parallel need to execute perl on remote hosts?

> It might be useful to get Ole's opinion on this.

Yes, could you please do this?

Thanks!

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2017-08-15  4:58 Problems with GNU Parallel and ulimit Eric Bavier
  2017-08-17 15:00 ` Arun Isaac
@ 2018-06-21 22:55 ` Clément Lassieur
  2018-08-11 10:04   ` Arun Isaac
  1 sibling, 1 reply; 11+ messages in thread
From: Clément Lassieur @ 2018-06-21 22:55 UTC (permalink / raw)
  To: Eric Bavier; +Cc: help-guix

Eric Bavier <ericbavier@centurylink.net> writes:

> Sending response to the list this time...
>
>> Arun Isaac <address@hidden> writes:
>> 
>> > Leo Famulari writes:
>> >
>> >> On Mon, Aug 14, 2017 at 12:46:25PM +1000, Ben Woodcroft wrote:
>> >>> On 14/08/17 12:31, Arun Isaac wrote:
>> >>> > Does `parallel' work for anybody? When I run
>> >>> > 
>> >>> > $ ls | parallel echo
>> >>> > 
>> >>> > I get the following error message, and nothing happens.
>> >>> It works fine for me using Guix on a foreign OS, and has since I
>> >>> can remember.
>> >>> ben
>> >>
>> >> Same here. Arun, can you give some more details about your system?
>> >
>> > This is probably a problem only on GuixSD systems. Could you try on
>> > GuixSD instead of using Guix on a foreign distro?
>> 
>> I tested it on GuixSD, works fine. Are you trying the command in a
>> folder with > 1024 files perhaps? What does `ulimit -n` print?
>
> The problem appears if you do not have 'perl' available in $PATH. GNU
> parallel calls it recursively for a few tasks:
>
> $ guix package -I | grep "perl"
> $ echo $?
> 1
> $ parallel echo ::: 1 2 3
> ...
> parallel: Warning: Raising ulimit -u or /etc/security/limits.conf may help.
> parallel: Error: No more processes: cannot run a single job. Something is wrong at 1.
> $ guix environment --ad-hoc perl
> [dev]$ parallel echo ::: 1 2 3
> 1
> 2
> 3
>
> There is some path patching that goes on in the package recipe, but it
> must have gone out of sync in latest versions.  The 'post-install-test'
> phase wouldn't catch this particular error without '(setenv "PATH" "")'.
>
> I wouldn't feel quite comfortable doing a "blind" update of the regex,
> since some mentions of "perl" I think are meant to be executed by
> remote hosts and so shouldn't be patched to absolute store references.

Hi,

This is fixed by 4e7950fdce5b83db018c44a80ec4a4059612d716 (it adds a
wrap-program phase).

Clément

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Problems with GNU Parallel and ulimit
  2018-06-21 22:55 ` Clément Lassieur
@ 2018-08-11 10:04   ` Arun Isaac
  0 siblings, 0 replies; 11+ messages in thread
From: Arun Isaac @ 2018-08-11 10:04 UTC (permalink / raw)
  To: Clément Lassieur; +Cc: help-guix


> This is fixed by 4e7950fdce5b83db018c44a80ec4a4059612d716 (it adds a
> wrap-program phase).

I just verified this. It works. Thanks! :-)

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2018-08-11 10:04 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-15  4:58 Problems with GNU Parallel and ulimit Eric Bavier
2017-08-17 15:00 ` Arun Isaac
2018-06-21 22:55 ` Clément Lassieur
2018-08-11 10:04   ` Arun Isaac
  -- strict thread matches above, loose matches on Subject: below --
2017-08-15  4:41 ericbavier
2017-08-14  2:31 Arun Isaac
2017-08-14  2:46 ` Ben Woodcroft
2017-08-14 16:14   ` Leo Famulari
2017-08-14 18:44     ` Arun Isaac
2017-08-14 19:16       ` Marius Bakke
2017-08-14 20:07       ` myglc2

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.