Hi Guix! I was looking to pause a long build today and asked on IRC how to accomplish pause/resume. It seems this is possible already with the following: kill --signal SIGSTOP|SIGCONT {pids-of-build-process-tree} There is already a command to list the processes associated to guix commands: guix processes. Perhaps pause/resume can be a subcommand or set of flags to guix processes. The following is the first thing that comes to mind: guix processes --pause package-name ... --resume package-name ... What do you think? Thanks! - John
Hi,
John Soo <jsoo1@asu.edu> skribis:
> I was looking to pause a long build today and asked on IRC how to
> accomplish pause/resume. It seems this is possible already with the
> following:
>
> kill --signal SIGSTOP|SIGCONT {pids-of-build-process-tree}
>
> There is already a command to list the processes associated to guix
> commands: guix processes. Perhaps pause/resume can be a subcommand or
> set of flags to guix processes. The following is the first thing that
> comes to mind:
>
> guix processes --pause package-name ... --resume package-name ...
>
> What do you think?
First, note that the daemon is unaware of “packages”, it only knows
about “derivations”.
Second, ‘guix processes’ is nice but it uses low-level heuristics to
determine what daemon sessions are open, what their clients are, and
what they’re building; it resorts to heuristics because the daemon as it
stands doesn’t have a way to communicate its current state. It works
well in practice, but still I wouldn’t go too far building around it.
Last, you’d need to send SIGTSTP to the whole process group of the
build, like so (I think, haven’t tried):
sudo kill -TSTP -123
where 123 is the “SessionPID” shown by ‘guix processes’. However, doing
so may affect build results: processes in the build environment might
handle SIGTSTP specially, which can have side effects. It’s an
observable action.
Conclusion: I don’t think we can implement this reliably.
HTH!
Ludo’.
Hello! I want to preface all this by saying this is not a huge priority to me. pause/resume would just be a nice quality of life improvement. Ludovic Courtès <ludo@gnu.org> writes: > First, note that the daemon is unaware of “packages”, it only knows > about “derivations”. Agreed. I was thinking mostly about the best interface for a user. But if derivations are too low-level I can see not being able to use the package name. > Second, ‘guix processes’ is nice but it uses low-level heuristics to > determine what daemon sessions are open, what their clients are, and > what they’re building; it resorts to heuristics because the daemon as it > stands doesn’t have a way to communicate its current state. It works > well in practice, but still I wouldn’t go too far building around it. > > Last, you’d need to send SIGTSTP to the whole process group of the > build, like so (I think, haven’t tried): > > sudo kill -TSTP -123 Ah right, I got that advice on IRC also but I don't actually know how to do it. Thanks! > where 123 is the “SessionPID” shown by ‘guix processes’. However, doing > so may affect build results: processes in the build environment might > handle SIGTSTP specially, which can have side effects. It’s an > observable action. I thought of the side effects after sending the email. Makes sense to me. Does that mean it is not worth including? Given that the source and state of a build do not change while paused, if a build tool changes its outputs after paused/resumed is it still reproducible? The other part that would make it difficult is that maybe different build-systems respond differently to different signals. A simpler to implement alternative might just be to provide a way to send an arbitrary signal to the process tree. Given some builds might respond un-reproducibly to it, a warning could be provided. In any case, such a subcommand not a high priority to me as a user, though sometimes I long to pause guix build ungoogled-chromium. Thanks again, John
[-- Attachment #1: Type: text/plain, Size: 1137 bytes --] Ludo', Ludovic Courtès 写道: > First, note that the daemon is unaware of “packages”, it only > knows > about “derivations”. Derivations have a (file) name, which can be matched with a regex allowing one to, say, ‘pause libreoffice’. It works in practice. I do this often & it's *extremely* convenient. Sometimes even necessary, because I have a habit of starting too many builds ;-) It's nice not to lose 6h of work. Not every handy hack needs to be upstreamed though. ‘It's fugly’ is a strong argument in Guixland, and I like that. However, this is FUD: > Last, you’d need to send SIGTSTP to the whole process group of > the > build, like so (I think, haven’t tried): > > sudo kill -TSTP -123 > > where 123 is the “SessionPID” shown by ‘guix processes’. > However, doing > so may affect build results: processes in the build environment > might > handle SIGTSTP specially, which can have side effects. It’s an > observable action. Kind regards, T G-R, now thinking about acronyms like ‘CRIU’ and what a next-level hack would look like... [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --]
Hello Tobias :), Tobias Geerinckx-Rice <me@tobias.gr> writes: > Ludo', > > Ludovic Courtès 写道: >> First, note that the daemon is unaware of “packages”, it only knows >> about “derivations”. > > Derivations have a (file) name, which can be matched with a regex > allowing one to, say, ‘pause libreoffice’. It works in practice. > I do this often & it's *extremely* convenient. This feels close to little sed/awk pipelines. Which is not to be entirely dismissive. I like the compositionality of these tools. In fact I mentioned earlier that it might be good to send arbitrary signals. But why not let kill (shell or scheme) do that? All we would need is to filter and format pids in a composable way (on the scheme side and the shell side). That has the benefits of remaining agnostic on side effects in builds (let the user decide what they are comfortable with) and being more composable. Maybe flags like this would be enough: guix processes --session=<derivation-regex> ... to get something like 5555 1212 343434 ... > Sometimes even necessary, because I have a habit of starting too many > builds ;-) It's nice not to lose 6h of work. Indeed. This already saved me hours after learning it yesterday. > Not every handy hack needs to be upstreamed though. ‘It's fugly’ is a > strong argument in Guixland, and I like that. Same. > T G-R, now thinking about acronyms like ‘CRIU’ and what a next-level > hack would look like... Yeah seems like functional build systems would ideally be order-independent, pause/resumable, and the build steps time-travelable themselves. But that's a cool aside for now. Thanks for the tips! John
[-- Attachment #1: Type: text/plain, Size: 831 bytes --] After playing around with the daemon I agree that it plays a little too loose with processes to make this upstream material ATM (groups? shepherd? *shrug*). To close: Tobias Geerinckx-Rice 写道: > However, this is FUD: > >> Last, you’d need to send SIGTSTP to the whole process group of >> the >> build, like so (I think, haven’t tried): Re-reading this I wonder if you misread SIGSTOP as SIGTSTP? The two are not related. SIGSTOP cannot be handled or blocked. It's possible to *detect*, like one can detect almost anything because, but you'd be going out of your way to do so. I still think the concern is oddly specific & theoretical. Build processes *could* be using rowhammer to deliberately crash if you're browsing Reddit, too, and who could blame them. Kind regards, T G-R [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --]
After further review, I realize that guix processes already formats for recutils. I never think to reach for that tool, but it seems good.
After even further investigation, Does guix processes output the desired rec format? It seems hard to select the child process pid: ChildProcess: 16923: guile --no-auto-compile -L /gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import /gnu/store/mh1fkn1d9c9mg6hihxvjngxmn3qjmp38-ungoogled-chromium-86.0.4240.111-0.c34a56d-guile-builder ChildProcess: 31859: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real substitute --query ChildProcess: 31869: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real offload x86_64-linux 0 1 0 After perusing info recutils some, I expected the output to be more like: ChildProcessPID: 16923 ChildProcessCommand: guile --no-auto-compile -L /gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import /gnu/store/mh1fkn1d9c9mg6hihxvjngxmn3qjmp38-ungoogled-chromium-86.0.4240.111-0.c34a56d-guile-builder ChildProcessPID: 31859 ChildProcessCommand: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real substitute --query ChildProcessPID: 31869 ChildProcessCommand: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real offload x86_64-linux 0 1 0 But even that does not capture that the child processes all belong to the session. I must be missing something about the rec format or querying. Any clues? Thanks, John
Hi all, On +2020-11-03 14:53:07 +0100, Ludovic Courtès wrote: > Hi, > > John Soo <jsoo1@asu.edu> skribis: > > > I was looking to pause a long build today and asked on IRC how to > > accomplish pause/resume. It seems this is possible already with the > > following: > > > > kill --signal SIGSTOP|SIGCONT {pids-of-build-process-tree} > > > > There is already a command to list the processes associated to guix > > commands: guix processes. Perhaps pause/resume can be a subcommand or > > set of flags to guix processes. The following is the first thing that > > comes to mind: > > > > guix processes --pause package-name ... --resume package-name ... > > > > What do you think? > > First, note that the daemon is unaware of “packages”, it only knows > about “derivations”. > What if you turned the problem inside out, and made the derivation volunteer to be paused (at sensible places in its progress)? I.e., if you defined part of a given derivation "hash-prefixed-derivation-file-name-in-question.drv" to do a check for the existence of /var/guix/daemon-ctl/hash-prefixed-derivation-file-name-in-question.drv.sfx (.sfx appended to signify special effects :) ISTM that could open the door for some easy hacks, (and probably some dangers to watch for :) E.g. a proof of concept might be just to sleep 6 seconds (say) and repeat sleep/check until the file disappears. IWG this should not change anything for non-volunteering derivations other than the load-relief of not running the sleeping process(es). Then the person wanting to pause the derivation "hash-prefixed-derivation-file-name-in-question.drv" could do so simply by touch /var/guix/daemon-ctl/hash-prefixed-derivation-file-name-in-question.drv.sfx and deleting it when wanting to allow it to continue. Later, if that works, .sfx files could have content, for as yet unimagined purposes ;) [...] > > Conclusion: I don’t think we can implement this reliably. > IDK from the outside, but inside-out, WDYT? > HTH! > > Ludo’. > -- Regards, Bengt Richter
[-- Attachment #1: Type: text/plain, Size: 384 bytes --] Hello Guix, I just sent a patch to normalize the output of processes with bug number 44460. The allows me to compose recutils with kill to get the desired effect of pausing all process trees for the things I want without any convoluted implementation. I think I would be satisfied with such a patch and nothing else. Thanks again! John [-- Attachment #2: Type: text/html, Size: 557 bytes --]
Hi,
John Soo <jsoo1@asu.edu> skribis:
> This feels close to little sed/awk pipelines. Which is not to be
> entirely dismissive. I like the compositionality of these tools. In
> fact I mentioned earlier that it might be good to send arbitrary
> signals. But why not let kill (shell or scheme) do that? All we would
> need is to filter and format pids in a composable way (on the scheme
> side and the shell side). That has the benefits of remaining agnostic on
> side effects in builds (let the user decide what they are comfortable
> with) and being more composable.
>
> Maybe flags like this would be enough:
>
> guix processes --session=<derivation-regex> ...
>
> to get something like
>
> 5555
> 1212
> 343434
> ...
You can filter by piping ‘guix processes’ output through ‘recsel’:
--8<---------------cut here---------------start------------->8---
$ sudo guix processes | recsel -p SessionPID,LockHeld -e 'LockHeld ~ "chromium"'
SessionPID: 31410
LockHeld: /gnu/store/kdsp1pjj6znaxzs3d0vfwdcddc436g7f-ungoogled-chromium-86.0.4240.183-0.b68e17f.lock
SessionPID: 3455
LockHeld: /gnu/store/bhy3c5damrpzx7hdp8bam1lk2rk7789r-ungoogled-chromium-86.0.4240.183-0.b68e17f.lock
--8<---------------cut here---------------end--------------->8---
HTH,
Ludo’.
Hi,
John Soo <jsoo1@asu.edu> skribis:
> After even further investigation,
>
> Does guix processes output the desired rec format? It seems hard to
> select the child process pid:
>
> ChildProcess: 16923: guile --no-auto-compile -L /gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import /gnu/store/mh1fkn1d9c9mg6hihxvjngxmn3qjmp38-ungoogled-chromium-86.0.4240.111-0.c34a56d-guile-builder
> ChildProcess: 31859: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real substitute --query
> ChildProcess: 31869: /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4/bin/guile \ /gnu/store/xi3jc6476vyv2vmcsfa1xkpqbxp1apk6-guix-1.1.0-27.1c21468/bin/.guix-real offload x86_64-linux 0 1 0
>
> After perusing info recutils some, I expected the output to be more
> like:
>
> ChildProcessPID: 16923
> ChildProcessCommand: guile --no-auto-compile -L /gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import /gnu/store/mh1fkn1d9c9mg6hihxvjngxmn3qjmp38-ungoogled-chromium-86.0.4240.111-0.c34a56d-guile-builder
Yes, that would be nicer.
However, note that if you fiddle with child processes, your warranty is
void! :-)
You’d be interfering with build processes, thus potentially changing
their result in an uncontrolled way.
Ludo’.
Hi Ludovic, Ludovic Courtès <ludo@gnu.org> writes: > Last, you’d need to send SIGTSTP to the whole process group of the > build, like so (I think, haven’t tried): > > sudo kill -TSTP -123 > > where 123 is the “SessionPID” shown by ‘guix processes’. However, doing > so may affect build results: processes in the build environment might > handle SIGTSTP specially, which can have side effects. It’s an > observable action. What's the rationale for using SIGTSTP instead of SIGSTOP here? FWIW, on a few occasions I've paused builds by sending SIGSTOP to the relevant process group, and later SIGCONT, and it has worked for me. As I recall, I've done this while building rust, webkitgtk, and icecat. However, I suspect that if I paused a build while running tests, the test suite might ultimately fail due to a "timeout". > Conclusion: I don’t think we can implement this reliably. Agreed. Thanks, Mark
Hello everyone, Ludovic Courtès <ludo@gnu.org> writes: >> After perusing info recutils some, I expected the output to be more >> like: >> >> ChildProcessPID: 16923 >> ChildProcessCommand: guile --no-auto-compile -L >> /gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import >> /gnu/store/mh1fkn1d9c9mg6hihxvjngxmn3qjmp38-ungoogled-chromium-86.0.4240.111-0.c34a56d-guile-builder > > Yes, that would be nicer. I sent in patch https://issues.guix.gnu.org/issue/44460 to address it. With that I don't see any reason to consider a subcommand. > However, note that if you fiddle with child processes, your warranty is > void! :-) Definitely agreed. > You’d be interfering with build processes, thus potentially changing > their result in an uncontrolled way. Like Mark said, I have seen tests potentially time out, and I have to agree. I am done discussing a subcommand. I'll shift attention to #44460. Thanks for listening to little old me :). Have a nice weekend, John
Hi Mark,
Mark H Weaver <mhw@netris.org> skribis:
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Last, you’d need to send SIGTSTP to the whole process group of the
>> build, like so (I think, haven’t tried):
>>
>> sudo kill -TSTP -123
>>
>> where 123 is the “SessionPID” shown by ‘guix processes’. However, doing
>> so may affect build results: processes in the build environment might
>> handle SIGTSTP specially, which can have side effects. It’s an
>> observable action.
>
> What's the rationale for using SIGTSTP instead of SIGSTOP here?
No good reason; SIGSTOP is safer because it cannot be handled (info
"(libc) Job Control Signals"), and thus my argument above is moot.
Thanks for the correction!
Ludo’.