unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Move switch-symlinks to (guix build utils)
@ 2022-06-01  7:34 Arun Isaac
  2022-06-01 12:09 ` Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: Arun Isaac @ 2022-06-01  7:34 UTC (permalink / raw)
  To: Guix Devel


Hi Guix,

The switch-symlinks function from (guix utils) is often required in
activation-service G-expressions. But, only (guix build utils) and not
(guix utils) is available to such G-expressions. See, for example, the
pcscd-activation G-expression in (gnu services security-token) where
switch-symlinks gets
redefined. https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/security-token.scm#n70

Could we move switch-symlinks to (guix build utils)? Would such a change
require a core-updates cycle?

Thanks!
Arun


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-01  7:34 Move switch-symlinks to (guix build utils) Arun Isaac
@ 2022-06-01 12:09 ` Maxime Devos
  2022-06-02 15:06   ` Arun Isaac
  0 siblings, 1 reply; 13+ messages in thread
From: Maxime Devos @ 2022-06-01 12:09 UTC (permalink / raw)
  To: Arun Isaac, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

Arun Isaac schreef op wo 01-06-2022 om 13:04 [+0530]:
> Hi Guix,
> 
> The switch-symlinks function from (guix utils) is often required in
> activation-service G-expressions. But, only (guix build utils) and not
> (guix utils) is available to such G-expressions. See, for example, the
> pcscd-activation G-expression in (gnu services security-token) where
> switch-symlinks gets
> redefined. https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/security-token.scm#n70
> 
> Could we move switch-symlinks to (guix build utils)? Would such a change
> require a core-updates cycle?

To avoid a world-rebuild, you could for now make a module (guix build
symlinks) or such?  An alternative is (gnu build activation), but then
some (guix ...) modules would depend on (gnu ...).

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-01 12:09 ` Maxime Devos
@ 2022-06-02 15:06   ` Arun Isaac
  2022-06-02 15:13     ` Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: Arun Isaac @ 2022-06-02 15:06 UTC (permalink / raw)
  To: Maxime Devos, Guix Devel


Hi Maxime,

> To avoid a world-rebuild, you could for now make a module (guix build
> symlinks) or such?  An alternative is (gnu build activation), but then
> some (guix ...) modules would depend on (gnu ...).

I don't really mind waiting for the next core-updates cycle. This
problem has been around for a long time. It wouldn't hurt much to wait a
bit longer. If there is consensus on this change, I can send a
patch. WDYT?

Thanks,
Arun


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-02 15:06   ` Arun Isaac
@ 2022-06-02 15:13     ` Maxime Devos
  2022-06-03 16:38       ` Ludovic Courtès
  0 siblings, 1 reply; 13+ messages in thread
From: Maxime Devos @ 2022-06-02 15:13 UTC (permalink / raw)
  To: Arun Isaac, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

Arun Isaac schreef op do 02-06-2022 om 20:36 [+0530]:
> Hi Maxime,
> 
> > To avoid a world-rebuild, you could for now make a module (guix build
> > symlinks) or such?  An alternative is (gnu build activation), but then
> > some (guix ...) modules would depend on (gnu ...).
> 
> I don't really mind waiting for the next core-updates cycle. This
> problem has been around for a long time. It wouldn't hurt much to wait a
> bit longer. If there is consensus on this change, I can send a
> patch. WDYT?

Downside of (guix build utils): it makes it harder to modify switch-
symlinks.  I don't know if that's too much of a downside in practice. 
Also, there is a bug in switch-symlinks:

(use-modules (guix utils))
;; simulate an interrupted switch-symlinks (C-c, power interruption, ...)
(symlink "target" "link.new")
;; now try again
(switch-symlinks "link" "target")

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure symlink: Bestand bestaat al: "link.new"

If we move it to (guix build utils), I'd prefer the bug to be addressed first.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-02 15:13     ` Maxime Devos
@ 2022-06-03 16:38       ` Ludovic Courtès
  2022-06-03 17:09         ` Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: Ludovic Courtès @ 2022-06-03 16:38 UTC (permalink / raw)
  To: Maxime Devos; +Cc: Arun Isaac, Guix Devel

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Arun Isaac schreef op do 02-06-2022 om 20:36 [+0530]:
>> Hi Maxime,
>> 
>> > To avoid a world-rebuild, you could for now make a module (guix build
>> > symlinks) or such?  An alternative is (gnu build activation), but then
>> > some (guix ...) modules would depend on (gnu ...).
>> 
>> I don't really mind waiting for the next core-updates cycle. This
>> problem has been around for a long time. It wouldn't hurt much to wait a
>> bit longer. If there is consensus on this change, I can send a
>> patch. WDYT?
>
> Downside of (guix build utils): it makes it harder to modify switch-
> symlinks.  I don't know if that's too much of a downside in practice. 
> Also, there is a bug in switch-symlinks:
>
> (use-modules (guix utils))
> ;; simulate an interrupted switch-symlinks (C-c, power interruption, ...)
> (symlink "target" "link.new")
> ;; now try again
> (switch-symlinks "link" "target")
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure symlink: Bestand bestaat al: "link.new"

That’s EACCES?

> If we move it to (guix build utils), I'd prefer the bug to be addressed first.

Yes, better be cautious before “setting it in stone”.  Do you have a fix
in mind?

With this issue addressed, I think it’s fine to move to (guix build
utils) on ‘core-updates’.  We’ll have to run ‘make check’ to make sure
all users are updated, probably with #:select to clarify what binding
they need from (guix build utils).

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 16:38       ` Ludovic Courtès
@ 2022-06-03 17:09         ` Maxime Devos
  2022-06-03 19:30           ` Ludovic Courtès
                             ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Maxime Devos @ 2022-06-03 17:09 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Arun Isaac, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

Ludovic Courtès schreef op vr 03-06-2022 om 18:38 [+0200]:
> That’s EACCES?

It's EEXIST.

> > If we move it to (guix build utils), I'd prefer the bug to be addressed first.
> 
> Yes, better be cautious before “setting it in stone”.  Do you have a fix
> in mind?

Maybe replace

   (symlink target pivot)

by (symlink/remove-old target pivot)

where

(define (symlink/remove-old target link)
  "Make a symbolic link named LINK pointing to TARGET.
If LINK already exists, it will be removed first.
This is not an atomic operation."
  (catch 'system-error
    (lambda ()
      (symlink target link))
    (lambda stuff
      (if (= (system-error-errno stuff) EEXIST)
          (begin
            ;; remove old link and retry
            (delete-file link)
            (symlink/remove-old link target))
          (apply throw stuff)))))

?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 17:09         ` Maxime Devos
@ 2022-06-03 19:30           ` Ludovic Courtès
  2022-06-03 19:30           ` Ludovic Courtès
  2022-06-06 11:27           ` Arun Isaac
  2 siblings, 0 replies; 13+ messages in thread
From: Ludovic Courtès @ 2022-06-03 19:30 UTC (permalink / raw)
  To: Maxime Devos; +Cc: Arun Isaac, Guix Devel

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Maybe replace
>
>    (symlink target pivot)
>
> by (symlink/remove-old target pivot)
>
> where
>
> (define (symlink/remove-old target link)
>   "Make a symbolic link named LINK pointing to TARGET.
> If LINK already exists, it will be removed first.
> This is not an atomic operation."
>   (catch 'system-error
>     (lambda ()
>       (symlink target link))
>     (lambda stuff
>       (if (= (system-error-errno stuff) EEXIST)
>           (begin
>             ;; remove old link and retry
>             (delete-file link)
>             (symlink/remove-old link target))
>           (apply throw stuff)))))

Alright, SGTM (this procedure would be kept private).

So Arun, the floor is yours!  :-)

Ludo’.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 17:09         ` Maxime Devos
  2022-06-03 19:30           ` Ludovic Courtès
@ 2022-06-03 19:30           ` Ludovic Courtès
  2022-06-03 23:55             ` bokr
  2022-06-06 11:27           ` Arun Isaac
  2 siblings, 1 reply; 13+ messages in thread
From: Ludovic Courtès @ 2022-06-03 19:30 UTC (permalink / raw)
  To: Maxime Devos; +Cc: Arun Isaac, Guix Devel

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Maybe replace
>
>    (symlink target pivot)
>
> by (symlink/remove-old target pivot)
>
> where
>
> (define (symlink/remove-old target link)
>   "Make a symbolic link named LINK pointing to TARGET.
> If LINK already exists, it will be removed first.
> This is not an atomic operation."
>   (catch 'system-error
>     (lambda ()
>       (symlink target link))
>     (lambda stuff
>       (if (= (system-error-errno stuff) EEXIST)
>           (begin
>             ;; remove old link and retry
>             (delete-file link)
>             (symlink/remove-old link target))
>           (apply throw stuff)))))

Alright, SGTM (this procedure would be kept private).

So Arun, the floor is yours!  :-)

Ludo’.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 19:30           ` Ludovic Courtès
@ 2022-06-03 23:55             ` bokr
  2022-06-04 10:17               ` Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: bokr @ 2022-06-03 23:55 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Maxime Devos, Arun Isaac, Guix Devel

Hi,

I am not expert on kernel link internals, but if
you need/prefer atomic change to a specific link,
does my log [1] below suggest a way?

On +2022-06-03 21:30:39 +0200, Ludovic Courtès wrote:
> Hi,
> 
> Maxime Devos <maximedevos@telenet.be> skribis:
> 
> >, Maybe replace
> >
> >    (symlink target pivot)
> >
> > by (symlink/remove-old target pivot)
> >
> > where
> >
> > (define (symlink/remove-old target link)
> >   "Make a symbolic link named LINK pointing to TARGET.
> > If LINK already exists, it will be removed first.
> > This is not an atomic operation."
> >   (catch 'system-error
> >     (lambda ()
> >       (symlink target link))
> >     (lambda stuff
> >       (if (= (system-error-errno stuff) EEXIST)
> >           (begin
> >             ;; remove old link and retry
> >             (delete-file link)
> >             (symlink/remove-old link target))
> >           (apply throw stuff)))))
> 
> Alright, SGTM (this procedure would be kept private).
> 
> So Arun, the floor is yours!  :-)
> 
> Ludo’.
>

[0]: (some "<--= note" notes edited in)
BTW: notice that Size: is number of chars in TARGET name, not the LINK name
--8<---------------cut here---------------start------------->8---
$ ln -sT TARGET LINK
$ stat LINK
  File: LINK -> TARGET
  Size: 6               Blocks: 0          IO Block: 4096   symbolic link
Device: fe00h/65024d    Inode: 19401553    Links: 1   # <--= note inode number: mv will not change it
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/    bokr)   Gid: ( 1000/    bokr)
Access: 2022-06-04 00:57:47.680832044 +0200
Modify: 2022-06-04 00:57:47.680832044 +0200
Change: 2022-06-04 00:57:47.680832044 +0200
 Birth: -
$ mv -v LINK WAS-LINK
renamed 'LINK' -> 'WAS-LINK'
$ stat WAS-LINK
  File: WAS-LINK -> TARGET
  Size: 6               Blocks: 0          IO Block: 4096   symbolic link # <--= TARGET is still 6 chars :)
Device: fe00h/65024d    Inode: 19401553    Links: 1    # <--= note inode number: did not change with mv
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/    bokr)   Gid: ( 1000/    bokr)
Access: 2022-06-04 00:57:47.680832044 +0200
Modify: 2022-06-04 00:57:47.680832044 +0200
Change: 2022-06-04 00:58:33.617240104 +0200
 Birth: -
$ ln -sT NEW-TARGET LINK
$ stat LINK 
  File: LINK -> NEW-TARGET
  Size: 10              Blocks: 0          IO Block: 4096   symbolic link # <--= NEW-TARGET is 10 chars :)
Device: fe00h/65024d    Inode: 19401638    Links: 1    <--= note inode number: new for new use of LINK name
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/    bokr)   Gid: ( 1000/    bokr)
Access: 2022-06-04 01:00:10.630121441 +0200
Modify: 2022-06-04 01:00:10.630121441 +0200
Change: 2022-06-04 01:00:10.630121441 +0200
 Birth: -
$ rm -v WAS-LINK      <--= note no OS objection to removing renamed old LINK
removed 'WAS-LINK'
$ stat *LINK*
  File: LINK -> NEW-TARGET
  Size: 10              Blocks: 0          IO Block: 4096   symbolic link
Device: fe00h/65024d    Inode: 19401638    Links: 1
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/    bokr)   Gid: ( 1000/    bokr)
Access: 2022-06-04 01:00:10.630121441 +0200
Modify: 2022-06-04 01:00:10.630121441 +0200
Change: 2022-06-04 01:00:10.630121441 +0200
 Birth: -
$ 
--8<---------------cut here---------------end--------------->8---

I'm guessing the inode rewrite involved in mv -v LINK WAS-LINK should be atomic?
Maybe some kernels have to struggle without good hardware support, but still atomic?

Maybe a kernel insider will chime in?
HTH some way.
--
Regards,
Bengt Richter


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 23:55             ` bokr
@ 2022-06-04 10:17               ` Maxime Devos
  0 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-06-04 10:17 UTC (permalink / raw)
  To: bokr, Ludovic Courtès; +Cc: Arun Isaac, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 738 bytes --]

bokr@bokr.com schreef op za 04-06-2022 om 01:55 [+0200]:
> I am not expert on kernel link internals, but if
> you need/prefer atomic change to a specific link,
> does my log [1] below suggest a way?
> [...]

Only the replacing of an old by a new symlink needs to be atomic, and
this is already the case.  The only non-atomicity is that if
interrupted, an extra symlink [...].new can be created, but except for
tidyness this is not really a problem AFAICT.

Speaking of atomicity, I wonder if 'fsync'ing the symlink or the
directory is required?  E.g., is it possible after a power interruption
for the contents (= target) of the new symlink to be truncated without
doing a 'fsync' before the rename?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-03 17:09         ` Maxime Devos
  2022-06-03 19:30           ` Ludovic Courtès
  2022-06-03 19:30           ` Ludovic Courtès
@ 2022-06-06 11:27           ` Arun Isaac
  2022-06-06 12:20             ` Maxime Devos
  2 siblings, 1 reply; 13+ messages in thread
From: Arun Isaac @ 2022-06-06 11:27 UTC (permalink / raw)
  To: Maxime Devos, Ludovic Courtès; +Cc: Guix Devel


> Maybe replace
>
>    (symlink target pivot)
>
> by (symlink/remove-old target pivot)
>
> where
>
> (define (symlink/remove-old target link)
>   "Make a symbolic link named LINK pointing to TARGET.
> If LINK already exists, it will be removed first.
> This is not an atomic operation."
>   (catch 'system-error
>     (lambda ()
>       (symlink target link))
>     (lambda stuff
>       (if (= (system-error-errno stuff) EEXIST)
>           (begin
>             ;; remove old link and retry
>             (delete-file link)
>             (symlink/remove-old link target))
>           (apply throw stuff)))))
>
> ?

Could we implement symlink/remove-old without catch and throw? Something
like:

--8<---------------cut here---------------start------------->8---
(define (symlink/remove-old target link)
  "Make a symbolic link named LINK pointing to TARGET. If LINK already
exists, it will be removed first.  This is not an atomic operation."
  (when (file-exists? link)
    (delete-file link))
  (symlink target link))
--8<---------------cut here---------------end--------------->8---

We might also inline this into switch-symlinks like so.

--8<---------------cut here---------------start------------->8---
(define (switch-symlinks link target)
  "Atomically switch LINK, a symbolic link, to point to TARGET.  Works
both when LINK already exists and when it does not."
  (let ((pivot (string-append link ".new")))
    ;; Delete pivot link in case it already exists. This can happen if
    ;; a previous switch-symlinks was interrupted.
    (when (file-exists? pivot)
      (delete-file pivot))
    (symlink target pivot)
    (rename-file pivot link)))
--8<---------------cut here---------------end--------------->8---

WDYT?


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-06 11:27           ` Arun Isaac
@ 2022-06-06 12:20             ` Maxime Devos
  2022-06-08 20:12               ` Arun Isaac
  0 siblings, 1 reply; 13+ messages in thread
From: Maxime Devos @ 2022-06-06 12:20 UTC (permalink / raw)
  To: Arun Isaac, Ludovic Courtès; +Cc: Guix Devel

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

Arun Isaac schreef op ma 06-06-2022 om 16:57 [+0530]:
> Could we implement symlink/remove-old without catch and throw? Something
> like:
> 
> --8<---------------cut here---------------start------------->8---
> (define (symlink/remove-old target link)
>   "Make a symbolic link named LINK pointing to TARGET. If LINK already
> exists, it will be removed first.  This is not an atomic operation."
>   (when (file-exists? link)
>     (delete-file link))
>   (symlink target link))
> --8<---------------cut here---------------end--------------->8---

file-exists? does not check if the file exists -- for a broken symlink,
it will return #false instead of #true.

Also, this introduces additional non-atomicity (though in this
particular case, this does not seem to be a problem).  It would also
make things a tiny bit slower, because file-exists? makes an additional
syscall.

For these three reasons, I would go for the more robust catch+throw.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Move switch-symlinks to (guix build utils)
  2022-06-06 12:20             ` Maxime Devos
@ 2022-06-08 20:12               ` Arun Isaac
  0 siblings, 0 replies; 13+ messages in thread
From: Arun Isaac @ 2022-06-08 20:12 UTC (permalink / raw)
  To: Maxime Devos, Ludovic Courtès; +Cc: Guix Devel


Hi Maxime,

Thanks for the explanation! I am working on a patch. I'll send something
soon.

Regards,
Arun


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-06-08 20:27 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-01  7:34 Move switch-symlinks to (guix build utils) Arun Isaac
2022-06-01 12:09 ` Maxime Devos
2022-06-02 15:06   ` Arun Isaac
2022-06-02 15:13     ` Maxime Devos
2022-06-03 16:38       ` Ludovic Courtès
2022-06-03 17:09         ` Maxime Devos
2022-06-03 19:30           ` Ludovic Courtès
2022-06-03 19:30           ` Ludovic Courtès
2022-06-03 23:55             ` bokr
2022-06-04 10:17               ` Maxime Devos
2022-06-06 11:27           ` Arun Isaac
2022-06-06 12:20             ` Maxime Devos
2022-06-08 20:12               ` Arun Isaac

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).