unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#70877: guix-daemon fails to copy 4+GB file to store
@ 2024-05-11 10:52 Ricardo Wurmus
  2024-05-12  7:12 ` Efraim Flashner
  2024-05-13 10:10 ` Ludovic Courtès
  0 siblings, 2 replies; 6+ messages in thread
From: Ricardo Wurmus @ 2024-05-11 10:52 UTC (permalink / raw)
  To: 70877; +Cc: ludo

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

The guix-daemon's libutil/util.cc uses copy_file_range to copy a
downloaded file into the store.  copy_file_range fails on files larger
than 4GB with an error like this:

    guix build: error: short write in copy_file_range `15' to `16': No such file or directory

The man page for copy_file_range says that it could return EFBIG when
the range exceeds the maximum range.  The daemon code does not check any
limits and will attempt to copy the whole file.

I believe our code ought to check the value of st.size and fall back to
a boring copy if it exceeds some "reasonable" value.

This is where copy_file_range is used:
https://git.savannah.gnu.org/cgit/guix.git/tree/nix/libutil/util.cc#n382

Here is a little reproducer:


[-- Attachment #2: bug.scm --]
[-- Type: text/plain, Size: 425 bytes --]

(use-modules (guix download)
             (guix packages)
             (guix build-system trivial))

(package
  (name "chungus")
  (version "1")
  (source
   (origin
     (method url-fetch)
     (uri "http://localhost:1111/chungus")
     (sha256
      (base32 "0nx67d4ls2nfwcfdmg81vf240z6lpwpdqypssr1wzn3hyz4szci4"))))
  (build-system trivial-build-system)
  (home-page "")
  (synopsis "")
  (description "")
  (license #f))

[-- Attachment #3: Type: text/plain, Size: 468 bytes --]


--8<---------------cut here---------------start------------->8---
# generate a big file
dd bs=1M count=4096 if=/dev/zero of=/tmp/chungus
# serve it
guix shell woof -- woof -i 127.0.0.1 -p 1111 -c 1 /tmp/chungus
# build the source derivation
guix build --no-grafts -Sf bug.scm
# observe the error
# guix build: error: short write in copy_file_range `15' to `16': No such file or directory
--8<---------------cut here---------------end--------------->8---

-- 
Ricardo

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2024-05-13 16:25 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-11 10:52 bug#70877: guix-daemon fails to copy 4+GB file to store Ricardo Wurmus
2024-05-12  7:12 ` Efraim Flashner
2024-05-13 10:10 ` Ludovic Courtès
2024-05-13 12:09   ` Ricardo Wurmus
2024-05-13 14:34     ` Ludovic Courtès
2024-05-13 16:24       ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).