[bug#53388] New package: gallery-dl

[bug#53388] New package: gallery-dl

[Bird]

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

Hello,

The attached patch adds a new package: gallery-dl in
gnu/packages/video.scm

Thanks!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-gallery-dl-Add-a-new-package.patch --]
[-- Type: text/x-patch, Size: 1735 bytes --]

From 799d589e6b5d76b4cac6272d22f87603729fb9e1 Mon Sep 17 00:00:00 2001
From: Bird <birdsite@airmail.cc>
Date: Thu, 20 Jan 2022 11:42:24 +0000
Subject: [PATCH] gnu: gallery-dl: Add a new package. *
 gnu/packages/video.scm(gallery-dl): Add package.

---
 gnu/packages/video.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 461ccbb950..237cb931d0 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -2245,6 +2245,31 @@ (define-public smplayer
 the last played position, etc.")
     (license license:gpl2+)))
 
+(define-public gallery-dl
+  (package
+    (name "gallery-dl")
+    (version "1.20.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/mikf/gallery-dl"
+                                  "/releases/download/v"
+                                  version
+                                  "/gallery_dl-"
+                                  version
+                                  ".tar.gz"))
+              (sha256
+               (base32
+                "0qkz8aznvybdqrjxsl6ir319ras05mi8l0sal4mgi18l70jndh51"))))
+    (build-system python-build-system)
+    (propagated-inputs (list python-requests))
+    (home-page "https://github.com/mikf/gallery-dl")
+    (synopsis "Command-line program to download images from several sites")
+    (description "Command-line program to download image galleries
+ and collections from several image hosting sites")
+    (license gpl2+)
+    (arguments
+     '(#:tests? #f))))
+
 (define-public gnome-mpv
   (deprecated-package "gnome-mpv" celluloid))
 

base-commit: 2b6af630d61dd5b16424be55088de2b079e9fbaf
-- 
2.34.0

[bug#53388] New package: gallery-dl

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1569 bytes --]

Hi,

Bird schreef op do 20-01-2022 om 11:48 [+0000]:
> +    (propagated-inputs (list python-requests))

The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
and ‘ffmpeg‘ for some conversion, so maybe these could be added?

Also, propagation can lead to profile collisions which can be
complicated to resolve, so can this be made a regular input instead of
a propagated input?  Some wrap-program might be necessary.

> +    (home-page "https://github.com/mikf/gallery-dl")
> +    (synopsis "Command-line program to download images from several
> sites")
> +    (description "Command-line program to download image galleries
> + and collections from several image hosting sites")

Does it only support images, or also videos?  From the ‘ffmpeg’ and
‘youtube-dl’ dependency, I would assume the latter.  Also, when I'm
reading this description, it reminds me a lot of youtube-dl -- perhaps
the description could explain it supports more video sites, or less
video sites, or not only video but also images, or something along
these lines.

> +    (license gpl2+)
> +    (arguments
> +     '(#:tests? #f))))

Why are tests disabled here?

Also, 'gallery-dl' (indirectly) uses openssl for the S in HTTPS,
so you may need to add SSL_CERT_DIR (or SSL_CERT_FILE, not sure) to
native-search-paths to make "guix shell --pure yt-dlp nss-certs" work.

Although there appears to be some disagreement in that area, see e.g.
some discussion in #53324, so maybe not.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#53388] New package: gallery-dl

[Bird]

Maxime Devos <maximedevos@telenet.be> writes:

> Hi,
>
> Bird schreef op do 20-01-2022 om 11:48 [+0000]:
>> +    (propagated-inputs (list python-requests))
>
> The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
> and ‘ffmpeg‘ for some conversion, so maybe these could be added?
>
> Also, propagation can lead to profile collisions which can be
> complicated to resolve, so can this be made a regular input instead of
> a propagated input?  Some wrap-program might be necessary.

Can programs call to regular inputs that are not propagated?  Since
gallery-dl will need ffmpeg and python-requests (and possibly yt-dlp),
would it need to be in user's profile?

>
>> +    (home-page "https://github.com/mikf/gallery-dl")
>> +    (synopsis "Command-line program to download images from several
>> sites")
>> +    (description "Command-line program to download image galleries
>> + and collections from several image hosting sites")
>
> Does it only support images, or also videos?  From the ‘ffmpeg’ and
> ‘youtube-dl’ dependency, I would assume the latter.  Also, when I'm
> reading this description, it reminds me a lot of youtube-dl -- perhaps
> the description could explain it supports more video sites, or less
> video sites, or not only video but also images, or something along
> these lines.
>

It's really similar to youtube-dl but targets images primarily, i'll
make that clearer in the next patch

>
>> +    (license gpl2+)
>> +    (arguments
>> +     '(#:tests? #f))))
>
> Why are tests disabled here?
>

Sorry, I just assumed it will fail the tests due to network-lessness of
build environment, it actually passes the tests.

>
> Also, 'gallery-dl' (indirectly) uses openssl for the S in HTTPS,
> so you may need to add SSL_CERT_DIR (or SSL_CERT_FILE, not sure) to
> native-search-paths to make "guix shell --pure yt-dlp nss-certs" work.
>
> Although there appears to be some disagreement in that area, see e.g.
> some discussion in #53324, so maybe not.
>

I did a quick grep of SSL_CERT_FILE and SSL_CERT_DIR in gnu/packages/
but didn't find anything, could you please explain what i should do
here? 

>
> Greetings,
> Maxime.
>

Thanks,
Bird

[bug#53388] New package: gallery-dl

[Leo Famulari]

On Thu, Jan 20, 2022 at 05:42:17PM +0000, Bird wrote:
> > The README says it can use ‘youtube-dl’ or ‘yt-dlp’ for vide downloads
> > and ‘ffmpeg‘ for some conversion, so maybe these could be added?
> >
> > Also, propagation can lead to profile collisions which can be
> > complicated to resolve, so can this be made a regular input instead of
> > a propagated input?  Some wrap-program might be necessary.
> 
> Can programs call to regular inputs that are not propagated?  Since
> gallery-dl will need ffmpeg and python-requests (and possibly yt-dlp),
> would it need to be in user's profile?

They can, but they need to learn how to find them. Usually that happens
by making the package record the "store path" of the input at
build-time.

On the other hand, propagated-inputs appear on $PATH, so if the calling
program does the normal thing it will find them.

But, the drawback of propagating, for example, FFmpeg, is that it makes
it impossible for users to `guix install` a custom FFmpeg package
alongside the propagated FFmpeg.

> >> +    (home-page "https://github.com/mikf/gallery-dl")

I looked at the source code, and this program respects some
configuration options to find the location of youtube-dl and ffmpeg:

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#ugoiraffmpeg-location

It's a matter of opinion but, to me, both of those programs are things
that users may customize or choose particular versions of, so it's okay
to expect users to install them and configure gallery-dl properly.

If we want to improve this package in that regard... that's for a
followup patch!

[bug#53388] New package: gallery-dl

[Leo Famulari]

On Thu, Jan 20, 2022 at 11:48:25AM +0000, Bird wrote:
> The attached patch adds a new package: gallery-dl in
> gnu/packages/video.scm

> From 799d589e6b5d76b4cac6272d22f87603729fb9e1 Mon Sep 17 00:00:00 2001
> From: Bird <birdsite@airmail.cc>
> Date: Thu, 20 Jan 2022 11:42:24 +0000
> Subject: [PATCH] gnu: gallery-dl: Add a new package. *
>  gnu/packages/video.scm(gallery-dl): Add package.

Hello, and thanks for the patch!

I applied it to a Git checkout of the Guix source code, and tried
rebuilding Guix, as described here in the Contributing chapter of the
manual:

https://guix.gnu.org/manual/devel/en/html_node/Building-from-Git.html

However, it crashes like this:

------
ice-9/eval.scm:293:34: error: gpl2+: unbound variable
hint: Did you forget `(use-modules (guix licenses))'?

make[2]: *** [Makefile:7380: make-packages-go] Error 1
make[2]: Leaving directory '/home/leo/work/guix'
make[1]: *** [Makefile:6453: all-recursive] Error 1
make[1]: Leaving directory '/home/leo/work/guix'
make: *** [Makefile:3982: all] Error 2
------

The reason for this error is that licenses in the gnu/packages/video.scm
module are "prefixed" to avoid a namespace collision between packages
and licenses with the same name, such as zlib and expat:

https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/video.scm?id=f223535e1c2f052f671e44a6c546d0ebde3591b1#n80

You can check other packages in that module for examples of how to
prefix the license.

> +    (license gpl2+)

I checked several of the .py files in the source code, and setup.py, and
they all seem to specify that the program is made available under the
terms of version 2 of the GPL, but no later versions. So, this package
should use "gpl2", not "gpl2+".

> +    (arguments
> +     '(#:tests? #f))))

We always run upstream test suites in Guix packages, because it helps
validate our packaging and finds bugs to assist upstream development.
If we do not run the tests, we add a code comment explaining why. In
Scheme, comments begin with the ; (semicolon) character.

So, can you send a revised patch that adjusts the license and either
runs the test suite or explains why not?

Thanks in advance!

[bug#53388] New package: gallery-dl

[Bird]

[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]

[Added Maxime because this patch should fix his worries too]

Leo Famulari <leo@famulari.name> writes:

> On Thu, Jan 20, 2022 at 11:48:25AM +0000, Bird wrote:

> I applied it to a Git checkout of the Guix source code, and tried
> rebuilding Guix, as described here in the Contributing chapter of the
> manual:
>
> https://guix.gnu.org/manual/devel/en/html_node/Building-from-Git.html
>
> However, it crashes like this:
[...]
> The reason for this error is that licenses in the gnu/packages/video.scm
> module are "prefixed" to avoid a namespace collision between packages
> and licenses with the same name, such as zlib and expat:
>
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/video.scm?id=f223535e1c2f052f671e44a6c546d0ebde3591b1#n80
>
> You can check other packages in that module for examples of how to
> prefix the license.
>
>> +    (license gpl2+)

Thanks, fixed.

>
> I checked several of the .py files in the source code, and setup.py,
>and
> they all seem to specify that the program is made available under the
> terms of version 2 of the GPL, but no later versions. So, this package
> should use "gpl2", not "gpl2+".

I was confused due to the example on LICENSE containing phrase 'or (at
your option) any later version' but turns out that was an example.

>
>> +    (arguments
>> +     '(#:tests? #f))))
>
> We always run upstream test suites in Guix packages, because it helps
> validate our packaging and finds bugs to assist upstream development.
> If we do not run the tests, we add a code comment explaining why. In
> Scheme, comments begin with the ; (semicolon) character.
>
> So, can you send a revised patch that adjusts the license and either
> runs the test suite or explains why not?

It is attached in this mail.

>
> Thanks in advance!
>

Thanks!



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-gallery-dl-Add-a-new-package.patch --]
[-- Type: text/x-patch, Size: 1831 bytes --]

From 8a3ab5dba4f85dd1fb2ddaf683f9da7e5ec9854e Mon Sep 17 00:00:00 2001
From: Bird <birdsite@airmail.cc>
Date: Fri, 21 Jan 2022 04:32:39 +0000
Subject: [PATCH] gnu: gallery-dl: Add a new package. *
 gnu/packages/video.scm(gallery-dl): Add package.

---
 gnu/packages/video.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 461ccbb950..47f33ed3ac 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -2245,6 +2245,31 @@ (define-public smplayer
 the last played position, etc.")
     (license license:gpl2+)))
 
+(define-public gallery-dl
+  (package
+    (name "gallery-dl")
+    (version "1.20.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/mikf/gallery-dl"
+                                  "/releases/download/v"
+                                  version
+                                  "/gallery_dl-"
+                                  version
+                                  ".tar.gz"))
+              (sha256
+               (base32
+                "0qkz8aznvybdqrjxsl6ir319ras05mi8l0sal4mgi18l70jndh51"))))
+    (build-system python-build-system)
+    (inputs (list python-requests ffmpeg))
+    (home-page "https://github.com/mikf/gallery-dl")
+    (synopsis "Command-line program to download images from several sites")
+    (description "Command-line program to download image galleries
+and collections from several image hosting sites
+While this package can use youtube-dl or yt-dlp packages to download videos,
+the focus is more on images and image hosting sites.")
+    (license license:gpl2)))
+
 (define-public gnome-mpv
   (deprecated-package "gnome-mpv" celluloid))
 

base-commit: 2b6af630d61dd5b16424be55088de2b079e9fbaf
-- 
2.34.0

bug#53388: New package: gallery-dl

[Leo Famulari]

On Fri, Jan 21, 2022 at 04:46:59AM +0000, Bird wrote:
> From 8a3ab5dba4f85dd1fb2ddaf683f9da7e5ec9854e Mon Sep 17 00:00:00 2001
> From: Bird <birdsite@airmail.cc>
> Date: Fri, 21 Jan 2022 04:32:39 +0000
> Subject: [PATCH] gnu: gallery-dl: Add a new package. *
>  gnu/packages/video.scm(gallery-dl): Add package.

Thanks! I made some minor cosmetic changes to indentation and
standardized the commit message and pushed as commit
cc74d19d663b78a9ec8207d7e3a41cabfbe1a100.

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=cc74d19d663b78a9ec8207d7e3a41cabfbe1a100

I forgot to make sure your authorship was recorded in the copyright
header, so I did that in a followup commit.