bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Nathan Dehnel]

bash-5.0$ youtube-dl -x https://www.youtube.com/watch?v=7Ijd_iN9Blk
[youtube] 7Ijd_iN9Blk: Downloading webpage
[download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been downloaded
[download] 100% of 15.45MiB
ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please install one.

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

Hi Nathan,

Nathan Dehnel 写道：
> bash-5.0$ youtube-dl -x 
> https://www.youtube.com/watch?v=7Ijd_iN9Blk
> [youtube] 7Ijd_iN9Blk: Downloading webpage
> [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been 
> downloaded
> [download] 100% of 15.45MiB
> ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please 
> install one.

It works fine without ‘-x’:

  λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
  [youtube] 7Ijd_iN9Blk: Downloading webpage
  [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
  [download] 100% of 37.09MiB in 00:02

This *is* functioning correctly in my view: that advanced options 
optionally require run-time dependencies is not a bug but a 
feature.  The programme clearly explains which package could be 
installed to continue, and the user can chose which one they 
prefer - including neither, which suffices for the majority of 
youtube-dling.

Matters would be different if the error message were less clear, 
or perhaps if ffmpeg weren't so insanely great:

  λ guix size youtube-dl | tail -n1
  total: 186.9 MiB
  λ guix size youtube-dl ffmpeg | tail -n1
  total: 811.2 MiB

(!)

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[raingloom]

On Tue, 15 Sep 2020 14:06:11 +0200
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
wrote:

> Hi Nathan,
> 
> Nathan Dehnel 写道：
> > bash-5.0$ youtube-dl -x 
> > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been 
> > downloaded
> > [download] 100% of 15.45MiB
> > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please 
> > install one.  
> 
> It works fine without ‘-x’:
> 
>   λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
>   [youtube] 7Ijd_iN9Blk: Downloading webpage
>   [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
>   [download] 100% of 37.09MiB in 00:02
> 
> This *is* functioning correctly in my view: that advanced options 
> optionally require run-time dependencies is not a bug but a 
> feature.  The programme clearly explains which package could be 
> installed to continue, and the user can chose which one they 
> prefer - including neither, which suffices for the majority of 
> youtube-dling.
> 
> Matters would be different if the error message were less clear, 
> or perhaps if ffmpeg weren't so insanely great:
> 
>   λ guix size youtube-dl | tail -n1
>   total: 186.9 MiB
>   λ guix size youtube-dl ffmpeg | tail -n1
>   total: 811.2 MiB
> 
> (!)
> 
> Kind regards,
> 
> T G-R

Maybe it's time we added an "optional dependencies" field?
There seems to be a bug report or help request like every week that
just boils down to "this package has an undocumented dependency".

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Leo Famulari]

On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
> Matters would be different if the error message were less clear, or perhaps
> if ffmpeg weren't so insanely great:
> 
>  λ guix size youtube-dl | tail -n1
>  total: 186.9 MiB
>  λ guix size youtube-dl ffmpeg | tail -n1
>  total: 811.2 MiB

I wonder, should we expect FFmpeg to already be referenced by somebody's
profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
The use case of "download video and watch it on another machine (or
never watch it)" seems somewhat esoteric.

On the other hand, youtube-dl has no explicit dependencies — I guess
that everything it needs is from Python. That's beautiful, and I
understand the reluctance to add a dependency, especially on a "kitchen
sink" package like FFmpeg.

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Nathan Dehnel]

Ludovic wanted to add useflags to guix.
https://lists.gnu.org/archive/html/guix-devel/2019-05/msg00285.html
I don't know what became of it.

On Tue, Sep 15, 2020 at 2:44 PM raingloom <raingloom@riseup.net> wrote:
>
> On Tue, 15 Sep 2020 14:06:11 +0200
> Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
> wrote:
>
> > Hi Nathan,
> >
> > Nathan Dehnel 写道：
> > > bash-5.0$ youtube-dl -x
> > > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been
> > > downloaded
> > > [download] 100% of 15.45MiB
> > > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please
> > > install one.
> >
> > It works fine without ‘-x’:
> >
> >   λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
> >   [youtube] 7Ijd_iN9Blk: Downloading webpage
> >   [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
> >   [download] 100% of 37.09MiB in 00:02
> >
> > This *is* functioning correctly in my view: that advanced options
> > optionally require run-time dependencies is not a bug but a
> > feature.  The programme clearly explains which package could be
> > installed to continue, and the user can chose which one they
> > prefer - including neither, which suffices for the majority of
> > youtube-dling.
> >
> > Matters would be different if the error message were less clear,
> > or perhaps if ffmpeg weren't so insanely great:
> >
> >   λ guix size youtube-dl | tail -n1
> >   total: 186.9 MiB
> >   λ guix size youtube-dl ffmpeg | tail -n1
> >   total: 811.2 MiB
> >
> > (!)
> >
> > Kind regards,
> >
> > T G-R
>
> Maybe it's time we added an "optional dependencies" field?
> There seems to be a bug report or help request like every week that
> just boils down to "this package has an undocumented dependency".

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Mark H Weaver]

Leo Famulari <leo@famulari.name> writes:

> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
>> Matters would be different if the error message were less clear, or perhaps
>> if ffmpeg weren't so insanely great:
>> 
>>  λ guix size youtube-dl | tail -n1
>>  total: 186.9 MiB
>>  λ guix size youtube-dl ffmpeg | tail -n1
>>  total: 811.2 MiB
>
> I wonder, should we expect FFmpeg to already be referenced by somebody's
> profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
> The use case of "download video and watch it on another machine (or
> never watch it)" seems somewhat esoteric.

I was just about to disagree with you, on the grounds that GStreamer,
and in particular Totem (a.k.a. GNOME Videos) does not depend on FFmpeg.
It is, in fact, possible to watch videos downloaded using 'youtube-dl',
with only gst-plugins-{base,good} installed, provided that you request
freedom-respecting formats like VP9 and Opus.

How to request freedom-respecting formats?  Unfortunately, the obvious
"--prefer-free-formats" usually doesn't work in my experience, but you
can run "youtube-dl -F <URL>" to see a list of available formats, and
then something like "youtube-dl -f 247+251 <URL>" (for non-live youtube
videos of sufficient resolution) to download VP9 and Opus and combine
them into a file.

However, in my recent experience, youtube usually provides the
freedom-respecting formats only as separate audio and video streams
which must be recombined by youtube-dl, and in that case the 'ffmpeg'
command line tool seems to be required.

So, ultimately I tend to agree with Leo that 'youtube-dl' is unlikely to
be useful without 'ffmpeg', and therefore it might as well depend on it
somehow.

I see two possible approaches: ffmpeg could be added to youtube-dl's
propagated-inputs, or it could be instead be added to 'inputs', and we
could substitute hardcoded paths to 'ffmpeg' in the built 'youtube-dl'.
I generally tend to favor the latter approach, but there are valid
arguments for each approach and I don't feel strongly about it.

What do you think?

       Mark

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 1684 bytes --]

Leo, Mark,

Mark H Weaver 写道：
> Leo Famulari <leo@famulari.name> writes:
>
>> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice 
>> via Bug reports for GNU Guix wrote:
>>> Matters would be different if the error message were less 
>>> clear, or perhaps
>>> if ffmpeg weren't so insanely great:
>>> 
>>>  λ guix size youtube-dl | tail -n1
>>>  total: 186.9 MiB
>>>  λ guix size youtube-dl ffmpeg | tail -n1
>>>  total: 811.2 MiB
>>
>> I wonder, should we expect FFmpeg to already be referenced by 
>> somebody's
>> profile if they are using youtube-dl? VLC and mpv both depend 
>> on FFmpeg.
>> The use case of "download video and watch it on another machine 
>> (or
>> never watch it)" seems somewhat esoteric.

I don't see why my use case should be sidelined as ‘esoteric’. 
;-)

> However, in my recent experience, youtube usually provides the
> freedom-respecting formats only as separate audio and video 
> streams
> which must be recombined by youtube-dl, and in that case the 
> 'ffmpeg'
> command line tool seems to be required.

This is a good argument.  If including ffmpeg by default makes it 
easier to request & mux freeër formats, let's do that.

> I see two possible approaches: ffmpeg could be added to 
> youtube-dl's
> propagated-inputs, or it could be instead be added to 'inputs', 
> and we
> could substitute hardcoded paths to 'ffmpeg' in the built 
> 'youtube-dl'.
> I generally tend to favor the latter approach, but there are 
> valid
> arguments for each approach and I don't feel strongly about it.

Which advantages does the former have?  I wasn't aware of any.

Thanks!

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

Addresses <https://issues.guix.gnu.org/43418>.

* gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
[arguments]: Fall back to input ffmpeg instead of searching $PATH.
---
 gnu/packages/video.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 8ebe7dbcdf..50f751f8da 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -2184,6 +2184,13 @@ To load this plugin, specify the following option when starting mpv:
      ;; 'youtube-dl.bash-completion'.
      `(#:tests? #f ; Many tests fail. The test suite can be run with pytest.
        #:phases (modify-phases %standard-phases
+                  (add-after 'unpack 'default-to-the-ffmpeg-input
+                    (lambda _
+                      ;; Continue respecting the --ffmpeg-location argument.
+                      (substitute* "youtube_dl/postprocessor/ffmpeg.py"
+                        (("\\.get\\('ffmpeg_location'\\)" match)
+                         (format #f "~a or '~a'" match (which "ffmpeg"))))
+                      #t))
                   (add-before 'install 'fix-the-data-directories
                     (lambda* (#:key outputs #:allow-other-keys)
                       (let ((prefix (assoc-ref outputs "out")))
@@ -2207,6 +2214,8 @@ To load this plugin, specify the following option when starting mpv:
                         (copy-file "youtube-dl.zsh"
                                    (string-append zsh "/_youtube-dl"))
                         #t))))))
+    (inputs
+     `(("ffmpeg" ,ffmpeg)))
     (synopsis "Download videos from YouTube.com and other sites")
     (description
      "Youtube-dl is a small command-line program to download videos from
-- 
2.28.0

bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

[Mark H Weaver]

Hi Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Mark H Weaver 写道：
>
>> I see two possible approaches: ffmpeg could be added to youtube-dl's
>> propagated-inputs, or it could be instead be added to 'inputs', and
>> we could substitute hardcoded paths to 'ffmpeg' in the built
>> 'youtube-dl'.  I generally tend to favor the latter approach, but
>> there are valid arguments for each approach and I don't feel strongly
>> about it.
>
> Which advantages does the former have?  I wasn't aware of any.

The former approach is arguably simpler, easier to maintain, and if the
user upgrades only 'ffmpeg' while leaving 'youtube-dl' unchanged, with
the latter approach 'youtube-dl' will continue using the old 'ffmpeg'
with its potential security holes, which users unfamiliar with Guix
might not expect.

I still generally prefer the latter approach for several reasons, most
notably because it means fewer dynamically-determined dependencies,
which are contrary to our goals for reproducible behavior, i.e. that if
the software in a given store item works today for your use case, it
will work the same way tomorrow.

However, it's worth noting that this property is less useful for a
program like 'youtube-dl', because a copy of 'youtube-dl' that works
today might well stop working tomorrow because of changes made on the
server side, regardless of our efforts.

      Mark

bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.

[Mark H Weaver]

Hi Tobias,

> Addresses <https://issues.guix.gnu.org/43418>.
>
> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
> [arguments]: Fall back to input ffmpeg instead of searching $PATH.

Looks good to me.  Thank you!

     Mark

bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

Mark, Simon,

Mark H Weaver 写道：
>> Addresses <https://issues.guix.gnu.org/43418>.
>>
>> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
>> [arguments]: Fall back to input ffmpeg instead of searching 
>> $PATH.
>
> Looks good to me.  Thank you!

Thanks, pushed.

> The usage seems more Fixes than Addresses.

I'm still of the opinion that this was no bug but a feature 
request, and there was nothing to fix.  I promise to add it to the 
release notes if you think it's necessary, but would rather keep 
those limited to ‘real’ bugs as well.  It's not like we're short 
on those ;-)

Whatever it was, it's now closed, and we'll be spared of 
duplicates from now on.

Thanks all,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.

[zimoun]

On Wed, 16 Sep 2020 at 13:54, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> > The usage seems more Fixes than Addresses.
>
> I'm still of the opinion that this was no bug but a feature
> request, and there was nothing to fix.  I promise to add it to the
> release notes if you think it's necessary, but would rather keep
> those limited to ‘real’ bugs as well.  It's not like we're short
> on those ;-)

I am fine with whatever the convention is.  Just one that we
collectively share. :-)
It helps to dig with the tandem "git log --grep" and "M-x debbugs", IMHO.
I mean it could make sense to have Fixes for "real" bugs and Addresses
for anything else tracked by debbugs (whishlist, typo, etc.).

Cheers,
simon