Video subtitles

Video subtitles

[Laura Lazzati]

Hi!

I have been playing with aegisub today and I would like to share my
experience with that tool, your ideas about how video subtitles should
be included, and your advice too.

I installed aegisub from guix, and when I open it, it has a GUI. There
you load your video, in my case I loaded the final video - the one
everybody loved with Dennis pics, that already had my voice - and
started from there. And ended up with a a final video with subtitles
included. But don't stop reading here.

The first question I need to answer is if subtitles are going to be
present all the time as part of the video, or they should be able to
be disabled. We had that discussion somewhere, but if they are going
to be present as part of the video (hardsubs) only .srt and .ass - the
ones created by default by aegisub - are supported by ffmpeg:
https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo

I remembered how much time each audio file lasted, so I added one
subtitle for each svg file that appears in the video - they were just
three- with the full content of what is said in the audio. The
drawback of this is that each svg file in the video has the full
subtitle text before the speaker mentions those words - I will ask
Björn tomorrow to upload it to their IPFS so that you can see what I
mean.

When choosing the format of the subtitles, Ricardo mentioned that it
was better using usf because it had support for styling. The good news
is that aegisub allows us to do so in their own interface, but  in
their .ass format, one of the two that supports ffmpeg,  which is not
compatible,  I guess, with the formats mentioned in po4a for
translation. However the .ass subtitle can be exported as .ttxt -
which is an XML, and that can be converted back to .ass.

I find a solution for not getting the whole text of the subtitle at
once, that is creating each svg/cli session/whatever
appears-in-the-video-if-needed with their audio and subtitle
separately and glue every piece of video as a last step with ffmpeg.
Apparently, that is possible according to
https://trac.ffmpeg.org/wiki/Concatenate - have not tried it yet.
However, I don't know if, in case of wanting to have the subtitles
separately - not  hardsubs-  in this latter case if it is possible to
concatenate them and use them outside the video.

Sorry if the mail is too long, or if sth is messy, please, let me know.

Regards :)
Laura

Re: Video subtitles

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1270 bytes --]

Hi Laura,

> I find a solution for not getting the whole text of the subtitle at
> once, that is creating each svg/cli session/whatever
> appears-in-the-video-if-needed with their audio and subtitle
> separately and glue every piece of video as a last step with ffmpeg.
> Apparently, that is possible according to
> https://trac.ffmpeg.org/wiki/Concatenate - have not tried it yet.
> However, I don't know if, in case of wanting to have the subtitles
> separately - not  hardsubs-  in this latter case if it is possible to
> concatenate them and use them outside the video.

I've done this many times and it works very well.  Off the top of my head,
something like

--8<---------------cut here---------------start------------->8---
  ffmpeg -i movie.mkv -i sub1.usf -i sub2.usf -map 0 -c:s copy -c:v copy -c:a copy output.mkv
--8<---------------cut here---------------end--------------->8---

should cut it.

I would not recommend hard-subbing.  It's less flexible and does not allow for
multiple languages.  Instead, you can use the .mkv format which allows for
embedding multiple subtitles, choosing a default subtitle, and finally leaves it
to the user to disable them when playing.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Video subtitles

[Ricardo Wurmus]

Hi Laura,

> The first question I need to answer is if subtitles are going to be
> present all the time as part of the video, or they should be able to
> be disabled. We had that discussion somewhere, but if they are going
> to be present as part of the video (hardsubs) only .srt and .ass - the
> ones created by default by aegisub - are supported by ffmpeg:
> https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo

The subtitles should be provided as a separate file, so that they can
easily be swapped out or disabled.  Some container formats allow
embedding the subtitles, but we would need to embed all subtitles in all
language variants of all video files, and this seems like more work.
Having the subtitles as separate files is more convenient for
processing.

> I remembered how much time each audio file lasted, so I added one
> subtitle for each svg file that appears in the video - they were just
> three- with the full content of what is said in the audio.

It is more common to have a single subtitle file per video.

> The drawback of this is that each svg file in the video has the full
> subtitle text before the speaker mentions those words - I will ask
> Björn tomorrow to upload it to their IPFS so that you can see what I
> mean.

You can time the appearance of text.  I’ve only ever written srt
subtitle files by hand before, but they are all similar in that they
associate a piece of text with a timestamp.  The text is then displayed
at the specified time only.

You can figure out the appropriate timestamps by playing back the video
and recording the times when something is said that should be titled.

> When choosing the format of the subtitles, Ricardo mentioned that it
> was better using usf because it had support for styling. The good news
> is that aegisub allows us to do so in their own interface, but  in
> their .ass format, one of the two that supports ffmpeg,  which is not
> compatible,  I guess, with the formats mentioned in po4a for
> translation. However the .ass subtitle can be exported as .ttxt -
> which is an XML, and that can be converted back to .ass.

“.ass” files are very popular and very rich in styling features.  “.usf”
files are less commonly seen in the wild.

I have no strong opinions about using “.usf”.  “.ass” may just be fine.
Apparently, it can be converted to po files for translation with
“sub2po”, which is part of the “translate” toolkit.  I have no
experience with this, and I think it would be better to use a format
that didn’t require even more tools to work with.

This shouldn’t block you, though.  It is fine to use Aegisub manually to
create the initial subtitle and then convert it to another format.
Aegisub does not need to be part of the automated process to compile the
videos.

--
Ricardo

Re: Video subtitles

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]


> Some container formats allow embedding the subtitles, but we would need to
> embed all subtitles in all language variants of all video files, and this
> seems like more work.

.mkv does and while ffmpeg needs to copy the whole stream to embed subtitles,
mkvtoolnix can do in place, which means it would take a fraction of a second to
insert/remove subtitles.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Video subtitles

[Laura Lazzati]

Hi!

> I have no strong opinions about using “.usf”.  “.ass” may just be fine.
> Apparently, it can be converted to po files for translation with
> “sub2po”, which is part of the “translate” toolkit.  I have no
> experience with this, and I think it would be better to use a format
> that didn’t require even more tools to work with.
I have taken a look at sub2po. I understand that it translates .srt or
.sub files only.
I read it from here:
http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/subtitles.html

The following formats are supported for subtitles:
MicroDVD
MPL2
MPsub
SubRip (.srt)
SubViewer 2.0 (.sub)
TMPlayer
Sub Station Alpha
Advanced Sub Station Alpha
BTW, is there a player that you suggest for adding the subtitles?
> Aegisub does not need to be part of the automated process to compile the
> videos.
Thanks, I was quite worried about having to automate the part of
creating the subtitles..  What about Pierre's suggestion of
mkvtoolnix?

Regards!
Laura

Re: Video subtitles

[Ricardo Wurmus]

Hi Laura,

> BTW, is there a player that you suggest for adding the subtitles?

You can use mpv with a command like this:

    mpv --sub-file my-subtitle.ass my-video.ogv

> What about Pierre's suggestion of mkvtoolnix?

We can reconsider this near the end of the internship, I think.  Whether
to embed subtitles or not isn’t something that we need to decide now.
It’s an optional final step.

--
Ricardo

Re: Video subtitles

[Laura Lazzati]

Hi!
>     mpv --sub-file my-subtitle.ass my-video.ogv
Is there a reason for the .ogv extension?

Regards :)

Re: Video subtitles

[Ricardo Wurmus]

Hi Laura,

>>     mpv --sub-file my-subtitle.ass my-video.ogv
> Is there a reason for the .ogv extension?

No particular reason; ogv is the extension for a patent-unencumbered
video encoding.  I don’t know enough about video encodings to give a
recommendation what to use for the project.  Luckily, ffmpeg supports a
very wide range of formats, so you needn’t worry about this now.

--
Ricardo

Re: Video subtitles

[Gábor Boskovits]

Hello,

Ricardo Wurmus <rekado@elephly.net> ezt írta (időpont: 2018. dec. 20.,
Cs, 9:26):
>
>
> Hi Laura,
>
> > The first question I need to answer is if subtitles are going to be
> > present all the time as part of the video, or they should be able to
> > be disabled. We had that discussion somewhere, but if they are going
> > to be present as part of the video (hardsubs) only .srt and .ass - the
> > ones created by default by aegisub - are supported by ffmpeg:
> > https://trac.ffmpeg.org/wiki/HowToBurnSubtitlesIntoVideo
>
> The subtitles should be provided as a separate file, so that they can
> easily be swapped out or disabled.  Some container formats allow
> embedding the subtitles, but we would need to embed all subtitles in all
> language variants of all video files, and this seems like more work.
> Having the subtitles as separate files is more convenient for
> processing.
>

Yes, and the embeding into the container can be automated at the end if needed.

> > I remembered how much time each audio file lasted, so I added one
> > subtitle for each svg file that appears in the video - they were just
> > three- with the full content of what is said in the audio.
>
> It is more common to have a single subtitle file per video.

It would be ok to have a single subtitle file.

>
> > The drawback of this is that each svg file in the video has the full
> > subtitle text before the speaker mentions those words - I will ask
> > Björn tomorrow to upload it to their IPFS so that you can see what I
> > mean.
>
> You can time the appearance of text.  I’ve only ever written srt
> subtitle files by hand before, but they are all similar in that they
> associate a piece of text with a timestamp.  The text is then displayed
> at the specified time only.
>
> You can figure out the appropriate timestamps by playing back the video
> and recording the times when something is said that should be titled.
>

So to make this more explicit, you could break down the text to smaller
chunks, then time the appearance of those.

> > When choosing the format of the subtitles, Ricardo mentioned that it
> > was better using usf because it had support for styling. The good news
> > is that aegisub allows us to do so in their own interface, but  in
> > their .ass format, one of the two that supports ffmpeg,  which is not
> > compatible,  I guess, with the formats mentioned in po4a for
> > translation. However the .ass subtitle can be exported as .ttxt -
> > which is an XML, and that can be converted back to .ass.
>
> “.ass” files are very popular and very rich in styling features.  “.usf”
> files are less commonly seen in the wild.
>
> I have no strong opinions about using “.usf”.  “.ass” may just be fine.
> Apparently, it can be converted to po files for translation with
> “sub2po”, which is part of the “translate” toolkit.  I have no
> experience with this, and I think it would be better to use a format
> that didn’t require even more tools to work with.
>
> This shouldn’t block you, though.  It is fine to use Aegisub manually to
> create the initial subtitle and then convert it to another format.
> Aegisub does not need to be part of the automated process to compile the
> videos.
>
> --
> Ricardo
>
>

I believe that we don't want to do anything fancy with the subtitles, so I
guess any format would be just fine, provided we have tool support.

Best regards,
g_bor