Texlive and native-inputs

Texlive and native-inputs

[Andreas Enge]

Currently, texlive has a certain number of native inputs:
   (native-inputs
    `(("perl" ,perl)
      ("pkg-config" ,pkg-config)
      ("python" ,python-2) ; incompatible with Python 3 (print syntax)
      ("tcsh" ,tcsh)))
But I think these are not needed during build time, but to patch-shebang
scripts that are installed into the bin directory. So should they not be
normal inputs?

This is part of commit c4c4cc05979f2a2d0212963c5fe1b940d63a0958 which was
a mass-move from inputs to native-inputs. I wonder if these need to be
verified one by one by hand? How does one know without going through every
line of the build logs whether an interpreter is used during the build
or in installed scripts? Or what happens if both is the case?

Did I misunderstand anything?

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1926 bytes --]

On Wed, Oct 29, 2014 at 06:00:02PM +0100, Andreas Enge wrote:
     Currently, texlive has a certain number of native inputs:
        (native-inputs
         `(("perl" ,perl)
           ("pkg-config" ,pkg-config)
           ("python" ,python-2) ; incompatible with Python 3 (print syntax)
           ("tcsh" ,tcsh)))
     But I think these are not needed during build time, but to patch-shebang
     scripts that are installed into the bin directory. So should they not be
     normal inputs?
     

If they were "normal" inputs and you were cross compiling, then the packages which
are made available, would be those for the target system, not the native one.  Hence
they could not run, and the build would break.

     This is part of commit c4c4cc05979f2a2d0212963c5fe1b940d63a0958 which was
     a mass-move from inputs to native-inputs. I wonder if these need to be
     verified one by one by hand? How does one know without going through every
     line of the build logs whether an interpreter is used during the build
     or in installed scripts? 

You are probably right - to be sure they should be manually checked.  An 
alternative would be to attempt cross building all the affected packages.  That 
said, I cannot envisage a scenario where a non-native pkg-config would be  
needed.  In the case of perl, python, etc I recall that previous discussions 
concluded that if they are needed in installed scripts, then it should be up to the 
user to install them.

     Or what happens if both is the case?

In that case, the package would need to be declared as both an input and a native-input.
     
     Did I misunderstand anything?

From your first paragraph, I think you did.
     
     
J'

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Texlive and native-inputs

[Andreas Enge]

On Wed, Oct 29, 2014 at 06:13:09PM +0100, John Darrington wrote:
> If they were "normal" inputs and you were cross compiling, then the packages which
> are made available, would be those for the target system, not the native one.  Hence
> they could not run, and the build would break.

Well, my point is that probably the scripts are not run during the build
process, but on the target system.

For instance, texlive installs a file
   texlive-2014-data/texmf-dist/scripts/psutils/psjoin.pl ,
which I suppose does what its name says.

Because we have perl as a native input, patch-shebangs replaces its first
line by
   #!/gnu/store/5x0h6n8ln2fvaqk1q2ji79x08y8bdr35-perl-5.16.1/bin/perl
Then if we are cross-compiling with perl as a native input, this is the perl
of the build and not of the target machine, and the user calling psjoin.pl
on the target machine has a dangling interpreter here.

So I think perl, python and tcsh should be normal inputs; as far as I know,
they are not used during the build process of texlive.

>      Or what happens if both is the case?
> In that case, the package would need to be declared as both an input and a native-input.

Would it work? One would need to patch-shebang with the normal input and
use the native-input for scripts that are used during the build process.
Is this distinguished somehow? (Well, there is of course the problem of
scripts that are used during the build process _and_ installed on the user's
machine, which means that the package is simply not cross-compilable in
our setting. One could imagine, for instance, that psjoin.pl is used for
concatenating a set of postscript files to create the documentation depending
on how the package is configured, and that it is also included in the
package for installation as a handy tool.)

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 2353 bytes --]

On Wed, Oct 29, 2014 at 06:39:48PM +0100, Andreas Enge wrote:
     On Wed, Oct 29, 2014 at 06:13:09PM +0100, John Darrington wrote:
     > If they were "normal" inputs and you were cross compiling, then the packages which
     > are made available, would be those for the target system, not the native one.  Hence
     > they could not run, and the build would break.
     
     Well, my point is that probably the scripts are not run during the build
     process, but on the target system.
     
     For instance, texlive installs a file
        texlive-2014-data/texmf-dist/scripts/psutils/psjoin.pl ,
     which I suppose does what its name says.
     
     Because we have perl as a native input, patch-shebangs replaces its first
     line by
        #!/gnu/store/5x0h6n8ln2fvaqk1q2ji79x08y8bdr35-perl-5.16.1/bin/perl
     Then if we are cross-compiling with perl as a native input, this is the perl
     of the build and not of the target machine, and the user calling psjoin.pl
     on the target machine has a dangling interpreter here.
     
     So I think perl, python and tcsh should be normal inputs; as far as I know,
     they are not used during the build process of texlive.

Obviously, if they are not used during the build, then, as you say they should 
not be native-inputs.

I guess you would have to ask Ludovic whether they should be inputs at all, or if we should
just rely on the user to install the relevant package.  I know in similar cases we've done the
latter.
     
     >      Or what happens if both is the case?
     > In that case, the package would need to be declared as both an input and a native-input.
     
     Would it work? One would need to patch-shebang with the normal input and
     use the native-input for scripts that are used during the build process.
     Is this distinguished somehow? 

Now I'm starting to get out of my depth, but as I understand it, this could be resolved thus:

 `(inputs (("perl" ,perl)))
 `(native-inputs (("native-perl" ,perl)))

 (do-something-with (assoc-ref %build-inputs "perl"))
 (do-something-else-with (assoc-ref %build-inputs "native-perl"))

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Texlive and native-inputs

[Andreas Enge]

On Wed, Oct 29, 2014 at 06:39:48PM +0100, Andreas Enge wrote:
> Would it work? One would need to patch-shebang with the normal input and
> use the native-input for scripts that are used during the build process.
> Is this distinguished somehow? (Well, there is of course the problem of
> scripts that are used during the build process _and_ installed on the user's
> machine, which means that the package is simply not cross-compilable in
> our setting.

In a sense this is what happens with texlive. "make check" uses perl scripts,
and other perl scripts are installed.

But then I suppose that "make check" does not make much sense anyway when
cross-compiling? Do we activate it then?

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1402 bytes --]

On Wed, Oct 29, 2014 at 07:51:44PM +0100, Andreas Enge wrote:
     On Wed, Oct 29, 2014 at 06:39:48PM +0100, Andreas Enge wrote:
     > Would it work? One would need to patch-shebang with the normal input and
     > use the native-input for scripts that are used during the build process.
     > Is this distinguished somehow? (Well, there is of course the problem of
     > scripts that are used during the build process _and_ installed on the user's
     > machine, which means that the package is simply not cross-compilable in
     > our setting.
     
     In a sense this is what happens with texlive. "make check" uses perl scripts,
     and other perl scripts are installed.
     
     But then I suppose that "make check" does not make much sense anyway when
     cross-compiling? Do we activate it then?

Supposing it was possible to run "make check" on a cross-compiled build (for 
example using an emulator), then we would need the non-native perl.
The installed perl scripts also need the non-native perl.  

So it looks as if the the native perl is not needed at all.   


What do these scripts do anyway?  Are they fundamental to TeXlive or are they for some bells and whistles?

J'

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Texlive and native-inputs

[Andreas Enge]

On Wed, Oct 29, 2014 at 08:55:55PM +0100, John Darrington wrote:
> What do these scripts do anyway?  Are they fundamental to TeXlive or are they for some bells and whistles?

I never use them as far as I know, so they do not seem to be fundamental.

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

On Wed, Oct 29, 2014 at 11:31:18PM +0100, Andreas Enge wrote:
     On Wed, Oct 29, 2014 at 08:55:55PM +0100, John Darrington wrote:
     > What do these scripts do anyway?  Are they fundamental to TeXlive or are they for some bells and whistles?
     
     I never use them as far as I know, so they do not seem to be fundamental.
     
In that case, it would seem to me, that perl should not be in input at all.

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Texlive and native-inputs

[Andreas Enge]

On Thu, Oct 30, 2014 at 08:53:24AM +0100, John Darrington wrote:
> In that case, it would seem to me, that perl should not be in input at all.

It should. Perl is needed for "make check". But even if it were not, we
always patch-shebang script files in our packages. Otherwise they would not
be usable, since we do not have /usr/bin/perl. That I personally do not
use these texlive scripts is not a reason to make them unusable!

Now if this has not been done for octave, maybe we should rediscuss it and
add the needed script interpreters to its inputs.

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

On Thu, Oct 30, 2014 at 09:02:10AM +0100, Andreas Enge wrote:
     On Thu, Oct 30, 2014 at 08:53:24AM +0100, John Darrington wrote:
     > In that case, it would seem to me, that perl should not be in input at all.
     
     It should. Perl is needed for "make check". But even if it were not, we
     always patch-shebang script files in our packages. Otherwise they would not
     be usable, since we do not have /usr/bin/perl. That I personally do not
     use these texlive scripts is not a reason to make them unusable!
     

It could be patched to "perl".  That way, so long as the user has done 
guix package -i perl  it will work for him.
     


-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Texlive and native-inputs

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> But then I suppose that "make check" does not make much sense anyway when
> cross-compiling? Do we activate it then?

For Automake-generated makefiles, ‘make check’ does nothing when
cross-compiling, so no extra action needs to be taken.

Ludo’.

Re: Texlive and native-inputs

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Wed, Oct 29, 2014 at 06:13:09PM +0100, John Darrington wrote:
>> If they were "normal" inputs and you were cross compiling, then the packages which
>> are made available, would be those for the target system, not the native one.  Hence
>> they could not run, and the build would break.
>
> Well, my point is that probably the scripts are not run during the build
> process, but on the target system.
>
> For instance, texlive installs a file
>    texlive-2014-data/texmf-dist/scripts/psutils/psjoin.pl ,
> which I suppose does what its name says.
>
> Because we have perl as a native input, patch-shebangs replaces its first
> line by
>    #!/gnu/store/5x0h6n8ln2fvaqk1q2ji79x08y8bdr35-perl-5.16.1/bin/perl
> Then if we are cross-compiling with perl as a native input, this is the perl
> of the build and not of the target machine, and the user calling psjoin.pl
> on the target machine has a dangling interpreter here.

Then you have evidence that Perl must be in ‘inputs’.  It might/may be
that it also needs to be in ‘native-inputs’ (there’s a chance that some
build scripts use it.)

> So I think perl, python and tcsh should be normal inputs; as far as I know,
> they are not used during the build process of texlive.

OK.

>>      Or what happens if both is the case?
>> In that case, the package would need to be declared as both an input and a native-input.
>
> Would it work?

Search paths specifications for native and target inputs are disjoint.
So for instance, when cross-compiling, only native inputs are added to
$PATH.

However, I just realized that our ‘patch-shebangs’ phase (the one that
runs at the end) is buggy, since it patches according to what’s in
$PATH, even when cross-compiling.  I’ve filed a bug.

Anyway, apart from that, it would work.  :-)

Ludo’.

Re: Texlive and native-inputs

[Andreas Enge]

On Wed, Oct 29, 2014 at 06:13:09PM +0100, John Darrington wrote:
> You are probably right - to be sure they should be manually checked.  An 
> alternative would be to attempt cross building all the affected packages. 

In the case that native inputs should instead be normal, if I understand
correctly, cross-builds will succeed, but the built package will not be
usable on the target system.

Your suggestion detects cases where normal inputs should be native.

> In the case of perl, python, etc I recall that previous discussions 
> concluded that if they are needed in installed scripts, then it should be up to the 
> user to install them.

No, as I understand it, the patch-shebang is there so that scripts run
without installing additional interpreters.

Andreas

Re: Texlive and native-inputs

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1843 bytes --]

On Wed, Oct 29, 2014 at 06:43:03PM +0100, Andreas Enge wrote:
     On Wed, Oct 29, 2014 at 06:13:09PM +0100, John Darrington wrote:
     > You are probably right - to be sure they should be manually checked.  An 
     > alternative would be to attempt cross building all the affected packages. 
     
     In the case that native inputs should instead be normal, if I understand
     correctly, cross-builds will succeed, but the built package will not be
     usable on the target system.
     
     Your suggestion detects cases where normal inputs should be native.

It should also do the inverse.  If, say, libpng is declared as native-input, when it
should be input, then it will fail at build stage.  Specifically at the linking stage when
the liker tries to link the target binary against the native -lpng.

However in the case of things like perl, I suppose this relates to what I was saying earlier 
about putting the onus on the user to install it rather than making it an input.

     
     > In the case of perl, python, etc I recall that previous discussions 
     > concluded that if they are needed in installed scripts, then it should be up to the 
     > user to install them.
     
     No, as I understand it, the patch-shebang is there so that scripts run
     without installing additional interpreters.

I certainly remember in the case of octave we decided not to patch-shebang and have the user 
install the additional interpreters, but this may not be a universal policy.  I think it depends
on if the additional interpreters are there as part of the core functionality or as an 
optional feature.
     
J'

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]