getting started with guix import nix

getting started with guix import nix

[Paul Garlick]

[-- Attachment #1: Type: text/plain, Size: 4405 bytes --]

Hi,

I am attempting to package the OpenCASCADE-OCE library for Guix using
the Nix opencascade_oce package as a starting point.  I have a standard
Guix installation, a standard Nix installation plus the Guix source to
work on separately.  opencascade_oce is not installed in the Nix store.
 The Nix setup is:

$ echo $NIX_PATH
nixpkgs=/home/paul/.nix-defexpr/channels/nixpkgs

$ which nix-instantiate
/home/paul/.nix-profile/bin/nix-instantiate

I try:

$ export NIX_REMOTE=daemon
$ ./pre-inst-env guix import nix $HOME/.nix-defexpr/channels/nixpkgs
opencascade_oce

i get:

;;; SSAX warning: Skipping PI: xml

trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
Backtrace:
In guix/import/snix.scm:
 190: 19 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 18 [loop #<input: #{read pipe}# 10> () ...]
 190: 17 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 16 [loop #<input: #{read pipe}# 10> () ...]
 190: 15 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 14 [loop #<input: #{read pipe}# 10> () ...]
 190: 13 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 12 [loop #<input: #{read pipe}# 10> () ...]
 190: 11 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 10 [loop #<input: #{read pipe}# 10> () ...]
 190: 9 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 8 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 7 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 6 [handle-start-tag function #<input: #{read pipe}# 10> ...]
In sxml/upstream/SSAX.scm:
1523: 5 [#<procedure fda8a0 at sxml/upstream/SSAX.scm:1522:1 (tag-head
port elems entities namespaces)> function ...]
1246: 4 [#<procedure 16e2800 at sxml/upstream/SSAX.scm:1233:2 (port
entities)> #<input: #{read pipe}# 10> ...]
1190: 3 [read-attrib-value #\" #<input: #{read pipe}# 10> () ()]
In sxml/ssax/input-parse.scm:
 103: 2 [next-token () (#\" #\space #\newline ...) ...]
In ice-9/rdelim.scm:
  88: 1 [read-delimited "\" \n\t\r<&" #<input: #{read pipe}# 10> peek]
In unknown file:
   ?: 0 [%read-delimited! "\" \n\t\r<&" ...]

ERROR: In procedure %read-delimited!:
ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack overflow"
())'.
Backtrace:
In guix/import/snix.scm:
 190: 19 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 18 [loop #<input: #{read pipe}# 10> () ...]
 190: 17 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 16 [loop #<input: #{read pipe}# 10> () ...]
 190: 15 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 14 [loop #<input: #{read pipe}# 10> () ...]
 190: 13 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 12 [loop #<input: #{read pipe}# 10> () ...]
 190: 11 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 10 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 9 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
pairs>)]
 190: 8 [handle-start-tag function #<input: #{read pipe}# 10> ...]
In sxml/upstream/SSAX.scm:
1523: 7 [#<procedure fda8a0 at sxml/upstream/SSAX.scm:1522:1 (tag-head
port elems entities namespaces)> function ...]
1246: 6 [#<procedure 16e2800 at sxml/upstream/SSAX.scm:1233:2 (port
entities)> #<input: #{read pipe}# 10> ...]
1190: 5 [read-attrib-value #\" #<input: #{read pipe}# 10> () ()]
In sxml/ssax/input-parse.scm:
 103: 4 [next-token () (#\" #\space #\newline ...) ...]
In ice-9/rdelim.scm:
  88: 3 [read-delimited "\" \n\t\r<&" #<input: #{read pipe}# 10> peek]
In unknown file:
   ?: 2 [%read-delimited! "\" \n\t\r<&" ...]
In ice-9/boot-9.scm:
 108: 1 [#<procedure e06100 at ice-9/boot-9.scm:100:6 (thrown-k .
args)> vm-error ...]
  68: 0 [abort-to-prompt catch14 vm-error vm-run "VM: Stack overflow"
()]

ice-9/boot-9.scm:68:2: In procedure abort-to-prompt:
ice-9/boot-9.scm:68:2: Throw to key `vm-error' with args `(vm-run "VM:
Stack overflow" ())'.
paul$ error: stack overflow (possible infinite recursion)

I can see references to snix.scm but I am not sure how to interpret the
error output.  Bug or operator error?##SELECTION_END##

Best,

Paul.

[-- Attachment #2: Type: text/html, Size: 5862 bytes --]

Re: getting started with guix import nix

[Ludovic Courtès]

Hello!

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

> I am attempting to package the OpenCASCADE-OCE library for Guix using
> the Nix opencascade_oce package as a starting point.  I have a standard
> Guix installation, a standard Nix installation plus the Guix source to
> work on separately.  opencascade_oce is not installed in the Nix store.
>  The Nix setup is:
>
> $ echo $NIX_PATH
> nixpkgs=/home/paul/.nix-defexpr/channels/nixpkgs
>
> $ which nix-instantiate
> /home/paul/.nix-profile/bin/nix-instantiate
>
> I try:
>
> $ export NIX_REMOTE=daemon
> $ ./pre-inst-env guix import nix $HOME/.nix-defexpr/channels/nixpkgs
> opencascade_oce

You’re doing it right.  Here I get:

--8<---------------cut here---------------start------------->8---
$ NIX_REMOTE=daemon ./pre-inst-env guix import nix /data/src/nixpkgs/ opencascade_oce
;;; SSAX warning: Skipping PI: xml

trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
trace: `types.list' is deprecated; use `types.listOf' instead
;; converted from /data/src/nixpkgs/pkgs/development/libraries/opencascade/oce.nix:5

(package
  (name "opencascade-oce")
  (version "0.16")
  (source
    (origin
      (method url-fetch)
      (uri (string-append
             "https://github.com/tpaviot/oce/archive/OCE-"
             version
             ".tar.gz"))
      (sha256
        (base32
          "05bmg1cjz827bpq8s0hp96byirm4c3zc9vx26qz76kjsg8ry87w4"))))
  (build-system gnu-build-system)
  (inputs
    `(("cmake" ,cmake)
      ("freetype" ,freetype)
      ("ftgl" ,ftgl)
      ("qt" ,qt)
      ("libtool" ,libtool)
      ("libXmu" ,libXmu)
      ("file" ,file)
      ("tk" ,tk)
      ("tcl" ,tcl)
      ("mesa" ,mesa)))
  (home-page "http://www.opencascade.org/")
  (synopsis
    "Open CASCADE Technology, libraries for 3D modeling and numerical simulation")
  (description #f)
  (license #f))
$ git describe
v0.11.0-3412-ge8a463e
$ (cd /data/src/nixpkgs/ ; git describe)
release-16.03-start-8750-gdde259d
--8<---------------cut here---------------end--------------->8---

> i get:
>
> ;;; SSAX warning: Skipping PI: xml
>
> trace: lib.zip is deprecated, use lib.zipAttrsWith instead
> trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
> Backtrace:
> In guix/import/snix.scm:
>  190: 19 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 18 [loop #<input: #{read pipe}# 10> () ...]
>  190: 17 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 16 [loop #<input: #{read pipe}# 10> () ...]
>  190: 15 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 14 [loop #<input: #{read pipe}# 10> () ...]
>  190: 13 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 12 [loop #<input: #{read pipe}# 10> () ...]
>  190: 11 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 10 [loop #<input: #{read pipe}# 10> () ...]
>  190: 9 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 8 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 7 [loop #<input: #{read pipe}# 10> () #f (() . #<vhash 1c6c320 5
> pairs>)]
>  190: 6 [handle-start-tag function #<input: #{read pipe}# 10> ...]
> In sxml/upstream/SSAX.scm:
> 1523: 5 [#<procedure fda8a0 at sxml/upstream/SSAX.scm:1522:1 (tag-head
> port elems entities namespaces)> function ...]
> 1246: 4 [#<procedure 16e2800 at sxml/upstream/SSAX.scm:1233:2 (port
> entities)> #<input: #{read pipe}# 10> ...]
> 1190: 3 [read-attrib-value #\" #<input: #{read pipe}# 10> () ()]
> In sxml/ssax/input-parse.scm:
>  103: 2 [next-token () (#\" #\space #\newline ...) ...]
> In ice-9/rdelim.scm:
>   88: 1 [read-delimited "\" \n\t\r<&" #<input: #{read pipe}# 10> peek]
> In unknown file:
>    ?: 0 [%read-delimited! "\" \n\t\r<&" ...]
>
> ERROR: In procedure %read-delimited!:
> ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack overflow"

It may be that we’re not getting the same XML output from
‘nix-instantiate’ and that somehow confuses the XML parser in your case.

To investigate, you’d need to run the same ‘nix-instantiate’ command
that snix.scm invokes, same its XML output, and then try parsing it with
‘xml->sxml’ from (sxml simple).

HTH!

Ludo’.

Re: getting started with guix import nix

[Paul Garlick]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

Thank you, Ludo.
I have:
$ git describe
v0.11.0-3373-g2df3d14
My Nixpkgs directory is not managed by git.  It was installed by the Nix install script.  It seems to have the same structure as the Github repository:
$ ls /home/paul/.nix-defexpr/channels/nixpkgs
doc  maintainers  pkgs     README.md    svn-revision
lib  nixos        COPYING  default.nix
> It may be that we’re not getting the same XML output from
> ‘nix-instantiate’ and that somehow confuses the XML parser in your
> case.
> 
> To investigate, you’d need to run the same ‘nix-instantiate’ command
> that snix.scm invokes, same its XML output, and then try parsing it
> with
> ‘xml->sxml’ from (sxml simple).
> 
Following the snix.scm code, I have tried:
$ nix-instantiate --strict --eval --xml -A opencascade_oce
/home/paul/.nix-defexpr/channels/nixpkgs
This command sends xml to standard output for a few minutes, before I
kill the process.  Is it common for this process to need a long time?
 I have noticed in the Nix manual this comment on the --strict option:
Warning: This option can cause non-termination, because lazy
    data structures can be infinitely large.
Maybe this is a Nix problem?
Paul.

[-- Attachment #2: Type: text/html, Size: 1749 bytes --]

Re: getting started with guix import nix

[Paul Garlick]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

I have tried updating my local  <nixpkgs> directory with 'nix-channel
--update'.  

No change; the full nix-instantiate command does not terminate.  

The git commit of <nixpkgs> (in /home/paul/.nix-
defexpr/channels/nixpkgs/.version-suffix) is now 497e6d2.

Also, if I remove '--strict' from the nix-instantiate command I get:

<?xml version='1.0' encoding='utf-8'?>
<expr>
  <derivation>
    <repeated />
  </derivation>
</expr>

Terse##SELECTION_END##! but at least the process completes.

I have noticed that the non-termination problem is not confined to the
opencascage_oce package.  If I try 

$ nix-instantiate --strict --eval --xml -A guile /home/paul/.nix-
defexpr/channels/nixpkgs

that does not terminate either.

Best,

Paul.

[-- Attachment #2: Type: text/html, Size: 1167 bytes --]

Re: getting started with guix import nix

[Ludovic Courtès]

Hi!

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

> I have noticed that the non-termination problem is not confined to the
> opencascage_oce package. If I try 
>
> $ nix-instantiate --strict --eval --xml -A guile
> /home/paul/.nix-defexpr/channels/nixpkgs
>
> that does not terminate either.

Does it just take long, or does it simply not terminate?

I get:

--8<---------------cut here---------------start------------->8---
$ time NIX_REMOTE=daemon nix-instantiate -A guile /data/src/nixpkgs --strict --eval --xml | wc -c
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
trace: `types.list' is deprecated; use `types.listOf' instead
7786158

real	0m0.668s
user	0m0.348s
sys	0m0.160s
$ (cd /data/src/nixpkgs/; git describe)
release-16.03-start-8750-gdde259d
$ nix-instantiate --version
nix-instantiate (Nix) 1.11.2
--8<---------------cut here---------------end--------------->8---

As you can see it produces 7 MiB of XML data, but that’s pretty fast.

HTH,
Ludo’.

Re: getting started with guix import nix

[Paul Garlick]

[-- Attachment #1: Type: text/plain, Size: 3611 bytes --]

Hi Ludo,
> > 
> > I have noticed that the non-termination problem is not confined to
> > the
> > opencascage_oce package. If I try 
> > 
> > $ nix-instantiate --strict --eval --xml -A guile
> > /home/paul/.nix-defexpr/channels/nixpkgs
> > 
> > that does not terminate either.
> Does it just take long, or does it simply not terminate?
After a few (approximately five) minutes the process is still producing
xml.  Looking at the output, after killing the process, there is a
large amount of blank space plus the occasional line of xml, such as:
<function column="15" line="47"
path="/nix/store/k3jqgxqfwg6q9bgkys63c25lbjprl3gq-nixpkgs-
17.03pre96825.497e6d2/nixpkgs/lib/types.nix">
> 
> $ nix-instantiate --version
> nix-instantiate (Nix) 1.11.2
> 
> 

For my nix installation:
$ nix-instantiate --version
nix-instantiate (Nix) 1.11.4
Also, I have cloned the nixpkgs repository so that I can change the
version more easily.  To set the nixpkgs directory to be the same
version as the version you are using:
$ git checkout dde259d
Then I get the same output:
$ export NIX_REMOTE=daemon
$ ./pre-inst-env guix import nix /data/paul/sourceCode/nixpkgs
opencascade_oce
;;; SSAX warning: Skipping PI: xml
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
trace: `types.list' is deprecated; use `types.listOf' instead
;; converted from
/data/paul/sourceCode/nixpkgs/pkgs/development/libraries/opencascade/oc
e.nix:5
(package
  (name "opencascade-oce")
  (version "0.16")
  (source
    (origin
      (method url-fetch)
      (uri (string-append
             "https://github.com/tpaviot/oce/archive/OCE-"
             version
             ".tar.gz"))
      (sha256
        (base32
          "05bmg1cjz827bpq8s0hp96byirm4c3zc9vx26qz76kjsg8ry87w4"))))
  (build-system gnu-build-system)
  (inputs
    `(("cmake" ,cmake)
      ("freetype" ,freetype)
      ("ftgl" ,ftgl)
      ("qt" ,qt)
      ("libtool" ,libtool)
      ("libXmu" ,libXmu)
      ("file" ,file)
      ("tk" ,tk)
      ("tcl" ,tcl)
      ("mesa" ,mesa)))
  (home-page "http://www.opencascade.org/")
  (synopsis
    "Open CASCADE Technology, libraries for 3D modeling and numerical
simulation")
  (description #f)
  (license #f))
paul$ cat
/data/paul/sourceCode/nixpkgs/pkgs/development/libraries/opencascade/oc
e.nix
{stdenv, fetchurl, mesa, tcl, tk, file, libXmu, cmake, libtool, qt4,
ftgl, freetype}:
stdenv.mkDerivation rec {
  name = "opencascade-oce-0.16";
  src = fetchurl {
    url = https://github.com/tpaviot/oce/archive/OCE-0.16.tar.gz;
    sha256 = "05bmg1cjz827bpq8s0hp96byirm4c3zc9vx26qz76kjsg8ry87w4";
  };
  buildInputs = [ mesa tcl tk file libXmu libtool qt4 ftgl freetype
cmake ];
  preConfigure = ''
    cmakeFlags="$cmakeFlags -DOCE_INSTALL_PREFIX=$out"
  '';
  # https://bugs.freedesktop.org/show_bug.cgi?id=83631
  NIX_CFLAGS_COMPILE = "-DGLX_GLXEXT_LEGACY";
  enableParallelBuilding = true;
  meta = {
    description = "Open CASCADE Technology, libraries for 3D modeling
and numerical simulation";
    homepage = http://www.opencascade.org/;
    maintainers = with stdenv.lib.maintainers; [viric];
    platforms = with stdenv.lib.platforms; linux;
  };
}
This is almost usable, except the version of the file oce.nix has
changed since the dde259d commit.  Also, it may be nix-instantiate does
not terminate for many other packages in recent versions of nixpkgs.
Best,
Paul.

[-- Attachment #2: Type: text/html, Size: 5892 bytes --]

Re: getting started with guix import nix

[Ludovic Courtès]

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

>> > I have noticed that the non-termination problem is not confined to
>> > the
>> > opencascage_oce package. If I try 
>> > 
>> > $ nix-instantiate --strict --eval --xml -A guile
>> > /home/paul/.nix-defexpr/channels/nixpkgs
>> > 
>> > that does not terminate either.
>> Does it just take long, or does it simply not terminate?
> After a few (approximately five) minutes the process is still producing
> xml.  Looking at the output, after killing the process, there is a
> large amount of blank space plus the occasional line of xml, such as:
> <function column="15" line="47"
> path="/nix/store/k3jqgxqfwg6q9bgkys63c25lbjprl3gq-nixpkgs-
> 17.03pre96825.497e6d2/nixpkgs/lib/types.nix">

OK.

> $ ./pre-inst-env guix import nix /data/paul/sourceCode/nixpkgs
> opencascade_oce
> ;;; SSAX warning: Skipping PI: xml
> trace: lib.zip is deprecated, use lib.zipAttrsWith instead
> trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
> trace: `types.list' is deprecated; use `types.listOf' instead
> ;; converted from
> /data/paul/sourceCode/nixpkgs/pkgs/development/libraries/opencascade/oc
> e.nix:5
> (package
>   (name "opencascade-oce")

Good!

> This is almost usable, except the version of the file oce.nix has
> changed since the dde259d commit.  Also, it may be nix-instantiate does
> not terminate for many other packages in recent versions of nixpkgs.

Indeed, with Nixpkgs 16.09-beta-6955-g0c7afce, ‘guix import nix’ crashes
with a stack overflow in the XML parser after a few seconds.  Likewise:

--8<---------------cut here---------------start------------->8---
$ time NIX_REMOTE=daemon nix-instantiate -A guile /data/src/nixpkgs --strict --eval --xml | wc -c
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: `mkStrict' is obsolete; use `mkOverride 0' instead.
error: stack overflow (possible infinite recursion)
1055021551

real    0m5.959s
user    0m5.660s
sys     0m0.700s
--8<---------------cut here---------------end--------------->8---

IOW, it crashes after having spit 1 GB of XML.  Woow.  :-)
Could you report it to the Nix folks?

Now, I don’t think ‘guix import nix’ is that important to be honest.  I
think its value decreases as the number of packages in Guix increases.
Also, all it gives is a rough template and not the harder details; so
you may find that packaging OpenCascade won’t take much longer without
‘guix import nix’.

Thanks,
Ludo’.