NPM breakthrough with test packages and weird error

NPM breakthrough with test packages and weird error

[swedebugia]

Hi

I have good (and bad) news!

Today I worked intensely on the node-build-system, importer and made
some improvements as I ran into multiple errors using the importer and
building the packages.

I tried first importing tape (test package, quite popular) and succeded
with a few adjustments.

Then I tried tap, its a hairy mess unfortunately and pulls in del,
typescript, grunt, nyc, and others

I write this message mostly because I now receive a really weird error
from guix.

sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K node-grunt
guix build: error: build failed: invalid hash
`72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K
node-tap-mocha-reporter
guix build: error: build failed: invalid hash
`72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'

The error is the same across multiple packages. Builds "succeede" as
usual if they are already in the store or unrelated to grunt and nyc.

After investigating with strace I found this in the end:

[pid  8371] read(13, "download", 8)     = 8
[pid  8371] write(13,
"\10\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0minimatch-3.0.4."..., 944) = 944
[pid  8371] read(13, "ptxc\0\0\0\0", 8) = 8
[pid  8371] read(13, "K\0\0\0\0\0\0\0", 8) = 8
[pid  8371] read(13, "invalid hash `72bd690b3fbd3be123"..., 75) = 75
[pid  8371] read(13, "\0\0\0\0\0", 5)   = 5
[pid  8371] read(13, "\1", 1)           = 1
[pid  8371] read(13, "\0\0\0\0\0\0\0", 7) = 7
[pid  8371] close(13)                   = 0
[pid  8371] write(2, "guix build: error: build failed:"..., 109guix
build: error: build failed: invalid hash
`72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
) = 109
[pid  8371] exit_group(1)               = ?
[pid  8376] <... read resumed> <unfinished ...>) = ?
[pid  8375] <... read resumed> <unfinished ...>) = ?
[pid  8376] +++ exited with 1 +++
[pid  8375] +++ exited with 1 +++
[pid  8374] <... futex resumed>)        = ?
[pid  8374] +++ exited with 1 +++
+++ exited with 1 +++

How did I get a corrupted store? No idea. How to delete single items? 

I found this email from 宋文武 from 2016

"> How do I get rid of these ca. 30 outdated store items?
I think call gc for each one will work, eg:

for i in /gnu/store/*teensy*; do guix gc -d $i; done"

I adapted it to *minimatch* and deleted them all and checked the store
manually for any minimatch left and... it still did not work.

Hm.

Removed the hash for minimatch from the node.scm. Tried again. No luck.

Time to repair the store? Update the daemon via reconfigure?

I run vanilla 0.16.

-- 
Cheers 
Swedebugia

Re: NPM breakthrough with test packages and weird error

[swedebugia]

On 2019-01-12 18:57, swedebugia@riseup.net wrote:
> Hi
> 
> I have good (and bad) news!
> 
> Today I worked intensely on the node-build-system, importer and made
> some improvements as I ran into multiple errors using the importer and
> building the packages.
> 
> I tried first importing tape (test package, quite popular) and succeded
> with a few adjustments.
> 
> Then I tried tap, its a hairy mess unfortunately and pulls in del,
> typescript, grunt, nyc, and others
> 
> I write this message mostly because I now receive a really weird error
> from guix.
> 
> sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K node-grunt
> guix build: error: build failed: invalid hash
> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K
> node-tap-mocha-reporter
> guix build: error: build failed: invalid hash
> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> 
> The error is the same across multiple packages. Builds "succeede" as
> usual if they are already in the store or unrelated to grunt and nyc.
> 
> After investigating with strace I found this in the end:
> 
> [pid  8371] read(13, "download", 8)     = 8
> [pid  8371] write(13,
> "\10\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0minimatch-3.0.4."..., 944) = 944
> [pid  8371] read(13, "ptxc\0\0\0\0", 8) = 8
> [pid  8371] read(13, "K\0\0\0\0\0\0\0", 8) = 8
> [pid  8371] read(13, "invalid hash `72bd690b3fbd3be123"..., 75) = 75
> [pid  8371] read(13, "\0\0\0\0\0", 5)   = 5
> [pid  8371] read(13, "\1", 1)           = 1
> [pid  8371] read(13, "\0\0\0\0\0\0\0", 7) = 7
> [pid  8371] close(13)                   = 0
> [pid  8371] write(2, "guix build: error: build failed:"..., 109guix
> build: error: build failed: invalid hash
> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> ) = 109
> [pid  8371] exit_group(1)               = ?
> [pid  8376] <... read resumed> <unfinished ...>) = ?
> [pid  8375] <... read resumed> <unfinished ...>) = ?
> [pid  8376] +++ exited with 1 +++
> [pid  8375] +++ exited with 1 +++
> [pid  8374] <... futex resumed>)        = ?
> [pid  8374] +++ exited with 1 +++
> +++ exited with 1 +++
> 
> How did I get a corrupted store? No idea. How to delete single items? 
> 
> I found this email from 宋文武 from 2016
> 
> "> How do I get rid of these ca. 30 outdated store items?
> I think call gc for each one will work, eg:
> 
> for i in /gnu/store/*teensy*; do guix gc -d $i; done"
> 
> I adapted it to *minimatch* and deleted them all and checked the store
> manually for any minimatch left and... it still did not work.
> 
> Hm.
> 
> Removed the hash for minimatch from the node.scm. Tried again. No luck.
> 
> Time to repair the store? Update the daemon via reconfigure?
> 
> I run vanilla 0.16.

Pushed my changes here for anybody interested in taking a look
https://gitlab.com/swedebugia/guix/commits/npm2

(All my work build on the previous work of Jelle and Julien. Big thanks
to you!)

Resume of its status: 
* recursive import works now.
* cyclic dependencies are not detected by the importer
* you have to import an earlier version by hand if a cycle is
encountered.

I imported 400+ packages in one day and most of them work out of the box
:)

-- 
Cheers 
Swedebugia

Re: NPM breakthrough with test packages and weird error

[Julien Lepiller]

Le Sat, 12 Jan 2019 11:08:03 -0800,
swedebugia@riseup.net a écrit :

> On 2019-01-12 18:57, swedebugia@riseup.net wrote:
> > Hi
> > 
> > I have good (and bad) news!
> > 
> > Today I worked intensely on the node-build-system, importer and made
> > some improvements as I ran into multiple errors using the importer
> > and building the packages.
> > 
> > I tried first importing tape (test package, quite popular) and
> > succeded with a few adjustments.
> > 
> > Then I tried tap, its a hairy mess unfortunately and pulls in del,
> > typescript, grunt, nyc, and others
> > 
> > I write this message mostly because I now receive a really weird
> > error from guix.
> > 
> > sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K node-grunt
> > guix build: error: build failed: invalid hash
> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> > sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K
> > node-tap-mocha-reporter
> > guix build: error: build failed: invalid hash
> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> > 
> > The error is the same across multiple packages. Builds "succeede" as
> > usual if they are already in the store or unrelated to grunt and
> > nyc.
> > 
> > After investigating with strace I found this in the end:
> > 
> > [pid  8371] read(13, "download", 8)     = 8
> > [pid  8371] write(13,
> > "\10\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0minimatch-3.0.4."..., 944) = 944
> > [pid  8371] read(13, "ptxc\0\0\0\0", 8) = 8
> > [pid  8371] read(13, "K\0\0\0\0\0\0\0", 8) = 8
> > [pid  8371] read(13, "invalid hash `72bd690b3fbd3be123"..., 75) = 75
> > [pid  8371] read(13, "\0\0\0\0\0", 5)   = 5
> > [pid  8371] read(13, "\1", 1)           = 1
> > [pid  8371] read(13, "\0\0\0\0\0\0\0", 7) = 7
> > [pid  8371] close(13)                   = 0
> > [pid  8371] write(2, "guix build: error: build failed:"..., 109guix
> > build: error: build failed: invalid hash
> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
> > ) = 109
> > [pid  8371] exit_group(1)               = ?
> > [pid  8376] <... read resumed> <unfinished ...>) = ?
> > [pid  8375] <... read resumed> <unfinished ...>) = ?
> > [pid  8376] +++ exited with 1 +++
> > [pid  8375] +++ exited with 1 +++
> > [pid  8374] <... futex resumed>)        = ?
> > [pid  8374] +++ exited with 1 +++
> > +++ exited with 1 +++
> > 
> > How did I get a corrupted store? No idea. How to delete single
> > items? 
> > 
> > I found this email from 宋文武 from 2016
> >   
> > "> How do I get rid of these ca. 30 outdated store items?  
> > I think call gc for each one will work, eg:
> > 
> > for i in /gnu/store/*teensy*; do guix gc -d $i; done"
> > 
> > I adapted it to *minimatch* and deleted them all and checked the
> > store manually for any minimatch left and... it still did not work.
> > 
> > Hm.
> > 
> > Removed the hash for minimatch from the node.scm. Tried again. No
> > luck.
> > 
> > Time to repair the store? Update the daemon via reconfigure?
> > 
> > I run vanilla 0.16.  
> 
> Pushed my changes here for anybody interested in taking a look
> https://gitlab.com/swedebugia/guix/commits/npm2
> 
> (All my work build on the previous work of Jelle and Julien. Big
> thanks to you!)
> 
> Resume of its status: 
> * recursive import works now.
> * cyclic dependencies are not detected by the importer
> * you have to import an earlier version by hand if a cycle is
> encountered.
> 
> I imported 400+ packages in one day and most of them work out of the
> box :)
> 

Nice!

For the hash issue, I just had a similar one because I somehow removed
the hash in an origin record like so:

(sha256 (base32 ""))

The error is weird, but maybe the importer did something wrong?

Re: NPM breakthrough with test packages and weird error

[swedebugia]

[-- Attachment #1.1: Type: text/html, Size: 5245 bytes --]

[-- Attachment #1.2: Type: text/plain, Size: 4670 bytes --]

Julien Lepiller <julien@lepiller.eu> skrev: (12 januari 2019 23:26:10 CET)
>Le Sat, 12 Jan 2019 11:08:03 -0800,
>swedebugia@riseup.net a écrit :
>
>> On 2019-01-12 18:57, swedebugia@riseup.net wrote:
>> > Hi
>> > 
>> > I have good (and bad) news!
>> > 
>> > Today I worked intensely on the node-build-system, importer and
>made
>> > some improvements as I ran into multiple errors using the importer
>> > and building the packages.
>> > 
>> > I tried first importing tape (test package, quite popular) and
>> > succeded with a few adjustments.
>> > 
>> > Then I tried tap, its a hairy mess unfortunately and pulls in del,
>> > typescript, grunt, nyc, and others
>> > 
>> > I write this message mostly because I now receive a really weird
>> > error from guix.
>> > 
>> > sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K node-grunt
>> > guix build: error: build failed: invalid hash
>> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> > sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K
>> > node-tap-mocha-reporter
>> > guix build: error: build failed: invalid hash
>> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> > 
>> > The error is the same across multiple packages. Builds "succeede"
>as
>> > usual if they are already in the store or unrelated to grunt and
>> > nyc.
>> > 
>> > After investigating with strace I found this in the end:
>> > 
>> > [pid  8371] read(13, "download", 8)     = 8
>> > [pid  8371] write(13,
>> > "\10\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0minimatch-3.0.4."..., 944) = 944
>> > [pid  8371] read(13, "ptxc\0\0\0\0", 8) = 8
>> > [pid  8371] read(13, "K\0\0\0\0\0\0\0", 8) = 8
>> > [pid  8371] read(13, "invalid hash `72bd690b3fbd3be123"..., 75) =
>75
>> > [pid  8371] read(13, "\0\0\0\0\0", 5)   = 5
>> > [pid  8371] read(13, "\1", 1)           = 1
>> > [pid  8371] read(13, "\0\0\0\0\0\0\0", 7) = 7
>> > [pid  8371] close(13)                   = 0
>> > [pid  8371] write(2, "guix build: error: build failed:"..., 109guix
>> > build: error: build failed: invalid hash
>> > `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> > ) = 109
>> > [pid  8371] exit_group(1)               = ?
>> > [pid  8376] <... read resumed> <unfinished ...>) = ?
>> > [pid  8375] <... read resumed> <unfinished ...>) = ?
>> > [pid  8376] +++ exited with 1 +++
>> > [pid  8375] +++ exited with 1 +++
>> > [pid  8374] <... futex resumed>)        = ?
>> > [pid  8374] +++ exited with 1 +++
>> > +++ exited with 1 +++
>> > 
>> > How did I get a corrupted store? No idea. How to delete single
>> > items? 
>> > 
>> > I found this email from 宋文武 from 2016
>> >   
>> > "> How do I get rid of these ca. 30 outdated store items?  
>> > I think call gc for each one will work, eg:
>> > 
>> > for i in /gnu/store/*teensy*; do guix gc -d $i; done"
>> > 
>> > I adapted it to *minimatch* and deleted them all and checked the
>> > store manually for any minimatch left and... it still did not work.
>> > 
>> > Hm.
>> > 
>> > Removed the hash for minimatch from the node.scm. Tried again. No
>> > luck.
>> > 
>> > Time to repair the store? Update the daemon via reconfigure?
>> > 
>> > I run vanilla 0.16.  
>> 
>> Pushed my changes here for anybody interested in taking a look
>> https://gitlab.com/swedebugia/guix/commits/npm2
>> 
>> (All my work build on the previous work of Jelle and Julien. Big
>> thanks to you!)
>> 
>> Resume of its status: 
>> * recursive import works now.
>> * cyclic dependencies are not detected by the importer
>> * you have to import an earlier version by hand if a cycle is
>> encountered.
>> 
>> I imported 400+ packages in one day and most of them work out of the
>> box :)
>> 
>
>Nice!
>
>For the hash issue, I just had a similar one because I somehow removed
>the hash in an origin record like so:
>
>(sha256 (base32 ""))
>
>The error is weird, but maybe the importer did something wrong?

Thanks for the information. 

I switched to my guix on parabola and cloned first yours and them my repo.

Then I imported minimatch again to make sure the hash is right and it worked!

Status:

I now got these working 95 % built with guix:
* tap
* grunt
* coffee-script
* @babel/core and /cli

In total these have 300 dependencies that now work.

I got stuck with tape because one of the dependencies need typescript to build and typescript depends on gulp.

Gulp is a nightmare (304 dep with multiple circles if versions are not considered).

If I disable building on this one dependency tape would probably work.


-- 
Sent from my p≡p for Android.

[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 3825 bytes --]

Re: NPM breakthrough with test packages and weird error

[Julien Lepiller]

Le 2019-01-14 13:27, swedebugia a écrit :
> Julien Lepiller <julien@lepiller.eu> skrev: (12 januari 2019 23:26:10
> CET)
> 
>> Le Sat, 12 Jan 2019 11:08:03 -0800,
>> swedebugia@riseup.net a écrit :
>> 
>> On 2019-01-12 18:57, swedebugia@riseup.net wrote:
>> Hi
>> 
>> I have good (and bad) news!
>> 
>> Today I worked intensely on the node-build-system, importer and made
>> some improvements as I ran into multiple errors using the importer
>> and building the packages.
>> 
>> I tried first importing tape (test package, quite popular) and
>> succeded with a few adjustments.
>> 
>> Then I tried tap, its a hairy mess unfortunately and pulls in del,
>> typescript, grunt, nyc, and others
>> 
>> I write this message mostly because I now receive a really weird
>> error from guix.
>> 
>> sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K node-grunt
>> guix build: error: build failed: invalid hash
>> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> sdb@antelope ~/src/guix$ ./pre-inst-env guix build -K
>> node-tap-mocha-reporter
>> guix build: error: build failed: invalid hash
>> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> 
>> The error is the same across multiple packages. Builds "succeede" as
>> usual if they are already in the store or unrelated to grunt and
>> nyc.
>> 
>> After investigating with strace I found this in the end:
>> 
>> [pid 8371] read(13, "download", 8) = 8
>> [pid 8371] write(13,
>> "\10\0\0\0\0\0\0\0\27\0\0\0\0\0\0\0minimatch-3.0.4."..., 944) = 944
>> [pid 8371] read(13, "ptxc\0\0\0\0", 8) = 8
>> [pid 8371] read(13, "K\0\0\0\0\0\0\0", 8) = 8
>> [pid 8371] read(13, "invalid hash `72bd690b3fbd3be123"..., 75) = 75
>> [pid 8371] read(13, "\0\0\0\0\0", 5) = 5
>> [pid 8371] read(13, "\1", 1) = 1
>> [pid 8371] read(13, "\0\0\0\0\0\0\0", 7) = 7
>> [pid 8371] close(13) = 0
>> [pid 8371] write(2, "guix build: error: build failed:"..., 109guix
>> build: error: build failed: invalid hash
>> `72bd690b3fbd3be123e66c0c1932d856b294931d134cd07d4d4cd6aa1e07'
>> ) = 109
>> [pid 8371] exit_group(1) = ?
>> [pid 8376] <... read resumed> <unfinished ...>) = ?
>> [pid 8375] <... read resumed> <unfinished ...>) = ?
>> [pid 8376] +++ exited with 1 +++
>> [pid 8375] +++ exited with 1 +++
>> [pid 8374] <... futex resumed>) = ?
>> [pid 8374] +++ exited with 1 +++
>> +++ exited with 1 +++
>> 
>> How did I get a corrupted store? No idea. How to delete single
>> items?
>> 
>> I found this email from 宋文武 from 2016
>> 
>> "> How do I get rid of these ca. 30 outdated store items?
>> I think call gc for each one will work, eg:
>> 
>> for i in /gnu/store/*teensy*; do guix gc -d $i; done"
>> 
>> I adapted it to *minimatch* and deleted them all and checked the
>> store manually for any minimatch left and... it still did not work.
>> 
>> Hm.
>> 
>> Removed the hash for minimatch from the node.scm. Tried again. No
>> luck.
>> 
>> Time to repair the store? Update the daemon via reconfigure?
>> 
>> I run vanilla 0.16.
>> 
>> Pushed my changes here for anybody interested in taking a look
>> https://gitlab.com/swedebugia/guix/commits/npm2
>> 
>> (All my work build on the previous work of Jelle and Julien. Big
>> thanks to you!)
>> 
>> Resume of its status:
>> * recursive import works now.
>> * cyclic dependencies are not detected by the importer
>> * you have to import an earlier version by hand if a cycle is
>> encountered.
>> 
>> I imported 400+ packages in one day and most of them work out of the
>> box :)
> 
> Nice!
> 
> For the hash issue, I just had a similar one because I somehow removed
> the hash in an origin record like so:
> 
> (sha256 (base32 ""))
> 
> The error is weird, but maybe the importer did something wrong?
> 
> Thanks for the information.
> 
> I switched to my guix on parabola and cloned first yours and them my
> repo.
> 
> Then I imported minimatch again to make sure the hash is right and it
> worked!
> 
> Status:
> 
> I now got these working 95 % built with guix:
> * tap
> * grunt
> * coffee-script

Sorry, but coffeescript is a compiler. Like almost every compiler these 
days,
it's written in its own language, and isn't bootstrapped (in the sense 
of
https://bootstrappable.org). They provide a directory with javascript
"sources" (generated from the previous version in the best case, or an
unreleased version quite often), but the actual sources are written in
coffeescript. I've already tried to build a version from the old ruby
sources, but even they weren't able to build the compiler on the 
following
commit…

If we want to bootstrap coffeescript, we'll need an alternative 
implementation
of a coffeescript compiler or interpreter.

> * @babel/core and /cli

Let's make sure babel doesn't suffer from the same flaw...

> 
> In total these have 300 dependencies that now work.
> 
> I got stuck with tape because one of the dependencies need typescript
> to build and typescript depends on gulp.
> 
> Gulp is a nightmare (304 dep with multiple circles if versions are not
> considered).
> 
> If I disable building on this one dependency tape would probably work.
> 
> --
> Sent from my p≡p for Android.