Re: How to make GNU Guile more successful

Re: How to make GNU Guile more successful

[Michael Vehrs]

As a late-comer to this discussion, here are my two cents. The thing I 
miss most is a central package repository like Eggs Unlimited 
(http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the 
Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course. 
Sure, a bespoke package manager might be nifty, but a single curated 
list of packages would be a game-changer.


Regards


Michael

Re: How to make GNU Guile more successful

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]


Michael Vehrs <Michael.Burschik@gmx.de> writes:

> As a late-comer to this discussion, here are my two cents. The thing I 
> miss most is a central package repository like Eggs Unlimited 
> (http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the 
> Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course. 
> Sure, a bespoke package manager might be nifty, but a single curated 
> list of packages would be a game-changer.

In theory we have guildhall for this: https://github.com/ijp/guildhall

In practice it does not provide a web interface for uploading packages.

If you want to do something truly exciting, you could take wingos fibers
and build a high performance web interface for guildhall with them.

This is something I’d love to do but I fear that it’s not high enough in
my todo list that I’ll actually get it done.¹

Best wishes,
Arne

¹: I became Freenet Release Manager a few months ago. That took away the
   last free time I could allocate to new challenging projects.
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: How to make GNU Guile more successful

[Michael Vehrs]

On 02/20/2017 09:41 PM, Arne Babenhauserheide wrote:
> Michael Vehrs <Michael.Burschik@gmx.de> writes:
>
>> As a late-comer to this discussion, here are my two cents. The thing I
>> miss most is a central package repository like Eggs Unlimited
>> (http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
>> Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
>> Sure, a bespoke package manager might be nifty, but a single curated
>> list of packages would be a game-changer.
> In theory we have guildhall for this: https://github.com/ijp/guildhall

In theory, yes. But there isn't actually very much available in the 
repository.

>
> In practice it does not provide a web interface for uploading packages.
>
> If you want to do something truly exciting, you could take wingos fibers
> and build a high performance web interface for guildhall with them.

High performance is not really important in this case. We are not 
talking about gazillions of npm packages.

>
> This is something I’d love to do but I fear that it’s not high enough in
> my todo list that I’ll actually get it done.¹
>
> Best wishes,
> Arne
>
> ¹: I became Freenet Release Manager a few months ago. That took away the
>     last free time I could allocate to new challenging projects.

If someone had a realistic plan of building something like that, I might 
be able to contribute. I am not going to tackle it alone.


Regards

Michael

Re: How to make GNU Guile more successful

[Arne Babenhauserheide]

Michael Vehrs writes:

> On 02/20/2017 09:41 PM, Arne Babenhauserheide wrote:
>> Michael Vehrs <Michael.Burschik@gmx.de> writes:
>>
>>> As a late-comer to this discussion, here are my two cents. The thing I
>>> miss most is a central package repository like Eggs Unlimited
>>> (http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
>>> Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
>>> Sure, a bespoke package manager might be nifty, but a single curated
>>> list of packages would be a game-changer.
>> In theory we have guildhall for this: https://github.com/ijp/guildhall
>
> In theory, yes. But there isn't actually very much available in the 
> repository.

I think the main reasons for that are (a) that it’s unmaintained (you
can’t just get your package added), (b) that it’s underdocumented (it
doesn’t say how to add a package or create a repo in less than 5 steps),
and (c) that there is no web interface for uploading a package.

>> In practice it does not provide a web interface for uploading packages.
>>
>> If you want to do something truly exciting, you could take wingos fibers
>> and build a high performance web interface for guildhall with them.
>
> High performance is not really important in this case. We are not 
> talking about gazillions of npm packages.

That’s true. That’s why I wrote exciting :)

You could also use the existing (web server) tools to build such a site.

Best wishes,
Arne

Re: How to make GNU Guile more successful

[Amirouche]

Le 21/02/2017 à 07:01, Michael Vehrs a écrit :
> On 02/20/2017 09:41 PM, Arne Babenhauserheide wrote:
>> Michael Vehrs <Michael.Burschik@gmx.de> writes:
>>
>>> As a late-comer to this discussion, here are my two cents. The thing I
>>> miss most is a central package repository like Eggs Unlimited
>>> (http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
>>> Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
>>> Sure, a bespoke package manager might be nifty, but a single curated
>>> list of packages would be a game-changer.
>> In theory we have guildhall for this: https://github.com/ijp/guildhall
>
> In theory, yes. But there isn't actually very much available in the 
> repository.
>
>>
>> In practice it does not provide a web interface for uploading packages.
>>
>> If you want to do something truly exciting, you could take wingos fibers
>> and build a high performance web interface for guildhall with them.
>
> High performance is not really important in this case. We are not 
> talking about gazillions of npm packages.
>
>>
>> This is something I’d love to do but I fear that it’s not high enough in
>> my todo list that I’ll actually get it done.¹
>>
>> Best wishes,
>> Arne
>>
>> ¹: I became Freenet Release Manager a few months ago. That took away the
>>     last free time I could allocate to new challenging projects.
>
> If someone had a realistic plan of building something like that, I 
> might be able to contribute. I am not going to tackle it alone.
>

What's the status of guildhall, does it work? where are the latest changes?

amirouche ~ amz3

Re: How to make GNU Guile more successful

[Amirouche]

Le 21/02/2017 à 18:18, Arne Babenhauserheide a écrit :
> Michael Vehrs writes:
>
>> On 02/20/2017 09:41 PM, Arne Babenhauserheide wrote:
>>> Michael Vehrs <Michael.Burschik@gmx.de> writes:
>>>> As a late-comer to this discussion, here are my two ce
>>> In practice it does not provide a web interface for uploading packages.
>>>
>>> If you want to do something truly exciting, you could take wingos fibers
>>> and build a high performance web interface for guildhall with them.
>> High performance is not really important in this case. We are not
>> talking about gazillions of npm packages.
> That’s true. That’s why I wrote exciting :)
>
> You could also use the existing (web server) tools to build such a site.

We can make it exciting and use fibers web server nonetheless.

The interface of the *fibers* web server is the same as guile web
server interface.

Here is it [0]:

```

(use-modules (fibers web server))

(define (handler request body)
   (values '((content-type . (text/plain)))
           "Hello, World!"))

(run-server handler)

```

Basically you just need to change the use-modules line in your current
project.

[0]

Re: How to make GNU Guile more successful

[Mike Gran]

>>> If you want to do something truly exciting, you could take wingos fibers

>>> and build a high performance web interface for guildhall with them.
>> High performance is not really important in this case. We are not
>> talking about gazillions of npm packages.
> That’s true. That’s why I wrote exciting :)
>
> You could also use the existing (web server) tools to build such a site.



The fastest thing would be to create a git project with a
{meta}/{maintainer}/{project} type structure and let people hack away unsupervised
in their {maintainer} subdirectories.  Then add a thin libraries that
allows one to pick a specific Git commit UUID as a version number.

It would be chaos, but, we don't need professionalism when there is nothing
about which to be professional.


Or, to be fancier, clone metacpan. It exists.


-Mike

Re: How to make GNU Guile more successful

[Amirouche]

Le 21/02/2017 à 19:31, Mike Gran a écrit :
>
>>>> If you want to do something truly exciting, you could take wingos fibers
>>>> and build a high performance web interface for guildhall with them.
>>> High performance is not really important in this case. We are not
>>> talking about gazillions of npm packages.
>> That’s true. That’s why I wrote exciting :)
>>
>> You could also use the existing (web server) tools to build such a site.
>
>
> The fastest thing would be to create a git project with a
> {meta}/{maintainer}/{project} type structure and let people hack away unsupervised
> in their {maintainer} subdirectories.  Then add a thin libraries that
> allows one to pick a specific Git commit UUID as a version number.
>
> It would be chaos, but, we don't need professionalism when there is nothing
> about which to be professional.
>
>
> Or, to be fancier, clone metacpan. It exists.

Sorry, I don't understand you want to use git as the package index?

Re: How to make GNU Guile more successful

[Mike Gran]

On Tuesday, February 21, 2017 10:33 AM, Amirouche <amirouche@hypermove.net> wrote:


> Sorry, I don't understand you want to use git as the package index?


Just have a top-level README.md that everyone edits as the package index.
It would suffice until so long as the number of packages remains in the low
hundreds.

Re: How to make GNU Guile more successful

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2098 bytes --]


Amirouche <amirouche@hypermove.net> writes:
>> If someone had a realistic plan of building something like that, I 
>> might be able to contribute. I am not going to tackle it alone.
>
> What's the status of guildhall, does it work? where are the latest changes?

What we need is someone who regularly puts the package folder onto a
static webserver (starnge that I did not think of just using a repo
instead of implementing a full web interface with access rights and
such).

Guildhall works with guile-2.0, and there are no changes (though I have
a local version in which `guild hall` outputs the default installation
locations).

However I get an error with guile 2.1.5:

$ make
./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable -Warity-mismatch -Wformat -o "guildhall/cli/config.go" "guildhall/cli/config.scm"
Backtrace:
In ice-9/boot-9.scm:
  2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
  2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide …)
  2724:10 17 (_ (guildhall config) _ _ #:ensure _)
  3000:16 16 (try-module-autoload _ _)
   2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot…>)
  3020:22 14 (_)
In unknown file:
          13 (primitive-load-path "guildhall/config" #<procedure d47…>)
In guildhall/config.scm:
     25:0 12 (_)
In ice-9/boot-9.scm:
  2799:17 11 (resolve-interface (guildhall destination fhs) #:select …)
  2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
  3000:16  9 (try-module-autoload _ _)
   2336:4  8 (save-module-excursion #<procedure da8930 at ice-9/boot…>)
  3020:22  7 (_)
In unknown file:
           6 (primitive-load-path "guildhall/destination/fhs" #<proc…>)
In guildhall/destination/fhs.scm:
    152:0  5 (_)
In guildhall/ext/inc/irregex-r6rs.scm:
  1505:17  4 (sre->irregex _ . _)
  2448:15  3 (sre->nfa _ 0)
  2391:29  2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
  2301:52  1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
In ice-9/boot-9.scm:
   762:26  0 (dispatch-exception _ _ _)

Best wishes,
Arne

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: How to make GNU Guile more successful

[Michael Vehrs]

On 02/21/2017 07:01 AM, Michael Vehrs wrote:
> On 02/20/2017 09:41 PM, Arne Babenhauserheide wrote:
>> Michael Vehrs <Michael.Burschik@gmx.de> writes:
>>
>>> As a late-comer to this discussion, here are my two cents. The thing I
>>> miss most is a central package repository like Eggs Unlimited
>>> (http://wiki.call-cc.org/chicken-projects/egg-index-4.html), or the
>>> Racket Package List (http://pkgs.racket-lang.org/), or CPAN, of course.
>>> Sure, a bespoke package manager might be nifty, but a single curated
>>> list of packages would be a game-changer.
>> In theory we have guildhall for this: https://github.com/ijp/guildhall
>
> In theory, yes. But there isn't actually very much available in the 
> repository.
>
>>
>> In practice it does not provide a web interface for uploading packages.
>>
>> If you want to do something truly exciting, you could take wingos fibers
>> and build a high performance web interface for guildhall with them.
>
> High performance is not really important in this case. We are not 
> talking about gazillions of npm packages.
>
>>
>> This is something I’d love to do but I fear that it’s not high enough in
>> my todo list that I’ll actually get it done.¹
>>
>> Best wishes,
>> Arne
>>
>> ¹: I became Freenet Release Manager a few months ago. That took away the
>>     last free time I could allocate to new challenging projects.
>
> If someone had a realistic plan of building something like that, I 
> might be able to contribute. I am not going to tackle it alone.
>
>
> Regards
>
> Michael
>
>

By the way, this (http://sph.mn/content/3e73) is the most complete list 
of guile projects I have seen so far.


Regards

Michael

Re: How to make GNU Guile more successful

[Amirouche]

Le 21/02/2017 à 20:25, Arne Babenhauserheide a écrit :
> Amirouche <amirouche@hypermove.net> writes:
>>> If someone had a realistic plan of building something like that, I
>>> might be able to contribute. I am not going to tackle it alone.
>> What's the status of guildhall, does it work? where are the latest changes?
> What we need is someone who regularly puts the package folder onto a
> static webserver

What does it mean?

> (starnge that I did not think of just using a repo
> instead of implementing a full web interface with access rights and
> such).

The issue I have with the web repository (I assume you meant git repo),
is that it requires for the maintainer of the repository to push a button,
accept new packages (at the very least). Which would mean that I offer
some kind of guarantees over the content of the software that you
download. Which maybe right now I can assume, but if there is as much
as patch in the repo as there is in guix I will need help...

>
> Guildhall works with guile-2.0, and there are no changes (though I have
> a local version in which `guild hall` outputs the default installation
> locations).
>
> However I get an error with guile 2.1.5:
>
> $ make
> ./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable -Warity-mismatch -Wformat -o "guildhall/cli/config.go" "guildhall/cli/config.scm"
> Backtrace:
> In ice-9/boot-9.scm:
>    2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
>    2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide …)
>    2724:10 17 (_ (guildhall config) _ _ #:ensure _)
>    3000:16 16 (try-module-autoload _ _)
>     2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot…>)
>    3020:22 14 (_)
> In unknown file:
>            13 (primitive-load-path "guildhall/config" #<procedure d47…>)
> In guildhall/config.scm:
>       25:0 12 (_)
> In ice-9/boot-9.scm:
>    2799:17 11 (resolve-interface (guildhall destination fhs) #:select …)
>    2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
>    3000:16  9 (try-module-autoload _ _)
>     2336:4  8 (save-module-excursion #<procedure da8930 at ice-9/boot…>)
>    3020:22  7 (_)
> In unknown file:
>             6 (primitive-load-path "guildhall/destination/fhs" #<proc…>)
> In guildhall/destination/fhs.scm:
>      152:0  5 (_)
> In guildhall/ext/inc/irregex-r6rs.scm:
>    1505:17  4 (sre->irregex _ . _)
>    2448:15  3 (sre->nfa _ 0)
>    2391:29  2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
>    2301:52  1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
> In ice-9/boot-9.scm:
>     762:26  0 (dispatch-exception _ _ _)
>
> Best wishes,
> Arne

Re: How to make GNU Guile more successful

[Nala Ginrut]

I think we have to elaborate the question clearer.

1. How to make guile-scheme more successful?
I think this is similar to ask "how to make scheme more successful".
This is the big question to the whole scheme community.

2. How to make guile platform more successful?
I this case, let me ask a question, if we have guile-python3
(compatible with most of Python3 features), and provide more
convenient FFI helper function to help bind more libraries in short
time, is it possible to attract more people from Python land? Given
we'll have good JIT compiler in the future.


On Thu, Mar 2, 2017 at 3:25 AM, Amirouche <amirouche@hypermove.net> wrote:
>
>
> Le 21/02/2017 à 20:25, Arne Babenhauserheide a écrit :
>>
>> Amirouche <amirouche@hypermove.net> writes:
>>>>
>>>> If someone had a realistic plan of building something like that, I
>>>> might be able to contribute. I am not going to tackle it alone.
>>>
>>> What's the status of guildhall, does it work? where are the latest
>>> changes?
>>
>> What we need is someone who regularly puts the package folder onto a
>> static webserver
>
>
> What does it mean?
>
>> (starnge that I did not think of just using a repo
>> instead of implementing a full web interface with access rights and
>> such).
>
>
> The issue I have with the web repository (I assume you meant git repo),
> is that it requires for the maintainer of the repository to push a button,
> accept new packages (at the very least). Which would mean that I offer
> some kind of guarantees over the content of the software that you
> download. Which maybe right now I can assume, but if there is as much
> as patch in the repo as there is in guix I will need help...
>
>
>>
>> Guildhall works with guile-2.0, and there are no changes (though I have
>> a local version in which `guild hall` outputs the default installation
>> locations).
>>
>> However I get an error with guile 2.1.5:
>>
>> $ make
>> ./env /home/arne/.local/bin//guile-tools compile -Wunbound-variable
>> -Warity-mismatch -Wformat -o "guildhall/cli/config.go"
>> "guildhall/cli/config.scm"
>> Backtrace:
>> In ice-9/boot-9.scm:
>>    2860:10 19 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
>>    2799:17 18 (resolve-interface (guildhall config) #:select _ #:hide …)
>>    2724:10 17 (_ (guildhall config) _ _ #:ensure _)
>>    3000:16 16 (try-module-autoload _ _)
>>     2336:4 15 (save-module-excursion #<procedure 6d5d80 at ice-9/boot…>)
>>    3020:22 14 (_)
>> In unknown file:
>>            13 (primitive-load-path "guildhall/config" #<procedure d47…>)
>> In guildhall/config.scm:
>>       25:0 12 (_)
>> In ice-9/boot-9.scm:
>>    2799:17 11 (resolve-interface (guildhall destination fhs) #:select …)
>>    2724:10 10 (_ (guildhall destination fhs) _ _ #:ensure _)
>>    3000:16  9 (try-module-autoload _ _)
>>     2336:4  8 (save-module-excursion #<procedure da8930 at ice-9/boot…>)
>>    3020:22  7 (_)
>> In unknown file:
>>             6 (primitive-load-path "guildhall/destination/fhs" #<proc…>)
>> In guildhall/destination/fhs.scm:
>>      152:0  5 (_)
>> In guildhall/ext/inc/irregex-r6rs.scm:
>>    1505:17  4 (sre->irregex _ . _)
>>    2448:15  3 (sre->nfa _ 0)
>>    2391:29  2 (lp ((* any) (: "." ($ (+ (~ numeric))))) 1 0 0)
>>    2301:52  1 (lp ("." ($ (+ (~ numeric)))) 1 0 _)
>> In ice-9/boot-9.scm:
>>     762:26  0 (dispatch-exception _ _ _)
>>
>> Best wishes,
>> Arne
>
>
>

Re: How to make GNU Guile more successful

[David Kastrup]

Nala Ginrut <nalaginrut@gmail.com> writes:

> I think we have to elaborate the question clearer.
>
> 1. How to make guile-scheme more successful?
> I think this is similar to ask "how to make scheme more successful".
> This is the big question to the whole scheme community.
>
> 2. How to make guile platform more successful?
> I this case, let me ask a question, if we have guile-python3
> (compatible with most of Python3 features), and provide more
> convenient FFI helper function to help bind more libraries in short
> time, is it possible to attract more people from Python land? Given
> we'll have good JIT compiler in the future.

Frankly, I doubt that migration of large Python-based applications is
going to be a thing when nobody can even be bothered with immersing
himself in the problems with migrating LilyPond from Guile-1.8 to
Guile-2.

-- 
David Kastrup

Re: How to make GNU Guile more successful

[Nala Ginrut]

On Fri, Mar 3, 2017 at 5:18 PM, David Kastrup <dak@gnu.org> wrote:
> Frankly, I doubt that migration of large Python-based applications is
> going to be a thing when nobody can even be bothered with immersing
> himself in the problems with migrating LilyPond from Guile-1.8 to
> Guile-2.

No, I don't think so.
If we have guile-python3, the migration work becomes attractive to
Guile community. Because each time you migrate a library, it can be
used in all languages implemented on Guile platform. Write only once
for all supported languages. Even if we don't care about Python (do
we?), we have to write many libraries for Guile to make it more
useful. It's the work Guile community has to do anyway.

And LilyPond is not a good case here, since not everybody needs it. I
think the best way to push a community is to provide convenient way to
let users who care certain library to contribute it. But we don't have
it now. For example, the documentation or tools to help 1.8->2.0.
Python has tools for Python2->Python3 and documents for it. It is the
management of Guile community, not technical problem.
Fortunately, it's just management problem, and it's easier to improve
than technical one, only if we found a persistent way to push and
there's enough contributors.

Re: How to make GNU Guile more successful

[David Kastrup]

Nala Ginrut <nalaginrut@gmail.com> writes:

> On Fri, Mar 3, 2017 at 5:18 PM, David Kastrup <dak@gnu.org> wrote:
>
>> Frankly, I doubt that migration of large Python-based applications is
>> going to be a thing when nobody can even be bothered with immersing
>> himself in the problems with migrating LilyPond from Guile-1.8 to
>> Guile-2.
>
> No, I don't think so.
> If we have guile-python3, the migration work becomes attractive to
> Guile community. Because each time you migrate a library, it can be
> used in all languages implemented on Guile platform.

The .go organization and call gate costs (for example constant string
conversions) and memory organization and foreign string hardiness issues
bogging down LilyPond will affect interfacing to every external library
with a high call rate for processing.

> I think the best way to push a community is to provide convenient way
> to let users who care certain library to contribute it. But we don't
> have it now. For example, the documentation or tools to help 1.8->2.0.
> Python has tools for Python2->Python3 and documents for it. It is the
> management of Guile community, not technical problem.  Fortunately,
> it's just management problem, and it's easier to improve than
> technical one, only if we found a persistent way to push and there's
> enough contributors.

The technical problems won't go away by themselves.

So which migration of a large Python-based application do you expect to
become a thing without addressing significant amounts of technical
problems first?  Or how do I have to interpret your "No, I don't think
so."?

-- 
David Kastrup

Re: How to make GNU Guile more successful

[Nala Ginrut]

On Fri, Mar 3, 2017 at 8:19 PM, David Kastrup <dak@gnu.org> wrote:
> The .go organization and call gate costs (for example constant string
> conversions) and memory organization and foreign string hardiness issues
> bogging down LilyPond will affect interfacing to every external library
> with a high call rate for processing.

I respect LilyPond contributors and people who put effort on it. But
it's not on my hack-line.
If someone really care about it, it'll be done someday definitely.

> The technical problems won't go away by themselves.
>
> So which migration of a large Python-based application do you expect to
> become a thing without addressing significant amounts of technical
> problems first?  Or how do I have to interpret your "No, I don't think
> so."?
>

If I don't say "no, I don't think so", should I say "well, yes, Guile
has no chance, let's remove it".? ;-)
I think most of the people in Guile community knows the problems and
difficulties, it's unnecessary to repeat them again, what we need is
to find the solution.
I gave part of solution, and I'm enjoying to go with it.
If you're afraid of "large python based apps", how about get a seat
and see how others solve it?
Anyway, talking the problems and difficulties people already know
again and again is a contribution too. Then we don't need to write
TODO.

Do you think I know nothing about the complexity? I'll tell you that I
love that complexity so I'm in.
When I came to Guile community 7 years ago, I know nothing as a pure newbie.
But now I know I can learn many things each time when I challenge the
difficulties.
If you feel depressed when you face LilyPond, please don't forget I
feel frustrated when I'm debugging the Artanis server-core and our JIT
compiler too.
No one is facing the easy job.

Frankly, I have better idea than migrating all Python pacakges: just
migrate when you need it, or write a better one.
I'm trying to use Guile things in real products, so I'm serious.

I still want to ask "why do we have to compare to Python?"
I'm going to write guile-python3 because some young people in my team
has difficult to use Scheme directly, and we don't want them to
rewrite whole things in Scheme again.
But Guile will bring many potential advantages rather than our current
Python and C++14 combination.

Be more confident, folks.
That's why the answer from me is "no, I don't think so".

Best regards.

Re: How to make GNU Guile more successful

[Matt Wette]

> On Mar 2, 2017, at 9:28 PM, Nala Ginrut <nalaginrut@gmail.com> wrote:

> 2. How to make guile platform more successful?
> I this case, let me ask a question, if we have guile-python3
> (compatible with most of Python3 features), and provide more
> convenient FFI helper function to help bind more libraries in short
> time, is it possible to attract more people from Python land? Given
> we'll have good JIT compiler in the future.

Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.

Matt

Re: How to make GNU Guile more successful

[Amirouche]

Le 03/03/2017 à 18:21, Matt Wette a écrit :
>> On Mar 2, 2017, at 9:28 PM, Nala Ginrut <nalaginrut@gmail.com> wrote:
>> 2. How to make guile platform more successful?
>> I this case, let me ask a question, if we have guile-python3
>> (compatible with most of Python3 features), and provide more
>> convenient FFI helper function to help bind more libraries in short
>> time, is it possible to attract more people from Python land? Given
>> we'll have good JIT compiler in the future.
> Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.
>
> Matt
Tx for working on that!

Re: How to make GNU Guile more successful

[Amirouche]

Le 03/03/2017 à 18:21, Matt Wette a écrit :
>> On Mar 2, 2017, at 9:28 PM, Nala Ginrut <nalaginrut@gmail.com> wrote:
>> 2. How to make guile platform more successful?
>> I this case, let me ask a question, if we have guile-python3
>> (compatible with most of Python3 features), and provide more
>> convenient FFI helper function to help bind more libraries in short
>> time, is it possible to attract more people from Python land? Given
>> we'll have good JIT compiler in the future.
> Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.
>
> Matt
>
>
Are you aware of scheme-bytestructures [0] that's what we are using
in guile-git. If you target that API to wrap C structs, it would be helpful.

[0] https://github.com/TaylanUB/scheme-bytestructures

Re: How to make GNU Guile more successful

[Mike Gran]

> On Friday, March 3, 2017 11:18 AM, Amirouche <amirouche@hypermove.net> wrote:

>> Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.


> Are you aware of scheme-bytestructures [0] that's what we are using> in guile-git. If you target that API to wrap C structs, it would be helpful.

SWIG used to work for Guile.  It was a rather complete solution to
the wrapping problem.

Re: How to make GNU Guile more successful

[Matt Wette]

> On Mar 3, 2017, at 11:16 AM, Amirouche <amirouche@hypermove.net> wrote:
>> Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.
> Are you aware of scheme-bytestructures [0] that's what we are using
> in guile-git. If you target that API to wrap C structs, it would be helpful.
> 
> [0] https://github.com/TaylanUB/scheme-bytestructures <https://github.com/TaylanUB/scheme-bytestructures>
Cloned it.  Thanks for the reference. — Matt

Re: How to make GNU Guile more successful

[Matt Wette]

> On Mar 3, 2017, at 11:24 AM, Mike Gran <spk121@yahoo.com> wrote:
> 
>> On Friday, March 3, 2017 11:18 AM, Amirouche <amirouche@hypermove.net> wrote:
> 
>>> Now that the C preprocessor in NYACC/C99 is more robust I could start looking at generating some sort of  “FFI helper” functionality.  I have been thinking about this problem on and off for a while.  Completely automating things might be tough, but a “helper” would probably go a long way.
> 
> 
>> Are you aware of scheme-bytestructures [0] that's what we are using> in guile-git. If you target that API to wrap C structs, it would be helpful.
> 
> SWIG used to work for Guile.  It was a rather complete solution to
> the wrapping problem.

I had forgotten about that.  Seems targeted to 1.X but might be worth trying to autogenerate swig template or equivalent to start.

Re: How to make GNU Guile more successful

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 17190 bytes --]


David Kastrup <dak@gnu.org> writes:

> Nala Ginrut <nalaginrut@gmail.com> writes:
>
>> I think we have to elaborate the question clearer.
>>
>> 1. How to make guile-scheme more successful?
>> I think this is similar to ask "how to make scheme more successful".
>> This is the big question to the whole scheme community.
>>
>> 2. How to make guile platform more successful?
>> I this case, let me ask a question, if we have guile-python3
>> (compatible with most of Python3 features), and provide more
>> convenient FFI helper function to help bind more libraries in short
>> time, is it possible to attract more people from Python land? Given
>> we'll have good JIT compiler in the future.
>
> Frankly, I doubt that migration of large Python-based applications is
> going to be a thing when nobody can even be bothered with immersing
> himself in the problems with migrating LilyPond from Guile-1.8 to
> Guile-2.

I worked on testing Lilypond with user installed Guile 2.x, does that
count?

Just to have it recorded somewhere, here’s a patch to lilypond along
with a copy of the bash history of the setup (cleaned up, it was many
times as long):

## patch

From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)

---
 configure.ac         | 7 ++++++-
 lily/pdf-scheme.cc   | 4 ++++
 scm/memory-trace.scm | 3 ++-
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/configure.ac b/configure.ac
index d77ea15..393976b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
 STEPMAKE_WINDOWS
 
 # guile executable for some scripts
-STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+if test "$GUILEv2" = "yes"
+then
+    STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
+else
+    STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
+fi
 
 # perl for help2man and for mf2pt1.pl
 STEPMAKE_PERL(REQUIRED)
diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..f5ae70c 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
    *                  (string->utf16 str 'big)
    */
   if (g)
+#if GUILEV2
+    return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
+#else
     return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+#endif
   else
     return str;
 }
diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
index d8ffeb9..9ebd722 100644
--- a/scm/memory-trace.scm
+++ b/scm/memory-trace.scm
@@ -2,7 +2,8 @@
 
 (define-module (scm memory-trace))
 (use-modules (lily)
-             (ice-9 format))
+             (ice-9 format)
+             (ice-9 threads))
 
 (define-public (mtrace:start-trace freq)
   (set! usecond-interval (inexact->exact (/ 1000000 freq)))
-- 
2.10.2


## bash history

sudo emerge media-fonts/tex-gyre
sudo nano /etc/portage/package.keywords/sonstiges 
sudo emerge media-fonts/tex-gyre
sudo pmerge dblatex
git clone git://git.sv.gnu.org/lilypond.git
cd lilypond/
./autogen.sh --prefix ~/.local --enable-guile2
mkdir -p ~/texmf/lh
cp ~/Downloads/lhfnt35g-source.zip ~/texmf/lh/
unzip lhfnt35g-source.zip 
cd ~/texmf/source/latex/lh
latex lcyfonts.ins 
latex ot2fonts.ins; latex t2ccfonts.ins
ls ~/texmf/examples/
mkdir ~/texmf/lh
cp *sty ~/texmf/lh/
./autogen.sh --prefix ~/.local --enable-guile2
sudo emerge dev-texlive/texlive-langcyrillic
sudo pmerge dev-texlive/texlive-langcyrillic
make out/lilypond-invoke-editor
cd scripts/
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor --no-discard-stderr
/home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor 
./autogen.sh --prefix ~/.local --enable-guile2
make
LD_LIBRARY_PATH=/home/arne/.local/lib/guile/2.2/ g++ -o out/lilypond  ./out/modified-font-metric.o ./out/sequential-iterator.o ./out/hara-kiri-group-spanner.o ./out/completion-note-heads-engraver.o ./out/slur-score-parameters.o ./out/flag.o ./out/span-arpeggio-engraver.o ./out/beam-quanting.o ./out/dynamic-performer.o ./out/parse-scm.o ./out/tie.o ./out/bend-engraver.o ./out/staff-spacing.o ./out/rest-collision-engraver.o ./out/simple-spacer-scheme.o ./out/audio-item.o ./out/pdf-scheme.o ./out/paper-score.o ./out/slur-configuration.o ./out/constrained-breaking.o ./out/score-engraver.o ./out/unpure-pure-container.o ./out/multi-measure-rest-engraver.o ./out/system-start-delimiter-engraver.o ./out/context-property.o ./out/freetype.o ./out/interval-minefield.o ./out/kievan-ligature.o ./out/completion-rest-engraver.o ./out/pure-from-neighbor-engraver.o ./out/horizontal-bracket-engraver.o ./out/grob-closure.o ./out/default-bar-line-engraver.o ./out/control-track-performer.o ./out/file-name-map.o ./out/audio-staff.o ./out/line-spanner.o ./out/grob-info.o ./out/spring.o ./out/slur.o ./out/music-scheme.o ./out/program-option.o ./out/fretboard-engraver.o ./out/hyphen-engraver.o ./out/page-marker.o ./out/global-vars.o ./out/stream-event-scheme.o ./out/script-column-engraver.o ./out/bezier-bow.o ./out/page-breaking-scheme.o ./out/lily-version.o ./out/repeat-tie-engraver.o ./out/tweak-engraver.o ./out/staff-symbol-referencer.o ./out/moment-scheme.o ./out/note-heads-engraver.o ./out/paper-outputter.o ./out/music-wrapper.o ./out/grob.o ./out/performance-scheme.o ./out/bar-line.o ./out/page-turn-engraver.o ./out/dimensions-scheme.o ./out/tab-note-heads-engraver.o ./out/piano-pedal-bracket.o ./out/spaceable-grob.o ./out/event-iterator.o ./out/lyric-combine-music-iterator.o ./out/percent-repeat-iterator.o ./out/audio-element-info.o ./out/midi-cc-announcer.o ./out/font-metric.o ./out/note-spacing-engraver.o ./out/duration-scheme.o ./out/accidental-placement.o ./out/music-iterator.o ./out/rhythmic-head.o ./out/grob-array.o ./out/tab-tie-follow-engraver.o ./out/context.o ./out/dispatcher.o ./out/scale.o ./out/chord-name-engraver.o ./out/auto-change-iterator.o ./out/tie-engraver.o ./out/grid-line-interface.o ./out/forbid-break-engraver.o ./out/note-column.o ./out/lookup.o ./out/global-ctor.o ./out/custos-engraver.o ./out/piano-pedal-engraver.o ./out/simple-music-iterator.o ./out/pitch-scheme.o ./out/mensural-ligature-engraver.o ./out/articulations.o ./out/horizontal-bracket.o ./out/script-column.o ./out/global-context.o ./out/slur-performer.o ./out/guile-init.o ./out/stencil-scheme.o ./out/font-size-engraver.o ./out/rest-engraver.o ./out/skyline.o ./out/percent-repeat-item.o ./out/enclosing-bracket.o ./out/module-scheme.o ./out/input.o ./out/paper-outputter-scheme.o ./out/spacing-interface.o ./out/break-align-engraver.o ./out/chord-tremolo-iterator.o ./out/general-scheme.o ./out/least-squares.o ./out/audio-element.o ./out/paper-system.o ./out/lily-lexer-scheme.o ./out/beam.o ./out/music-sequence.o ./out/music.o ./out/stem.o ./out/relative-octave-check.o ./out/keyword.o ./out/translator-ctors.o ./out/stencil-integral.o ./out/arpeggio-engraver.o ./out/extender-engraver.o ./out/prob-scheme.o ./out/ligature-engraver.o ./out/lyric-hyphen.o ./out/simple-spacer.o ./out/output-property-engraver.o ./out/drum-note-performer.o ./out/fingering-column-engraver.o ./out/midi-item.o ./out/undead.o ./out/repeat-acknowledge-engraver.o ./out/grace-engraver.o ./out/figured-bass-position-engraver.o ./out/tie-formatting-problem.o ./out/context-mod-scheme.o ./out/pfb.o ./out/fingering-engraver.o ./out/profile.o ./out/break-alignment-interface.o ./out/trill-spanner-engraver.o ./out/multi-measure-rest.o ./out/collision-engraver.o ./out/system-start-delimiter.o ./out/one-line-auto-height-breaking.o ./out/minimal-page-breaking.o ./out/dot-column-engraver.o ./out/note-name-engraver.o ./out/breathing-sign.o ./out/paper-book.o ./out/sustain-pedal.o ./out/tie-configuration.o ./out/staff-grouper-interface.o ./out/output-def.o ./out/nested-property.o ./out/timing-translator.o ./out/protected-scm.o ./out/volta-bracket.o ./out/beam-performer.o ./out/beaming-pattern.o ./out/prob.o ./out/part-combine-engraver.o ./out/smobs.o ./out/system.o ./out/figured-bass-engraver.o ./out/paper-def.o ./out/performer-group.o ./out/rhythmic-column-engraver.o ./out/volta-engraver.o ./out/paper-column-engraver.o ./out/scm-hash.o ./out/beam-collision-engraver.o ./out/apply-context-iterator.o ./out/skyline-pair.o ./out/gdb.o ./out/spacing-spanner.o ./out/stanza-number-align-engraver.o ./out/line-interface.o ./out/measure-grouping-spanner.o ./out/staff-symbol-referencer-scheme.o ./out/stem-engraver.o ./out/font-metric-scheme.o ./out/beam-engraver.o ./out/freetype-error.o ./out/instrument-name-engraver.o ./out/partial-iterator.o ./out/lily-lexer.o ./out/metronome-engraver.o ./out/warn-scheme.o ./out/includable-lexer.o ./out/item.o ./out/spacing-engraver.o ./out/relocate.o ./out/pointer-group-interface.o ./out/measure-grouping-engraver.o ./out/grace-music.o ./out/staff-performer.o ./out/slur-engraver.o ./out/function-documentation.o ./out/dots.o ./out/performer.o ./out/page-spacing.o ./out/separating-line-group-engraver.o ./out/axis-group-interface-scheme.o ./out/box.o ./out/spacing-determine-loose-columns.o ./out/duration.o ./out/stencil-expression.o ./out/phrasing-slur-engraver.o ./out/side-position-interface.o ./out/pitch-interval.o ./out/accidental.o ./out/pure-from-neighbor-interface.o ./out/optimal-page-breaking.o ./out/spanner-break-forbid-engraver.o ./out/drum-note-engraver.o ./out/stanza-number-engraver.o ./out/midi-walker.o ./out/ly-module.o ./out/span-bar-stub-engraver.o ./out/book-scheme.o ./out/grid-point-engraver.o ./out/scheme-engraver.o ./out/dot-column.o ./out/note-head-scheme.o ./out/lily-imports.o ./out/font-config-scheme.o ./out/open-type-font-scheme.o ./out/hairpin.o ./out/quote-iterator.o ./out/ledger-line-engraver.o ./out/auto-beam-engraver.o ./out/pango-font.o ./out/misc.o ./out/repeated-music.o ./out/figured-bass-continuation.o ./out/grace-iterator.o ./out/cue-clef-engraver.o ./out/pango-select.o ./out/main.o ./out/music-wrapper-iterator.o ./out/pitch-squash-engraver.o ./out/tuplet-iterator.o ./out/stencil-interpret.o ./out/key-signature-interface.o ./out/pitched-trill-engraver.o ./out/score.o ./out/context-mod.o ./out/fingering-column.o ./out/music-function-scheme.o ./out/bar-number-engraver.o ./out/break-substitution.o ./out/sources.o ./out/piano-pedal-performer.o ./out/performance.o ./out/volta-repeat-iterator.o ./out/slash-repeat-engraver.o ./out/font-config.o ./out/lily-guile.o ./out/balloon.o ./out/text-spanner-engraver.o ./out/bezier.o ./out/stencil.o ./out/clef.o ./out/spacing-basic.o ./out/dynamic-engraver.o ./out/time-signature-engraver.o ./out/ottava-engraver.o ./out/music-output.o ./out/lily-parser.o ./out/ligature-bracket-engraver.o ./out/translator-dispatch-list.o ./out/lily-parser-scheme.o ./out/engraver-scheme.o ./out/dynamic-align-engraver.o ./out/part-combine-iterator.o ./out/custos.o ./out/translator-scheme.o ./out/slur-scoring.o ./out/accidental-engraver.o ./out/rest-collision.o ./out/staff-symbol-engraver.o ./out/spanner.o ./out/note-column-scheme.o ./out/piano-pedal-align-engraver.o ./out/clef-modifier.o ./out/note-performer.o ./out/pango-font-scheme.o ./out/melody-spanner.o ./out/tie-column.o ./out/tuplet-bracket.o ./out/episema-engraver.o ./out/lyric-extender.o ./out/grob-property.o ./out/lyric-combine-music.o ./out/vaticana-ligature.o ./out/cluster-engraver.o ./out/instrument-switch-engraver.o ./out/part-combine-part-iterator.o ./out/audio-column.o ./out/key-performer.o ./out/gregorian-ligature-engraver.o ./out/context-specced-music-iterator.o ./out/pango-select-scheme.o ./out/translator-group.o ./out/book.o ./out/stream-event.o ./out/staff-symbol.o ./out/font-interface.o ./out/lyric-engraver.o ./out/page-breaking.o ./out/engraver-group.o ./out/grob-smob.o ./out/grob-array-scheme.o ./out/rod.o ./out/moment.o ./out/dispatcher-scheme.o ./out/lilypond-version.o ./out/tab-staff-symbol-engraver.o ./out/font-select.o ./out/grob-interface-scheme.o ./out/tuplet-engraver.o ./out/balloon-engraver.o ./out/template5.o ./out/paper-column.o ./out/script-row-engraver.o ./out/clef-engraver.o ./out/one-page-breaking.o ./out/midi-chunk.o ./out/context-def.o ./out/paper-score-scheme.o ./out/axis-group-engraver.o ./out/cluster.o ./out/translator.o ./out/item-scheme.o ./out/double-percent-repeat-engraver.o ./out/separation-item.o ./out/key-engraver.o ./out/keep-alive-together-engraver.o ./out/source-file.o ./out/all-font-metrics-scheme.o ./out/self-alignment-interface.o ./out/axis-group-interface.o ./out/midi-cc-performer.o ./out/input-scheme.o ./out/lyric-performer.o ./out/grid-line-span-engraver.o ./out/glissando-engraver.o ./out/event-chord-iterator.o ./out/chord-name.o ./out/spring-smob.o ./out/score-performer.o ./out/paper-system-scheme.o ./out/coherent-ligature-engraver.o ./out/context-scheme.o ./out/tie-performer.o ./out/semi-tie.o ./out/chord-tremolo-engraver.o ./out/pitch.o ./out/listener.o ./out/percent-repeat-engraver.o ./out/note-head.o ./out/relative-octave-music.o ./out/note-spacing.o ./out/dimension-cache.o ./out/spacing-loose-columns.o ./out/melody-engraver.o ./out/page-spacing-result.o ./out/script-engraver.o ./out/ttf.o ./out/dot-formatting-problem.o ./out/all-font-metrics.o ./out/ledger-line-spanner.o ./out/staff-collecting-engraver.o ./out/spacing-options.o ./out/time-signature-performer.o ./out/pfb-scheme.o ./out/property-iterator.o ./out/vaticana-ligature-engraver.o ./out/script-interface.o ./out/score-scheme.o ./out/one-line-page-breaking.o ./out/page-marker-scheme.o ./out/spanner-scheme.o ./out/music-function.o ./out/ambitus-engraver.o ./out/new-fingering-engraver.o ./out/bar-check-iterator.o ./out/change-iterator.o ./out/context-handle.o ./out/parenthesis-engraver.o ./out/align-interface.o ./out/change-sequence-iterator.o ./out/column-x-positions.o ./out/output-def-scheme.o ./out/global-context-scheme.o ./out/tie-details.o ./out/rest.o ./out/note-collision.o ./out/dot-configuration.o ./out/tempo-performer.o ./out/translator-group-ctors.o ./out/ottava-bracket.o ./out/stem-tremolo.o ./out/mensural-ligature.o ./out/grob-pq-engraver.o ./out/lily-modules.o ./out/page-turn-page-breaking.o ./out/gregorian-ligature.o ./out/engraver.o ./out/rhythmic-music-iterator.o ./out/mark-engraver.o ./out/page-layout-problem-scheme.o ./out/vertical-align-engraver.o ./out/kievan-ligature-engraver.o ./out/tie-specification.o ./out/tuplet-number.o ./out/laissez-vibrer-engraver.o ./out/dots-engraver.o ./out/directional-element-interface.o ./out/concurrent-hairpin-engraver.o ./out/line-interface-scheme.o ./out/paper-book-scheme.o ./out/pointer-group-interface-scheme.o ./out/grob-interface.o ./out/open-type-font.o ./out/span-bar-engraver.o ./out/midi-stream.o ./out/page-layout-problem.o ./out/input-smob.o ./out/note-head-line-engraver.o ./out/bar-engraver.o ./out/text-interface.o ./out/text-engraver.o ./out/grob-scheme.o ./out/arpeggio.o ./out/semi-tie-column.o ./out/program-option-scheme.o ./out/footnote-engraver.o ./out/simultaneous-music-iterator.o ./out/breathing-sign-engraver.o ./out/grace-spacing-engraver.o ./out/lexer.o ./out/parser.o ./out/../../flower/out/library.a  -ldl   -L/home/arne/.local/lib -L/home/arne/.local/lib/guile/2.2 -lguile-2.2 -lgc  -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype  -lfontconfig -lfreetype  -lfreetype -v 2>&1 | grep -i lib
nm out/lilypond 
LD_LIBRARY_PATH=~/.local/lib/  out/lilypond 
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:../:../scm LD_LIBRARY_PATH=~/.local/lib/  out/lilypond 
wget http://infinite-hands.draketo.de/infinite_hands_sheet.ly
wget -O mf/out/emmentaler-20.otf https://github.com/saebekassebil/subito/raw/master/resources/gonville/lilyfonts/otf/emmentaler-20.otf
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
cp lilypond-data_2.18.2-4.1_all.deb /tmp
scp kav:/usr/share/lilypond/2.16.2/fonts/otf/*otf mf/out/
GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
git add configure.ac lily/general-scheme.cc lily/pdf-scheme.cc scm/memory-trace.scm
time GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ infinite_hands_sheet.ly 
cat 0001-Build-fixes-for-Guile-2.1.x-not-yet-functional.patch
cat > 1 <<EOF
diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
index 1168ee9..2df63fc 100644
--- a/lily/general-scheme.cc
+++ b/lily/general-scheme.cc
@@ -275,7 +275,8 @@ LY_DEFINE (ly_protects, "ly:protects",
 #if SCM_MAJOR_VERSION < 2 || SCM_MAJOR_VERSION == 2 && SCM_MINOR_VERSION < 1
   return scm_protects;
 #else
-  return programming_error ("ly:protects is not supported in Guile 2.1");
+  // return programming_error ("ly:protects is not supported in Guile 2.1");
+  return 0; // FIXME: Evil hack just to get this to build.
 #endif
 }
EOF

Best wishes,
Arne

PS: If this provides at least a small step towards guile 2 in lilypond,
    it’s worth its while.
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: How to make GNU Guile more successful

[Thomas Morley]

2017-03-05 0:44 GMT+01:00 Arne Babenhauserheide <arne_bab@web.de>:
>
> David Kastrup <dak@gnu.org> writes:
>
>> Nala Ginrut <nalaginrut@gmail.com> writes:
>>
>>> I think we have to elaborate the question clearer.
>>>
>>> 1. How to make guile-scheme more successful?
>>> I think this is similar to ask "how to make scheme more successful".
>>> This is the big question to the whole scheme community.
>>>
>>> 2. How to make guile platform more successful?
>>> I this case, let me ask a question, if we have guile-python3
>>> (compatible with most of Python3 features), and provide more
>>> convenient FFI helper function to help bind more libraries in short
>>> time, is it possible to attract more people from Python land? Given
>>> we'll have good JIT compiler in the future.
>>
>> Frankly, I doubt that migration of large Python-based applications is
>> going to be a thing when nobody can even be bothered with immersing
>> himself in the problems with migrating LilyPond from Guile-1.8 to
>> Guile-2.
>
> I worked on testing Lilypond with user installed Guile 2.x, does that
> count?
>
> Just to have it recorded somewhere, here’s a patch to lilypond along
> with a copy of the bash history of the setup (cleaned up, it was many
> times as long):
>
> ## patch
>
> From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
> Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)
>
> ---
>  configure.ac         | 7 ++++++-
>  lily/pdf-scheme.cc   | 4 ++++
>  scm/memory-trace.scm | 3 ++-
>  3 files changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/configure.ac b/configure.ac
> index d77ea15..393976b 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
>  STEPMAKE_WINDOWS
>
>  # guile executable for some scripts
> -STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
> +if test "$GUILEv2" = "yes"
> +then
> +    STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
> +else
> +    STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
> +fi
>
>  # perl for help2man and for mf2pt1.pl
>  STEPMAKE_PERL(REQUIRED)
> diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
> index da2ce2c..f5ae70c 100644
> --- a/lily/pdf-scheme.cc
> +++ b/lily/pdf-scheme.cc
> @@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
>     *                  (string->utf16 str 'big)
>     */
>    if (g)
> +#if GUILEV2
> +    return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
> +#else
>      return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
> +#endif
>    else
>      return str;
>  }
> diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
> index d8ffeb9..9ebd722 100644
> --- a/scm/memory-trace.scm
> +++ b/scm/memory-trace.scm
> @@ -2,7 +2,8 @@
>
>  (define-module (scm memory-trace))
>  (use-modules (lily)
> -             (ice-9 format))
> +             (ice-9 format)
> +             (ice-9 threads))
>
>  (define-public (mtrace:start-trace freq)
>    (set! usecond-interval (inexact->exact (/ 1000000 freq)))
> --
> 2.10.2
>
>
> ## bash history
>
> sudo emerge media-fonts/tex-gyre
> sudo nano /etc/portage/package.keywords/sonstiges
> sudo emerge media-fonts/tex-gyre
> sudo pmerge dblatex
> git clone git://git.sv.gnu.org/lilypond.git
> cd lilypond/
> ./autogen.sh --prefix ~/.local --enable-guile2
> mkdir -p ~/texmf/lh
> cp ~/Downloads/lhfnt35g-source.zip ~/texmf/lh/
> unzip lhfnt35g-source.zip
> cd ~/texmf/source/latex/lh
> latex lcyfonts.ins
> latex ot2fonts.ins; latex t2ccfonts.ins
> ls ~/texmf/examples/
> mkdir ~/texmf/lh
> cp *sty ~/texmf/lh/
> ./autogen.sh --prefix ~/.local --enable-guile2
> sudo emerge dev-texlive/texlive-langcyrillic
> sudo pmerge dev-texlive/texlive-langcyrillic
> make out/lilypond-invoke-editor
> cd scripts/
> /home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor --no-discard-stderr
> /home/arne/lilypond/scripts/build/out/help2man out/lilypond-invoke-editor
> ./autogen.sh --prefix ~/.local --enable-guile2
> make
> LD_LIBRARY_PATH=/home/arne/.local/lib/guile/2.2/ g++ -o out/lilypond  ./out/modified-font-metric.o ./out/sequential-iterator.o ./out/hara-kiri-group-spanner.o ./out/completion-note-heads-engraver.o ./out/slur-score-parameters.o ./out/flag.o ./out/span-arpeggio-engraver.o ./out/beam-quanting.o ./out/dynamic-performer.o ./out/parse-scm.o ./out/tie.o ./out/bend-engraver.o ./out/staff-spacing.o ./out/rest-collision-engraver.o ./out/simple-spacer-scheme.o ./out/audio-item.o ./out/pdf-scheme.o ./out/paper-score.o ./out/slur-configuration.o ./out/constrained-breaking.o ./out/score-engraver.o ./out/unpure-pure-container.o ./out/multi-measure-rest-engraver.o ./out/system-start-delimiter-engraver.o ./out/context-property.o ./out/freetype.o ./out/interval-minefield.o ./out/kievan-ligature.o ./out/completion-rest-engraver.o ./out/pure-from-neighbor-engraver.o ./out/horizontal-bracket-engraver.o ./out/grob-closure.o ./out/default-bar-line-engraver.o ./out/control-track-performer.o ./out/file-name-map.o ./out/audio-staff.o ./out/line-spanner.o ./out/grob-info.o ./out/spring.o ./out/slur.o ./out/music-scheme.o ./out/program-option.o ./out/fretboard-engraver.o ./out/hyphen-engraver.o ./out/page-marker.o ./out/global-vars.o ./out/stream-event-scheme.o ./out/script-column-engraver.o ./out/bezier-bow.o ./out/page-breaking-scheme.o ./out/lily-version.o ./out/repeat-tie-engraver.o ./out/tweak-engraver.o ./out/staff-symbol-referencer.o ./out/moment-scheme.o ./out/note-heads-engraver.o ./out/paper-outputter.o ./out/music-wrapper.o ./out/grob.o ./out/performance-scheme.o ./out/bar-line.o ./out/page-turn-engraver.o ./out/dimensions-scheme.o ./out/tab-note-heads-engraver.o ./out/piano-pedal-bracket.o ./out/spaceable-grob.o ./out/event-iterator.o ./out/lyric-combine-music-iterator.o ./out/percent-repeat-iterator.o ./out/audio-element-info.o ./out/midi-cc-announcer.o ./out/font-metric.o ./out/note-spacing-engraver.o ./out/duration-scheme.o ./out/accidental-placement.o ./out/music-iterator.o ./out/rhythmic-head.o ./out/grob-array.o ./out/tab-tie-follow-engraver.o ./out/context.o ./out/dispatcher.o ./out/scale.o ./out/chord-name-engraver.o ./out/auto-change-iterator.o ./out/tie-engraver.o ./out/grid-line-interface.o ./out/forbid-break-engraver.o ./out/note-column.o ./out/lookup.o ./out/global-ctor.o ./out/custos-engraver.o ./out/piano-pedal-engraver.o ./out/simple-music-iterator.o ./out/pitch-scheme.o ./out/mensural-ligature-engraver.o ./out/articulations.o ./out/horizontal-bracket.o ./out/script-column.o ./out/global-context.o ./out/slur-performer.o ./out/guile-init.o ./out/stencil-scheme.o ./out/font-size-engraver.o ./out/rest-engraver.o ./out/skyline.o ./out/percent-repeat-item.o ./out/enclosing-bracket.o ./out/module-scheme.o ./out/input.o ./out/paper-outputter-scheme.o ./out/spacing-interface.o ./out/break-align-engraver.o ./out/chord-tremolo-iterator.o ./out/general-scheme.o ./out/least-squares.o ./out/audio-element.o ./out/paper-system.o ./out/lily-lexer-scheme.o ./out/beam.o ./out/music-sequence.o ./out/music.o ./out/stem.o ./out/relative-octave-check.o ./out/keyword.o ./out/translator-ctors.o ./out/stencil-integral.o ./out/arpeggio-engraver.o ./out/extender-engraver.o ./out/prob-scheme.o ./out/ligature-engraver.o ./out/lyric-hyphen.o ./out/simple-spacer.o ./out/output-property-engraver.o ./out/drum-note-performer.o ./out/fingering-column-engraver.o ./out/midi-item.o ./out/undead.o ./out/repeat-acknowledge-engraver.o ./out/grace-engraver.o ./out/figured-bass-position-engraver.o ./out/tie-formatting-problem.o ./out/context-mod-scheme.o ./out/pfb.o ./out/fingering-engraver.o ./out/profile.o ./out/break-alignment-interface.o ./out/trill-spanner-engraver.o ./out/multi-measure-rest.o ./out/collision-engraver.o ./out/system-start-delimiter.o ./out/one-line-auto-height-breaking.o ./out/minimal-page-breaking.o ./out/dot-column-engraver.o ./out/note-name-engraver.o ./out/breathing-sign.o ./out/paper-book.o ./out/sustain-pedal.o ./out/tie-configuration.o ./out/staff-grouper-interface.o ./out/output-def.o ./out/nested-property.o ./out/timing-translator.o ./out/protected-scm.o ./out/volta-bracket.o ./out/beam-performer.o ./out/beaming-pattern.o ./out/prob.o ./out/part-combine-engraver.o ./out/smobs.o ./out/system.o ./out/figured-bass-engraver.o ./out/paper-def.o ./out/performer-group.o ./out/rhythmic-column-engraver.o ./out/volta-engraver.o ./out/paper-column-engraver.o ./out/scm-hash.o ./out/beam-collision-engraver.o ./out/apply-context-iterator.o ./out/skyline-pair.o ./out/gdb.o ./out/spacing-spanner.o ./out/stanza-number-align-engraver.o ./out/line-interface.o ./out/measure-grouping-spanner.o ./out/staff-symbol-referencer-scheme.o ./out/stem-engraver.o ./out/font-metric-scheme.o ./out/beam-engraver.o ./out/freetype-error.o ./out/instrument-name-engraver.o ./out/partial-iterator.o ./out/lily-lexer.o ./out/metronome-engraver.o ./out/warn-scheme.o ./out/includable-lexer.o ./out/item.o ./out/spacing-engraver.o ./out/relocate.o ./out/pointer-group-interface.o ./out/measure-grouping-engraver.o ./out/grace-music.o ./out/staff-performer.o ./out/slur-engraver.o ./out/function-documentation.o ./out/dots.o ./out/performer.o ./out/page-spacing.o ./out/separating-line-group-engraver.o ./out/axis-group-interface-scheme.o ./out/box.o ./out/spacing-determine-loose-columns.o ./out/duration.o ./out/stencil-expression.o ./out/phrasing-slur-engraver.o ./out/side-position-interface.o ./out/pitch-interval.o ./out/accidental.o ./out/pure-from-neighbor-interface.o ./out/optimal-page-breaking.o ./out/spanner-break-forbid-engraver.o ./out/drum-note-engraver.o ./out/stanza-number-engraver.o ./out/midi-walker.o ./out/ly-module.o ./out/span-bar-stub-engraver.o ./out/book-scheme.o ./out/grid-point-engraver.o ./out/scheme-engraver.o ./out/dot-column.o ./out/note-head-scheme.o ./out/lily-imports.o ./out/font-config-scheme.o ./out/open-type-font-scheme.o ./out/hairpin.o ./out/quote-iterator.o ./out/ledger-line-engraver.o ./out/auto-beam-engraver.o ./out/pango-font.o ./out/misc.o ./out/repeated-music.o ./out/figured-bass-continuation.o ./out/grace-iterator.o ./out/cue-clef-engraver.o ./out/pango-select.o ./out/main.o ./out/music-wrapper-iterator.o ./out/pitch-squash-engraver.o ./out/tuplet-iterator.o ./out/stencil-interpret.o ./out/key-signature-interface.o ./out/pitched-trill-engraver.o ./out/score.o ./out/context-mod.o ./out/fingering-column.o ./out/music-function-scheme.o ./out/bar-number-engraver.o ./out/break-substitution.o ./out/sources.o ./out/piano-pedal-performer.o ./out/performance.o ./out/volta-repeat-iterator.o ./out/slash-repeat-engraver.o ./out/font-config.o ./out/lily-guile.o ./out/balloon.o ./out/text-spanner-engraver.o ./out/bezier.o ./out/stencil.o ./out/clef.o ./out/spacing-basic.o ./out/dynamic-engraver.o ./out/time-signature-engraver.o ./out/ottava-engraver.o ./out/music-output.o ./out/lily-parser.o ./out/ligature-bracket-engraver.o ./out/translator-dispatch-list.o ./out/lily-parser-scheme.o ./out/engraver-scheme.o ./out/dynamic-align-engraver.o ./out/part-combine-iterator.o ./out/custos.o ./out/translator-scheme.o ./out/slur-scoring.o ./out/accidental-engraver.o ./out/rest-collision.o ./out/staff-symbol-engraver.o ./out/spanner.o ./out/note-column-scheme.o ./out/piano-pedal-align-engraver.o ./out/clef-modifier.o ./out/note-performer.o ./out/pango-font-scheme.o ./out/melody-spanner.o ./out/tie-column.o ./out/tuplet-bracket.o ./out/episema-engraver.o ./out/lyric-extender.o ./out/grob-property.o ./out/lyric-combine-music.o ./out/vaticana-ligature.o ./out/cluster-engraver.o ./out/instrument-switch-engraver.o ./out/part-combine-part-iterator.o ./out/audio-column.o ./out/key-performer.o ./out/gregorian-ligature-engraver.o ./out/context-specced-music-iterator.o ./out/pango-select-scheme.o ./out/translator-group.o ./out/book.o ./out/stream-event.o ./out/staff-symbol.o ./out/font-interface.o ./out/lyric-engraver.o ./out/page-breaking.o ./out/engraver-group.o ./out/grob-smob.o ./out/grob-array-scheme.o ./out/rod.o ./out/moment.o ./out/dispatcher-scheme.o ./out/lilypond-version.o ./out/tab-staff-symbol-engraver.o ./out/font-select.o ./out/grob-interface-scheme.o ./out/tuplet-engraver.o ./out/balloon-engraver.o ./out/template5.o ./out/paper-column.o ./out/script-row-engraver.o ./out/clef-engraver.o ./out/one-page-breaking.o ./out/midi-chunk.o ./out/context-def.o ./out/paper-score-scheme.o ./out/axis-group-engraver.o ./out/cluster.o ./out/translator.o ./out/item-scheme.o ./out/double-percent-repeat-engraver.o ./out/separation-item.o ./out/key-engraver.o ./out/keep-alive-together-engraver.o ./out/source-file.o ./out/all-font-metrics-scheme.o ./out/self-alignment-interface.o ./out/axis-group-interface.o ./out/midi-cc-performer.o ./out/input-scheme.o ./out/lyric-performer.o ./out/grid-line-span-engraver.o ./out/glissando-engraver.o ./out/event-chord-iterator.o ./out/chord-name.o ./out/spring-smob.o ./out/score-performer.o ./out/paper-system-scheme.o ./out/coherent-ligature-engraver.o ./out/context-scheme.o ./out/tie-performer.o ./out/semi-tie.o ./out/chord-tremolo-engraver.o ./out/pitch.o ./out/listener.o ./out/percent-repeat-engraver.o ./out/note-head.o ./out/relative-octave-music.o ./out/note-spacing.o ./out/dimension-cache.o ./out/spacing-loose-columns.o ./out/melody-engraver.o ./out/page-spacing-result.o ./out/script-engraver.o ./out/ttf.o ./out/dot-formatting-problem.o ./out/all-font-metrics.o ./out/ledger-line-spanner.o ./out/staff-collecting-engraver.o ./out/spacing-options.o ./out/time-signature-performer.o ./out/pfb-scheme.o ./out/property-iterator.o ./out/vaticana-ligature-engraver.o ./out/script-interface.o ./out/score-scheme.o ./out/one-line-page-breaking.o ./out/page-marker-scheme.o ./out/spanner-scheme.o ./out/music-function.o ./out/ambitus-engraver.o ./out/new-fingering-engraver.o ./out/bar-check-iterator.o ./out/change-iterator.o ./out/context-handle.o ./out/parenthesis-engraver.o ./out/align-interface.o ./out/change-sequence-iterator.o ./out/column-x-positions.o ./out/output-def-scheme.o ./out/global-context-scheme.o ./out/tie-details.o ./out/rest.o ./out/note-collision.o ./out/dot-configuration.o ./out/tempo-performer.o ./out/translator-group-ctors.o ./out/ottava-bracket.o ./out/stem-tremolo.o ./out/mensural-ligature.o ./out/grob-pq-engraver.o ./out/lily-modules.o ./out/page-turn-page-breaking.o ./out/gregorian-ligature.o ./out/engraver.o ./out/rhythmic-music-iterator.o ./out/mark-engraver.o ./out/page-layout-problem-scheme.o ./out/vertical-align-engraver.o ./out/kievan-ligature-engraver.o ./out/tie-specification.o ./out/tuplet-number.o ./out/laissez-vibrer-engraver.o ./out/dots-engraver.o ./out/directional-element-interface.o ./out/concurrent-hairpin-engraver.o ./out/line-interface-scheme.o ./out/paper-book-scheme.o ./out/pointer-group-interface-scheme.o ./out/grob-interface.o ./out/open-type-font.o ./out/span-bar-engraver.o ./out/midi-stream.o ./out/page-layout-problem.o ./out/input-smob.o ./out/note-head-line-engraver.o ./out/bar-engraver.o ./out/text-interface.o ./out/text-engraver.o ./out/grob-scheme.o ./out/arpeggio.o ./out/semi-tie-column.o ./out/program-option-scheme.o ./out/footnote-engraver.o ./out/simultaneous-music-iterator.o ./out/breathing-sign-engraver.o ./out/grace-spacing-engraver.o ./out/lexer.o ./out/parser.o ./out/../../flower/out/library.a  -ldl   -L/home/arne/.local/lib -L/home/arne/.local/lib/guile/2.2 -lguile-2.2 -lgc  -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype  -lfontconfig -lfreetype  -lfreetype -v 2>&1 | grep -i lib
> nm out/lilypond
> LD_LIBRARY_PATH=~/.local/lib/  out/lilypond
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:../:../scm LD_LIBRARY_PATH=~/.local/lib/  out/lilypond
> wget http://infinite-hands.draketo.de/infinite_hands_sheet.ly
> wget -O mf/out/emmentaler-20.otf https://github.com/saebekassebil/subito/raw/master/resources/gonville/lilyfonts/otf/emmentaler-20.otf
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
> cp lilypond-data_2.18.2-4.1_all.deb /tmp
> scp kav:/usr/share/lilypond/2.16.2/fonts/otf/*otf mf/out/
> GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ test.ly
> git add configure.ac lily/general-scheme.cc lily/pdf-scheme.cc scm/memory-trace.scm
> time GUILE_LOAD_PATH=$GUILE_LOAD_PATH:.:scm LD_LIBRARY_PATH=~/.local/lib/  lily/out/lilypond -I ly/ -I mf/out/ -I ps/ infinite_hands_sheet.ly
> cat 0001-Build-fixes-for-Guile-2.1.x-not-yet-functional.patch
> cat > 1 <<EOF
> diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
> index 1168ee9..2df63fc 100644
> --- a/lily/general-scheme.cc
> +++ b/lily/general-scheme.cc
> @@ -275,7 +275,8 @@ LY_DEFINE (ly_protects, "ly:protects",
>  #if SCM_MAJOR_VERSION < 2 || SCM_MAJOR_VERSION == 2 && SCM_MINOR_VERSION < 1
>    return scm_protects;
>  #else
> -  return programming_error ("ly:protects is not supported in Guile 2.1");
> +  // return programming_error ("ly:protects is not supported in Guile 2.1");
> +  return 0; // FIXME: Evil hack just to get this to build.
>  #endif
>  }
> EOF
>
> Best wishes,
> Arne
>
> PS: If this provides at least a small step towards guile 2 in lilypond,
>     it’s worth its while.
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken

Hi Arne,

many thanks for your work!
I'll have a closer look tomorrow (it's in the middle of the night here)...

The main problem is not that we can't build lilypond with 2.0.14 or 2.1.7
Checkout the branch dev/guile-v2-work in the lilypond-repository (you
will need to rebase it against current master) and you'll be able to
build LilyPond with guile-2.0.13, so I'm pretty confident it will work
with 2.0.14. For building with guile 2.1.7 an additional patch is
needed, currently I've simply deleted all about ly:protects
(brute-force, I know...).

Though the perfomance of LilyPond is dramatically slowed down. Making
it effectively unusable for any longer input.
I'm currently testing this for builds with guile 1.8.8, 2.0.14 and
2.1.7 and will report tomorrow.

Cheers,
  Harm

Re: How to make GNU Guile more successful

[Thomas Morley]

Hi Arne,

as promised here my thoughts/findings.
All tested with a LilyPond build from the rebased branch
dev/guile-v2-work, with some additional patches on top of it:

diff --git a/lily/general-scheme.cc b/lily/general-scheme.cc
index 1168ee9..db376d11 100644
--- a/lily/general-scheme.cc
+++ b/lily/general-scheme.cc
@@ -267,6 +267,7 @@ LY_DEFINE (ly_dimension_p, "ly:dimension?", 1, 0,
0, (SCM d),
 /*
   Debugging mem leaks:
 */
+/*
 LY_DEFINE (ly_protects, "ly:protects",
            0, 0, 0, (),
            "Return hash of protected objects.")
@@ -278,6 +279,7 @@ LY_DEFINE (ly_protects, "ly:protects",
   return programming_error ("ly:protects is not supported in Guile 2.1");
 #endif
 }
+*/

 LY_DEFINE (ly_gettext, "ly:gettext",
            1, 0, 0, (SCM original),
diff --git a/scm/lily.scm b/scm/lily.scm
index d3164e4..23b1647 100644
--- a/scm/lily.scm
+++ b/scm/lily.scm
@@ -836,10 +836,11 @@ messages into errors.")

 (define-public (dump-gc-protects)
   (set! gc-protect-stat-count (1+ gc-protect-stat-count))
-  (let* ((protects (sort (hash-table->alist (ly:protects))
-                         (lambda (a b)
-                           (< (object-address (car a))
-                              (object-address (car b))))))
+  (let* (;(protects (sort (hash-table->alist (ly:protects))
+         ;                (lambda (a b)
+         ;                  (< (object-address (car a))
+         ;                     (object-address (car b))))))
+         (protects '())
          (out-file-name (string-append
                          "gcstat-" (number->string gc-protect-stat-count)
                          ".scm"))


diff --git a/ly/scheme-sandbox.ly b/ly/scheme-sandbox.ly
index 7dec0dc..1f233c0 100644
--- a/ly/scheme-sandbox.ly
+++ b/ly/scheme-sandbox.ly
@@ -13,4 +13,8 @@
 % requirements may be different.

 #(newline)
-#(scm-style-repl)
+#(if (guile-v2)
+     (begin
+       (use-modules (system repl repl))
+       (start-repl))
+     (scm-style-repl))



The ly-file used for testing can be found at
http://lists.gnu.org/archive/html/lilypond-user/2016-11/msg00948.html
As the author says, this file was not finished, so several errors and
warnings because of suboptimal user-input happen.
Quite typical for ongoing work and good for testings.
Compiling the file Urtext.ly returns a 159-pages-pdf.

2017-03-05 3:05 GMT+01:00 Thomas Morley <thomasmorley65@gmail.com>:
> 2017-03-05 0:44 GMT+01:00 Arne Babenhauserheide <arne_bab@web.de>:

>> I worked on testing Lilypond with user installed Guile 2.x, does that
>> count?
>>
>> Just to have it recorded somewhere, here’s a patch to lilypond along
>> with a copy of the bash history of the setup (cleaned up, it was many
>> times as long):
>>
>> ## patch
>>
>> From bd2ffea6f4c4c1ede13f5ac82d0a8ce31ccfe3c7 Mon Sep 17 00:00:00 2001
>> Subject: [PATCH] Build fixes for Guile 2.1.x (not yet functional)
>>
>> ---
>>  configure.ac         | 7 ++++++-
>>  lily/pdf-scheme.cc   | 4 ++++
>>  scm/memory-trace.scm | 3 ++-
>>  3 files changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/configure.ac b/configure.ac
>> index d77ea15..393976b 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -267,7 +267,12 @@ STEPMAKE_FREETYPE2(freetype2, REQUIRED, 2.1.10)
>>  STEPMAKE_WINDOWS
>>
>>  # guile executable for some scripts
>> -STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
>> +if test "$GUILEv2" = "yes"
>> +then
>> +    STEPMAKE_GUILE(OPTIONAL, 2.0.7, 2.2.0)
>> +else
>> +    STEPMAKE_GUILE(OPTIONAL, 1.8.2, 1.9.0)
>> +fi

The above is already in, see
$ git log -p 91ff9563ebe1c1cd720ad1a44890e7375fd83da8


>>  # perl for help2man and for mf2pt1.pl
>>  STEPMAKE_PERL(REQUIRED)
>> diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
>> index da2ce2c..f5ae70c 100644
>> --- a/lily/pdf-scheme.cc
>> +++ b/lily/pdf-scheme.cc
>> @@ -91,7 +91,11 @@ LY_DEFINE (ly_encode_string_for_pdf, "ly:encode-string-for-pdf",
>>     *                  (string->utf16 str 'big)
>>     */
>>    if (g)
>> +#if GUILEV2
>> +    return scm_take_locale_stringn (g, bytes_written); // scm_take_str eventually frees g!
>> +#else
>>      return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
>> +#endif
>>    else
>>      return str;
>>  }

Instead of the above, currently this code is in:

diff --git a/lily/pdf-scheme.cc b/lily/pdf-scheme.cc
index da2ce2c..61cf382 100644
--- a/lily/pdf-scheme.cc
+++ b/lily/pdf-scheme.cc
@@ -84,14 +84,16 @@ LY_DEFINE (ly_encode_string_for_pdf,
"ly:encode-string-for-pdf",
   free (p);

   /* Convert back to SCM object and return it */
-  /* FIXME guile-2.0: With guile 2.0 the internal representation of a string
-   *                  has changed (char vector rather than binary bytes in
-   *                  UTF-8). However, with guile 2.0, ly:encode-string-for-pdf
-   *                  is no longer needed and can be replaced by the new
-   *                  (string->utf16 str 'big)
-   */
   if (g)
-    return scm_take_str (g, bytes_written); // scm_take_str eventually frees g!
+    {
+      /*
+       * Return the raw byte representation of the UTF-16BE encoded string,
+       * in a locale independent way.
+       */
+      SCM string = scm_from_latin1_stringn (g, bytes_written);
+      free(g);
+      return string;
+    }
   else
     return str;
 }
See:
git log -p d15c38c0ddd4c04edcf82cda50ca30f6dc4941fa

Though, I tested your suggestion as well.
It changes one of our regtests, utf-8.ly. But I have to admit I tested
only this one (not all regtests) and did not try to build the docs.
(All this stuff is pretty heavy for my admittedly weak laptop, it
lasts ages...)
Compiling the above mentioned Urtext.ly-file with your proposal shows
no significant difference for timings (see below)


>> diff --git a/scm/memory-trace.scm b/scm/memory-trace.scm
>> index d8ffeb9..9ebd722 100644
>> --- a/scm/memory-trace.scm
>> +++ b/scm/memory-trace.scm
>> @@ -2,7 +2,8 @@
>>
>>  (define-module (scm memory-trace))
>>  (use-modules (lily)
>> -             (ice-9 format))
>> +             (ice-9 format)
>> +             (ice-9 threads))
>>
>>  (define-public (mtrace:start-trace freq)
>>    (set! usecond-interval (inexact->exact (/ 1000000 freq)))
>> --
>> 2.10.2

Thanks for the above.
It fixes an annoying warning when LilyPond uses guile2.
I'll have to test, but I guess it's compatible with guile-1.
If so I'll likely check it in, ofcourse giving proper credits ;)

[...]
>>
>> PS: If this provides at least a small step towards guile 2 in lilypond,
>>     it’s worth its while.
>> --
>> Unpolitisch sein
>> heißt politisch sein
>> ohne es zu merken
>
> Hi Arne,
>
> many thanks for your work!
> I'll have a closer look tomorrow (it's in the middle of the night here)...
>
> The main problem is not that we can't build lilypond with 2.0.14 or 2.1.7
> Checkout the branch dev/guile-v2-work in the lilypond-repository (you
> will need to rebase it against current master) and you'll be able to
> build LilyPond with guile-2.0.13, so I'm pretty confident it will work
> with 2.0.14. For building with guile 2.1.7 an additional patch is
> needed, currently I've simply deleted all about ly:protects
> (brute-force, I know...).
>
> Though the perfomance of LilyPond is dramatically slowed down. Making
> it effectively unusable for any longer input.
> I'm currently testing this for builds with guile 1.8.8, 2.0.14 and
> 2.1.7 and will report tomorrow.

Here some timing values

(1)
lilypond-2.19.52 using guile 1.8.7
(I would have prefered to build lilypond with a guile-1.8.8 build from
the guile-repository. Though my try to build it from the
branch_release-1-8 failed. Instead attempting to fix it, I then used a
released lilypond-version)

real    8m16.191s
user    6m39.864s
sys    0m10.860s

(2)
guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
lilypond-2.19.56, build from local branch dev/guile-v2.2-work

real    34m11.762s
user    45m11.316s
sys    0m5.604s

(3)
guile-2.1.7 build from guile-git-repository, branch master
(I've got this warning:
configure: WARNING: *** GNU Readline is too old on your system.
configure: WARNING: *** You need readline version 2.1 or later.
No idea whether this may have an impact on lilyponds compiling-time
I'll have to test.)
lilypond-2.19.56, build from local branch dev/guile-v2.2-work

real    67m29.132s
user    93m14.812s
sys    0m7.332s


Thanks,
  Harm

Re: How to make GNU Guile more successful

[David Kastrup]

Thomas Morley <thomasmorley65@gmail.com> writes:

> Here some timing values
>
> (1)
> lilypond-2.19.52 using guile 1.8.7
> (I would have prefered to build lilypond with a guile-1.8.8 build from
> the guile-repository. Though my try to build it from the
> branch_release-1-8 failed. Instead attempting to fix it, I then used a
> released lilypond-version)
>
> real    8m16.191s
> user    6m39.864s
> sys    0m10.860s
>
> (2)
> guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>
> real    34m11.762s
> user    45m11.316s
> sys    0m5.604s
>
> (3)
> guile-2.1.7 build from guile-git-repository, branch master
> (I've got this warning:
> configure: WARNING: *** GNU Readline is too old on your system.
> configure: WARNING: *** You need readline version 2.1 or later.
> No idea whether this may have an impact on lilyponds compiling-time
> I'll have to test.)
> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>
> real    67m29.132s
> user    93m14.812s
> sys    0m7.332s

Same compilation options?  This looks like a surprisingly serious
regression compared to 2.0.

-- 
David Kastrup

Re: How to make GNU Guile more successful

[Thomas Morley]

2017-03-05 15:09 GMT+01:00 David Kastrup <dak@gnu.org>:
> Thomas Morley <thomasmorley65@gmail.com> writes:
>
>> Here some timing values
>>
>> (1)
>> lilypond-2.19.52 using guile 1.8.7
>> (I would have prefered to build lilypond with a guile-1.8.8 build from
>> the guile-repository. Though my try to build it from the
>> branch_release-1-8 failed. Instead attempting to fix it, I then used a
>> released lilypond-version)
>>
>> real    8m16.191s
>> user    6m39.864s
>> sys    0m10.860s
>>
>> (2)
>> guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
>> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>>
>> real    34m11.762s
>> user    45m11.316s
>> sys    0m5.604s
>>
>> (3)
>> guile-2.1.7 build from guile-git-repository, branch master
>> (I've got this warning:
>> configure: WARNING: *** GNU Readline is too old on your system.
>> configure: WARNING: *** You need readline version 2.1 or later.
>> No idea whether this may have an impact on lilyponds compiling-time
>> I'll have to test.)
>> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>>
>> real    67m29.132s
>> user    93m14.812s
>> sys    0m7.332s
>
> Same compilation options?

Yep.
To get comparable results I always did exactly the same, for building
the guile-versions as well as for building lilypond.
Doing all tests with a fresh restarted computer.


Cheers,
  Harm

> This looks like a surprisingly serious
> regression compared to 2.0.
>
> --
> David Kastrup

Re: How to make GNU Guile more successful

[Thomas Morley]

2017-03-05 15:01 GMT+01:00 Thomas Morley <thomasmorley65@gmail.com>:

> The above is already in, see
> $ git log -p 91ff9563ebe1c1cd720ad1a44890e7375fd83da8
[...}
> See:
> git log -p d15c38c0ddd4c04edcf82cda50ca30f6dc4941fa

Aargh, those are from my local _rebased_ branch.
Try instead:
e23d7b1763fdc2e7815c7069d5d1702f1a132383
59197ffbf370b12986ef3f3ee225322048110477

Sorry,
  Harm

Lilypond speed (was Re: How to make GNU Guile more successful)

[Andy Wingo]

On Sun 05 Mar 2017 15:01, Thomas Morley <thomasmorley65@gmail.com> writes:

> Here some timing values
>
> (1)
> lilypond-2.19.52 using guile 1.8.7
> (I would have prefered to build lilypond with a guile-1.8.8 build from
> the guile-repository. Though my try to build it from the
> branch_release-1-8 failed. Instead attempting to fix it, I then used a
> released lilypond-version)
>
> real    8m16.191s
> user    6m39.864s
> sys    0m10.860s
>
> (2)
> guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>
> real    34m11.762s
> user    45m11.316s
> sys    0m5.604s
>
> (3)
> guile-2.1.7 build from guile-git-repository, branch master
> (I've got this warning:
> configure: WARNING: *** GNU Readline is too old on your system.
> configure: WARNING: *** You need readline version 2.1 or later.
> No idea whether this may have an impact on lilyponds compiling-time
> I'll have to test.)
> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>
> real    67m29.132s
> user    93m14.812s
> sys    0m7.332s

Oh, that's interesting.  IME the only thing that is slower on 2.2
compared to 2.0 is the compiler; everything else is significantly
faster.  Could it be that Lilypond is spending time compiling things
somehow?  Perhaps via scm_load without disabling autocompilation or
something?  Guessing at this point tho.

But that's getting ahead of myself -- do you have a document that
exhibits a similar performance series (1.8 < 2.0 < 2.2) but which
doesn't take so long to run?  That can make perf investigation a bit
more tractable :)

Andy

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Thomas Morley]

[-- Attachment #1: Type: text/plain, Size: 4000 bytes --]

2017-03-06 21:41 GMT+01:00 Andy Wingo <wingo@pobox.com>:
> On Sun 05 Mar 2017 15:01, Thomas Morley <thomasmorley65@gmail.com> writes:
>
>> Here some timing values
>>
>> (1)
>> lilypond-2.19.52 using guile 1.8.7
>> (I would have prefered to build lilypond with a guile-1.8.8 build from
>> the guile-repository. Though my try to build it from the
>> branch_release-1-8 failed. Instead attempting to fix it, I then used a
>> released lilypond-version)
>>
>> real    8m16.191s
>> user    6m39.864s
>> sys    0m10.860s
>>
>> (2)
>> guile-2.0.14 build from guile-git-repository, branch remotes/origin/stable-2.0
>> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>>
>> real    34m11.762s
>> user    45m11.316s
>> sys    0m5.604s
>>
>> (3)
>> guile-2.1.7 build from guile-git-repository, branch master
>> (I've got this warning:
>> configure: WARNING: *** GNU Readline is too old on your system.
>> configure: WARNING: *** You need readline version 2.1 or later.
>> No idea whether this may have an impact on lilyponds compiling-time
>> I'll have to test.)
>> lilypond-2.19.56, build from local branch dev/guile-v2.2-work
>>
>> real    67m29.132s
>> user    93m14.812s
>> sys    0m7.332s
>
> Oh, that's interesting.  IME the only thing that is slower on 2.2
> compared to 2.0 is the compiler; everything else is significantly
> faster.  Could it be that Lilypond is spending time compiling things
> somehow?  Perhaps via scm_load without disabling autocompilation or
> something?  Guessing at this point tho.

At least git-grepping for scm_load inthe lilypond-repository returns nothing.

>
> But that's getting ahead of myself -- do you have a document that
> exhibits a similar performance series (1.8 < 2.0 < 2.2) but which
> doesn't take so long to run?  That can make perf investigation a bit
> more tractable :)
>
> Andy

Attached you'll find short-lily-guile-test.ly.
Two overrides, foo and fooI, are defined to show embedding lily in
guile and vice versa as well as some simple string-operations.
(The overrides themselves print silly things, but it's typical
user-generated code, just to demonstrate the method)
Below you'll find timing values for lilypond with guile 1.8.7, 2.0.14 and 2.1.7.
Those three lily-versions are tested without foo and fooI, with foo
and with foo _and_ fooI

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.56 guile 1.8.7
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real    0m2.207s
user    0m2.120s
sys        0m0.076s

with foo:

real    0m6.707s
user    0m6.336s
sys        0m0.372s

with foo and fooI:

real    0m9.749s
user    0m9.268s
sys        0m0.472s


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.57 guile 2.0.14
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real    0m11.843s
user    0m12.452s
sys        0m0.172s

with foo:

real    0m34.020s
user    0m35.284s
sys        0m0.168s


with foo and fooI:

real    0m38.819s
user    0m40.020s
sys        0m0.396s


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
lilypond 2.19.57 guile 2.1.7
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

real    0m8.594s
user    0m9.424s
sys        0m0.180s

with foo:

real    0m14.626s
user    0m15.924s
sys        0m0.204s

with foo and fooI:

real    0m19.422s
user    0m20.960s
sys        0m0.304s


So guile 2.1.7 is indeed faster than 2.0.14 with this test-file, otoh
I've redone testings with the other file and can confirm 2.1.7 being
slower there.
Currently I've no clue why.


Btw, I've improved my local setup to be able to test lilypond more
quickly with different guile versions. Though I wasn't able to compile
1.8.8, neither from the repository nor from the tarball downloaded
from
https://www.gnu.org/software/guile/download/
Due to:
async.c: In function 'scm_i_queue_async_cell':
async.c:243:14: error: variable 'count' set but not used
[-Werror=unused-but-set-variable]
       size_t count;
              ^

Am I missing something?
I'm aware noone is interested in developing 1.8.8 further, though I
would have prefered to build lilypond with that version as well, like
the other test-versions.

Cheers,
  Harm

[-- Attachment #2: short-lily-guile-test.ly --]
[-- Type: text/x-lilypond, Size: 1425 bytes --]

\version "2.19.56"

%% Two overrides, attempting to demonstrate/trigger how embedding lily in guile
%% and vice versa works.
%% Both doing some simple string-operations

foo = {
\override Score.BarNumber.break-visibility = ##(#t #t #t)
\override Score.BarNumber.before-line-breaking =
  #(lambda (grob)
     (let ((b-nmbr 
             (string->number (markup->string (ly:grob-property grob 'text)))))
  
       (ly:grob-set-property! grob 'self-alignment-X CENTER)
       (ly:grob-set-property! grob 'text
         #{ 
            \markup 
             #(if (and (number? b-nmbr) (odd? b-nmbr))
                  (format #f "guile ~a" (version))
                  (format #f "lilypond ~a" (lilypond-version)))
         #})))
}

fooII =
\override NoteHead.after-line-breaking =
#(lambda (grob)
  (if (positive? 
        (ly:pitch-octave 
          (ly:prob-property (ly:grob-property grob 'cause) 'pitch)))
      (begin
        (ly:grob-set-property! grob 'stencil
          (ly:stencil-aligned-to
            (grob-interpret-markup grob
              #{ 
                 \markup #(string-concatenate '("名" "字" "-" "♥" "-" "字" "名")) 
              #})
            CENTER 
            X))
        (ly:grob-set-property! grob 'color red))
      (ly:grob-set-property! grob 'color green)))
  
{
    %% successively uncomment:
    %\foo
    %\fooII
    \repeat unfold 20 { \repeat unfold 4 { c'4 e' g' c'' } \break }
}

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[David Kastrup]

Thomas Morley <thomasmorley65@gmail.com> writes:

> So guile 2.1.7 is indeed faster than 2.0.14 with this test-file, otoh
> I've redone testings with the other file and can confirm 2.1.7 being
> slower there.
> Currently I've no clue why.

Lot's of output?  The output files are generated in
lily/paper-outputter.cc with

    SCM
    Paper_outputter::dump_string (SCM scm)
    {
      return scm_display (scm, file ());
    }

    SCM
    Paper_outputter::scheme_to_string (SCM scm)
    {
      return scm_eval (scm, output_module_);
    }

    SCM
    Paper_outputter::module () const
    {
      return output_module_;
    }

    SCM
    Paper_outputter::output_scheme (SCM scm)
    {
      SCM str = scheme_to_string (scm);
      if (scm_is_string (str))
        dump_string (str);
      return str;
    }

    SCM
    paper_outputter_dump (void *po, SCM x)
    {
      Paper_outputter *me = (Paper_outputter *) po;
      return me->output_scheme (x);
    }

    void
    Paper_outputter::output_stencil (Stencil stil)
    {
      interpret_stencil_expression (stil.expr (), paper_outputter_dump,
                                    (void *) this, Offset (0, 0));
    }

So every output item is generated by running a humongous expression
through scm_eval and then calling scm_display on the expression (when it
turns out it is a string).

For PDF output, those strings are generated in the PostScript backend in
scm/output-ps.scm, typically with stuff like:

(define (char font i)
  (ly:format "~a (\\~a) show"
             (ps-font-command font)
             (ly:inexact->string i 8)))

(define (circle radius thick fill)
  (ly:format
   "~a ~4f ~4f draw_circle"
   (if fill
       "true"
       "false")
   radius thick))

with ly:format defined in C and consequently ping-ponging strings
through the SCM API (using scm_to_locale_stringbuf and
scm_take_locale_stringn).

So the basic question is whether the output generation phase (after all
typesetting and page breaking has finished) is significantly involved in
the total slowdown or not.

> Btw, I've improved my local setup to be able to test lilypond more
> quickly with different guile versions. Though I wasn't able to compile
> 1.8.8, neither from the repository nor from the tarball downloaded
> from
> https://www.gnu.org/software/guile/download/
> Due to:
> async.c: In function 'scm_i_queue_async_cell':
> async.c:243:14: error: variable 'count' set but not used
> [-Werror=unused-but-set-variable]
>        size_t count;
>               ^
>
> Am I missing something?

Remove compilation option -Wall here?

> I'm aware noone is interested in developing 1.8.8 further,

There was just a question on the developer list (I think) how to best
maintain a fork of 1.8.

-- 
David Kastrup

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]


() Thomas Morley <thomasmorley65@gmail.com>
() Thu, 9 Mar 2017 00:17:12 +0100

   Btw, I've improved my local setup to be able to test lilypond
   more quickly with different guile versions.  Though I wasn't
   able to compile 1.8.8, neither from the repository

Strange, because the repo should have...

   nor from the tarball downloaded from
   https://www.gnu.org/software/guile/download/
   Due to:
   async.c: In function 'scm_i_queue_async_cell':
   async.c:243:14: error: variable 'count' set but not used
   [-Werror=unused-but-set-variable]
          size_t count;
                 ^

   Am I missing something?

...a fix installed from 2012-05-01:

 http://git.savannah.gnu.org/cgit/guile.git/commit/?h=branch_release-1-8&id=e2547476441

among other, more recent, changes.  Note that this is on branch
‘branch_release-1-8’.  Perhaps you were on a different branch?

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Ludovic Courtès]

Hello,

Thomas Morley <thomasmorley65@gmail.com> skribis:

> Btw, I've improved my local setup to be able to test lilypond more
> quickly with different guile versions. Though I wasn't able to compile
> 1.8.8, neither from the repository nor from the tarball downloaded
> from
> https://www.gnu.org/software/guile/download/
> Due to:
> async.c: In function 'scm_i_queue_async_cell':
> async.c:243:14: error: variable 'count' set but not used
> [-Werror=unused-but-set-variable]
>        size_t count;
>               ^
>
> Am I missing something?

Could you try configuring like this:

  ./configure --disable-error-on-warning

?

> I'm aware noone is interested in developing 1.8.8 further, though I
> would have prefered to build lilypond with that version as well, like
> the other test-versions.

The performance gap in LilyPond between 1.8 and 2.0 is terrible.  I
suppose LilyPond uses ‘eval’ to run Scheme code?  What fraction of the
Scheme code being run for this benchmark is pre-compiled (as a .go
file)?  Is auto-compilation enabled, and could it be that the figures
include auto-compilation time?

Thanks,
Ludo’.

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Paul]

On 03/09/2017 07:13 AM, Ludovic Courtès wrote:

> What fraction of the Scheme code being run for this benchmark is pre-compiled (as a .go file)?

I don't think any of LilyPond's Scheme code is pre-compiled at this point...

Yep, as David Kastrup wrote in the "GNU Guile 2.1.7 released (beta)" thread on Feb 28, 2017:

<quote>
Regular read and eval, however, is used a lot during the parsing of
files and startup of LilyPond.  But at least under Guile-1.8, the
parsing and preprocessing took up a rather small part of the overall
runtime (in the order of 15% or so), so it is unlikely to be responsible
for the bulk of the slowdown.

My personal guess is that the largest performance impact at the moment
will be due to an absence of generation and installation of .go files.
Since .go files are target-dependent (if I am not mistaken) and LilyPond
is cross-compiled for a number of architectures with different byte
orders and type sizes, it seems tricky to get this under wraps.

The next largest performance impact will be redecoding issues.
</quote>

-Paul

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Ludovic Courtès]

Paul <paul@paulwmorris.com> skribis:

> On 03/09/2017 07:13 AM, Ludovic Courtès wrote:
>
>> What fraction of the Scheme code being run for this benchmark is pre-compiled (as a .go file)?
>
> I don't think any of LilyPond's Scheme code is pre-compiled at this point...
>
> Yep, as David Kastrup wrote in the "GNU Guile 2.1.7 released (beta)" thread on Feb 28, 2017:
>
> <quote>
> Regular read and eval, however, is used a lot during the parsing of
> files and startup of LilyPond.  But at least under Guile-1.8, the
> parsing and preprocessing took up a rather small part of the overall
> runtime (in the order of 15% or so), so it is unlikely to be responsible
> for the bulk of the slowdown.
>
> My personal guess is that the largest performance impact at the moment
> will be due to an absence of generation and installation of .go files.
> Since .go files are target-dependent (if I am not mistaken) and LilyPond
> is cross-compiled for a number of architectures with different byte
> orders and type sizes, it seems tricky to get this under wraps.
>
> The next largest performance impact will be redecoding issues.
> </quote>

Thanks.  As Andy wrote in that thread, it would be beneficial if
LilyPond could pre-compile as much as possible of its core Scheme code.

Ludo’.

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Paul]

On 03/10/2017 11:18 AM, Ludovic Courtès wrote:

> Thanks.  As Andy wrote in that thread, it would be beneficial if
> LilyPond could pre-compile as much as possible of its core Scheme code.

Hi, Yeah, it seems like that would be the next step in addressing the 
performance questions.  That brings up how to compile for different 
targets (from that other thread):

David: Since .go files are target-dependent (if I am not mistaken) and LilyPond
is cross-compiled for a number of architectures with different byte
orders and type sizes, it seems tricky to get this under wraps.

Andy: Yeah.  In both 2.0 and 2.2 there are only four "targets" really (32-bit
and 64-bit, big- and little-endian), so it's somewhat manageable.
"guild compile" does support cross-compilation, and I think there are
some projects that do so; but yep, wiring that up can be tricky like you
say.


So maybe sorting out and/or documenting the "best practices" for 
addressing this trickiness would be a mutually beneficial thing for both 
LilyPond and Guile?  (For anyone looking for ways to help.) Anyone know 
of projects that are examples of how to manage this kind of 
cross-compilation?

(I'm an occasional LilyPond contributor who learned Scheme from hacking 
with/on LilyPond, and I'd love to see LilyPond working well with 
Guile2.  Ultimately, it seems like that would be in the best interests 
of both projects, especially in the long run.)

Cheers,
-Paul

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Thomas Morley]

Hi,

2017-03-09 13:13 GMT+01:00 Ludovic Courtès <ludo@gnu.org>:
> Hello,
>
> Thomas Morley <thomasmorley65@gmail.com> skribis:
>
>> Btw, I've improved my local setup to be able to test lilypond more
>> quickly with different guile versions. Though I wasn't able to compile
>> 1.8.8, neither from the repository nor from the tarball downloaded
>> from
>> https://www.gnu.org/software/guile/download/
>> Due to:
>> async.c: In function 'scm_i_queue_async_cell':
>> async.c:243:14: error: variable 'count' set but not used
>> [-Werror=unused-but-set-variable]
>>        size_t count;
>>               ^
>>
>> Am I missing something?
>
> Could you try configuring like this:
>
>   ./configure --disable-error-on-warning
>
> ?

Works.
Thanks for the hint.

>> I'm aware noone is interested in developing 1.8.8 further, though I
>> would have prefered to build lilypond with that version as well, like
>> the other test-versions.
>
> The performance gap in LilyPond between 1.8 and 2.0 is terrible.  I
> suppose LilyPond uses ‘eval’ to run Scheme code?  What fraction of the
> Scheme code being run for this benchmark is pre-compiled (as a .go
> file)?  Is auto-compilation enabled, and could it be that the figures
> include auto-compilation time?



I think/hope Paul answered already sufficiently.
Let me add, I'd be interested in examples of cross-compiled
applications having already done so, as well.
The guile2-manual says nothing about the best practise or perhaps I missed it.


Thanks,
  Harm

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Ludovic Courtès]

Thomas Morley <thomasmorley65@gmail.com> skribis:

> Let me add, I'd be interested in examples of cross-compiled
> applications having already done so, as well.

It boils down to having a makefile rule along the lines of:

  %.go: %.scm
          guild compile --target="$(host)" -o $@ $<

where $host is the cross-compilation target triplet, such as
arm-linux-gnueaihf.  See
<https://gitlab.com/gnutls/gnutls/blob/master/guile/Makefile.am#L78> for
an example.

HTH,
Ludo’.

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Matt Wette]

If lilypond is performing a lot of eval or lambda generation would turning off optimization help?

	(compile expr  #:opts ‘(#:partial-eval? #f #:cse? #f))

Matt

Re: Lilypond speed (was Re: How to make GNU Guile more successful)

[Andy Wingo]

On Mon 13 Mar 2017 00:20, Matt Wette <matt.wette@gmail.com> writes:

> If lilypond is performing a lot of eval or lambda generation would turning off optimization help?
>
> 	(compile expr  #:opts ‘(#:partial-eval? #f #:cse? #f))

I think Lilypond is currently not going through the compiler at all, so
no partial evaluation or anything else is running -- it's just
macroexpand-then-interpret.

Incidentally in 2.2 the options you need to turn off optimization are a
little more complicated.  We have "guild compile -O0" which will produce
the right set of options but nothing like #:optimize-level 0 or
something that you can pass to `compile'.

Andy

Controlling optimizations in 2.2

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> Incidentally in 2.2 the options you need to turn off optimization are a
> little more complicated.  We have "guild compile -O0" which will produce
> the right set of options but nothing like #:optimize-level 0 or
> something that you can pass to `compile'.

What would you recommend as the main optimization to turn off in 2.2 if
one is to reduce compile time?

I’m asking in the context of Guix, where there’s no much to optimize in
files that just define packages.

Ludo’.

Re: Controlling optimizations in 2.2

[Andy Wingo]

On Mon 13 Mar 2017 15:10, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> Incidentally in 2.2 the options you need to turn off optimization are a
>> little more complicated.  We have "guild compile -O0" which will produce
>> the right set of options but nothing like #:optimize-level 0 or
>> something that you can pass to `compile'.
>
> What would you recommend as the main optimization to turn off in 2.2 if
> one is to reduce compile time?
>
> I’m asking in the context of Guix, where there’s no much to optimize in
> files that just define packages.

Use the equivalent of -O0.  See "guild compile -Ohelp" and what "guild
compile" does.

Andy