Org Build System (aka Makefile)

Org Build System (aka Makefile)

[Achim Gratz]

At long last the promised documentation for the build system starts to
materialize on Worg:

http://orgmode.org/worg/dev/org-build-system.html

It undoubtedly needs improvement, so please let me know where and/or add
to it yourself.  Thank you.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: Org Build System (aka Makefile)

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

> At long last the promised documentation for the build system starts to
> materialize on Worg:
>
> http://orgmode.org/worg/dev/org-build-system.html

Great.  Thanks for writing this up!

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

I've just pushed a change to the Makefile to more easily allow
customization of compilation methods.  See

http://orgmode.org/worg/dev/org-build-system.html#sec-3-2-1

for what is available.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org Build System (aka Makefile)

[Bastien]

Hi Achim,

Achim Gratz <Stromeko@nexgo.de> writes:

> I've just pushed a change to the Makefile to more easily allow
> customization of compilation methods.  See
>
> http://orgmode.org/worg/dev/org-build-system.html#sec-3-2-1
>
> for what is available.

Thanks for taking care of this page.

Please make the default "make" procedure display all warnings that the
user would see by compiling Emacs itself.

I know we disagree about this: you think that compiler warnings are for
the developers, not for the users.  I think the default "make" should
send as much warnings as Emacs sends with its own default "make".

Moreover, I think we don't know who is a developer and who is a user.
For example, I'm a developer, Eric is a developer, and we both ignored
that the current "make" was hiding warnings.  Even developers sometimes
don't run "make helpall" -- only those who wants to use something else
than the default compilation method.

If a user wants the compilation to go faster, he can always use another
instruction (the current "make" -- renamed "make quiet"?)

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
> Please make the default "make" procedure display all warnings that the
> user would see by compiling Emacs itself.

That isn't even possible, you'd need to use Emacs' build system (which,
btw gives inconsistent results for repeated compiles).

> I know we disagree about this: you think that compiler warnings are for
> the developers, not for the users.  I think the default "make" should
> send as much warnings as Emacs sends with its own default "make".

You continue to misunderstand what I was saying or at least trying to
say.  The primary function of Org's build system is to, well, build Org
with the minimum fuzz.  That's what it was designed to do and that is
what it does — it can do other things as well, but you'll have to
configure it to do that.

Getting more warnings is a secondary function, however useful they might
be.  Now, Emacs Lisp as a dynamic language is notoriously difficult in
the static checks department (warnings are but a small part of that) and
Emacs lacks functions to do this thoroughly in an automated manner.
There is _no_ complete tool for doing dependency checks at the source
level (I'd love to be proved wrong) for instance.  Emacs does have elint
since version 23, which is properly separated from building, but again,
is still incomplete.  I've added two compilation methods that use elint,
but they certainly aren't for casual use — the first one takes 25 times
as long as a simple build and the other one 275 times.

> If a user wants the compilation to go faster, he can always use another
> instruction (the current "make" -- renamed "make quiet"?)

And by the same argument, everybody can just as well add the line

_COMPILE_=single

to local.mk if wanted.  This gives _different_ warnings (and in general
more), but it is still no substitute for static checks and testing.  I'm
not going to degrade the build performance for everyone to save a
handful of people the bother of doing that.  If you insist, do the
change yourself (it's defined in default.mk).



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: Org Build System (aka Makefile)

[Eric Schulte]

>> I know we disagree about this: you think that compiler warnings are for
>> the developers, not for the users.  I think the default "make" should
>> send as much warnings as Emacs sends with its own default "make".
>
> You continue to misunderstand what I was saying or at least trying to
> say.  The primary function of Org's build system is to, well, build Org
> with the minimum fuzz.  That's what it was designed to do and that is
> what it does — it can do other things as well, but you'll have to
> configure it to do that.
>
> Getting more warnings is a secondary function, however useful they might
> be.

But we certainly shouldn't (and currently aren't?) inhibit the display
of any warnings when the default make is run.  I was surprised to run
make compile-source and see additional warnings which weren't shown
during regular make.  What is the difference between "make" and "make
compile-source" which results in different warnings?

> Now, Emacs Lisp as a dynamic language is notoriously difficult in the
> static checks department (warnings are but a small part of that) and
> Emacs lacks functions to do this thoroughly in an automated manner.
> There is _no_ complete tool for doing dependency checks at the source
> level (I'd love to be proved wrong) for instance.  Emacs does have
> elint since version 23, which is properly separated from building, but
> again, is still incomplete.  I've added two compilation methods that
> use elint, but they certainly aren't for casual use — the first one
> takes 25 times as long as a simple build and the other one 275 times.
>

After some time digging through the make files, it looks to me like one
must edit the local.mk file to run these.  I'd propose that they are
added as a separate Makefile target (mentioned by "make help") so that
they can be easily run.  Very few people (users or developers) are
willing to edit make configuration files.

Perhaps these elint build options should be used to build when "make
check" is run.  If a user is willing to run the test suite they should
be willing to endure a slower build for more thorough warnings.

Best,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Org Build System (aka Makefile)

[Achim Gratz]

Eric Schulte writes:
> But we certainly shouldn't (and currently aren't?) inhibit the display
> of any warnings when the default make is run.  I was surprised to run
> make compile-source and see additional warnings which weren't shown
> during regular make.

These warnings aren't reliable — the byte compiler doesn't really try to
find and report problems.

> What is the difference between "make" and "make
> compile-source" which results in different warnings?

make -n compile
make -n _COMPILE_=single compile

The difference is starting a single Emacs and then compiling all files
vs. starting a fresh Emacs instance for each file to be compiled.  The
change was originally triggered by some differences to the builds in
package manager (ELPA) and solidified due to the fact that this is the
only method that does function with only Emacs available.  Should have
been discussed around November last year, IIRC.

> After some time digging through the make files, it looks to me like one
> must edit the local.mk file to run these.

You are welcome to dig through whatever files, but maybe you might
consult the documentation first?  As you would read there and can see
above, you can do it all on the command line if you wish.  If you want
to enact that change permanently, you should edit local.mk — that's the
only reason it exists.

> I'd propose that they are added as a separate Makefile target
> (mentioned by "make help") so that they can be easily run.

If you want additional make targets you can also implement those in
local.mk; run `make helpall´ some time and ask yourself if you really
need more.

> Very few people (users or developers) are willing to edit make
> configuration files.

Those same people that have no problem to edit the sources?  Come on,
you can't be serious.

> Perhaps these elint build options should be used to build when "make
> check" is run.  If a user is willing to run the test suite they should
> be willing to endure a slower build for more thorough warnings.

If they want to, they can edit local.mk.  But since it is not necessary
for the build and there won't be any warnings to see if the developers
do a good job, it's not a useful default.  It is maybe useful as an
additional configuration for release tests (just as it is useful to have
multiple configurations to be able to test different versions of Emacs).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org Build System (aka Makefile)

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:

> Eric Schulte writes:
>> But we certainly shouldn't (and currently aren't?) inhibit the display
>> of any warnings when the default make is run.  I was surprised to run
>> make compile-source and see additional warnings which weren't shown
>> during regular make.
>
> These warnings aren't reliable — the byte compiler doesn't really try to
> find and report problems.
>

Understood, however they are still a useful check, especially after
large code changes or during code cleanup.  In fact, because I never run
compiled Org-mode, these checks are one of my main uses of the Makefile.

>
>> What is the difference between "make" and "make
>> compile-source" which results in different warnings?
>
> make -n compile
> make -n _COMPILE_=single compile
>
> The difference is starting a single Emacs and then compiling all files
> vs. starting a fresh Emacs instance for each file to be compiled.  The
> change was originally triggered by some differences to the builds in
> package manager (ELPA) and solidified due to the fact that this is the
> only method that does function with only Emacs available.  Should have
> been discussed around November last year, IIRC.
>

Thanks for clarifying.  It certainly does sound like we are using the
correct default build option.  It would be preferable if the Emacs
compilation process produced more predictable warnings, but I doubt this
is a high priority for the busy core Emacs devs.

>
>> After some time digging through the make files, it looks to me like one
>> must edit the local.mk file to run these.
>
> You are welcome to dig through whatever files, but maybe you might
> consult the documentation first?

I don't find the strings "single compile", "compile-source" or "elint"
anywhere in the Org documentation.  Perhaps there is different
documentation for the Makefile?

> As you would read there and can see above, you can do it all on the
> command line if you wish.  If you want to enact that change
> permanently, you should edit local.mk — that's the only reason it
> exists.
>
>> I'd propose that they are added as a separate Makefile target
>> (mentioned by "make help") so that they can be easily run.
>
> If you want additional make targets you can also implement those in
> local.mk; run `make helpall´ some time and ask yourself if you really
> need more.
>

Despite the huge number of Makefile targets (do we really need 12
different versions of "make clean"), I do think that a make target to
run a static code check would be useful.

>
>> Very few people (users or developers) are willing to edit make
>> configuration files.
>
> Those same people that have no problem to edit the sources?  Come on,
> you can't be serious.
>

I am one of these people and I am completely serious.  This is the first
time I've looked at Org-mode's make system -- beyond my help with the
test infrastructure.

The Makefile uses different languages and has different goals than the
source code and I think there are many who feel comfortable editing one
but not the other.  If you'll permit me an exaggerated metaphor, asking
developers to edit a Makefile is like asking a watch maker to rebuild
the table in her workshop.  She will likely find the task to be a waste
of time, to be outside of her core competency, and not directly related
to her real work, even if it results in a more comfortable work
environment.

>
>> Perhaps these elint build options should be used to build when "make
>> check" is run.  If a user is willing to run the test suite they should
>> be willing to endure a slower build for more thorough warnings.
>
> If they want to, they can edit local.mk.

As I continue to contend, editing local.mk simply will not happen in
most cases.

> But since it is not necessary for the build and there won't be any
> warnings to see if the developers do a good job, it's not a useful
> default.  It is maybe useful as an additional configuration for
> release tests (just as it is useful to have multiple configurations to
> be able to test different versions of Emacs).
>

Having spent some time playing around with the elint options, I withdraw
my suggestion that this should be the default for running "make test" as
it is often simply too slow, however, I do think it should be exposed as
a make file target (listed by "make help"), or in some way made possible
without requiring esoteric configuration (either on the command line or
in local.mk).

Thanks for your patient explanations and consideration.

>
>
> Regards,
> Achim.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Org Build System (aka Makefile)

[Bastien]

Let's summarize.

It is no a matter of "exposing the user to the warnings" or not.

It is a matter of exposing the user to the warnings that might be 
useful to him -- i.e. the ones he might want to report to the list
just to let the developers know, or in the context of a bug hunt.

The warnings you get by compiling Elisp files within a single Emacs
process are *not* useful to the users because these warnings happen
during context-free compilation, and as such, they are not relevant to
what the user evaluates when he loads Org (even worse: those warnings
may lead him to misinterpret what is missing at run time, whereas they
warn about things that are missing at comile time.)

So let's stick to the current default `make' for compilation.

However, I would suggest these changes to the current default.mk:

- Have a target `make single' (useful for developers)

- `make test' should use the default make compilation (within one
  Emacs process), because this target is primarily for performing 
  tests, not for checking warnings.

- `make test single' should run the tests *and* compile each file
  in a separate Emacs process -- so that it gives as much info as
  the developers may desire.

- `make elint' would run the current `make _COMPILE_=slint3'.

- `make elint single' would run the current `make _COMPILE_=slint4'
  (even though I would not complain if we get rid of this target.)

- Let's get rid of the _COMPILE_ variable: it is not handy to have 
  to edit this just for running checks.


I hope our brains now all compile fine within the same context.  :)

Achim, if you are okay with the suggestions above, can you make the
relevant changes?

Thanks!

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
> However, I would suggest these changes to the current default.mk:

These changes do not belong into default.mk — default.mk is the fallback
for when no changes to local.mk have been made.

> - Have a target `make single' (useful for developers)
>
> - `make elint' would run the current `make _COMPILE_=slint3'.

I don't like such proliferation of toplevel targets since they can't be
overridden by users, things like this (an alias) are easy enough to set
up in local.mk:

--8<---------------cut here---------------start------------->8---
single:	_COMPILE_=single
single:	compile
elint:  _COMPILE_=slint3
elint:  compile
--8<---------------cut here---------------end--------------->8---

That was (and still is) the point of having local.mk — putting
customized functionality on top of the standard make mechanism.

> - `make test single' should run the tests *and* compile each file
>   in a separate Emacs process -- so that it gives as much info as
>   the developers may desire.
>
> - `make elint single' would run the current `make _COMPILE_=slint4'
>   (even though I would not complain if we get rid of this target.)

Make doesn't work that way, you can't give it two targets and hope for
it to combine these into something else.

> - Let's get rid of the _COMPILE_ variable: it is not handy to have 
>   to edit this just for running checks.

I need to keep it for communication with the sub-make in lisp/, although
I'll probably rename it to something shorter.  There's a reason
everything is set up the way it is and it actually becomes more
complicated rather than easier if I do it differently.

I'd rather provide a sample local.mk for developers that implements
these suggestions.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org Build System (aka Makefile)

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

Achim Gratz <Stromeko@nexgo.de> writes:

> Bastien writes:
>> However, I would suggest these changes to the current default.mk:
>
> These changes do not belong into default.mk — default.mk is the fallback
> for when no changes to local.mk have been made.
>
>> - Have a target `make single' (useful for developers)
>>
>> - `make elint' would run the current `make _COMPILE_=slint3'.
>
> I don't like such proliferation of toplevel targets since they can't be
> overridden by users,

I second the idea that a top level 'make elint' would be very useful for
developers (see the attached patch).  In my opinion this would be more
useful than a number of existing top-level targets, e.g., config-*,
update, update2, cleanall, cleandirs, cleancontrib, cleantesting,
cleanutils, cleanelc and targets.

So if we want to have fewer top level targets (which I think would also
be a good idea), perhaps one or more of the above could be removed
before an elint target is added.

> things like this (an alias) are easy enough to set up in local.mk:

But many more people will use such a target if it exists at the top
level and is mentioned by "make help".


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-adding-elint-top-level-Makefile-target.patch --]
[-- Type: text/x-patch, Size: 1146 bytes --]

From 39e1ce6e8f33561db94451248d1c17705bd8f4ee Mon Sep 17 00:00:00 2001
From: Eric Schulte <eric.schulte@gmx.com>
Date: Mon, 13 Aug 2012 16:42:59 -0600
Subject: [PATCH] adding elint top-level Makefile target

---
 Makefile   | 1 +
 targets.mk | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/Makefile b/Makefile
index 71e2765..0b9535e 100644
--- a/Makefile
+++ b/Makefile
@@ -29,6 +29,7 @@ help helpall::
 	$(info make compile        - build Org ELisp files)
 	$(info make autoloads      - create org-install.el to load Org in-place)
 	$(info make check          - build Org ELisp files and run test suite)
+	$(info make elint          - perform a static check of ELisp source files)
 helpall::
 	$(info make test           - ditto)
 	$(info make compile-dirty  - build only stale Org ELisp files)
diff --git a/targets.mk b/targets.mk
index 29b0aa5..7ddaff1 100644
--- a/targets.mk
+++ b/targets.mk
@@ -80,6 +80,9 @@ compile compile-dirty::
 all clean-install::
 	$(foreach dir, $(SUBDIRS), $(MAKE) -C $(dir) $@;)
 
+elint:
+	$(MAKE) -b _COMPILE_=slint3
+
 check test::	compile
 check test test-dirty::
 	-$(MKDIR) $(testdir)
-- 
1.7.11.4


[-- Attachment #3: Type: text/plain, Size: 46 bytes --]


-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Org Build System (aka Makefile)

[Achim Gratz]

Eric Schulte writes:
> I second the idea that a top level 'make elint' would be very useful for
> developers (see the attached patch).

I'll see to implement that when and if I get elint to process the Org
sources without throwing bogus warnings and errors because it runs into
some depth limit.  Until then I will not expose it on top level.

I take it you're not using `elint-current-buffer´ before checking your
edits in… which is how it was designed to be used, anyway.

> In my opinion this would be more useful than a number of existing
> top-level targets,
> config-*,

Try to replace that functionality any other way.  I could hide the
internal targets and document only the two or three that I want to be
used (see below).

> update

Compatibility.

> update2,

This was specifically requested, not that I like it very much.

> cleanall,

Compatibility.

> cleandirs, cleancontrib, cleantesting, cleanutils, cleanelc

Internal use and compatibility, I could remove the documentation if
Bastien changes his mind about all the target needing documentation.

> targets

Mandated by GNU Makefile standards which I try to adhere to.

> But many more people will use such a target if it exists at the top
> level and is mentioned by "make help".

Speculation.  I know that I won't use it very much because it simply
runs far too long on my machine.  An elint-dirty that just runs through
the files that have been changed would probably be much more useful, but
the time that could have been spent on trying to implement that went to
bikeshedding about which file to edit.  Thanks.

You found the time and energy to edit Makefile and targets.mk, so
presumably you might be able to edit local.mk as well as I suggested
numerous times.  So please go ahead and actually do it and then after
you've used elint for a while tell me how useful you find it from your
experience and if there are other things that need attending aside from
that depth limit.  Get other people to use it too, and have them chime
in.

> +elint:
> +	$(MAKE) -b _COMPILE_=slint3

This introduces a useless fork and GNU make doesn't even process the
'-b' option.  What happens then depends on what is the default target
(which may or may not include `compile´).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: Org Build System (aka Makefile)

[Eric Schulte]

Achim Gratz <Stromeko@nexgo.de> writes:
>
> You found the time and energy to edit Makefile and targets.mk, so
> presumably you might be able to edit local.mk as well as I suggested
> numerous times.  So please go ahead and actually do it and then after
> you've used elint for a while tell me how useful you find it from your
> experience and if there are other things that need attending aside from
> that depth limit.

Will do. Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Org Build System (aka Makefile)

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

>> cleandirs, cleancontrib, cleantesting, cleanutils, cleanelc
>
> Internal use and compatibility, I could remove the documentation if
> Bastien changes his mind about all the target needing documentation.

I'll stick to this: it is good to document all existing targets.

The question is whether a target should be displayed by `make helpall'
rather than just ̀make help'.  

The current make help is sober enough for me.

>> targets
>
> Mandated by GNU Makefile standards which I try to adhere to.

Sure, let's keep it.

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
> I'll stick to this: it is good to document all existing targets.
>
> The question is whether a target should be displayed by `make helpall'
> rather than just ̀make help'.  

As long as `make helpall´ was all the documentation that meant it had to
look like it does.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: Org Build System (aka Makefile)

[Bastien]

Hi Achim,

I reverted the commits introducing the _COMPILE_ variable and the elint
targets in the makefile.

I'm with Eric on thinking that even the casual developer should not have
to tweak his local.mk to run the equivalent of `make compile-single', as
it is directly useful to get some warnings.  Even further: if a user
reports a bug and we want to give him directions to get more information
on his environment, asking him to run `make compile-single' is simple.

As for elint, your last email shows this is still largely experimental 
and I doubt many developers use it to get useful warnings.  Maybe `make
elint' would be useful at some point, but for now developers can simply
do M-x elint-current-buffer RET if they want.

I also reduced the number of targets displayed by `make targets': only
the ones that may be directly useful to the user or the developers are
shown.  You were right on this -- I first needed all doc to be able to 
follow your development myself...

I hope you'll understand the choices above.

Best,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
[...]
> I hope you'll understand the choices above.

You should know the answer from the previous discussion, but I've
clearly failed to reach you.  Given your obvious desire to take over
direct control of the further development of the build system, I won't
do any further development unless you ask.  I'll see to complete the
documentation on Worg in the next few weeks.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org Build System (aka Makefile)

[Bastien]

Hi Achim,

it is not a matter of taking over the build system, it is a matter
of making it simple for the users and useful for the developers.

At least two of the core developers here want `make compile-single' 
and don't want to edit local.mk to do so.

The reverts I did were just for this to be the case.

If a majority of developers want a _COMPILE_ variable or whatever,
I'll happily let you implement it.

The decision I took of getting rid of the elint targets is perhaps
more controversial, but I think elint targets are more gadgets than
anything else right now, and potentially disconcerting ones.

You sound a bit angry at me, which I'm sorry to read.  FWIW, I 100%
acknowledge your sense of rigor and completeness and the way you can
handle complex stuff -- but as a maintainer, I also try to focus on
simplicity.

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Eric Schulte writes:
> I don't find the strings "single compile", "compile-source" or "elint"
> anywhere in the Org documentation.  Perhaps there is different
> documentation for the Makefile?

Yes, as mentioned several times in this thread:
http://orgmode.org/worg/dev/org-build-system.html

I'll add that link to `make help´.

> Despite the huge number of Makefile targets (do we really need 12
> different versions of "make clean"), I do think that a make target to
> run a static code check would be useful.

Most of these are used only internally or are provided for backwards
compatibility with the old Makefile.  Bastien insisted they should all
be documented, so they are.  The elint methods have just been
implemented, but for various reasons I don't really like the idea of yet
more toplevel targets.  If I add one it'll probably be what `slint3´
does.

>> Those same people that have no problem to edit the sources?  Come on,
>> you can't be serious.
>>
>
> I am one of these people and I am completely serious.  This is the first
> time I've looked at Org-mode's make system -- beyond my help with the
> test infrastructure.
>
> The Makefile uses different languages and has different goals than the
> source code and I think there are many who feel comfortable editing one
> but not the other.  If you'll permit me an exaggerated metaphor, asking
> developers to edit a Makefile is like asking a watch maker to rebuild
> the table in her workshop.  She will likely find the task to be a waste
> of time, to be outside of her core competency, and not directly related
> to her real work, even if it results in a more comfortable work
> environment.

To spin that metaphor further, that watchmaker should have no problem
telling a cabinetmaker exactly what her desk should be like and she'd
hopefully be wise enough not to tell him which tools to use or how to
join the table board to the legs.  ALso I think she might listen when
the cabinetmaker tells her how to take care of her new desk properly.

> As I continue to contend, editing local.mk simply will not happen in
> most cases.

So, how did your .emacs come about?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Org Build System (aka Makefile)

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

> http://orgmode.org/worg/dev/org-build-system.html
>
> I'll add that link to `make help´.

Good idea, thanks!

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Bastien]

Hi Achim,

Achim Gratz <Stromeko@nexgo.de> writes:

> You continue to misunderstand what I was saying or at least trying to
> say.  The primary function of Org's build system is to, well, build Org
> with the minimum fuzz.  That's what it was designed to do and that is
> what it does — it can do other things as well, but you'll have to
> configure it to do that.

This I understood perfectly :)

> Emacs does have elint
> since version 23, which is properly separated from building, but again,
> is still incomplete.  I've added two compilation methods that use elint,
> but they certainly aren't for casual use — the first one takes 25 times
> as long as a simple build and the other one 275 times.

Please refrain from adding such new targets to the Makefile for
now.  Let's settle the issue first.

> I'm not going to degrade the build performance for everyone to save a
> handful of people the bother of doing that.  If you insist, do the
> change yourself (it's defined in default.mk).

One thing I need to understand: what are the warnings that you have
when compiling within a single process and you don't when compiling
with one process per file?

The next thing I'd like to know is _why_ -- but even a rough answer
to the first question would help me take a decision about this.

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
> One thing I need to understand: what are the warnings that you have
> when compiling within a single process and you don't when compiling
> with one process per file?

Emacs Lisp as a dynamic language has no concept of a "well-formed"
program that can be verified by just looking at the source.  Correctness
of the program depends on entirely on the context, specifically any
bindings that have been made.  Now, for the same reason it also doesn't
really have a concept of "compilation" like other languages that need to
be translated.  Running the byte-compiler will not only byte-compile,
but also alter the context (predominantly via special forms that are
evaluated at compile-time; but in other ways, too).  So, running it with
one process per file provides the most minimal context, but not
necessarily the "best" or "correct" one.  I could just as well pre-load
org-install into the context of each compilation and things would look
different again.

The warnings you get from the byte-compiler are just noting when the
expected context and the encountered one differs (like the bytecompiler
seeing a function `foo´ being invoked, but doesn't know that is
defined).  This may or may not be an error at runtime, depending on what
the context looks like then.

> The next thing I'd like to know is _why_ -- but even a rough answer
> to the first question would help me take a decision about this.

There is no right or wrong answer to this, there are literally an
infinite number of ways to deal with that problem.  But it all comes
down to managing dependencies and ensuring proper setup of the context,
both at compile-time and run-time.  Org currently takes the heavy-handed
approach of (mostly) requiring all dependencies at compile-time, except
when that would create recursive requires.  This drags in a lot of
mostly useless context into each compilation and can even hide some
problems when some of the context is only indirectly required.  It also
subverts the goals of dynamic (auto-)loading at run-time, since instead
of just pulling in the functions needed it will pull in rather large
bodies of code.

I'd be in favor of automatic dependency management in the autoloads
style, but that would be a larger undertaking for a later time.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: Org Build System (aka Makefile)

[Bastien]

Please give me an example of a warning that is shown while compiling
within a single Emacs process and not shown while compiling files with
one Emacs process per file.

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien <bzg <at> gnu.org> writes:
> Please give me an example of a warning that is shown while compiling
> within a single Emacs process and not shown while compiling files with
> one Emacs process per file.

I don't know if something like that currently exists, if you want to check set
_COMPILE_=slint2 and compare the outputs of the three passes.  I doubt there is,
since the in-process compilation should be clean on current Git master.

Conceivably, you could have a defconst in file1 and the same symbol as a defvar
with initial value in file2, no requires in either file.  If you now compile
them in the order file1 and file2, you will get a warning when compiling in a
single process, but not when you compile them in isolation.  If they were both
defvars w/ initialization, you'd never get a warning even though it is still
wrong and the result at runtime depends on which file gets loaded first.


Regards,
Achim.

Re: Org Build System (aka Makefile)

[Bastien]

I tried _COMPILE_ = single

and I tried

~$ emacs -batch -Q --eval "(byte-compile-file \"~/install/git/org-mode/lisp/ob.el\")"

I get warnings in the second case, not in the first case.

Is there anything that _COMPILE_=single loads/expands on top of a 
bare Emacs when compiling using one Emacs process per file?

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien <bzg <at> gnu.org> writes:
> ~$ emacs -batch -Q --eval "(byte-compile-file
\"~/install/git/org-mode/lisp/ob.el\")"
> 
> I get warnings in the second case, not in the first case.

You should, because the command line you use does not set up the load-path
correctly.  The requires will now use the standard Emacs load-path, which
provides older files with the same name.


Regards,
Achim.

Re: Org Build System (aka Makefile)

[Bastien]

Achim Gratz <Stromeko@NexGo.DE> writes:

> Bastien <bzg <at> gnu.org> writes:
>> ~$ emacs -batch -Q --eval "(byte-compile-file
> \"~/install/git/org-mode/lisp/ob.el\")"
>> 
>> I get warnings in the second case, not in the first case.
>
> You should, because the command line you use does not set up the load-path
> correctly.  The requires will now use the standard Emacs load-path, which
> provides older files with the same name.

Here are the warnings I get:

,----
| In org-babel-sha1-hash:
| ob.el:1022:4:Warning: `(rm (lst) (dolist (p (quote ("replace" "silent"
|     "append" "prepend"))) (setq lst (remove p lst))) lst)' is a malformed
|     function
| ob.el:1033:54:Warning: reference to free variable `arg'
| 
| In org-babel-noweb-p:
| ob.el:2224:34:Warning: `(intersect (as bs) (when as (if (member (car as) bs)
|     (car as) (intersect (cdr as) bs))))' is a malformed function
| 
| In end of data:
| ob.el:2603:1:Warning: the following functions are not known to be defined:
|     org-labels, norm, arg, rm, intersect
| Wrote /home/guerry/install/git/org-mode/lisp/ob.elc
`----

(This is related to the use of cl-labels in ob.el.)

Do you get those warnings with 

~$ emacs -batch -Q --eval "(byte-compile-file \"~/install/git/org-mode/lisp/ob.el\")"

?

Do you get them with make 

~$ make _COMPILE_=single 

?

How do you set up the load-path when compiling with 
`make _COMPILE_=single'?

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Achim Gratz]

Bastien writes:
> Do you get them with make 
> ~$ make _COMPILE_=single 

Not now, but I've seen them before.  I think this is one of those cases
where an indirect require provides a dependency.

> How do you set up the load-path 

The current directory (which is lisp) is prepended to the load-path, so
that any require will find the correct files.  This is always the same
regardless the compilation method.

If you ever want to know what make is doing, just start a dry-run via
`make -n´ and you'll see.  The easiest way to compile a single file is
to remove the 


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Org Build System (aka Makefile)

[Bastien]

By the way, I just tried to compile with

_COMPILE_ = source 

in default.mk (instead of _COMPILE_ = dirall, which I commented) and it
seems to compile twice (as with "single+dirall") and it does not remove
the *elc files, as advertized.

Anything I miss here?

Thanks,

-- 
 Bastien

Re: Org Build System (aka Makefile)

[Samuel Wales]

Wait, I must have missed something.  Some warnings are not showing?

I do something like make >output 2>&1 ; grep -i 'error\|warning'
output. Good code has no warnings, I thought?

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com