relative load-file

relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]

For developing multi-file emacs packages, it would be helpful to have
something akin to require-relative in Ruby 1.9. That is, one wants to
load an Emacs Lisp file relative the file that issues the load which is
often in the same directory or a nearby directory.

This would be easy if there were a variable like __FILE__ that Ruby
provides. In Ruby 1.8, using __FILE__ one can simulate require-relative of
Ruby 1.9.

I figure it should be possible to write such a function or macro using
functions symbol-file, file-name-directory and variable load-file-name, but
the closest I've come so far is to pass in a symbol which is defined prior
in the loaded file. (defmacro involving (symbol-file (gensym)) doesn't seem
to work.)

The code I have is pretty short so I post it below. Any suggestions for
improving, specifically to remove the need to pass in a symbol?

Should this be part of Emacs where I suspect it would be easier to write?

(defun __FILE__ (symbol)
  "Return the string name of file of the currently running Emacs Lisp
program,
or nil.

SYMBOL should be some symbol defined in the file, but it is not
used if Emacs is currently running `load' of a file. The simplest
thing to do is call `provide' prior to this and use the value
given for that for SYMBOL. For example:
  (provide 'something)
  (__FILE__ 'something)
"
  (cond
     (load-file-name)
     ((symbol-file symbol))
     (t nil)))
(defun load-relative (file-or-list symbol)
  "Load an Emacs Lisp file relative to some other currently loaded
Emacs
Lisp file.

FILE-OR-LIST is Emacs Lisp either a string or a list of strings
containing files that you want to loaded. SYMBOL should a symbol
defined another Emacs Lisp file that you want FILE loaded relative
to. If this is called inside a `load', then SYMBOL is ignored and
`load-file-name' is used instead."

  (let ((prefix (file-name-directory (or (__FILE__ symbol) "./"))))
    (if (listp file-or-list)
        (mapcar (lambda(file) (load (concat prefix file))) file-or-list)
      (load (concat prefix file-or-list)))))

[-- Attachment #2: Type: text/html, Size: 2337 bytes --]

Re: relative load-file

[Tassilo Horn]

Rocky Bernstein <rocky@gnu.org> writes:

Hi Rocky!

> For developing multi-file emacs packages, it would be helpful to have
> something akin to require-relative in Ruby 1.9. That is, one wants to
> load an Emacs Lisp file relative the file that issues the load which
> is often in the same directory or a nearby directory.

`load-file' accepts relative paths.

Let's say your project looks like

      foo/foo.el
      foo/misc/foo-misc.el

then a (load-file "misc/foo-misc.el") in foo.el should work.

Bye,
Tassilo

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

On Wed, Nov 11, 2009 at 1:35 PM, Tassilo Horn <tassilo@member.fsf.org>wrote:

> Rocky Bernstein <rocky@gnu.org> writes:
>
> Hi Rocky!
>
> > For developing multi-file emacs packages, it would be helpful to have
> > something akin to require-relative in Ruby 1.9. That is, one wants to
> > load an Emacs Lisp file relative the file that issues the load which
> > is often in the same directory or a nearby directory.
>
> `load-file' accepts relative paths.
>
> Let's say your project looks like
>
>      foo/foo.el
>      foo/misc/foo-misc.el
>
> then a (load-file "misc/foo-misc.el") in foo.el should work.
>
> Bye,
> Tassilo
>

I don't see that this does what I need or meant to convey:
   one wants to load an Emacs Lisp file *relative [to] the* *[directory of
the]* *file that issues the load *

Here my test of the above suggestion..

$ find /tmp/proj -type f -print -exec cat {} \;
/tmp/proj/subdir/subload.el
       (load-file "../test1.el")
/tmp/proj/test1.el
      (load-file "test3.el")
/tmp/proj/test3.el
       (message "test3 here")

Now inside emacs I edit /tmp/proj/subdir/subload.el, and M-x
eval-current-buffer. I get:

load-file: Cannot open load file: /tmp/proj/subdir/test3.el

This is loading relative to the directory I started at, /tmp/proj/subdir,
not relative to the directory that of the file that issued the load,
/tmp/proj/test1.el.

[-- Attachment #2: Type: text/html, Size: 1952 bytes --]

Re: relative load-file

[Tassilo Horn]

Rocky Bernstein <rocky@gnu.org> writes:

Hi Rocky,

> I don't see that this does what I need or meant to convey:
>    one wants to load an Emacs Lisp file *relative [to] the* *[directory of
> the]* *file that issues the load *
>
> Here my test of the above suggestion..
>
> $ find /tmp/proj -type f -print -exec cat {} \;
> /tmp/proj/subdir/subload.el
>        (load-file "../test1.el")
> /tmp/proj/test1.el
>       (load-file "test3.el")
> /tmp/proj/test3.el
>        (message "test3 here")
>
> Now inside emacs I edit /tmp/proj/subdir/subload.el, and M-x
> eval-current-buffer. I get:
>
> load-file: Cannot open load file: /tmp/proj/subdir/test3.el
>
> This is loading relative to the directory I started at,
> /tmp/proj/subdir, not relative to the directory that of the file that
> issued the load, /tmp/proj/test1.el.

I see.  Well, generally those problems don't occur, because all elisp
projects I know put all the *.el files into one directory, and the files
are all named proj-xxx.el with one proj.el.  This prefixing is kind of a
convention (not only for files, but vars and funs, too), because there
are no namspaces in elisp.

Emacs itself has some subdirs, but all of them are added to the
load-path.  In both cases, the usual provide/require mechanics suffice.

Bye,
Tassilo

Re: relative load-file

[Stefan Monnier]

> This is loading relative to the directory I started at, /tmp/proj/subdir,
> not relative to the directory that of the file that issued the load,
> /tmp/proj/test1.el.

You can do

 (load (expand-file-name <relativename> (file-name-directory load-file-name)))

But until now we haven't needed to use such a scheme.


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2059 bytes --]

On Wed, Nov 11, 2009 at 3:17 PM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > This is loading relative to the directory I started at, /tmp/proj/subdir,
> > not relative to the directory that of the file that issued the load,
> > /tmp/proj/test1.el.
>
> You can do
>
>  (load (expand-file-name <relativename> (file-name-directory
> load-file-name)))
>

Unfortunately this doesn't work either because load-file-name might
sometimes be nil.  For example
where in test1.el I had:

   (load-file "test3.el")

Using the above suggestion, if you change that to

  (load (expand-file-name "test3.el" (file-name-directory load-file-name)))

And then M-x eval-current buffer inside "test1.el", you'll get an error in
(file-name-directory load-file-name) because load-file-name is nil.  That's
why in some cases I needed to use symbol-file of a defined symbol and even
failing that "./".



> But until now we haven't needed to use such a scheme.
>

 A long time ago I asked why none of the POSIX shells had a debugger.
Invariably someone gave an answer that shells are so simple, and an
interactive shell is so cool, and "set -x" tracing so awesome that you don't
need a debugger.  A long time ago I asked why all the media players like
xine, mplayer, and vlc rolled their own code with respect to CD handling
rather than use a common library; invariably the response I got  was that
things were better that way.  And I seem to recall a lot of consternation
over the issue of whether distributed emacs lisp code could span more than
one directory.

I don't want to get into get into a discussion of how one writes and
organizes code, or does program development. The Emacs community is content
and I am an outsider here. Please carry on with the great work you have been
doing.

For my own personal reasons I'd like to understand how close I can get to
having something analogous to __FILE__ and require_relative that Ruby has.
Thank you for your helping me figure this out and your tolerance of my
peculiar style of program development.



>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 2888 bytes --]

Re: relative load-file

[Stefan Monnier]

> Unfortunately this doesn't work either because load-file-name might
> sometimes be nil.  For example

Of course in the case of M-C-x or eval-buffer, it will be nil.
If you care about that case, refine it to

   (load (expand-file-name <foo> (if load-file-name
                                     (file-name-directory load-file-name))))
                                                          

-- Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

On Wed, Nov 11, 2009 at 6:06 PM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > Unfortunately this doesn't work either because load-file-name might
> > sometimes be nil.  For example
>
> Of course in the case of M-C-x or eval-buffer, it will be nil.
> If you care about that case, refine it to
>
>   (load (expand-file-name <foo> (if load-file-name
>                                     (file-name-directory load-file-name))))
>

I don't think this is quite right yet.

When load-file-name is nil,  this uses the name of default directory of
current buffer. But that's not the same as the directory of the file
containing the load-relative function.

For example, I could have eval'd a buffer that did a load-relative of a file
in a different directory. Even without this, I can change the default
directory using the cd function. And changing the default directory never
changes the file location of the file issuing load-relative.

Any other thoughts?


>
> -- Stefan
>

[-- Attachment #2: Type: text/html, Size: 1544 bytes --]

Re: relative load-file

[Stefan Monnier]

> When load-file-name is nil,  this uses the name of default directory of
> current buffer. But that's not the same as the directory of the file
> containing the load-relative function.

> For example, I could have eval'd a buffer that did a load-relative of a file
> in a different directory. Even without this, I can change the default
> directory using the cd function. And changing the default directory never
> changes the file location of the file issuing load-relative.

> Any other thoughts?

You obviously know enough about the problem to "fix it" further
(e.g. using buffer-file-name if you prefer it to default-directory).
I don't see any point in fixing it further, since there will always be
corner cases where it doesn't do what the user intended.


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2224 bytes --]

On Wed, Nov 11, 2009 at 8:20 PM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > When load-file-name is nil,  this uses the name of default directory of
> > current buffer. But that's not the same as the directory of the file
> > containing the load-relative function.
>
> > For example, I could have eval'd a buffer that did a load-relative of a
> file
> > in a different directory. Even without this, I can change the default
> > directory using the cd function. And changing the default directory never
> > changes the file location of the file issuing load-relative.
>
> > Any other thoughts?
>
> You obviously know enough about the problem to "fix it" further
>

Huh? If I knew a good solution to this short of changing the source code, I
*wouldn't* be asking.

(e.g. using buffer-file-name if you prefer it to default-directory).
>

What strikes me wrong about going in the direction of using buffer-file-name
or using buffers is that that we are introspecting about is the running
code.

I just took a look at the src/lread.c and although it is a somewhat bit
mysterious to me, from what I gather from readevalloop (and I could be *very
very* wrong here) is that when a file is read in, perhaps there *is
no*buffer created in the Emacs sense.

Avoiding buffer creation to read in a Lisp file make sense because one wants
reading Lisp code to be fast; furthemore there is no reason to store the
entire source contents as source contents after the file is read/eval'd.
(The vast majority of compilers don't save the source text).

I don't see any point in fixing it further, since there will always be
> corner cases where it doesn't do what the user intended.
>

A couple of things strike me as weird.

First, none of the examples I have given do I find really  corner case.
These and many more have come up. (Recall that I have been using this mode
of working, and I gather from prior remarks you haven't.)

Second, a reason one might try to encapsulate this in a library is to be
able to handle as many of the corner cases as possible so run-of-the-mill
users need not have to be concerned - they can use the feature and have
confidence that it does the right thing.

Thanks for the help.


>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 3211 bytes --]

Re: relative load-file

[Stefan Monnier]

> Huh? If I knew a good solution to this short of changing the source code, I
> *wouldn't* be asking.

Here it is then:

  (load (expand-file-name <foo> (file-name-directory
                                 (or load-file-name buffer-file-name))))

> What strikes me wrong about going in the direction of using
> buffer-file-name or using buffers is that that we are introspecting
> about is the running code.

You can't make it work in all cases.  It's simply not possible because
code exists outside of any notion of file, so "relative file name"
cannot always make sense.
So what strikes you as wrong is really a fundamental problem in what
you're requesting, rather than a problem in the solutions I proposed.

It's similar in a sense to the issue of a binary executable trying to
find associated files relative to its own location: in general the
executable cannot know its own location, so the best you can do is use
heuristics like look at $0 and search it in $PATH.

> First, none of the examples I have given do I find really corner case.

They work fine with the above code.

> These and many more have come up. (Recall that I have been using this mode
> of working, and I gather from prior remarks you haven't.)

This mode of working is not encouraged in Emacs.  The main reason is
probably just historical accident, but the fact that it can't work
reliably in all cases would be a good reason to retroactively justify
that choice.

> Second, a reason one might try to encapsulate this in a library is to be
> able to handle as many of the corner cases as possible so run-of-the-mill
> users need not have to be concerned - they can use the feature and have
> confidence that it does the right thing.

That would make sense if the functionality was important, but I haven't
seen evidence that it is.


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 4170 bytes --]

On Wed, Nov 11, 2009 at 11:22 PM, Stefan Monnier
<monnier@iro.umontreal.ca>wrote:

> > Huh? If I knew a good solution to this short of changing the source code,
> I
> > *wouldn't* be asking.
>
> Here it is then:
>
>  (load (expand-file-name <foo> (file-name-directory
>                                 (or load-file-name buffer-file-name))))
>
> > What strikes me wrong about going in the direction of using
> > buffer-file-name or using buffers is that that we are introspecting
> > about is the running code.
>
> You can't make it work in all cases.  It's simply not possible because
> code exists outside of any notion of file, so "relative file name"
> cannot always make sense.
>

This is a silly argument. It is neither necessary nor desirable to make
load-relative work in cases where it makes no sense. Surely one wouldn't
suggest banning "load" because code can get loaded into Emacs outside of
loading a file.


> So what strikes you as wrong is really a fundamental problem in what
> you're requesting, rather than a problem in the solutions I proposed.
>

Obviously, you think of it that way.

It is becoming clear now that this is something that can be addressed pretty
easily by making a small change to the Emacs source. However, failing that,
the mechanisms for simulating this are a little arcane.



> It's similar in a sense to the issue of a binary executable trying to
> find associated files relative to its own location: in general the
> executable cannot know its own location, so the best you can do is use
> heuristics like look at $0 and search it in $PATH.
>

But there are systems like Scala and Ruby where the running program can find
files relative to its own location! Perhaps part of the reason this feature
was provided is because it was felt that it too many programmers were using
those heuristics to create really ugly and unreliable code; yet it could be
done pretty simply and reliably from the interpreter.

Around the time that Emacs is in readevalloop, it knows very well the file
name that it is working with.

Since we are into analogies, here's how I think of this.

Suppose I have a programming language that doesn't have a debugger because
the run-time support for it is lacking. I suppose then that could say that
asking for debugging  in such a programming language is a fundamental
problem in the request. Verily there are many folks who live without
debuggers. Techniques like test-driven development, modular programming,
proving your program correct, interactive shells, dynamic loading and so on
reduce or I suppose some would say eliminate the need. So sure, one could
argue that the deficiency is in how the programmer programs, not the system.


By the way, in such systems it is sometimes possible to cull together
something that sort of feels like a debugger and so that sometimes happens.


>
> > First, none of the examples I have given do I find really corner case.
>
> They work fine with the above code.
>

Um, no. I cited two kinds of situations that fail with the above code and
both in fact do come up.


> > These and many more have come up. (Recall that I have been using this
> mode
> > of working, and I gather from prior remarks you haven't.)
>
> This mode of working is not encouraged in Emacs.  The main reason is
> probably just historical accident, but the fact that it can't work
> reliably in all cases would be a good reason to retroactively justify
> that choice.
>

Yes, I am very familiar with this kind of good reason. :-)


> > Second, a reason one might try to encapsulate this in a library is to be
> > able to handle as many of the corner cases as possible so run-of-the-mill
> > users need not have to be concerned - they can use the feature and have
> > confidence that it does the right thing.
>
> That would make sense if the functionality was important, but I haven't
> seen evidence that it is.
>

And I'm sorry, but I'm not going to try to convince you.

As I wrote before, folks are happy with the status quo, or at least not too
unhappy with it. So again, thanks for the help. I have a clearer idea of
where things stand now.

Keep up the good work!


>
>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 5857 bytes --]

Re: relative load-file

[spedrosa]

[-- Attachment #1: Type: text/plain, Size: 6581 bytes --]

I myself have a modular .emacs, with lots of scripts scattered among various
directories. I cannot just dump them all in a single directory, because some
of these are not even my scripts. Also, it is useful to organize files
according to their function(for instance, Ruby elisp scripts in their own
ruby directory, C in another, common files in yet another).

Nowadays, the main .emacs has to first add the load-paths so that the other
scripts can be loaded. A "load-relative" function would be helpful in this
case. To lessen that pain, I've started using everything relative to a
prefix that I set manually.

Other than that, it does work fine.


;; Lots of load path stuff omitted here

;;
(setq dotemacs-children-prefix "~/.emacs.children/")
(load "~/.emacs.children/dotemacs.el")
(dotemacs-load-children '("options"
              "theme"
              "recentf"
              "ido"
              "elisp"
              "ruby"
              "dictionary"
              "functions"
              "git"
              "twit"
              "svn"
              "scheme"
              "gnus"
              "bbdb"
              "w3"
              "anything"
              "company"
              "time_tracking"
              "jdee"
              "keymaps") )

--
Dotemacs package load status:

[OK] Finished loading file: options
[OK] Finished loading file: theme
[OK] Finished loading file: recentf
[OK] Finished loading file: ido
[OK] Finished loading file: elisp
[OK] Finished loading file: ruby
[OK] Finished loading file: dictionary
[OK] Finished loading file: functions
[OK] Finished loading file: git
[OK] Finished loading file: twit
[OK] Finished loading file: svn
[OK] Finished loading file: scheme
[OK] Finished loading file: gnus
[OK] Finished loading file: bbdb
[OK] Finished loading file: w3
[OK] Finished loading file: anything
[OK] Finished loading file: company
[OK] Finished loading file: time_tracking
[OK] Finished loading file: jdee
[ERROR] Unable to load file: jdee - (file-error Cannot open load file jde)
[OK] Finished loading file: keymaps



--Stephen

programmer, n:
       A red eyed, mumbling mammal capable of conversing with inanimate
monsters.


On Thu, Nov 12, 2009 at 10:01 AM, Rocky Bernstein <rocky@gnu.org> wrote:

>
>
> On Wed, Nov 11, 2009 at 11:22 PM, Stefan Monnier <monnier@iro.umontreal.ca
> > wrote:
>
>> > Huh? If I knew a good solution to this short of changing the source
>> code, I
>> > *wouldn't* be asking.
>>
>> Here it is then:
>>
>>  (load (expand-file-name <foo> (file-name-directory
>>                                 (or load-file-name buffer-file-name))))
>>
>> > What strikes me wrong about going in the direction of using
>> > buffer-file-name or using buffers is that that we are introspecting
>> > about is the running code.
>>
>> You can't make it work in all cases.  It's simply not possible because
>> code exists outside of any notion of file, so "relative file name"
>> cannot always make sense.
>>
>
> This is a silly argument. It is neither necessary nor desirable to make
> load-relative work in cases where it makes no sense. Surely one wouldn't
> suggest banning "load" because code can get loaded into Emacs outside of
> loading a file.
>
>
>> So what strikes you as wrong is really a fundamental problem in what
>> you're requesting, rather than a problem in the solutions I proposed.
>>
>
> Obviously, you think of it that way.
>
> It is becoming clear now that this is something that can be addressed
> pretty easily by making a small change to the Emacs source. However, failing
> that, the mechanisms for simulating this are a little arcane.
>
>
>
>> It's similar in a sense to the issue of a binary executable trying to
>> find associated files relative to its own location: in general the
>> executable cannot know its own location, so the best you can do is use
>> heuristics like look at $0 and search it in $PATH.
>>
>
> But there are systems like Scala and Ruby where the running program can
> find files relative to its own location! Perhaps part of the reason this
> feature was provided is because it was felt that it too many programmers
> were using those heuristics to create really ugly and unreliable code; yet
> it could be done pretty simply and reliably from the interpreter.
>
> Around the time that Emacs is in readevalloop, it knows very well the file
> name that it is working with.
>
> Since we are into analogies, here's how I think of this.
>
> Suppose I have a programming language that doesn't have a debugger because
> the run-time support for it is lacking. I suppose then that could say that
> asking for debugging  in such a programming language is a fundamental
> problem in the request. Verily there are many folks who live without
> debuggers. Techniques like test-driven development, modular programming,
> proving your program correct, interactive shells, dynamic loading and so on
> reduce or I suppose some would say eliminate the need. So sure, one could
> argue that the deficiency is in how the programmer programs, not the system.
>
>
> By the way, in such systems it is sometimes possible to cull together
> something that sort of feels like a debugger and so that sometimes happens.
>
>
>>
>> > First, none of the examples I have given do I find really corner case.
>>
>> They work fine with the above code.
>>
>
> Um, no. I cited two kinds of situations that fail with the above code and
> both in fact do come up.
>
>
>> > These and many more have come up. (Recall that I have been using this
>> mode
>> > of working, and I gather from prior remarks you haven't.)
>>
>> This mode of working is not encouraged in Emacs.  The main reason is
>> probably just historical accident, but the fact that it can't work
>> reliably in all cases would be a good reason to retroactively justify
>> that choice.
>>
>
> Yes, I am very familiar with this kind of good reason. :-)
>
>
>> > Second, a reason one might try to encapsulate this in a library is to be
>> > able to handle as many of the corner cases as possible so
>> run-of-the-mill
>> > users need not have to be concerned - they can use the feature and have
>> > confidence that it does the right thing.
>>
>> That would make sense if the functionality was important, but I haven't
>> seen evidence that it is.
>>
>
> And I'm sorry, but I'm not going to try to convince you.
>
> As I wrote before, folks are happy with the status quo, or at least not too
> unhappy with it. So again, thanks for the help. I have a clearer idea of
> where things stand now.
>
> Keep up the good work!
>
>
>>
>>
>>        Stefan
>>
>
>

[-- Attachment #2: Type: text/html, Size: 9192 bytes --]

Re: relative load-file

[Andreas Schwab]

"spedrosa@gmail.com" <spedrosa@gmail.com> writes:

> I myself have a modular .emacs, with lots of scripts scattered among various
> directories. I cannot just dump them all in a single directory, because some
> of these are not even my scripts. Also, it is useful to organize files
> according to their function(for instance, Ruby elisp scripts in their own
> ruby directory, C in another, common files in yet another).
>
> Nowadays, the main .emacs has to first add the load-paths so that the other
> scripts can be loaded.

Have you considered putting a subdirs.el file there?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: relative load-file

[Stefan Monnier]

> It is becoming clear now that this is something that can be addressed
> pretty easily by making a small change to the Emacs source.  However,
> failing that, the mechanisms for simulating this are a little arcane.

I have no idea what simple change you're talking about.

>> > First, none of the examples I have given do I find really corner case.
>> They work fine with the above code.
> Um, no. I cited two kinds of situations that fail with the above code and
> both in fact do come up.

Then I may have misunderstood something.  Can you state those cases
again where

  (load (expand-file-name <foo> (file-name-directory
                                 (or load-file-name buffer-file-name))))

won't do the right thing and yet those cases do show up?


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

Sorry for the delay. The usual excuse(s): real life, etc ...

On Thu, Nov 12, 2009 at 10:34 AM, Stefan Monnier
<monnier@iro.umontreal.ca>wrote:

> > It is becoming clear now that this is something that can be addressed
> > pretty easily by making a small change to the Emacs source.  However,
> > failing that, the mechanisms for simulating this are a little arcane.
>
> I have no idea what simple change you're talking about.
>

I looked again deeper at the code in* src/lread.c*, and see in fact *
load-file-name* comes pretty close.

But better is *(car current-load-list)* since *current-load-list* is set
from the C function *readevalloop()*. Since *readevalloop* is used from both
*load*-like calls and *eval*-like calls, it covers both kinds of ways to run
code. More important, in the *eval*-like calls, the value isn't changed by
evaluating buffer-changing operations as is the case with*(buffer-file-name)
*.

But since this is low-level and not a public API, I'll do a *stringp* test
and if that fails try the other hueristics.



> >> > First, none of the examples I have given do I find really corner case.
> >> They work fine with the above code.
> > Um, no. I cited two kinds of situations that fail with the above code and
> > both in fact do come up.
>
> Then I may have misunderstood something.  Can you state those cases
> again where
>
>  (load (expand-file-name <foo> (file-name-directory
>                                 (or load-file-name buffer-file-name))))
>
> won't do the right thing and yet those cases do show up?
>

One simple example characteristic of a class of things is to put  a *(find-file
...)* right before that load and evaluate the buffer.

However since the intent is to replace *require* (and *load*), the intended
uses are somewhat stereotypical. So commonly such statements get run early
close to the beginning of the invocation of code. I can add a
suggestion/warning to this effect.**

[-- Attachment #2: Type: text/html, Size: 2649 bytes --]

Re: relative load-file

[grischka]

 > [...]
 > That is, one wants to load an Emacs Lisp file relative the file
 > that issues the load which is often in the same directory or a
 > nearby directory.
 >
 > This would be easy if there were a variable like __FILE__ that Ruby
 > provides.

Why would you need __FILE__ if in emacs you can type much more easily
    #$
As in
    (load (concat (file-name-directory #$) "extra-stuff"))

--- grischka

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

On Fri, Nov 13, 2009 at 12:24 AM, grischka <grishka@gmx.de> wrote:

> > [...]
> > That is, one wants to load an Emacs Lisp file relative the file
>
> > that issues the load which is often in the same directory or a
> > nearby directory.
> >
> > This would be easy if there were a variable like __FILE__ that Ruby
> > provides.
>
> Why would you need __FILE__ if in emacs you can type much more easily
>   #$
> As in
>   (load (concat (file-name-directory #$) "extra-stuff"))
>

Never ascribe to willful intent that which can be easily explained by
ignorance :-)

I've just read up on this, and although it is appealing and another
heuristic that can be tried, it looks as though it too is sometimes gives
nil in cases where say (*car current-load-list)* works, just like *
load-file-name*. In fact I'd be curious to learn the differences between it
and *load-file-name.* Off hand, I'd guess that #$ persists after loading --
when it is set -- whereas *load-file-name* and the *car* don't, but
otherwise they are the same. Anyone know or care to comment?

Here is the test I tried. I put this in a file */tmp/foo.el:

*(message "++1 %s" #$)
(message "++2 %s" (car current-load-list))

When I eval-buffer I get:

++1 nil
++2 /tmp/foo.el

Of course if I* load-file* it, they both come out with the same file names.

Thanks for the idea and info!



> --- grischka
>
>

[-- Attachment #2: Type: text/html, Size: 2423 bytes --]

Re: relative load-file

[Stefan Monnier]

> Sorry for the delay. The usual excuse(s): real life, etc ...

There's no hurry: this is a mailing-list, not an IRC channel.

> I looked again deeper at the code in* src/lread.c*, and see in fact
> *load-file-name* comes pretty close.  But better is *(car
> current-load-list)* since *current-load-list* is set from the
> C function *readevalloop()*.

current-load-list is an internal undocumented variable.
I'd stay far away from it if I were you.  I can't think of any case
where it can give you better info than `load-file-name'.

>> Then I may have misunderstood something.  Can you state those cases
>> again where
>> 
>> (load (expand-file-name <foo> (file-name-directory
>> (or load-file-name buffer-file-name))))
>> 
>> won't do the right thing and yet those cases do show up?

> One simple example characteristic of a class of things is to put
> a *(find-file ...)* right before that load and evaluate the buffer.

This example seems to fail the "those cases do show up" test.  Not just
because the requires/loads tend to occur early in an Elisp buffer, but
also because a call to `find-file' (or set-buffer for that matter) at
the top-level of an Elisp buffer is extremely rare and strongly
discouraged by the convention that loading an Elisp file should not have
any "visible effect" (this convention is useful/necessary to allow
things like Customize to load files at will, e.g. just to get the needed
info to build a customization buffer).


        Stefan

Re: relative load-file

[Stefan Monnier]

> In fact I'd be curious to learn the differences between it
> and *load-file-name.*

The difference is the time at which the expansion takes place (either
read-time or eval-time).
E.g. try

    (defun foo () (interactive) (message "%s" #$))
and
    (defun foo () (interactive) (message "%s" load-file-name))

if you load a file containing one of the above lines and then you do
M-x foo RET you should see that the first returns the file name
(i.e. the value of load-file-name at the time the file was read) whereas
the other returns nil (the value of load-file-name at the time you
called `foo').


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2319 bytes --]

On Fri, Nov 13, 2009 at 9:22 AM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > Sorry for the delay. The usual excuse(s): real life, etc ...
>
> There's no hurry: this is a mailing-list, not an IRC channel.
>

> > I looked again deeper at the code in* src/lread.c*, and see in fact
> > *load-file-name* comes pretty close.  But better is *(car
> > current-load-list)* since *current-load-list* is set from the
> > C function *readevalloop()*.
>
> current-load-list is an internal undocumented variable.
> I'd stay far away from it if I were you.


Looks like that code has been there for over a decade, since '93 in fact.
But I certainly understand your concern and I too have some trepidation. I
can and will remove that condition leaving the remaining ones when it
becomes a problem.


> I can't think of any case
> where it can give you better info than `load-file-name'.
>

We've gone over this - inside eval-buffer.


> >> Then I may have misunderstood something.  Can you state those cases
> >> again where
> >>
> >> (load (expand-file-name <foo> (file-name-directory
> >> (or load-file-name buffer-file-name))))
> >>
> >> won't do the right thing and yet those cases do show up?
>
> > One simple example characteristic of a class of things is to put
> > a *(find-file ...)* right before that load and evaluate the buffer.
>
> This example seems to fail the "those cases do show up" test. Not just
>
because the requires/loads tend to occur early in an Elisp buffer, but
> also because a call to `find-file' (or set-buffer for that matter) at
> the top-level of an Elisp buffer is extremely rare and strongly
> discouraged by the convention that loading an Elisp file should not have
> any "visible effect" (this convention is useful/necessary to allow
> things like Customize to load files at will, e.g. just to get the needed
> info to build a customization buffer).
>

I see.  You seem to have strong and somewhat self-fulfilling views of what
programmers should do or not do in Emacs.

But given that you have said don't use this style and, worse, it is in those
class of things that you "discourage",  your opinion of what is likely to
occur for those who do use the style has less value than if you were coming
from this as and advocate or as someone who had used the style.




>
>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 3714 bytes --]

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

Thanks for the info. Based on what the differences described, it looks like
#$ is what I want to use.


On Fri, Nov 13, 2009 at 9:26 AM, Stefan Monnier <monnier@iro.umontreal.ca>wrote:

> > In fact I'd be curious to learn the differences between it
> > and *load-file-name.*
>
> The difference is the time at which the expansion takes place (either
> read-time or eval-time).
> E.g. try
>
>    (defun foo () (interactive) (message "%s" #$))
> and
>    (defun foo () (interactive) (message "%s" load-file-name))
>
> if you load a file containing one of the above lines and then you do
> M-x foo RET you should see that the first returns the file name
> (i.e. the value of load-file-name at the time the file was read) whereas
> the other returns nil (the value of load-file-name at the time you
> called `foo').
>
>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 1249 bytes --]

Re: relative load-file

[Stefan Monnier]

>> This example seems to fail the "those cases do show up" test. Not
>> just because the requires/loads tend to occur early in an Elisp
>> buffer, but also because a call to `find-file' (or set-buffer for
>> that matter) at the top-level of an Elisp buffer is extremely rare
>> and strongly discouraged by the convention that loading an Elisp file
>> should not have any "visible effect" (this convention is
>> useful/necessary to allow things like Customize to load files at
>> will, e.g. just to get the needed info to build a customization
>> buffer).
> I see.  You seem to have strong and somewhat self-fulfilling views of what
> programmers should do or not do in Emacs.

It's not restrictions about what programmers should do in Emacs, it's
restrictions about how to structure an Elisp package: the file itself
should be "declarative", such that the `load' itself won't affect the
behavior of Emacs.

This is a convention that doesn't come from me, but has appeared over
the years as being useful.  The example of `customize' is just one
of them.  Another case where we load a file and don't expect it to
change the behavior of the running Emacs session is when you
byte-compile a file that requires `foo': the byte-compiler should be
free to load `foo' without having to worry about it changing the
background color of the running session.

Note also that it's a convention that most packages have always followed
without even thinking about it.  And the few packages that didn't follow
it had no trouble adjusting.


        Stefan

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]

Ok. Thanks yet again for the useful information. I imagine this is
documented elsewhere as well.

When given the choice of writing code that works when conventions are
followed and code that works independent of whether conventions are
followed, I'll choose the later, even though I myself will try to follow
convention. (Except of course, there is good reason not to).


On Fri, Nov 13, 2009 at 11:17 AM, Stefan Monnier
<monnier@iro.umontreal.ca>wrote:

> >> This example seems to fail the "those cases do show up" test. Not
> >> just because the requires/loads tend to occur early in an Elisp
> >> buffer, but also because a call to `find-file' (or set-buffer for
> >> that matter) at the top-level of an Elisp buffer is extremely rare
> >> and strongly discouraged by the convention that loading an Elisp file
> >> should not have any "visible effect" (this convention is
> >> useful/necessary to allow things like Customize to load files at
> >> will, e.g. just to get the needed info to build a customization
> >> buffer).
> > I see.  You seem to have strong and somewhat self-fulfilling views of
> what
> > programmers should do or not do in Emacs.
>
> It's not restrictions about what programmers should do in Emacs, it's
> restrictions about how to structure an Elisp package: the file itself
> should be "declarative", such that the `load' itself won't affect the
> behavior of Emacs.
>
> This is a convention that doesn't come from me, but has appeared over
> the years as being useful.  The example of `customize' is just one
> of them.  Another case where we load a file and don't expect it to
> change the behavior of the running Emacs session is when you
> byte-compile a file that requires `foo': the byte-compiler should be
> free to load `foo' without having to worry about it changing the
> background color of the running session.
>
> Note also that it's a convention that most packages have always followed
> without even thinking about it.  And the few packages that didn't follow
> it had no trouble adjusting.
>
>
>        Stefan
>

[-- Attachment #2: Type: text/html, Size: 2619 bytes --]

Re: relative load-file

[Richard Stallman]

The basic idea of `require' is that you specify a feature which gets
found through a search of all libraries.  The idea is that these are
general features which anything might want to use.

You want something different:

    That is, one wants to
    load an Emacs Lisp file relative the file that issues the load which is
    often in the same directory or a nearby directory.

It seems to me that this is useful only for loading other parts of one
single multi-file program.

If you have a program which is in multiple files, you can write a
function similar to `require' which searches for files in any way you
like, and then you can use it.  The function can be in your program;
it does not need to be built-in.  It can load the chosen file using
`load'.

The files should each use `provide' in the usual way, and your
function should use the variable `features' to see if the desired
feature has already been loaded.  The only difference would be in
choosing which file to load.

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]

On Sat, Nov 14, 2009 at 6:24 AM, Richard Stallman <rms@gnu.org> wrote:

> The basic idea of `require' is that you specify a feature which gets
> found through a search of all libraries.  The idea is that these are
> general features which anything might want to use.
>

Require has* two* parts which are distinct. One part has the searching for a
file in it. The other part is the conditional load depending on whether or
not a feature has already been loaded


> You want something different:
>
>    That is, one wants to
>    load an Emacs Lisp file relative the file that issues the load which is
>    often in the same directory or a nearby directory.
>
> It seems to me that this is useful only for loading other parts of one
> single multi-file program.
>

No. A large modular program can be composed of other smaller modular
programs. In theory, one can package each file or subsets of files as their
own package, but often those smaller modules may be custom enough that it
doesn't make sense to do so. Or at least not initially.

However when developing or working on the program, one wants may want to
work with each of the subparts independently.

Here's an example that no doubt you are familiar with: a compile has a
parser, code generator and code optimizer. In a well-written modular
compiler one may want to work on and test the parser as an independent unit
even though I'm not sure you would want to package the front-end for a
specific language independently.

So what require-relative and load-relative allow one to do is create these
little independent units without the overhead of using a more elaborate
packaging systems. require/provide in fact does this too. However it pulls
in file searching which is not desired here. Instead, what we want to do in
a program built of custom modules but is just change how the file searching
is managed.

The second benefit of this is that we can develop out of the source tree
(that is without having to "install" the submodules) and have another
version or many versions "installed" at the same time. Starting with a
member of each version stays within that version of the code.


> If you have a program which is in multiple files, you can write a
> function similar to `require' which searches for files in any way you
> like, and then you can use it.  The function can be in your program;
> it does not need to be built-in.  It can load the chosen file using
> `load'.
>

> The files should each use `provide' in the usual way, and your
> function should use the variable `features' to see if the desired
> feature has already been loaded.  The only difference would be in
> choosing which file to load.
>


That's in fact what I have done<http://github.com/rocky/emacs-load-relative>.
The only "primitive" needed is Ruby's __FILE__  which is the same thing as
the C preprocessor  __FILE__.

$# and load-file-name are close; better is (car-safe current-load-list). How
these differ, and where __FILE__ is better, is that a macro substitution of
the file name is done at compile time inside the code the if compiled, or at
eval time if not.

[-- Attachment #2: Type: text/html, Size: 4087 bytes --]

Re: relative load-file

[M Jared Finder]

Let me state this clearly, because I'm not sure you're getting it.

A function that is being defined does not necessarily get defined from a 
file.

For example, you can send code directly to the interpreter.  Like this:
(eval '(defun foo () (message "Hello")))

What should __FILE__ return in this case?

eval-current-buffer and friends all are using this code path.  require 
uses load which is why load-file-name and #$ work.



Oh and by the way, (car-safe current-load-list) doesn't do anything 
useful in Emacs 22.

   -- MJF

Rocky Bernstein wrote:
> 
> 
> On Sat, Nov 14, 2009 at 6:24 AM, Richard Stallman <rms@gnu.org 
> <mailto:rms@gnu.org>> wrote:
> 
>     The basic idea of `require' is that you specify a feature which gets
>     found through a search of all libraries.  The idea is that these are
>     general features which anything might want to use.
> 
> 
> Require has/ two/ parts which are distinct. One part has the searching 
> for a file in it. The other part is the conditional load depending on 
> whether or not a feature has already been loaded
> 
> 
>     You want something different:
> 
>        That is, one wants to
>        load an Emacs Lisp file relative the file that issues the load
>     which is
>        often in the same directory or a nearby directory.
> 
>     It seems to me that this is useful only for loading other parts of one
>     single multi-file program.
> 
> 
> No. A large modular program can be composed of other smaller modular 
> programs. In theory, one can package each file or subsets of files as 
> their own package, but often those smaller modules may be custom enough 
> that it doesn't make sense to do so. Or at least not initially.
> 
> However when developing or working on the program, one wants may want to 
> work with each of the subparts independently.
> 
> Here's an example that no doubt you are familiar with: a compile has a 
> parser, code generator and code optimizer. In a well-written modular 
> compiler one may want to work on and test the parser as an independent 
> unit even though I'm not sure you would want to package the front-end 
> for a specific language independently.
> 
> So what require-relative and load-relative allow one to do is create 
> these little independent units without the overhead of using a more 
> elaborate packaging systems. require/provide in fact does this too. 
> However it pulls in file searching which is not desired here. Instead, 
> what we want to do in a program built of custom modules but is just 
> change how the file searching is managed.
> 
> The second benefit of this is that we can develop out of the source tree 
> (that is without having to "install" the submodules) and have another 
> version or many versions "installed" at the same time. Starting with a 
> member of each version stays within that version of the code.
> 
> 
>     If you have a program which is in multiple files, you can write a
>     function similar to `require' which searches for files in any way you
>     like, and then you can use it.  The function can be in your program;
>     it does not need to be built-in.  It can load the chosen file using
>     `load'.
> 
> 
>     The files should each use `provide' in the usual way, and your
>     function should use the variable `features' to see if the desired
>     feature has already been loaded.  The only difference would be in
>     choosing which file to load.
> 
> 
>  
> That's in fact what I have done 
> <http://github.com/rocky/emacs-load-relative>.  The only "primitive" 
> needed is Ruby's __FILE__  which is the same thing as the C 
> preprocessor  __FILE__.
> 
> $# and load-file-name are close; better is (car-safe current-load-list). 
> How these differ, and where __FILE__ is better, is that a macro 
> substitution of the file name is done at compile time inside the code 
> the if compiled, or at eval time if not.
> 

Re: relative load-file

[Richard Stallman]

    So what require-relative and load-relative allow one to do is create these
    little independent units without the overhead of using a more elaborate
    packaging systems.

I do not follow your point.  I think I understand what you mean by a
program with subparts.  Can you explain why you think
`require-relative' is particularly helpful for that?

    That's in fact what I have done<http://github.com/rocky/emacs-load-relative>.
    The only "primitive" needed is Ruby's __FILE__  which is the same thing as
    the C preprocessor  __FILE__.

Why do you think that making it relative to the location of the loading
file is particularly necessary?

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2322 bytes --]

On Sun, Nov 15, 2009 at 5:38 PM, Richard Stallman <rms@gnu.org> wrote:

>    So what require-relative and load-relative allow one to do is create
> these
>    little independent units without the overhead of using a more elaborate
>    packaging systems.
>
> I do not follow your point.  I think I understand what you mean by a
> program with subparts.  Can you explain why you think
> `require-relative' is particularly helpful for that?
>

Particularly helpful? Let's just go with helpful. ;-)

require-relative allows me to load or eval and test any one of the subparts
without mucking with the load path yet ensure I am getting the right files
loaded.

One can do something similar by saving, modifying and restoring load-path.
I've done that in many languages for many years. And after all these years
it is still clumsy, cumbersome, and fragile.

(In require-relative, after the check for a feature, if the feature is
loaded an additional check can be done to see if the source location is the
same as the location requested)

Others have noticed issues with path-searching mechanisms; so programming
languages like Ruby, Scala, and Python have been moving in the direction of
a something like require-relative even though they've long had something
like load-path.


>    That's in fact what I have done<
> http://github.com/rocky/emacs-load-relative>.
>     The only "primitive" needed is Ruby's __FILE__  which is the same thing
> as
>    the C preprocessor  __FILE__.
>
> Why do you think that making it relative to the location of the loading
> file is particularly necessary?
>
>
Again let's drop the "particularly" part. No doubt there are many ways to do
things and I don't represent that this is the only to work. Possibly not the
best way either, although I think an improvement for me over what I've seen
codified so far.

When I create source code, I very well know what the relative directory/file
structure looks like. So when I want to load or require another module,
using the relative name is petty simple, straightforward, and clear.
Irrelevant is what the load-path or current directory might be is set to at
run-time. The only information Emacs needs to fill in is the directory of
the source code.

That said, I suspect there are other situations that a macro-expanded
__FILE__ would be useful.

[-- Attachment #2: Type: text/html, Size: 3028 bytes --]

Re: relative load-file

[Stefan Monnier]

> Let me state this clearly, because I'm not sure you're getting it.
> A function that is being defined does not necessarily get defined
> from a file.

I think he gets it, mind you.  His usage pattern is just different.
His request makes sense.  But as long as his code is written properly
(expand-file-name <foo> (file-name-directory (or load-file-name
buffer-file-name))) should give the right result both during load and
during eval-buffer.  We could also change #$ to return (or load-file-name
buffer-file-name) rather than load-file-name so that
(expand-file-name <foo> (file-name-directory #$)) does the trick even
more reliably.


        Stefan

Re: relative load-file

[Richard Stallman]

    require-relative allows me to load or eval and test any one of the subparts
    without mucking with the load path yet ensure I am getting the right files
    loaded.

I see.  It is for when the program is NOT installed.

But suppose it were enough had to add one parent directory to
load-path, for instance ~/elisp, and you put all your directories of
Lisp programs under that.  Suppose that you specify `foo/bar', and it
would find that under ~/elisp.

Would that be easy and reliable enough?

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 2975 bytes --]

On Wed, Nov 18, 2009 at 7:10 AM, Richard Stallman <rms@gnu.org> wrote:

>    require-relative allows me to load or eval and test any one of the
> subparts
>    without mucking with the load path yet ensure I am getting the right
> files
>    loaded.
>
> I see.  It is for when the program is NOT installed.
>

I am not sure you see. For me, installation is kind of a sketchy thing -
several different versions of a program may be "installed".  Furthermore,
testing, debugging, or running subparts in one of those "installations" for
me is no different than testing when the program is "not installed".

So for me, one thing that make a programming development agile is reducing
the overhead and distinction of this "install" step.


>
> But suppose it were enough had to add one parent directory to
> load-path, for instance ~/elisp, and you put all your directories of
> Lisp programs under that.  Suppose that you specify `foo/bar', and it
> would find that under ~/elisp.
>

load-path may be fine for finding initially where to get a package. However
it is cumbersome and unnecessary for referring to the submodules of a
program from inside the source code.

In compiled languages this distinction is pretty clear. For example in C one
doesn't confuse PATH with INCLUDE_PATH. And note that I can write #include
"./stidio.h" as well as #include <stdio.h> which are very different things;
both are equally valid and used.


> Would that be easy and reliable enough?
>

Well, let's check out the possibilities.

Do I write my program with subparts to assume that "." is first in the load
path? I not, and I just assume "." is first, then my program can break *at
run time *if that assumption is not valid.

Okay, so now suppose I check, and possibly add something like "." (And here
__FILE__ not "." would the most reliable thing to use) and finally remove it
someplace in the code where I know I don't need to load any more files. Now
my program is more clumsy. Furthermore it is possible I might not get those
stereotypical steps right. And I suppose I need to worry about error cases:
if load-path is added and some error condition is raised, I should make sure
that my addition is removed.

For these two reasons it sounds like it would be good to write a package to
do these steps.  But then, why do I need load-path at all? For internal
module loading, what *benefit* is it serving?

And so  that's what I've done. What makes this a little unreliable right now
is the fact that there isn't a solid definition of __FILE__, though the
package I guess is providing one which over time will get better.

Since people's heads explode at the idea of adding __FILE__ because as has
been noted there are some situations where it makes no sense, suppose
load-file-name is *extended* to include more cases.

load-file-name already returns nil in some situations, so I guess people may
be less put off it it returns nil is a couple fewer cases such being called
from eval-buffer.

[-- Attachment #2: Type: text/html, Size: 3856 bytes --]

Re: relative load-file

[Richard Stallman]

I now have a better idea of what you're trying to do, and why you want
this to be relative.

However, it seems to me that a slightly different relative construct
might be better.  Instead of relative to "this file". it could be
relative to "this package's top directory".  To identify a directory
as counting in this way, you'd put a certain file name in it.  Emacs
would find the containing directory which has that name.  Then you
could specify a file name relative to that directory.

What do you think of that?

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 3570 bytes --]

On Sat, Nov 21, 2009 at 5:52 PM, Richard Stallman <rms@gnu.org> wrote:

> I now have a better idea of what you're trying to do, and why you want
> this to be relative.
>
> However, it seems to me that a slightly different relative construct
> might be better.  Instead of relative to "this file". it could be
> relative to "this package's top directory".  To identify a directory
> as counting in this way, you'd put a certain file name in it.  Emacs
> would find the containing directory which has that name.  Then you
> could specify a file name relative to that directory.
>
> What do you think of that?
>

I don't think it as good for a couple of reasons.

First, it seems like a little more work that isn't strictly needed - adding
that file.

Second, as files move over the course of the lifetime of a project, this
scheme will *always* require changing files more often than what I proposed
. Here is an example based on experience so far.

I am working on a debugger front end
<http://github.com/rocky/emacs-dbgr>package for GNU Emacs. Some files
are specific to specific debuggers (
bashdb <http://bashdb.sourceforge.net/>,
remake<http://github.com/rocky/remake>,
pydbgr <http://code.google.com/p/pydbgr/>,
rbdbgr<http://github.com/rocky/rbdbgr>,
etc.).  For now, I find it convenient to split the parts of the Ruby
debugger *rbdgr* into several of files: *rbdbgr-core*, *rbdbgr-track-mode*and
*rbdbgr*. *rbdbgr* requires *rbdbgr-track-mode* and *rbdbgr-core*. *
rbdbgr-track-mode* requires *rbdbgr-core*. Initially life was good. But then
as I started adding more code and more debuggers, I wanted to have Ruby
debugger code in a separate Ruby folder.

With the scheme I am currently using, I don't have to change the requires
when they stay the same relative to other required files: *rbdbgr* still
requires *rbdbgr-core* and *rbdbgr-track-mode* which were located in the
same directory and continue to be so. Irrelevant is how they are accessed
from the top. But the scheme you propose, I think I would have had to change
the *require* string.

Right now, I am planning on putting code for a second debugger
ruby-debug<http://rubyforge.org/projects/ruby-debug/>in the Ruby
directory. Given this change, perhaps I want a directory for
each debugger. Perhaps that is off of the top-level, or perhaps this is
another directory underneath Ruby. Again with the scheme I currently use,
there are fewer changes which makes it easier for me to try out the various
possibilities.

If there is a __FILE__ variable/macro/function, implementing what I propose
is very simple. Aside from lingering the bugs that I will discover and fix
over time, what I want is done.

But here's another neat thing that I just realized one can do using __FILE__
(or an extended load-file-name if you prefer).  I've just added that
to the load-relative
package <http://github.com/rocky/emacs-load-relative>.

Rather than use: (provide '*feature-name*), via  __FILE__ one can write a *
provide-me* macro that allows you to omit having to give the feature name if
it is the same as the file's base name sans directory. (Personally, I think
using this file/feature naming convention is generally a good idea). The
macro is short and simple. So here it is:

(defmacro provide-me ()
  "Call `provide' with the feature's symbol name made from
the source-code's file basename sans extension. For example if you
write (provide-me) inside file ~/lisp/foo.el, this is the same as
writing: (provide 'foo)."
  `(provide (intern (file-name-sans-extension
      (file-name-nondirectory (__FILE__))))))

[-- Attachment #2: Type: text/html, Size: 4699 bytes --]

Re: relative load-file

[Richard Stallman]

    First, it seems like a little more work that isn't strictly needed - adding
    that file.

That is one file in your entire program, which we assume is so many
files that you have decided to split it into multiple directories.
It is negligible work.

    Second, as files move over the course of the lifetime of a project, this
    scheme will *always* require changing files more often than what I proposed
    . Here is an example based on experience so far.

With file-relative references, any time you move one file,
you have to change all its references to other files,
as well as all other files' references to it.

With main-directory-relative references, when you move a file
between subdirectories, you don't have to change its references
to other files.  You only change the way other files refer to it.

Thus, when moving one file, the main-directory-relative approach
always requires less change.

You present a scenario of moving many files together.  Indeed, in that
scenario, the file-relative approach avoids the need to change the way
these files refer to each other.  But it still requires changing the
way they refer to all other files.

I can't fully follow your scenario because

    For now, I find it convenient to split the parts of the Ruby
    debugger *rbdgr* into several of files: *rbdbgr-core*, *rbdbgr-track-mode*and
    *rbdbgr*

I don't understand the significance of the *'s.  Do you write *'s in
file names, or are they wildcards, or what?

Emacs convention is not to use *'s in any names, except for names
of special buffers.

Re: relative load-file

[Rocky Bernstein]

[-- Attachment #1: Type: text/plain, Size: 5794 bytes --]

On Sun, Nov 22, 2009 at 9:29 PM, Richard Stallman <rms@gnu.org> wrote:

>     First, it seems like a little more work that isn't strictly needed -
> adding
>    that file.
>
> That is one file in your entire program, which we assume is so many
> files that you have decided to split it into multiple directories.
> It is negligible work.
>

Yes, but it is still more work. I believe that declaring that this
file/module belongs to some specific package makes things worse for the
programmer in terms of agility and free software.

Why? Let's go back to the compiler project which has sub-part parser and
code generator. Initially, I have directories for the parser and code
generator parts, while compiler has some top-level glue routines like
compile-file and build-project. The language for my compiler is wildly
successful and so now people request to use the parser part without the
other stuff.

Ok. So now I want to turn that sub-part into its own project. If I were
clairvoyant, I might have realized this in the beginning and made parser as
its own project inside compiler. But not code generator, because no one is
going to ask about that and it looks like it is additional work to note
top-level sub-parts. If however I make compiler a sub-project, from the
internal-loading standpoint I have to constrain myself to stay inside the
parser subdirectory.  There may be other utility programs in the compiler
that I want to use that I haven't externalized yet. For example, perhaps it
is "load-relative".

So how in the scheme you propose do I handle this? Is this adding more
overhead compared to load-relative? If so, it is less agile.

But I also wrote that this is also bad from a standpoint of Free Software.
Why? Because I want to encourage others to use parts of my program, extend
and modify it. If I have declared in the file that this file/module belongs
to a specific package, then that is something everyone who wants to use that
part will have to change when they adapt for their use, but might not under
the scheme I propose.


>
>    Second, as files move over the course of the lifetime of a project, this
>    scheme will *always* require changing files more often than what I
> proposed
>    . Here is an example based on experience so far.
>
> With file-relative references, any time you move one file,
> you have to change all its references to other files,
> as well as all other files' references to it.
>

For other files' references to it, yes. Not exactly true for its references
to other files. See below.


> With main-directory-relative references, when you move a file
> between subdirectories, you don't have to change its references
> to other files.  You only change the way other files refer to it.
>

And that's sometimes true with file-relative references. If I move a file
from
top/part/a.el to top/another-part/a.el  and "a.el" only refers to things
above "part"
-- that is the references all start out ".." -- then there is no change.


> Thus, when moving one file, the main-directory-relative approach
> always requires less change.
>

Not true. See above. I'm not sure how one assess the value of simplifying
moving one file versus renaming a directory or moving a collection of
related files.


>
> You present a scenario of moving many files together.  Indeed, in that
> scenario, the file-relative approach avoids the need to change the way
> these files refer to each other.


And why that is likely to occur is that we agree that modular programs are
made of subparts which are in turn made of other subparts. The names of the
subparts and the locations within the project may change, but the relative
position will probably change less.

 But it still requires changing the
> way they refer to all other files.
>

In some cases, yes. But in some cases no - such as renaming a directory
which has components which are not referred to outside the
directory/subpart.

In my compiler example, I said I had top-level utility programs to compile a
file or build a project. If I push those programs down into a directory say
"utility" then yes, I have to change references. And under the scheme you
propose I wouldn't. However if later I decide to change split "utility" into
two directories at the same level under compilers, say "command-line" and
"gui", then no I don't have to change anything. Again, I am assuming here
that there are no references into  "command-line" or "gui" from say "parser"
or "code-generator" and that files inside "command-line" refer only to other
"command-line" files not the "gui" files or vice versa. But I think these
are reasonable and common situations. At any rate, this has been my
experience so far.


> I can't fully follow your scenario because
>
>    For now, I find it convenient to split the parts of the Ruby
>    debugger *rbdgr* into several of files: *rbdbgr-core*,
> *rbdbgr-track-mode*and
>     *rbdbgr*
>
I don't understand the significance of the *'s.  Do you write *'s in
> file names, or are they wildcards, or what?
>

Sorry. My mistake. *x* was supposed to indicate a name of a file in italic
font. See
http://lists.gnu.org/archive/html/emacs-devel/2009-11/msg00940.html which
seems to show this. However, since this is a mailing list, in the future
I'll use quotes for file names, like have have above.

Although I didn't intend wildcards, now that you mention it that's a good
thing to add to the load-relative package. For example, right now I have an
"init" directory which contains initialization code for each of the
debuggers that the debugger front-end supports. Right now I have "do-list"
to load each of those. Better though would be to put the looping inside the
load-relative package.


> Emacs convention is not to use *'s in any names, except for names
> of special buffers.
>

[-- Attachment #2: Type: text/html, Size: 7901 bytes --]

Re: relative load-file

[Richard Stallman]

    Ok. So now I want to turn that sub-part into its own project. If I were
    clairvoyant, I might have realized this in the beginning and made parser as
    its own project inside compiler. But not code generator, because no one is
    going to ask about that and it looks like it is additional work to note
    top-level sub-parts. If however I make compiler a sub-project, from the
    internal-loading standpoint I have to constrain myself to stay inside the
    parser subdirectory.  There may be other utility programs in the compiler
    that I want to use that I haven't externalized yet. For example, perhaps it
    is "load-relative".

I see the point.