Circular dependencies between libraries - what to do?

Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Hi List, 

I have libraries that depend on each other, and I'm not sure 

 (1) how to use require without introducing circular dependencies?

 (2) how to declare their dependencies as MELPA packages?

1. When I put in 

,----
| A
| (require 'B)
`----

and in 

,----
| B
| (require 'A)
`----

I'm probably asking for trouble, but B actually does not work without A,
and A calls B functions, so they do require each other. 


2. I'm struggling a bit with defining MELPA packages. Its pretty well
explained in 

,----
| https://github.com/milkypostman/melpa
`----

but first I don't understand where the package doku comes from when
pressing RET in the *Packages* buffer:

,----
| anything is an uninstalled package.
| 
|      Status: Available -- [Install]
|     Version: 20130605.1746
|     Summary: open anything / QuickSilver-like candidate-selection framework
| 
| Start with M-x anything, narrow the list by typing some pattern,
| select with up/down/pgup/pgdown/C-p/C-n/C-v/M-v, choose with enter,
| left/right moves between sources.  With TAB actions can be selected
| if the selected candidate has more than one possible action.
| 
| Note that anything.el provides only the framework and some example
| configurations for demonstration purposes.  See anything-config.el
| for practical, polished, easy to use configurations which can be
| used to assemble a custom personalized configuration.
`----

when the recipe looks e.g. like this:

,----
| (anything
|  :fetcher git
|  :url "http://repo.or.cz/r/anything-config.git"
|  :files ("*.el" "contrib/*" "extensions/*.el"))
`----

and second I don't see how I can declare dependencies between MELPA
packages, causing package B to be installed when A is installed - and
vice versa, but without circular loop ...

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Emanuel Berg]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> (1) how to use require without introducing circular
> dependencies?

At the top of a file, put the require:s, e.g.,

    (require 'message)
    (require 'moggle)

Then at the end of the file, put

    (provide 'message-my)

If you get a circular dependency (which I never did
with 50+ init files, but OK, I suppose I would solve it
like this) - break the file in two so you get clean
modules.

Here is an example:

http://user.it.uu.se/~embe8573/conf/emacs-init/gnus/message-my.el

Then compile everything - here is the makefile I use,
if you don't have one and need to write one -

http://user.it.uu.se/~embe8573/conf/emacs-init/Makefile

When compiling, it'll say what functions and variables
are (can be) uncovered. They aren't (most often),
actually, but fix it anyway to get even better code,
and to make the compiler shut up :)

> (2) how to declare their dependencies as MELPA
> packages?

I never used MELPA so I don't know. Do tell if when
find out :)

Good luck!

-- 
underground experts united

Re: Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Emanuel Berg <embe8573@student.uu.se> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> (1) how to use require without introducing circular
>> dependencies?
>
> At the top of a file, put the require:s, e.g.,
>
>     (require 'message)
>     (require 'moggle)
>
> Then at the end of the file, put
>
>     (provide 'message-my)

ok, I knew that ;)

> If you get a circular dependency (which I never did
> with 50+ init files, but OK, I suppose I would solve it
> like this) - break the file in two so you get clean
> modules.

this is not about init files but about libraries, i.e. the Outshine
Project libraries outshine.el, outorg.el and navi-mode.el. I'm preparing
release v2.0 right now, I'm almost done, but the former clear hierarchy
(standalone core lib outshine.el, dependent add-on libs outorg and
navi-mode) has vanished since I turned them into a kind of
'org-minor-mode', now they all depend on each other and use each
other. Breaking them up is not really an option.

I would just need advice

 -  if can make them all (require ...) each other without causing breakage?

 -  or if I should rather make one of them require the other two (which
    on - the highest level lib navi-mode, or the lowest-level lib
    outshine.el)?


>> (2) how to declare their dependencies as MELPA
>> packages?
>
> I never used MELPA so I don't know. Do tell if when
> find out :)
>
> Good luck!

thx ;)

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> I have libraries that depend on each other, and I'm not sure 
>
>  (1) how to use require without introducing circular dependencies?

ok, the byte-compiler did not even let me introduce circular
dependencies, so this one is solved, I had to decide where to put the
require statements.

>  (2) how to declare their dependencies as MELPA packages?

This one is still an open question ...

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Stefan Monnier]

> I'm probably asking for trouble, but B actually does not work without A,
> and A calls B functions, so they do require each other.

You can try to restructure the files, so as to get rid of
the circularity.

Or you can load the files differently.  E.g. instead of top-level
`require', you can use autoloads, so that loading A doesn't load B, but
when A calls functions from B, then B gets loaded.  Or instead of
autoloads, you can place a (require 'B) *inside* the functions of A that
call functions from B.


        Stefan

Re: Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I'm probably asking for trouble, but B actually does not work without A,
>> and A calls B functions, so they do require each other.
>
> You can try to restructure the files, so as to get rid of
> the circularity.
>
> Or you can load the files differently.  E.g. instead of top-level
> `require', you can use autoloads, so that loading A doesn't load B, but
> when A calls functions from B, then B gets loaded.  Or instead of
> autoloads, you can place a (require 'B) *inside* the functions of A that
> call functions from B.

ok, seems I found a way to 'distribute' the requires without introducing
circularity. Thanks for the tips.

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Emanuel Berg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> You can try to restructure the files, so as to get
> rid of the circularity.

Yes, if one entity depends on another entity, while
that depends on the first, I can't see any other way
than making more, smaller entities.

And as this is so easy, why consider something else?

Aside from solving the problem you get a better, more
modular design with neater program units which will
benefit you (the OP) in other situations as well.

By the way, you (the OP) said "this is a library
question, not an init file question" (pseudo-quote) -
but what is the difference? If an init makes available
a bunch of Elisp functions for a specific purpose, how
is that not a library?

-- 
underground experts united

RE: Circular dependencies between libraries - what to do?

[Drew Adams]

> I have libraries that depend on each other, and I'm not sure
>  (1) how to use require without introducing circular dependencies?
>  (2) how to declare their dependencies as MELPA packages?
> 
> 1. When I put in
> ,----
> | A
> | (require 'B)
> `----
> and in
> ,----
> | B
> | (require 'A)
> `----
> 
> I'm probably asking for trouble, but B actually does not work
> without A, and A calls B functions, so they do require each other.

If these are your own libraries, so that you can modify their code,
then break them into pieces, and then reorder and recombine pieces.

Here is the guiding principle:

  It is not *all* of A that requires *all* of B, and
  it is not *all* of B that requires *all* of A.

So look at the code carefully and find which pieces have hard
(i.e., logical) dependencies on which other pieces.  Keep pieces
that must be together together; separate (at least logically and
temporarily) things that can be separated.

When you have something that makes sense, and thus works, then
you can try combining pieces to end up with fewer and larger
ones.  You might find that you end up with more than two files.
Or you might find that you end up with only one file, but
better structured (e.g. reordered).

You will have, at least during the exercise, made any real
dependencies explicit.  You will understand your code better,
and you will make it easier to modify it in the future.

Yes, this can take some time.  Two aids are:
1. The byte-compiler, as others have mentioned.
2. Commenting-out sections of code temporarily, using, e.g.,
   `comment-region' (with `C-u' it uncomments).

Re: Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Drew Adams <drew.adams@oracle.com> writes:

>> I have libraries that depend on each other, and I'm not sure
>>  (1) how to use require without introducing circular dependencies?
>>  (2) how to declare their dependencies as MELPA packages?
>> 
>> 1. When I put in
>> ,----
>> | A
>> | (require 'B)
>> `----
>> and in
>> ,----
>> | B
>> | (require 'A)
>> `----
>> 
>> I'm probably asking for trouble, but B actually does not work
>> without A, and A calls B functions, so they do require each other.
>
> If these are your own libraries, so that you can modify their code,
> then break them into pieces, and then reorder and recombine pieces.
>
> Here is the guiding principle:
>
>   It is not *all* of A that requires *all* of B, and
>   it is not *all* of B that requires *all* of A.
>
> So look at the code carefully and find which pieces have hard
> (i.e., logical) dependencies on which other pieces.  Keep pieces
> that must be together together; separate (at least logically and
> temporarily) things that can be separated.
>
> When you have something that makes sense, and thus works, then
> you can try combining pieces to end up with fewer and larger
> ones.  You might find that you end up with more than two files.
> Or you might find that you end up with only one file, but
> better structured (e.g. reordered).
>
> You will have, at least during the exercise, made any real
> dependencies explicit.  You will understand your code better,
> and you will make it easier to modify it in the future.
>
> Yes, this can take some time.  Two aids are:
> 1. The byte-compiler, as others have mentioned.
> 2. Commenting-out sections of code temporarily, using, e.g.,
>    `comment-region' (with `C-u' it uncomments).

I ckecked my libraries and figured out that I actually mixed up two
kinds of dependencies (ignore the names, I just made them up):

 - 'technical dependency' -> lib A calls lib B's functions

 - 'functional dependency' -> lib A won't work on a buffer if lib B
   hasn't been activated

in other words, outshine.el requires outorg.el because it calls its
functions, but outorg.el only needs outline-minor-mode with outshine
extensions loaded to work, it does not technically require outshine
functions.  

So I could actually distribute the (require ...) calls in a sane way
over the libs and solve the issue. 

Thanks for the tips anyway, I will keep them in mind.

PS

I wonder why my announcement of Outshine 2.0 did not make it to this
list - thats like an 8h delay or so.

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Christoph Wedler]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Emanuel Berg <embe8573@student.uu.se> writes:
>
>> Thorsten Jolitz <tjolitz@gmail.com> writes:
>>
>>> (1) how to use require without introducing circular
>>> dependencies?
>>
>> At the top of a file, put the require:s, e.g.,
>>
>>     (require 'message)
>>     (require 'moggle)
>>
>> Then at the end of the file, put
>>
>>     (provide 'message-my)

You can put the `provide' also at the beginning of the file.

  circ-a.el:
    (provide 'circ-a)
    (require 'circ-b)

  circ-b.el:
    (provide 'circ-b)
    (require 'circ-a)

works fine.

- Christoph

Re: Circular dependencies between libraries - what to do?

[Thorsten Jolitz]

Christoph Wedler <christoph.wedler@sap.com> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> Emanuel Berg <embe8573@student.uu.se> writes:
>>
>>> Thorsten Jolitz <tjolitz@gmail.com> writes:
>>>
>>>> (1) how to use require without introducing circular
>>>> dependencies?
>>>
>>> At the top of a file, put the require:s, e.g.,
>>>
>>>     (require 'message)
>>>     (require 'moggle)
>>>
>>> Then at the end of the file, put
>>>
>>>     (provide 'message-my)
>
> You can put the `provide' also at the beginning of the file.
>
>   circ-a.el:
>     (provide 'circ-a)
>     (require 'circ-b)
>
>   circ-b.el:
>     (provide 'circ-b)
>     (require 'circ-a)
>
> works fine.

Interesting. Is this considered 'acceptable' coding style in the Emacs
world?

-- 
cheers,
Thorsten

Re: Circular dependencies between libraries - what to do?

[Stefan Monnier]

> Interesting. Is this considered 'acceptable' coding style in the Emacs
> world?

It's discouraged, although it is used by some packages included in
Emacs, IIRC.


        Stefan

Re: Circular dependencies between libraries - what to do?

[Emanuel Berg]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Interesting. Is this considered 'acceptable' coding
> style in the Emacs world?

The best solution, which is also the easiest one to
understand and the easiest one to implement, is to
break down your program or library into modules and
have them include each other one step at a time.

-- 
underground experts united

Re: Circular dependencies between libraries - what to do?

[Aurélien Aptel]

I was reading the `provide' source code in the context of my dynamic
loading branch [1] and it reminded me of this thread.

On Tue, Sep 30, 2014 at 9:05 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> Interesting. Is this considered 'acceptable' coding style in the Emacs
>> world?
>
> It's discouraged, although it is used by some packages included in
> Emacs, IIRC.

The `provide' call is usually placed at the end of the file because
`provide' also evaluates loading hooks (`eval-after-load' code). If
you place it at the start of the file, loading hooks will potentially
use undefined functions/vars despite being in the file.


1: code https://github.com/aaptel/emacs-dynamic-module
   issues/roadmap  https://github.com/aaptel/emacs-dynamic-module/issues