req-package

req-package

[Edward Knyshov]

Hello everyone.
My name is Edward.

I was asked about explanation of how does req-package work and which
problems does it solve. The question appeared in this list's discussion on
req-package moving to core package proposal.

So, req-package solves one single problem - make order of package
configurations in your init.el right without continuous reordering your
code while still providing ambrosian use-package goodness. It makes your
.emacs.d code more strict, modular and error prone. You can look here, how
I divided my code in separate modules and how simple does it looks
https://github.com/edvorg/emacs-configs/tree/master/init.d .

Remember, how often you tackled into problem, when you need to require one
package, do some configuration, then the same with second and so on.
Sometimes it becomes too complex. Especially in cases when one package have
more than one dependency. You can draw a graph of dependencies in your
configuration, and, I'm sure, it's complex. req-package creates this graph
for you and makes a correct traverse on it. The syntax is almost the same
as with use-package, but it provides a few additional keywords:
1) :require - a parameter to specify dependencies
2) :loader - an optional parameter to specify where to get package (el-get,
elpa, etc.)

Interesting thing is that packages are installed automatically once
req-package-finish function is executed. So there is no need for things
like cask or save-packages. You just write a configuration with packages
you need and they will be there. req-package will try to use elpa, el-get
or any package system provided by you to find and install your packages.

That's it. I'm not trying to push my package to core. I'm just answering a
question here.

Edward.

Re: req-package

[Stefan Monnier]

> Remember, how often you tackled into problem, when you need to require one
> package, do some configuration, then the same with second and so on.

Indeed, one of the problems I have here is to remember the last time
I had a problem like the one you describe.

Obviously, we're not using the same set of packages or the same kind
of configuration.  Can you give some examples you've bumped into?


        Stefan

Re: req-package

[Edward Knyshov]

Yes, sure. Sorry for late answer. Here is example. I configure clojure
development environment with req-package(use-package)

(req-package eldoc
​​ ;; this form call will download eldoc package from elpa or el-get as
needed

  :commands eldoc
​​
​​
-mode
​​ ;; this parameter delays loading of eldoc until eldo-mode function is
called

  :init (prog
​​
n (add-h
​​
ook 'emacs-lisp-mode-hook (lambda () (eldoc-mode 1)))
​ ​;; add eldoc to emacs lisp mode

​                      ​
  (add-hook 'lisp-interaction-mode-hook (lambda () (eldoc-mode 1)))))
​​
;; add eldoc to
​lisp based interaction modes
 modes
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​
​​

​​
​​
(req-package clojure-mode
​​
 ;; this form call will download
​clojure-mode
 package from elpa or el-get as needed
  :mode (("clj\\'" . clojure-mo
​​
de)
​ ;; this parameter delays loading of clojure-mode until ​

​                 ​
("cljs\\'" . clojure-mode)))
​ ;; file with any of these extensions is loaded​


(req-package cider
​​
 ;; this form call will download
​cider
 package from elpa or el-get as needed
  :require (clojure-m
​​
ode eldoc)
​ ;; this parameters makes sure that this req-package form is evaluated
after all of these packages are loaded​

  :commands cider-mode
​ ;; delay loading until cider-mode function is called​

  :init (progn (add-hook 'clojure-mode-h
​​
ook #'cider-mode)
​ ;; start cider in clojure mode​


​                      ​
(add-hook 'cider-mode-hook #'eldoc-mode)
​ ;; start eldoc after cider setups some of its internal things and calls
its hook​


​                      ​
(setq nrepl-log-messages t)))
​ ;; some logging stuff


(req-package-finish)​ ;; start loading

​​
This
​ example may bit a bit hard to understand in concrete, but you may get an
idea.​

On Tue, Aug 11, 2015 at 6:30 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Remember, how often you tackled into problem, when you need to require
> one
> > package, do some configuration, then the same with second and so on.
>
> Indeed, one of the problems I have here is to remember the last time
> I had a problem like the one you describe.
>
> Obviously, we're not using the same set of packages or the same kind
> of configuration.  Can you give some examples you've bumped into?
>
>
>         Stefan
>
>
>

Re: req-package

[Alexander Shukaev]

> Yes, sure. Sorry for late answer. Here is example. I configure clojure
> development environment with req-package(use-package)
>
> (req-package eldoc
> ;; this form call will download eldoc package from elpa or el-get as
> needed
>
>   :commands eldoc
>
>
> -mode
> ;; this parameter delays loading of eldoc until eldo-mode function is
> called
>
>   :init (prog
>
> n (add-h
>
> ook 'emacs-lisp-mode-hook (lambda () (eldoc-mode 1)))
> ;; add eldoc to emacs lisp mode
>
>
>   (add-hook 'lisp-interaction-mode-hook (lambda () (eldoc-mode 1)))))
>
> ;; add eldoc to
> lisp based interaction modes
>  modes
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> (req-package clojure-mode
>
>  ;; this form call will download
> clojure-mode
>  package from elpa or el-get as needed
>   :mode (("clj\\'" . clojure-mo
>
> de)
> ;; this parameter delays loading of clojure-mode until
>
>
> ("cljs\\'" . clojure-mode)))
> ;; file with any of these extensions is loaded
>
>
> (req-package cider
>
>  ;; this form call will download
> cider
>  package from elpa or el-get as needed
>   :require (clojure-m
>
> ode eldoc)
> ;; this parameters makes sure that this req-package form is evaluated
> after all of these packages are loaded
>
>   :commands cider-mode
> ;; delay loading until cider-mode function is called
>
>   :init (progn (add-hook 'clojure-mode-h
>
> ook #'cider-mode)
> ;; start cider in clojure mode
>
>
>
> (add-hook 'cider-mode-hook #'eldoc-mode)
> ;; start eldoc after cider setups some of its internal things and calls
> its hook
>
>
>
> (setq nrepl-log-messages t)))
> ;; some logging stuff
>
>
> (req-package-finish) ;; start loading
>
>
> This
> example may bit a bit hard to understand in concrete, but you may get an
> idea.

Hello Edward,

so, basically all that dance is to simply avoid writing

(with-eval-after-load 'closure-mode
  (add-hook 'clojure-mode-hook #'cider-mode))

(with-eval-after-load 'eldoc
  (add-hook 'cider-mode-hook #'eldoc-mode))

(while still using `use-package')?  Besides, if I understand it
correctly, if `closure-mode', for example, would not be there (and not
downloaded from archives), then the whole configuration will fail?

Re: req-package

[Edward Knyshov]

Basically yes. Also it provides you simplier api, because you do not pushed
to write config in reversed way. You write what depends on what, instead of
what is dependency of what. You can write much more modularised config in
req-package way, because each package is configured with its own
req-package form. Imaging how cumbersome your config will be with
eval-after-load, if 10 package are dependent on clojure-mode and some of
them dependent on helm and some of them on flycheck and so on. I'm not even
sure that it will be easy to support.

Yes, sure, it will fail, because not all dependencies are satisfied. Why
use cider without clojure-mode for example or why to try enable eldoc in
clojure-mode if clojure mode failes to install?

On Thu, Aug 13, 2015 at 4:44 PM Alexander Shukaev <haroogan@gmail.com>
wrote:

> > Yes, sure. Sorry for late answer. Here is example. I configure clojure
> > development environment with req-package(use-package)
> >
> > (req-package eldoc
> > ;; this form call will download eldoc package from elpa or el-get as
> > needed
> >
> >   :commands eldoc
> >
> >
> > -mode
> > ;; this parameter delays loading of eldoc until eldo-mode function is
> > called
> >
> >   :init (prog
> >
> > n (add-h
> >
> > ook 'emacs-lisp-mode-hook (lambda () (eldoc-mode 1)))
> > ;; add eldoc to emacs lisp mode
> >
> >
> >   (add-hook 'lisp-interaction-mode-hook (lambda () (eldoc-mode 1)))))
> >
> > ;; add eldoc to
> > lisp based interaction modes
> >  modes
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > (req-package clojure-mode
> >
> >  ;; this form call will download
> > clojure-mode
> >  package from elpa or el-get as needed
> >   :mode (("clj\\'" . clojure-mo
> >
> > de)
> > ;; this parameter delays loading of clojure-mode until
> >
> >
> > ("cljs\\'" . clojure-mode)))
> > ;; file with any of these extensions is loaded
> >
> >
> > (req-package cider
> >
> >  ;; this form call will download
> > cider
> >  package from elpa or el-get as needed
> >   :require (clojure-m
> >
> > ode eldoc)
> > ;; this parameters makes sure that this req-package form is evaluated
> > after all of these packages are loaded
> >
> >   :commands cider-mode
> > ;; delay loading until cider-mode function is called
> >
> >   :init (progn (add-hook 'clojure-mode-h
> >
> > ook #'cider-mode)
> > ;; start cider in clojure mode
> >
> >
> >
> > (add-hook 'cider-mode-hook #'eldoc-mode)
> > ;; start eldoc after cider setups some of its internal things and calls
> > its hook
> >
> >
> >
> > (setq nrepl-log-messages t)))
> > ;; some logging stuff
> >
> >
> > (req-package-finish) ;; start loading
> >
> >
> > This
> > example may bit a bit hard to understand in concrete, but you may get an
> > idea.
>
> Hello Edward,
>
> so, basically all that dance is to simply avoid writing
>
> (with-eval-after-load 'closure-mode
>   (add-hook 'clojure-mode-hook #'cider-mode))
>
> (with-eval-after-load 'eldoc
>   (add-hook 'cider-mode-hook #'eldoc-mode))
>
> (while still using `use-package')?  Besides, if I understand it
> correctly, if `closure-mode', for example, would not be there (and not
> downloaded from archives), then the whole configuration will fail?
>

Re: req-package

[Alexander Shukaev]

> Yes, sure, it will fail, because not all dependencies are satisfied. Why use
> cider without clojure-mode for example or why to try enable eldoc in
> clojure-mode if clojure mode failes to install?

It's all about the example you used.  Consider something like this:

(use-package recentf
  :defer
  :commands
  (recentf-mode)
  :init
  (recentf-mode)
  :config
  ;; *before*
  (with-eval-after-load 'evil
    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
    (evil-set-initial-state 'recentf-dialog-mode 'motion)
    (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files))
  ;; *after*
  )

Would you agree that in general, `evil' has nothing to do with
`recentf'?  I can have other configurations *before* and *after* the
`evil'-specific one (even involving more `with-eval-after-load' forms
depending on other packages).  As a result, if `evil' is not
present,then still the rest of the configuration of `recentf' will
function properly, and I don't get disturbed by an error message or
half-working configuration due to abortion.

It could still be avoided if one could write a boilerplate like

(req-package recentf
  :defer
  :commands
  (recentf-mode)
  :init
  (recentf-mode)
  :config
  ;; *before*
  )

(req-package recentf
  :defer
  :require (evil)
  :config
  (evil-make-overriding-map recentf-dialog-mode-map 'motion)
  (evil-set-initial-state 'recentf-dialog-mode 'motion)
  (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files))

(req-package recentf
  :defer
  :config
  ;; *after*
  )

Will this work with `req-package'?  To be more specific, I guess you
understand what I mean here: I expect only the middle one to fail if
`evil' is not present, but *before* and *after* should still work.

> Basically yes. Also it provides you simplier api, because you do not pushed
> to write config in reversed way. You write what depends on what, instead of
> what is dependency of what. You can write much more modularised config in
> req-package way, because each package is configured with its own req-package
> form. Imaging how cumbersome your config will be with eval-after-load, if 10
> package are dependent on clojure-mode and some of them dependent on helm and
> some of them on flycheck and so on. I'm not even sure that it will be easy
> to support.

The only problem that I currently have with `use-package' is the
following.  Assume you have packages `a' and `b'.  Sometimes, when
configuring `b' with `use-package', I want to be absolutely sure that
not only `a' is loaded, but also that all the `(use-package a ...)'
forms have already been evaluated before the `(use-package b ...)'.  I
know 2 solutions to this problem.  (1) The dumb one, to arrange all
the `(use-package x ...)' into the corresponding, for example,
`init-x.el' and then of course `(require 'init-x)' in the right order
in some global configuration file (e.g. `init.el').  That is, for the
current example, it would be

(require 'init-a)
(require 'init-b)

Once again, this is dumb, and I don't do this, but rather stick with
the second option.  (2) Arrange all the `(use-package x ...)' into the
corresponding, for example, `init-x.el' and then do
`(with-eval-after-load 'init-x ...)' where necessary.  That is, for
the current example, it could either be

(require 'init-b)
(require 'init-a)

where the order appears to be wrong, but inside `(use-package b ...)'
I would use `(with-eval-after-load 'init-a ...)' which solves the
ordering problem; or

(require 'init-a)
(require 'init-b)

where the order is right, but `(with-eval-after-load 'init-a ...)'
inside `(use-package b ...)' still does no harm.  In other words, with
the approach (2), one does not have to care about the order of
execution of `(use-package x ...)' forms.

Now the question is whether your `req-package' does that automatically
with the `:require' keyword?  In particular, will

(req-package b
  :require a
  ...)

ensure that

(req-package a
  ...)

is evaluated before

(req-package b
  :require a
  ...)

regardless of the order in which they appear in the configuration files?

Re: req-package

[Edward Knyshov]

Some words about your example with evil mode.
You wrote an example.

(use-package recentf
  :defer
  :commands
  (recentf-mode)
  :init
  (recentf-mode)
  :config
  ;; *before*
  (with-eval-after-load 'evil
    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
    (evil-set-initial-state 'recentf-dialog-mode 'motion)
    (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files))
  ;; *after*
  )

It's seems you do not properly get an idea. You should use only one
req-package call for one package. Imaging kind of graph, which you describe
with (req-package package :require dependencies).
Here is how it will be rewritten with req-package

(req-package recentf)

(req-package evil
  :require recentf
  :config (progn  (evil-make-overriding-map recentf-dialog-mode-map 'motion)
                          (evil-set-initial-state 'recentf-dialog-mode
'motion)
                          (evil-ex-define-cmd "rfm[enu]"
#'recentf-open-files)))

That's all. So I even wouldn't call this boilerplate, because it less than
first example.

Re: req-package

[Stefan Monnier]

> Here is how it will be rewritten with req-package
>
> (req-package recentf)
>
> (req-package evil
>   :require recentf
>   :config (progn  (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>                   (evil-set-initial-state 'recentf-dialog-mode 'motion)
>                   (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))

But wouldn't that misbehave if you decide to stop using recentf (in
which case you'd want to just comment out your (req-package recentf)
without having to tweak the `evil' part).


        Stefan

Re: req-package

[Edward Knyshov]

In this case you should remove dependency from evil config.

On Fri, Aug 14, 2015, 12:25 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Here is how it will be rewritten with req-package
> >
> > (req-package recentf)
> >
> > (req-package evil
> >   :require recentf
> >   :config (progn  (evil-make-overriding-map recentf-dialog-mode-map
> 'motion)
> >                   (evil-set-initial-state 'recentf-dialog-mode 'motion)
> >                   (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))
>
> But wouldn't that misbehave if you decide to stop using recentf (in
> which case you'd want to just comment out your (req-package recentf)
> without having to tweak the `evil' part).
>
>
>         Stefan
>

Re: req-package

[Edward Knyshov]

Also in this example you could write only evil form, because recentf does
nothing. Like that.

(req-package evil
  :require recentf
  :config (progn  (evil-make-overriding-map recentf-dialog-mode-map 'motion)
                          (evil-set-initial-state 'recentf-dialog-mode
'motion)
                          (evil-ex-define-cmd "rfm[enu]"
#'recentf-open-files)))

On Fri, Aug 14, 2015 at 12:25 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Here is how it will be rewritten with req-package
> >
> > (req-package recentf)
> >
> > (req-package evil
> >   :require recentf
> >   :config (progn  (evil-make-overriding-map recentf-dialog-mode-map
> 'motion)
> >                   (evil-set-initial-state 'recentf-dialog-mode 'motion)
> >                   (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))
>
> But wouldn't that misbehave if you decide to stop using recentf (in
> which case you'd want to just comment out your (req-package recentf)
> without having to tweak the `evil' part).
>
>
>         Stefan
>

Re: req-package

[Stefan Monnier]

> In this case you should remove dependency from evil config.

But if you remove the ":require recentf" the code will still fail
since (evil-make-overriding-map recentf-dialog-mode-map 'motion) will
signal an error if recentf is not loaded.


        Stefan

Re: req-package

[Alexander Shukaev]

>> In this case you should remove dependency from evil config.
>
> But if you remove the ":require recentf" the code will still fail
> since (evil-make-overriding-map recentf-dialog-mode-map 'motion) will
> signal an error if recentf is not loaded.

Exactly, and that's what I wanted to point out as design flaw, but I
see one good solution which I believe could improve `req-package' even
further.

Why not do the following:

(req-package evil
  :config
  ;; Here we configure only `evil' itself, e.g.:
  (let ((keymap evil-emacs-state-map))
    (setcdr keymap nil)
    (define-key keymap (read-kbd-macro evil-toggle-key)
#'evil-exit-emacs-state))
  (setq-default evil-ex-commands nil)
  (evil-ex-define-cmd "w[rite]" #'evil-write)
  :config recentf
  ;; Here we configure what is related to `recentf' (plus `evil' of course).
  ;; It's kind of `:require recentf' (in a sense that it should also
manage dependencies as
  ;; `:require recentf' does), but it also includes corresponding
configuration as well.  The benefit is
  ;; that if `recentf' is not available, then this code is simply
disabled/ignored (similar to how
  ;; `with-eval-after-load' functions).  This is more general and
flexible approach then `:require'.
  (evil-make-overriding-map recentf-dialog-mode-map 'motion)
  (evil-set-initial-state 'recentf-dialog-mode 'motion)
  (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)
  :config help
  ;; Again the same concept as with `recentf'.
  (let ((keymap evil-emacs-state-map))
    (define-key keymap (kbd "C-?") #'help))
  (evil-ex-define-cmd "h[elp]" #'help)
  :config (my minibuffer)
  ;; Again the same concept as with `recentf', but now two packages
are prerequisites.
  (my-add-hook 'minibuffer-setup-hook #'evil-insert-state))

Furthermore, those `:config' forms should execute exactly in the order
they appear in the code.  Why is it important?  Consider
`(setq-default evil-ex-commands nil)', this erases all of the default
Evil's ex-commands, then we add our own `(evil-ex-define-cmd "w[rite]"
#'evil-write)'.  However, later on, if `recentf' is present, then
`(evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)' should run and
it should definitely run after `(setq-default evil-ex-commands nil)'.
By the way, the same applies to how `evil-emacs-state-map' is managed
(first reset and then gradually repopulated).

To conclude, I think the above approach is next-level evolution step
for `req-package' and possibly can be pull-requested to `use-package'
at some point.  Edward, do you see the potential here to extend
`req-package' accordingly?

Re: req-package

[Edward Knyshov]

It's a good idea Alexander. It sounds good.
I'm not sure about pull requesting to use package, because its author
wanted to keep use package simple, last time I talked to him.
That's why req-package was created. But I will really consider adding this
improvement to req-package.
Could you, please, create an issue at http://github.com/edvorg/req-package ?

Edward.

On Fri, Aug 14, 2015 at 5:40 PM Alexander Shukaev <haroogan@gmail.com>
wrote:

> >> In this case you should remove dependency from evil config.
> >
> > But if you remove the ":require recentf" the code will still fail
> > since (evil-make-overriding-map recentf-dialog-mode-map 'motion) will
> > signal an error if recentf is not loaded.
>
> Exactly, and that's what I wanted to point out as design flaw, but I
> see one good solution which I believe could improve `req-package' even
> further.
>
> Why not do the following:
>
> (req-package evil
>   :config
>   ;; Here we configure only `evil' itself, e.g.:
>   (let ((keymap evil-emacs-state-map))
>     (setcdr keymap nil)
>     (define-key keymap (read-kbd-macro evil-toggle-key)
> #'evil-exit-emacs-state))
>   (setq-default evil-ex-commands nil)
>   (evil-ex-define-cmd "w[rite]" #'evil-write)
>   :config recentf
>   ;; Here we configure what is related to `recentf' (plus `evil' of
> course).
>   ;; It's kind of `:require recentf' (in a sense that it should also
> manage dependencies as
>   ;; `:require recentf' does), but it also includes corresponding
> configuration as well.  The benefit is
>   ;; that if `recentf' is not available, then this code is simply
> disabled/ignored (similar to how
>   ;; `with-eval-after-load' functions).  This is more general and
> flexible approach then `:require'.
>   (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>   (evil-set-initial-state 'recentf-dialog-mode 'motion)
>   (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)
>   :config help
>   ;; Again the same concept as with `recentf'.
>   (let ((keymap evil-emacs-state-map))
>     (define-key keymap (kbd "C-?") #'help))
>   (evil-ex-define-cmd "h[elp]" #'help)
>   :config (my minibuffer)
>   ;; Again the same concept as with `recentf', but now two packages
> are prerequisites.
>   (my-add-hook 'minibuffer-setup-hook #'evil-insert-state))
>
> Furthermore, those `:config' forms should execute exactly in the order
> they appear in the code.  Why is it important?  Consider
> `(setq-default evil-ex-commands nil)', this erases all of the default
> Evil's ex-commands, then we add our own `(evil-ex-define-cmd "w[rite]"
> #'evil-write)'.  However, later on, if `recentf' is present, then
> `(evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)' should run and
> it should definitely run after `(setq-default evil-ex-commands nil)'.
> By the way, the same applies to how `evil-emacs-state-map' is managed
> (first reset and then gradually repopulated).
>
> To conclude, I think the above approach is next-level evolution step
> for `req-package' and possibly can be pull-requested to `use-package'
> at some point.  Edward, do you see the potential here to extend
> `req-package' accordingly?
>

Re: req-package

[Alexander Shukaev]

> It's a good idea Alexander. It sounds good.

Thanks, Edward.

> I'm not sure about pull requesting to use package, because its author wanted
> to keep use package simple, last time I talked to him.

Maybe we could try to involve him into this discussion once again as
your initial feature with dependency/order management and now this one
would absolutely blow away the plain naked `use-package'.  As I said
before, plain `use-package' is just sugar and convenience rather than
a "smart" tool for package management.

> That's why req-package was created. But I will really consider adding this
> improvement to req-package.
> Could you, please, create an issue at http://github.com/edvorg/req-package ?

I'll do that soon.

Re: req-package

[John Wiegley]

>>>>> Alexander Shukaev <haroogan@gmail.com> writes:

>> I'm not sure about pull requesting to use package, because its author
>> wanted to keep use package simple, last time I talked to him.

> Maybe we could try to involve him into this discussion once again as your
> initial feature with dependency/order management and now this one would
> absolutely blow away the plain naked `use-package'. As I said before, plain
> `use-package' is just sugar and convenience rather than a "smart" tool for
> package management.

I believe there are ways we can integrate the development, while allow the
feature work to proceed separately. This is partly why I re-designed
use-package in 2.0: so new functionality would not require maintenance (by me)
in core, since others rely on features that I never use, and hence would never
test.

So I'd love to examine whether req-package and use-package can be integrated,
such that we each work on our respective parts, while users can benefit from
both.

John

Re: req-package

[Alexander Shukaev]

>>> I'm not sure about pull requesting to use package, because its author
>>> wanted to keep use package simple, last time I talked to him.
>
>> Maybe we could try to involve him into this discussion once again as your
>> initial feature with dependency/order management and now this one would
>> absolutely blow away the plain naked `use-package'. As I said before, plain
>> `use-package' is just sugar and convenience rather than a "smart" tool for
>> package management.
>
> I believe there are ways we can integrate the development, while allow the
> feature work to proceed separately. This is partly why I re-designed
> use-package in 2.0: so new functionality would not require maintenance (by me)
> in core, since others rely on features that I never use, and hence would never
> test.

Yes, indeed, I totally understand this rationale.  The 2.0 is very
well redesigned in terms of extensibility; thanks a lot for your
effort.  Still, I believe it would be to the benefit of the majority
to give `req-package' a shot and seek to add it as an optional
functionality to `use-package' while relying on the existing 2.0
framework.

> So I'd love to examine whether req-package and use-package can be integrated,
> such that we each work on our respective parts, while users can benefit from
> both.

That's wonderful!

By the way, the issue [http://github.com/edvorg/req-package/issues/18]
corresponding to this thread has been updated.  Feel free to join the
discussion.

Regards,
Alexander

Re: req-package

[John Wiegley]

>>>>> Alexander Shukaev <haroogan@gmail.com> writes:

> Still, I believe it would be to the benefit of the majority to give
> `req-package' a shot and seek to add it as an optional functionality to
> `use-package' while relying on the existing 2.0 framework.

Yes, this is just what I was suggesting, unless I misunderstood something.
use-package hopes to serve as a foundation, with other, more rich frameworks
(like ELPA integration, dependency resolution, etc) sitting on top of it.

John

Re: req-package

[Les Harris]

I am just a normal user when it comes to use-package but it has
completely changed how I manage configuration for emacs.  I've been
following the recent threads on here keenly.

I just wanted to say that this vision of use-package as a foundational
framework is very exciting from an end-user's standpoint and I look
forward to seeing what results may come.

-- 
Do they only stand
By ignorance, is that their happy state,
The proof of their obedience and their faith?

Re: req-package

[Rusi]

On Sunday, August 16, 2015 at 2:21:41 PM UTC+5:30, Les Harris wrote:
> I am just a normal user when it comes to use-package but it has
> completely changed how I manage configuration for emacs.  I've been
> following the recent threads on here keenly.
> 
> I just wanted to say that this vision of use-package as a foundational
> framework is very exciting from an end-user's standpoint and I look
> forward to seeing what results may come.

Yes
The one thing that is not coming out in these threads or the docs is that 
use-package (req-package??¹) is hi-level declarative/functional whereas the 
builtin methods are low-level imperative.

The closest that the builtin methods are functional is
(require 'feature)
vs the more imperative
(load "feature")

However as soon as one wants to go from there to something a bit more
sophisticated eg add-hook, autoload, eval-after-load etc it all becomes 
mind-numbingly sequence-sensitive.

---------------------------------------
¹ As is stands req-package adds a key functional feature - dependency graph
verification. However from what is currently being discussed here it looks to
be getting more imperative.

Re: req-package

[Edward Knyshov]

Hi.

First thing we need to figure out is does use-package provide flexible
enough api to implement req-package as new use-package keyword.
Currently it works by calling (use-package ...) forms when corresponding
are packages a ready to configure.
We need a full control over packages loading flow, because we need to
rearrange some stuff to get all this dance work.

Edward.

On Sun, Aug 16, 2015 at 12:10 PM Rusi <rustompmody@gmail.com> wrote:

> On Sunday, August 16, 2015 at 2:21:41 PM UTC+5:30, Les Harris wrote:
> > I am just a normal user when it comes to use-package but it has
> > completely changed how I manage configuration for emacs.  I've been
> > following the recent threads on here keenly.
> >
> > I just wanted to say that this vision of use-package as a foundational
> > framework is very exciting from an end-user's standpoint and I look
> > forward to seeing what results may come.
>
> Yes
> The one thing that is not coming out in these threads or the docs is that
> use-package (req-package??¹) is hi-level declarative/functional whereas the
> builtin methods are low-level imperative.
>
> The closest that the builtin methods are functional is
> (require 'feature)
> vs the more imperative
> (load "feature")
>
> However as soon as one wants to go from there to something a bit more
> sophisticated eg add-hook, autoload, eval-after-load etc it all becomes
> mind-numbingly sequence-sensitive.
>
> ---------------------------------------
> ¹ As is stands req-package adds a key functional feature - dependency graph
> verification. However from what is currently being discussed here it looks
> to
> be getting more imperative.
>

Re: req-package

[John Wiegley]

>>>>> Edward Knyshov <edvorg@gmail.com> writes:

> First thing we need to figure out is does use-package provide flexible
> enough api to implement req-package as new use-package keyword.
> Currently it works by calling (use-package ...) forms when corresponding
> are packages a ready to configure.
>
> We need a full control over packages loading flow, because we need to
> rearrange some stuff to get all this dance work.

So, use-package just expands a macro into code, and you can replace every
single keyword handler if you wish.  So given that the use-package macro could
just turn into a req-package invocation, you should have complete freedom to
do whatever it is that you need.

John

Re: req-package

[Rusi]

On Sunday, August 16, 2015 at 3:08:18 PM UTC+5:30, Edward Knyshov wrote:
> On Sun, Aug 16, 2015 at 12:10 PM Rusi  wrote:
> 
> > On Sunday, August 16, 2015 at 2:21:41 PM UTC+5:30, Les Harris wrote:
> > > I am just a normal user when it comes to use-package but it has
> > > completely changed how I manage configuration for emacs.  I've been
> > > following the recent threads on here keenly.
> > >
> > > I just wanted to say that this vision of use-package as a foundational
> > > framework is very exciting from an end-user's standpoint and I look
> > > forward to seeing what results may come.
> >
> > Yes
> > The one thing that is not coming out in these threads or the docs is that
> > use-package (req-package??¹) is hi-level declarative/functional whereas the
> > builtin methods are low-level imperative.
> >
> > The closest that the builtin methods are functional is
> > (require 'feature)
> > vs the more imperative
> > (load "feature")
> >
> > However as soon as one wants to go from there to something a bit more
> > sophisticated eg add-hook, autoload, eval-after-load etc it all becomes
> > mind-numbingly sequence-sensitive.
> >
> > ---------------------------------------
> > ¹ As is stands req-package adds a key functional feature - dependency graph
> > verification. However from what is currently being discussed here it looks
> > to
> > be getting more imperative.
> >

> Hi.
> 
> First thing we need to figure out is does use-package provide flexible
> enough api to implement req-package as new use-package keyword.
> Currently it works by calling (use-package ...) forms when corresponding
> are packages a ready to configure.
> We need a full control over packages loading flow, because we need to
> rearrange some stuff to get all this dance work.

Err... I need to change my hat :-)
As an ordinary programmer, functional is generally better than imperative.

However with implementer hat on I know that the more fancy-ly declarative
a language is the more complex the internals are going to be.
After all its a conservation law -- someone somewhere needs to think the
messy parts.  Just better to do it once-and-for-all and to forget thereafter.
So thanks you guys -- you and John -- for doing the messy work to help
us keep our inits short, sequence-tolerant logically organized and zippy.

And just to remind, I jumped into these threads because I would like to hand
out to my students, init-fragments that are not so long arcane and messy as they
currently have to be. [Anyways they think I am nuts for using emacs...
So not too many available degrees of freedom :-) 
]

Re: req-package

[Edward Knyshov]

Unfortunately we need to write imperative code sometimes because we need to
increase performance and reduce memory usage.
First version of req-package has been implemented in completely functional
style.
After my configuration grown to use near 150 packages, things gone wrong
and I started to tackle into recursion limits of emacs lisp.
Also performance was not that high as now. What is left for me is to keep
req-package api functional while providing good performance.

On Sun, Aug 16, 2015 at 1:05 PM Rusi <rustompmody@gmail.com> wrote:

> On Sunday, August 16, 2015 at 3:08:18 PM UTC+5:30, Edward Knyshov wrote:
> > On Sun, Aug 16, 2015 at 12:10 PM Rusi  wrote:
> >
> > > On Sunday, August 16, 2015 at 2:21:41 PM UTC+5:30, Les Harris wrote:
> > > > I am just a normal user when it comes to use-package but it has
> > > > completely changed how I manage configuration for emacs.  I've been
> > > > following the recent threads on here keenly.
> > > >
> > > > I just wanted to say that this vision of use-package as a
> foundational
> > > > framework is very exciting from an end-user's standpoint and I look
> > > > forward to seeing what results may come.
> > >
> > > Yes
> > > The one thing that is not coming out in these threads or the docs is
> that
> > > use-package (req-package??¹) is hi-level declarative/functional
> whereas the
> > > builtin methods are low-level imperative.
> > >
> > > The closest that the builtin methods are functional is
> > > (require 'feature)
> > > vs the more imperative
> > > (load "feature")
> > >
> > > However as soon as one wants to go from there to something a bit more
> > > sophisticated eg add-hook, autoload, eval-after-load etc it all becomes
> > > mind-numbingly sequence-sensitive.
> > >
> > > ---------------------------------------
> > > ¹ As is stands req-package adds a key functional feature - dependency
> graph
> > > verification. However from what is currently being discussed here it
> looks
> > > to
> > > be getting more imperative.
> > >
>
> > Hi.
> >
> > First thing we need to figure out is does use-package provide flexible
> > enough api to implement req-package as new use-package keyword.
> > Currently it works by calling (use-package ...) forms when corresponding
> > are packages a ready to configure.
> > We need a full control over packages loading flow, because we need to
> > rearrange some stuff to get all this dance work.
>
> Err... I need to change my hat :-)
> As an ordinary programmer, functional is generally better than imperative.
>
> However with implementer hat on I know that the more fancy-ly declarative
> a language is the more complex the internals are going to be.
> After all its a conservation law -- someone somewhere needs to think the
> messy parts.  Just better to do it once-and-for-all and to forget
> thereafter.
> So thanks you guys -- you and John -- for doing the messy work to help
> us keep our inits short, sequence-tolerant logically organized and zippy.
>
> And just to remind, I jumped into these threads because I would like to
> hand
> out to my students, init-fragments that are not so long arcane and messy
> as they
> currently have to be. [Anyways they think I am nuts for using emacs...
> So not too many available degrees of freedom :-)
> ]
>

Re: req-package

[Rusi]

On Sunday, August 16, 2015 at 3:32:23 PM UTC+5:30, Rusi wrote:
> On Sunday, August 16, 2015 at 3:08:18 PM UTC+5:30, Edward Knyshov wrote:
> > Hi.
> > 
> > First thing we need to figure out is does use-package provide flexible
> > enough api to implement req-package as new use-package keyword.
> > Currently it works by calling (use-package ...) forms when corresponding
> > are packages a ready to configure.
> > We need a full control over packages loading flow, because we need to
> > rearrange some stuff to get all this dance work.
> 
> Err... I need to change my hat :-)
> As an ordinary programmer, functional is generally better than imperative.
> 
> However with implementer hat on I know that the more fancy-ly declarative
> a language is the more complex the internals are going to be.
> After all its a conservation law -- someone somewhere needs to think the
> messy parts.  Just better to do it once-and-for-all and to forget thereafter.
> So thanks you guys -- you and John -- for doing the messy work to help
> us keep our inits short, sequence-tolerant logically organized and zippy.
> 
> And just to remind, I jumped into these threads because I would like to hand
> out to my students, init-fragments that are not so long arcane and messy as they
> currently have to be. [Anyways they think I am nuts for using emacs...
> So not too many available degrees of freedom :-) 
> ]

And one more thing.
I tried to say it in the 'recursion' thread... Maybe it was too long.
So let me try to repeat it in short:

Do consider a feature request for a batch-mode for req-package 
Ie.
- In batch-mode it should do the network downloads etc, verify dependencies etc
- In normal mode it should do imports as lazily as possible, bind keys etc --
the use-package functionality -- assuming packages are in place.
ie no elpa stuff here

IOW requesting some help for us users to partition this frightening list
(info "(elisp)Startup Summary") 
into manageable sections

Re: req-package

[Edward Knyshov]

Rusi, if I correctly understood your request, I think you already have this
functionality in req-package.
Req-package will do network downloads only in cases when package is now
available offline on your loadpath,
so it wil run correctly in offline, after you run it once with network
connection.
One thing I don't understand is why you need to avoid dependency management
in normal mode?
It runs offline and your config will be completely non-working if we start
to run without dependency related reordering.

Edward.

On Sun, Aug 16, 2015 at 1:20 PM Rusi <rustompmody@gmail.com> wrote:

> On Sunday, August 16, 2015 at 3:32:23 PM UTC+5:30, Rusi wrote:
> > On Sunday, August 16, 2015 at 3:08:18 PM UTC+5:30, Edward Knyshov wrote:
> > > Hi.
> > >
> > > First thing we need to figure out is does use-package provide flexible
> > > enough api to implement req-package as new use-package keyword.
> > > Currently it works by calling (use-package ...) forms when
> corresponding
> > > are packages a ready to configure.
> > > We need a full control over packages loading flow, because we need to
> > > rearrange some stuff to get all this dance work.
> >
> > Err... I need to change my hat :-)
> > As an ordinary programmer, functional is generally better than
> imperative.
> >
> > However with implementer hat on I know that the more fancy-ly declarative
> > a language is the more complex the internals are going to be.
> > After all its a conservation law -- someone somewhere needs to think the
> > messy parts.  Just better to do it once-and-for-all and to forget
> thereafter.
> > So thanks you guys -- you and John -- for doing the messy work to help
> > us keep our inits short, sequence-tolerant logically organized and zippy.
> >
> > And just to remind, I jumped into these threads because I would like to
> hand
> > out to my students, init-fragments that are not so long arcane and messy
> as they
> > currently have to be. [Anyways they think I am nuts for using emacs...
> > So not too many available degrees of freedom :-)
> > ]
>
> And one more thing.
> I tried to say it in the 'recursion' thread... Maybe it was too long.
> So let me try to repeat it in short:
>
> Do consider a feature request for a batch-mode for req-package
> Ie.
> - In batch-mode it should do the network downloads etc, verify
> dependencies etc
> - In normal mode it should do imports as lazily as possible, bind keys etc
> --
> the use-package functionality -- assuming packages are in place.
> ie no elpa stuff here
>
> IOW requesting some help for us users to partition this frightening list
> (info "(elisp)Startup Summary")
> into manageable sections
>

Re: req-package

[Rusi]

On Sunday, August 16, 2015 at 4:03:05 PM UTC+5:30, Edward Knyshov wrote:
> Rusi, if I correctly understood your request, I think you already have this
> functionality in req-package.
> Req-package will do network downloads only in cases when package is now
> available offline on your loadpath,
> so it wil run correctly in offline, after you run it once with network
> connection.
> One thing I don't understand is why you need to avoid dependency management
> in normal mode?
> It runs offline and your config will be completely non-working if we start
> to run without dependency related reordering.
> 
> Edward.

Ok I guess I am confusing something or other
I tried your emacsconfig from github.
First time it took some 15 minutes to download all the packages (near 60M).
Ok... One time downloads are fine

But thereafter I remember seeing network like calls every time I start emacs
Switched back to my init
Now I again switch to yours and I dont see them...
So as I said must be confusing something...

So in general is that if we switch to req-package, all that was originally
going into init.el now has to go into the init-real which is on the
after-init-hook?

Then what is run-emacs.el for?
Its all rather confusing...

Re: req-package

[Edward Knyshov]

Yep, it's a lot of packages under the hood of my set up.
This networking calls you seen is related to my config only, not to
req-package.
I have some code, which randomly select new theme from my favourite ones.
Each time you started emacs, new theme was downloaded.

Also configuration located in init-real is only my own thing you don't need
to do.
I was trying to fix some stuff I even don't remember which.
I got this advice at http://emacswiki.org/emacs/ELPA.

Edward

On Sun, Aug 16, 2015 at 4:10 PM Rusi <rustompmody@gmail.com> wrote:

> On Sunday, August 16, 2015 at 4:03:05 PM UTC+5:30, Edward Knyshov wrote:
> > Rusi, if I correctly understood your request, I think you already have
> this
> > functionality in req-package.
> > Req-package will do network downloads only in cases when package is now
> > available offline on your loadpath,
> > so it wil run correctly in offline, after you run it once with network
> > connection.
> > One thing I don't understand is why you need to avoid dependency
> management
> > in normal mode?
> > It runs offline and your config will be completely non-working if we
> start
> > to run without dependency related reordering.
> >
> > Edward.
>
> Ok I guess I am confusing something or other
> I tried your emacsconfig from github.
> First time it took some 15 minutes to download all the packages (near 60M).
> Ok... One time downloads are fine
>
> But thereafter I remember seeing network like calls every time I start
> emacs
> Switched back to my init
> Now I again switch to yours and I dont see them...
> So as I said must be confusing something...
>
> So in general is that if we switch to req-package, all that was originally
> going into init.el now has to go into the init-real which is on the
> after-init-hook?
>
> Then what is run-emacs.el for?
> Its all rather confusing...
>

Re: req-package

[Rusi]

On Sunday, August 16, 2015 at 6:58:38 PM UTC+5:30, Edward Knyshov wrote:
> Yep, it's a lot of packages under the hood of my set up.
> This networking calls you seen is related to my config only, not to
> req-package.
> I have some code, which randomly select new theme from my favourite ones.
> Each time you started emacs, new theme was downloaded.
> 
> Also configuration located in init-real is only my own thing you don't need
> to do.
> I was trying to fix some stuff I even don't remember which.
> I got this advice at http://emacswiki.org/emacs/ELPA.

In which case we are back to where we (I) started with req-package
-- my bug-report https://github.com/edvorg/req-package/issues/16
  which refers to the earlier issue that requests recipes for using req-package:
-- https://github.com/edvorg/req-package/issues/14

JFTR I reached req-package looking for recipes for using use-package
And I am fairly certain many hundreds of users old and new will migrate
away from the clunky eval-after-load... add-hook... mess-mode as soon as we are able

Re: req-package

[Stefan Monnier]

>   :config recentf
>   ;; Here we configure what is related to `recentf' (plus `evil' of course).
>   ;; It's kind of `:require recentf' (in a sense that it should also
> manage dependencies as
>   ;; `:require recentf' does), but it also includes corresponding
> configuration as well.  The benefit is
>   ;; that if `recentf' is not available, then this code is simply
> disabled/ignored (similar to how
>   ;; `with-eval-after-load' functions).  This is more general and
> flexible approach then `:require'.

I think this should not try to require recentf.  IOW it should only
execute that code if/when recentf if enabled.  So you can still disable
all of recentf by commenting out the (req-package recentf).


        Stefan

Re: req-package

[Alexander Shukaev]

>>   :config recentf
>>   ;; Here we configure what is related to `recentf' (plus `evil' of course).
>>   ;; It's kind of `:require recentf' (in a sense that it should also
>> manage dependencies as
>>   ;; `:require recentf' does), but it also includes corresponding
>> configuration as well.  The benefit is
>>   ;; that if `recentf' is not available, then this code is simply
>> disabled/ignored (similar to how
>>   ;; `with-eval-after-load' functions).  This is more general and
>> flexible approach then `:require'.
>
> I think this should not try to require recentf.  IOW it should only
> execute that code if/when recentf if enabled.  So you can still disable
> all of recentf by commenting out the (req-package recentf).

True.  It should behave similarly to the default `:config' which
simply translates to `eval-after-load' without `require'-ing anything.
BUT it should still build the graph in the similar way as `:require'
does in order to force evaluation of `(req-package recentf ...)'
BEFORE the current one (`(req-package evil ...)' in this case), so
that the default `:config' from `(req-package recentf ...)' is
definitely executed BEFORE the `:config recentf' from `(req-package
evil ...)'.  Note that the code under either default `:config' from
`(req-package recentf ...)' or from `:config recentf' from
`(req-package evil ...)' will really execute only when `recentf' is
really loaded (by autoload, manually, or whatever) thanks to
`eval-after-load' behind `:config'.  By the way, `(req-package recentf
...)' on its own without `:demand' (and that's the encouraged practice
to avoid `:demand', which is really `(require 'recentf)') does not
load `recentf'; and therefore, does not have to be touched at all even
if one does not want to use `recentf' anymore (i.e. you don't have to
comment it).

Re: req-package

[Stefan Monnier]

> (with-eval-after-load 'closure-mode
>   (add-hook 'clojure-mode-hook #'cider-mode))

> (with-eval-after-load 'eldoc
>   (add-hook 'cider-mode-hook #'eldoc-mode))

Why would you need to wrap those in with-eval-after-load?


        Stefan

Re: req-package

[Alexander Shukaev]

>> (with-eval-after-load 'closure-mode
>>   (add-hook 'clojure-mode-hook #'cider-mode))
>
>> (with-eval-after-load 'eldoc
>>   (add-hook 'cider-mode-hook #'eldoc-mode))
>
> Why would you need to wrap those in with-eval-after-load?

It's true that you don't have to.  This is just a bad example, Stefan.
Nevertheless, the rationale might still be available.  Why should I
define the `clojure-mode-hook' variable to populate it with
`cinder-mode', while the corresponding `clojure-mode' is never used
and is not even installed.

With `eldoc' it's clear.  Why should `cider-mode' call `eldoc-mode'
when `eldoc.el' is not even installed on one's machine?

Re: req-package

[Stefan Monnier]

> It's true that you don't have to.  This is just a bad example, Stefan.

That's my point: I'm having trouble finding good examples.


        Stefan

Re: req-package

[Alexander Shukaev]

>> It's true that you don't have to.  This is just a bad example, Stefan.
>
> That's my point: I'm having trouble finding good examples.

Consider my previous one:

(use-package recentf
  :defer
  :commands
  (recentf-mode)
  :config
  (with-eval-after-load 'evil
    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
    (evil-set-initial-state 'recentf-dialog-mode 'motion)
    (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))

Re: req-package

[Stefan Monnier]

> Consider my previous one:
> (use-package recentf
>   :defer
>   :commands
>   (recentf-mode)
>   :config
>   (with-eval-after-load 'evil
>     (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>     (evil-set-initial-state 'recentf-dialog-mode 'motion)
>     (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))

Can you explain what' are the issues here?

I guess one issue is with

   (evil-make-overriding-map recentf-dialog-mode-map 'motion)

since this can only be used after both recentf and evil are loaded.
But I don't see where use-package req-package helps you with this
problem (which you instead solve with with-eval-after-load, IIUC).

Can you clarify?


        Stefan

Re: req-package

[Alexander Shukaev]

>> Consider my previous one:
>> (use-package recentf
>>   :defer
>>   :commands
>>   (recentf-mode)
>>   :config
>>   (with-eval-after-load 'evil
>>     (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>>     (evil-set-initial-state 'recentf-dialog-mode 'motion)
>>     (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))
>
> Can you explain what' are the issues here?
>
> I guess one issue is with
>
>    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>
> since this can only be used after both recentf and evil are loaded.
> But I don't see where use-package req-package helps you with this
> problem (which you instead solve with with-eval-after-load, IIUC).
>
> Can you clarify?

Right now, `use-package' is nothing more, but a convenient way to
structure configurations of packages, especially in terms of autoloads
and deferring. Yes, right now it's more cosmetics and sugar, rather
than something that one cannot live without.  Still, I think there
were plenty of good examples in another thread (with the `:mode'
keyword, for instance).

In this thread, however, we want to go deeper and discuss package
"configuration-time" dependencies (which are not covered by
`use-package' by default), and `req-package' offers some features in
this regard, though I still don't quite understand them
comprehensively.  The biggest issue, that is not solved anywhere
(neither vanilla Emacs nor `use-package' provide facilities for this)
out of the box, is the one that I described two posts earlier with the
example on `a' and `b'.  Did you understand the issue?  If
`req-package' solves it, then this might be a good step forward in
evolving `use-package' and making it a "really" handy tool for
managing complex configurations.

Re: req-package

[Edward Knyshov]

Alexander, the answer to your a-b question is yes, req-package manages it.
That's the main idea. You do not need to rearrange your code.

On Thu, Aug 13, 2015 at 7:37 PM Alexander Shukaev <haroogan@gmail.com>
wrote:

> >> Consider my previous one:
> >> (use-package recentf
> >>   :defer
> >>   :commands
> >>   (recentf-mode)
> >>   :config
> >>   (with-eval-after-load 'evil
> >>     (evil-make-overriding-map recentf-dialog-mode-map 'motion)
> >>     (evil-set-initial-state 'recentf-dialog-mode 'motion)
> >>     (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))
> >
> > Can you explain what' are the issues here?
> >
> > I guess one issue is with
> >
> >    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
> >
> > since this can only be used after both recentf and evil are loaded.
> > But I don't see where use-package req-package helps you with this
> > problem (which you instead solve with with-eval-after-load, IIUC).
> >
> > Can you clarify?
>
> Right now, `use-package' is nothing more, but a convenient way to
> structure configurations of packages, especially in terms of autoloads
> and deferring. Yes, right now it's more cosmetics and sugar, rather
> than something that one cannot live without.  Still, I think there
> were plenty of good examples in another thread (with the `:mode'
> keyword, for instance).
>
> In this thread, however, we want to go deeper and discuss package
> "configuration-time" dependencies (which are not covered by
> `use-package' by default), and `req-package' offers some features in
> this regard, though I still don't quite understand them
> comprehensively.  The biggest issue, that is not solved anywhere
> (neither vanilla Emacs nor `use-package' provide facilities for this)
> out of the box, is the one that I described two posts earlier with the
> example on `a' and `b'.  Did you understand the issue?  If
> `req-package' solves it, then this might be a good step forward in
> evolving `use-package' and making it a "really" handy tool for
> managing complex configurations.
>
>

Re: req-package

[Edward Knyshov]

I have a very complex configuration with nearly 300 packages. I tackled
into reordering hell when it was about 100 packages. And now my
configuration grown three times bigger, but still works perfectly with ease
of supporting and adding new features.

On Thu, Aug 13, 2015 at 11:27 PM Edward Knyshov <edvorg@gmail.com> wrote:

> Alexander, the answer to your a-b question is yes, req-package manages it.
> That's the main idea. You do not need to rearrange your code.
>
> On Thu, Aug 13, 2015 at 7:37 PM Alexander Shukaev <haroogan@gmail.com>
> wrote:
>
>> >> Consider my previous one:
>> >> (use-package recentf
>> >>   :defer
>> >>   :commands
>> >>   (recentf-mode)
>> >>   :config
>> >>   (with-eval-after-load 'evil
>> >>     (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>> >>     (evil-set-initial-state 'recentf-dialog-mode 'motion)
>> >>     (evil-ex-define-cmd "rfm[enu]" #'recentf-open-files)))
>> >
>> > Can you explain what' are the issues here?
>> >
>> > I guess one issue is with
>> >
>> >    (evil-make-overriding-map recentf-dialog-mode-map 'motion)
>> >
>> > since this can only be used after both recentf and evil are loaded.
>> > But I don't see where use-package req-package helps you with this
>> > problem (which you instead solve with with-eval-after-load, IIUC).
>> >
>> > Can you clarify?
>>
>> Right now, `use-package' is nothing more, but a convenient way to
>> structure configurations of packages, especially in terms of autoloads
>> and deferring. Yes, right now it's more cosmetics and sugar, rather
>> than something that one cannot live without.  Still, I think there
>> were plenty of good examples in another thread (with the `:mode'
>> keyword, for instance).
>>
>> In this thread, however, we want to go deeper and discuss package
>> "configuration-time" dependencies (which are not covered by
>> `use-package' by default), and `req-package' offers some features in
>> this regard, though I still don't quite understand them
>> comprehensively.  The biggest issue, that is not solved anywhere
>> (neither vanilla Emacs nor `use-package' provide facilities for this)
>> out of the box, is the one that I described two posts earlier with the
>> example on `a' and `b'.  Did you understand the issue?  If
>> `req-package' solves it, then this might be a good step forward in
>> evolving `use-package' and making it a "really" handy tool for
>> managing complex configurations.
>>
>>

Re: req-package

[Stefan Monnier]

> Right now, `use-package' is nothing more, but a convenient way to
> structure configurations of packages, especially in terms of autoloads
> and deferring. Yes, right now it's more cosmetics and sugar, rather
> than something that one cannot live without.

Sure.  The autoloads part is mostly redundant for ELPA packages, but the
structuring part is indeed nice.  And convenience is something we all
appreciate.

> In this thread, however, we want to go deeper and discuss package
> "configuration-time" dependencies (which are not covered by
> `use-package' by default),

Exactly.

> The biggest issue, that is not solved anywhere (neither vanilla Emacs
> nor `use-package' provide facilities for this) out of the box, is the
> one that I described two posts earlier with the example on `a' and
> `b'.  Did you understand the issue?

I must have missed it.
[...going back in the thread...]
OK, found it.  But no, I don't understand the issue.  More specifically,
I understand your hypothetical example and the problem it introduces,
but I'm having a hard time imagining when that happens in practice.

We have plenty of concrete problems that I prefer not to worry too much
about the hypothetical ones (except the ones I like, of course ;-)
[ This said, sometimes the hypothetical problems are helpful to see the
  problem in its full generality and thus find a better solution.  ]

More specifically, Elisp being Turing complete (and side-effectingly
dirty and all), I know for a fact that I will always be able to come up
with some example where any neat solution won't be able to express the
specific configuration I need.

** Random thought 1

BTW, part of the problem with a solution like use-package is also that
it hides what it does.  And in some cases you need to know how it does
its job in order to be able to tweak it correctly.
E.g. depending on how the ":config" thingy (which IIUC runs after the
package is loaded) is implemented, a (with-eval-after-load <pkg> <code>)
might end up running either before or after use-package's :config code.

So, we have the usual problem (recently mentioned in the context of
ELPA), that when it works, it's great, but when it doesn't it makes
things harder because you have to look under the cover to see what happens.

** Random thought 2

The design of something like use-package necessary has to start with
the design of a convention that is simple enough to be attractive yet
powerful enough to cover the vast majority of existing cases, *and*
where most of the remaining cases can be accommodated by the package
author (i.e. the packages need to be tweaked to make configuration
easier).

To take the example of Customize, the design is something like:

   We only handle global variables that are only modified via Customize

This covers many cases, but far from all.  And it's inconvenient to
accommodate this limitation by adding more global variables
(e.g. splitting `foobar-hook' into two, one that is fully under the
control of Customize and another that can be modified by Elisp packages;
or adding `foo-bar-enable' to control whether the `bar' major mode
should enable the `foo' minor mode).


        Stefan