emacs packages

emacs packages

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

Hi,

I would like to know what people think about having an
'emacs-build-system' to build packages imported from an ELPA
repository.  Note that the package archives provided by such a
repository are slightly different from the '.tar.gz' files that you
download from the "real" project home.  One difference is that the
former include final ".info" and "dir" files. It is likely that there
are more differences that I didn't notice due to my limited knowledge
about Emacs packages.

To be concrete about what I'm proposing, attached you find:

- An ELPA package importer
- An 'emacs-build-system'

There are a lot of Emacs packages. For this reason I would like to
propose to prefix them with 'emacs-' as we do with Python, ...

I'm not sure if there is a way to run some tests.  So, currently there
isn't a check phase.

To make those packages automatically available in Emacs without the
need for any code in the user '.emacs' file, I would suggest to
include in our Emacs package site initialization file some custom code
(to activate our ELPA emacs packages) such as the one in the attached
'guix-elpa.el' file.

If the feedback is positive, I will write documentation and some
tests; and send proper patches.

Regards,
Fede

[-- Attachment #2: elpa.scm --]
[-- Type: text/x-scheme, Size: 8286 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix import elpa)
  #:use-module (ice-9 match)
  #:use-module (ice-9 rdelim)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-9)
  #:use-module (srfi srfi-9 gnu)
  #:use-module (srfi srfi-11)
  #:use-module ((guix download) #:select (download-to-store))
  #:use-module (guix import utils)
  #:use-module (guix store)
  #:use-module (guix hash)
  #:use-module (guix base32)
  #:use-module ((guix utils) #:select (call-with-temporary-output-file
                                       memoize))
  #:export (elpa->guix-package))

(define (elpa-dependencies->names deps)
  "Convert the list of dependencies from the ELPA format, to a list of string
names."
  (map (lambda (d) (symbol->string (first d))) deps))

(define (filter-dependencies names)
  "Remove the package names included with Emacs from the list of
NAMES (strings)."
  (let ((emacs-standard-libraries
         '("emacs" "cl-lib")))
    (filter (lambda (d) (not (member d emacs-standard-libraries)))
            names)))

(define (elpa-name->package-name name)
  "Given the NAME of an Emacs package, return the corresponding Guix name."
  (let ((package-name-prefix "emacs-"))
    (if (string-prefix? package-name-prefix name)
        (string-downcase name)
        (string-append package-name-prefix (string-downcase name)))))

(define* (elpa-url #:optional (repo "gnu"))
  "Retrun the URL of REPO."
  (let ((elpa-archives
         '(("gnu" . "http://elpa.gnu.org/packages")
           ("melpa-stable" . "http://stable.melpa.org/packages")
           ("melpa" . "http://melpa.org/packages"))))
    (assoc-ref elpa-archives repo)))

(define* (elpa-fetch-archive #:optional (repo "gnu"))
  "Retrive the archive with the list of packages available from REPO."
  (let ((url (string-append (elpa-url repo) "/archive-contents")))
    (fetch-and-call-with-input-file url read)))

;; Fetch URL, store the content in a temporary file and call PROC with that
;; file.
(define fetch-and-call-with-input-file
  (memoize
   (lambda* (url proc #:optional (err-msg "unavailable"))
     (call-with-temporary-output-file
      (lambda (temp port)
        (or (and (url-fetch url temp)
                 (call-with-input-file temp proc))
            err-msg))))))

(define* (elpa-package-info name #:optional (repo "gnu"))
  "Extract the information about the package NAME from the package archieve of
REPO."
  (let* ((archive (elpa-fetch-archive repo))
         (info (filter (lambda (p) (eq? (first p) (string->symbol name)))
                       (cdr archive))))
    (if (pair? info) (first info) #f)))

;; Object to store information about an ELPA package.
(define-record-type <elpa-package>
  (make-elpa-package name version inputs synopsis kind home-page description
                     source-url)
  elpa-package?
  (name elpa-package-name)
  (version elpa-package-version)
  (inputs elpa-package-inputs)
  (synopsis elpa-package-synopsis)
  (kind elpa-package-kind)
  (home-page elpa-package-home-page)
  (description elpa-package-description)
  (source-url elpa-package-source-url))

(set-record-type-printer! <elpa-package>
                          (lambda (package port)
                            (format port "#<elpa-package ~a-~a>"
                                      (elpa-package-name package)
                                      (elpa-package-version package))))

(define (elpa-version->string elpa-version)
  "Convert the package version as used in Emacs package files into a string."
  (if (pair? elpa-version)
      (let-values (((ms rest) (match elpa-version
                                ((ms . rest)
                                 (values ms rest)))))
        (fold (lambda (n s) (string-append s "." (number->string n)))
              (number->string ms) rest))
      #f))

(define (lookup-extra alist key)
  "Lookup KEY from the ALIST extra package information."
  (assq-ref alist key))

(define (package-home-page alist)
  "Extract the package home-page from ALIST."
  (or (lookup-extra alist ':url) "unspecified"))

(define (nil->empty alist)
  "If ALIST is the symbol 'nil return the empty list.  Otherwise, return ALIST."
  (if (eq? alist 'nil)
      '()
      alist))

(define (package-source-url kind name version repo)
  "Return the source URL of the package described the the strings NAME and
VERSION at REPO.  KIND is either the symbol 'single or 'tar."
  (case kind
    ((single) (full-url repo name ".el" version))
    ((tar) (full-url repo name ".tar" version))
    (else
     #f)))

(define* (full-url repo name suffix #:optional (version #f))
  "Return the full URL of the package NAME at REPO and the SUFFIX.  Maybe
include VERSION."
  (if version
      (string-append (elpa-url repo) "/" name "-" version suffix)
      (string-append (elpa-url repo) "/" name suffix)))

(define (fetch-package-description kind name repo)
  "Fetch the description of package NAME of type KIND from REPO."
  (let ((url (full-url repo name "-readme.txt")))
    (fetch-and-call-with-input-file url read-string)))

(define* (fetch-elpa-package name #:optional (repo "gnu"))
  "Fetch package NAME from REPO."
  (let ((pkg (elpa-package-info name repo)))
    (match pkg
      ((name version reqs synopsis kind . rest)
       (let* ((name (symbol->string name))
             (ver (elpa-version->string version))
             (url (package-source-url kind name ver repo)))
         (make-elpa-package name ver
                            (nil->empty reqs) synopsis kind
                            (package-home-page (first rest))
                            (fetch-package-description kind name repo)
                            url)))
      (_ #f))))

(define* (elpa-package->sexp pkg)
  "Return the `package' S-expression for the Emacs package PKG, a record of
type '<elpa-package>'."
  
  (define name (elpa-package-name pkg))
  
  (define version (elpa-package-version pkg))
  
  (define source-url (elpa-package-source-url pkg))
  
  (define dependencies
    (let* ((deps (elpa-package-inputs pkg))
           (names (filter-dependencies (elpa-dependencies->names deps))))
      (map (lambda (n)
             (let ((new-n (elpa-name->package-name n)))
               (list new-n (list 'unquote (string->symbol new-n)))))
           names)))
      
  (define (maybe-inputs input-type inputs)
    (match inputs
      (()
       '())
      ((inputs ...)
       (list (list input-type
                   (list 'quasiquote inputs))))))
  
  (let ((tarball (with-store store
                   (download-to-store store source-url))))
    `(package
       (name ,(elpa-name->package-name name))
       (version ,version)
       (source (origin
                 (method url-fetch)
                 (uri (string-append ,@(factorize-uri source-url version)))
                 (sha256
                  (base32
                   ,(if tarball
                        (bytevector->nix-base32-string (file-sha256 tarball))
                        "failed to download package")))))
       (build-system emacs-build-system)
       ,@(maybe-inputs 'inputs dependencies)
       (home-page ,(elpa-package-home-page pkg))
       (synopsis ,(elpa-package-synopsis pkg))
       (description ,(elpa-package-description pkg))
       (license license:gpl3+))))

(define* (elpa->guix-package name #:optional (repo "gnu"))
  "Fetch the package NAME from REPO and produce a Guix package S-expression."
  (let ((pkg (fetch-elpa-package name repo)))
    (and=> pkg elpa-package->sexp)))

;;; elpa.scm ends here

[-- Attachment #3: elpa.scm --]
[-- Type: text/x-scheme, Size: 3206 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix scripts import elpa)
  #:use-module (guix ui)
  #:use-module (guix utils)
  #:use-module (guix import elpa)
  #:use-module (guix scripts import)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-11)
  #:use-module (srfi srfi-37)
  #:use-module (ice-9 match)
  #:use-module (ice-9 format)
  #:export (guix-import-elpa))

\f
;;;
;;; Command-line options.
;;;

(define %default-options
  '((repo . "gnu")))

(define (show-help)
  (display (_ "Usage: guix import elpa PACKAGE-NAME
Import the latest package named PACKAGE-NAME from an ELPA repository.\n"))
  (display (_ "
  -a ARCHIVE, --archive=ARCHIVE  specify the archive repository"))
  (display (_ "
  -h, --help                     display this help and exit"))
  (display (_ "
  -V, --version                  display version information and exit"))
  (newline)
  (show-bug-report-information))

(define %options
  ;; Specification of the command-line options.
  (cons* (option '(#\h "help") #f #f
                 (lambda args
                   (show-help)
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
                   (show-version-and-exit "guix import elpa")))
         (option '(#\a "archive") #t #f
                 (lambda (opt name arg result)
                   (alist-cons 'repo arg (alist-delete 'repo result))))
         %standard-import-options))

\f
;;;
;;; Entry point.
;;;

(define (guix-import-elpa . args)
  (define (parse-options)
    ;; Return the alist of option values.
    (args-fold* args %options
                (lambda (opt name arg result)
                  (leave (_ "~A: unrecognized option~%") name))
                (lambda (arg result)
                  (alist-cons 'argument arg result))
                %default-options))

  (let* ((opts (parse-options))
         (args (filter-map (match-lambda
                            (('argument . value)
                             value)
                            (_ #f))
                           (reverse opts))))
    (match args
      ((package-name)
       (let ((sexp (elpa->guix-package package-name (assoc-ref opts 'repo))))
         (unless sexp
           (leave (_ "failed to download package '~a'~%") package-name))
         sexp))
      (()
       (leave (_ "too few arguments~%")))
      ((many ...)
       (leave (_ "too many arguments~%"))))))

;;; elpa.scm ends here

[-- Attachment #4: emacs-build-system.scm --]
[-- Type: text/x-scheme, Size: 6740 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix build emacs-build-system)
  #:use-module ((guix build gnu-build-system) #:prefix gnu:)
  #:use-module (guix build utils)
  #:use-module (guix build emacs-utils)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-26)
  #:use-module (ice-9 rdelim)
  #:use-module (ice-9 regex)
  #:use-module (ice-9 match)
  #:export (%standard-phases
            emacs-build))

;; Commentary:
;;
;; Builder-side code of the build procedure for ELPA Emacs packages.
;;
;; Code:

;; Path relative to 'out' where we install ELPA packages.  We avoid the
;; ".../elpa" path as Emacs expects to find the ELPA repository
;; 'archive-contents' file and the archive signature.
(define guix-elpa-packages-path "/share/emacs/site-lisp/guix.d")

(define* (build #:key outputs inputs #:allow-other-keys)
  "Compile .el files."
  (let* ((emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
         (out (assoc-ref outputs "out"))
         (name-ver (store-dir->elpa-name-version out))
         (el-dir (string-append out guix-elpa-packages-path "/" name-ver)))
    (setenv "SHELL" "sh")
    (with-directory-excursion el-dir
      (fold (lambda (f s)
              (and s (zero? (system* emacs "--batch" "-Q" "-L" el-dir
                                     "-f" "batch-byte-compile" f))))
            #t (find-files "." "\\.el$")))))

(define* (patch-el-files #:key outputs inputs #:allow-other-keys)
  "Substitute the right path for '/bin/sh' in .el files."
  (let* ((emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
         (out (assoc-ref outputs "out"))
         (name-ver (store-dir->elpa-name-version out))
         (el-dir (string-append out guix-elpa-packages-path "/" name-ver)))
    (setenv "LC_ALL" "en_US.UTF-8")
    (with-directory-excursion el-dir
      (substitute* (find-files "." ".*\\.el$") (("/bin/sh") (which "sh"))))
    #t))

(define* (copy #:key outputs #:allow-other-keys)
  "Copy the package content to the installation directory."
  (let* ((out (assoc-ref outputs "out"))
         (name-ver (store-dir->elpa-name-version out))
         (src-dir (getcwd))
         (tgt-dir (string-append out guix-elpa-packages-path "/" name-ver)))
    (copy-recursively src-dir tgt-dir)
    #t))

(define* (move-doc #:key outputs #:allow-other-keys)
  "Move info files from the ELPA package directory to the info directory."
  (let* ((out (assoc-ref outputs "out"))
         (name-ver (store-dir->name-version out))
         (elpa-name-ver (package-name-version->elpa-name-version name-ver))
         (doc-dir (string-append out "/share/info/" name-ver))
         (el-dir (string-append out guix-elpa-packages-path "/" elpa-name-ver))
         (info-files (find-files el-dir "\\.info$")))
    (unless (null? info-files)
      (mkdir-p doc-dir)
      (with-directory-excursion el-dir
        (when (file-exists? "dir") (delete-file "dir"))
        (for-each (lambda (f)
                    (copy-file f (string-append doc-dir "/" (basename f)))
                    (delete-file f))
                  info-files)))
    #t))

(define* (make-autoloads #:key outputs inputs #:allow-other-keys)
  "Generate the autoloads file."
  (let* ((out (assoc-ref outputs "out"))
         (emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
         (name-ver (store-dir->elpa-name-version out))
         (name (package-name->name+version name-ver))
         (lisp-dir (string-append out guix-elpa-packages-path "/" name-ver)))
    (parameterize ((%emacs emacs))
      (emacs-generate-autoloads name lisp-dir))
    #t))


(define (package-name-version->elpa-name-version name-ver)
  "Convert the Guix package NAME-VER to the corresponding ELPA name-version
format.  Essnetially drop the prefix used in Guix."
  (let ((name (store-dir->name-version name-ver)))
    (if (string-prefix? "emacs-" name-ver)
        (store-dir->name-version name-ver)
        name-ver)))

(define (store-dir->elpa-name-version store-dir)
  "Given a store directory STORE-DIR return the part of the basename after the
second hyphen.  This corresponds to 'name-version' as used in ELPA packages."
  ((compose package-name-version->elpa-name-version
            store-dir->name-version)
   store-dir))

(define (store-dir->name-version store-dir)
  "Given a store directory STORE-DIR return the part of the basename
after the first hyphen.  This corresponds to 'name-version' of the package."
  (let* ((base (basename store-dir)))
    (string-drop base
                 (+ 1 (string-index base #\-)))))

;; from (guix utils).  Should we put it in (guix build utils)?
(define (package-name->name+version name)
  "Given NAME, a package name like \"foo-0.9.1b\", return two values:
\"foo\" and \"0.9.1b\".  When the version part is unavailable, NAME and
#f are returned.  The first hyphen followed by a digit is considered to
introduce the version part."
  ;; See also `DrvName' in Nix.

  (define number?
    (cut char-set-contains? char-set:digit <>))

  (let loop ((chars   (string->list name))
             (prefix '()))
    (match chars
      (()
       (values name #f))
      ((#\- (? number? n) rest ...)
       (values (list->string (reverse prefix))
               (list->string (cons n rest))))
      ((head tail ...)
       (loop tail (cons head prefix))))))

(define %standard-phases
  (modify-phases gnu:%standard-phases
    (delete 'configure)
    (delete 'check)
    (delete 'install)
    (replace 'build build)
    (add-before 'build 'copy copy)
    (add-after 'copy 'make-autoloads make-autoloads)
    ;;(add-after 'make-autoloads 'patch-el-files patch-el-files)
    (add-after 'make-autoloads 'move-doc move-doc)))

(define* (emacs-build #:key inputs (phases %standard-phases)
                      #:allow-other-keys #:rest args)
  "Build the given Emacs package, applying all of PHASES in order."
  (apply gnu:gnu-build
         #:inputs inputs #:phases phases
         args))

;;; emacs-build-system.scm ends here

[-- Attachment #5: emacs.scm --]
[-- Type: text/x-scheme, Size: 5514 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix build-system emacs)
  #:use-module (guix store)
  #:use-module (guix utils)
  #:use-module (guix packages)
  #:use-module (guix derivations)
  #:use-module (guix search-paths)
  #:use-module (guix build-system)
  #:use-module (guix build-system gnu)
  #:use-module (ice-9 match)
  #:use-module (srfi srfi-26)
  #:export (%emacs-build-system-modules
            emacs-build
            emacs-build-system))

;; Commentary:
;;
;; Standard build procedure for Emacs packages.  This is implemented as an
;; extension of 'gnu-build-system'.
;;
;; Code:

(define %emacs-build-system-modules
  ;; Build-side modules imported by default.
  `((guix build emacs-build-system)
    (guix build emacs-utils)
    ,@%gnu-build-system-modules))

(define (default-emacs)
  "Return the default Emacs package."
  ;; Lazily resolve the binding to avoid a circular dependency.
  (let ((emacs-mod (resolve-interface '(gnu packages emacs))))
    ;; we use 'emacs' instead of 'emacs-no-x' because the latter appears not
    ;; to be loading some macros and causes problems to some packages.  For
    ;; example, with the latter AUCTeX gives the error message:
    ;; "(invalid-function dbus-ignore-errors)".
    (module-ref emacs-mod 'emacs)))

(define* (lower name
                #:key source inputs native-inputs outputs system target
                (emacs (default-emacs))
                #:allow-other-keys
                #:rest arguments)
  "Return a bag for NAME."
  (define private-keywords
    '(#:target #:emacs #:inputs #:native-inputs))

  (and (not target)                               ;XXX: no cross-compilation
       (bag
         (name name)
         (system system)
         (host-inputs `(,@(if source
                              `(("source" ,source))
                              '())
                        ,@inputs

                        ;; Keep the standard inputs of 'gnu-build-system'.
                        ,@(standard-packages)))
         (build-inputs `(("emacs" ,emacs)
                         ,@native-inputs))
         (outputs outputs)
         (build emacs-build)
         (arguments (strip-keyword-arguments private-keywords arguments)))))

(define* (emacs-build store name inputs
                      #:key source
                      (tests? #t)
                      (test-target "test")
                      (configure-flags ''())
                      (phases '(@ (guix build emacs-build-system)
                                  %standard-phases))
                      (outputs '("out"))
                      (search-paths '())
                      (system (%current-system))
                      (guile #f)
                      (imported-modules %emacs-build-system-modules)
                      (modules '((guix build emacs-build-system)
                                 (guix build utils)
                                 (guix build emacs-utils))))
  "Build SOURCE using EMACS, and with INPUTS."
  (define builder
    `(begin
       (use-modules ,@modules)
       (emacs-build #:name ,name
                    #:source ,(match (assoc-ref inputs "source")
                                (((? derivation? source))
                                 (derivation->output-path source))
                                ((source)
                                 source)
                                (source
                                 source))
                    #:configure-flags ,configure-flags
                    #:system ,system
                    #:test-target ,test-target
                    #:tests? ,tests?
                    #:phases ,phases
                    #:outputs %outputs
                    #:search-paths ',(map search-path-specification->sexp
                                          search-paths)
                    #:inputs %build-inputs)))
  
  (define guile-for-build
    (match guile
      ((? package?)
       (package-derivation store guile system #:graft? #f))
      (#f                                         ; the default
       (let* ((distro (resolve-interface '(gnu packages commencement)))
              (guile  (module-ref distro 'guile-final)))
         (package-derivation store guile system #:graft? #f)))))

  (build-expression->derivation store name builder
                                #:inputs inputs
                                #:system system
                                #:modules imported-modules
                                #:outputs outputs
                                #:guile-for-build guile-for-build))

(define emacs-build-system
  (build-system
    (name 'emacs)
    (description "The build system for Emacs packages")
    (lower lower)))

;;; emacs.scm ends here

[-- Attachment #6: emacs-elpa.el --]
[-- Type: text/x-emacs-lisp, Size: 2336 bytes --]


(defun guix-package-desc-name (pkg-desc)
  "Return the name of the package. PKG-DESC is the unprocessed
list read from the package-description file."
  (nth 1 pkg-desc))

(defun guix-package-description-file (dir)
  "Return the name of the description file of the package in
DIR. (From Emacs 'package--description-file' in 'startup.el')."
  (concat (let ((subdir (file-name-nondirectory
                         (directory-file-name dir))))
            (if (string-match "\\([^.].*?\\)-\\([0-9]+\\(?:[.][0-9]+\\|\\(?:pre\\|beta\\|alpha\\)[0-9]+\\)*\\)" subdir)
                (match-string 1 subdir) subdir))
          "-pkg.el"))

(defun guix-package-load-descriptor (pkg-dir)
  "Return the uninterpreted content of the description file in
directory PKG-DIR. (Adapted from Emacs 'package-load-descriptor'
in 'package.el'.)"
  (let ((pkg-file (expand-file-name (guix-package-description-file pkg-dir)
                                    pkg-dir)))
    (when (file-exists-p pkg-file)
      (with-temp-buffer
        (insert-file-contents pkg-file)
        (goto-char (point-min))
        (read (current-buffer))))))

(defun guix-package-load-all (dir)
  "Activate all packages below DIR. (Adapted from Emacs
'package-activate-1' in 'package.el'.)"
  (when (file-directory-p dir)
    (dolist (subdir (directory-files dir))
      (let ((pkg-dir (expand-file-name subdir dir)))
        (when (file-directory-p pkg-dir)
          (let* ((pkg-desc (guix-package-load-descriptor pkg-dir))
                 (name (guix-package-desc-name pkg-desc))
                 (pkg-file (expand-file-name
                            (format "%s-autoloads.el" name) pkg-dir)))
            (when (file-exists-p pkg-file)
              ;; For some packages to be properly loaded (e.g.,
              ;; AUCTeX), they must be on the load-path.  However,
              ;; some packages add themselves to the load-path.  Hence
              ;; the check to avoid duplications.
              (push pkg-dir load-path)
              (let ((old-lp load-path))
                (with-demoted-errors (load pkg-file nil t))
                (unless (eq old-lp load-path)
                  (setq load-path old-lp))))))))))

(let* ((dir (concat (getenv "HOME") ".guix-profile/share/emacs/site-lisp/guix.d")))
  (when (file-exists-p dir)
    (guix-package-load-all dir)))

Re: emacs packages

[Mathieu Lirzin]

This seems like a great idea.

Federico Beffa <beffa@ieee.org> writes:

> There are a lot of Emacs packages. For this reason I would like to
> propose to prefix them with 'emacs-' as we do with Python, ...

IMO an 'el-' prefix would be better since it refers to the actual
language, and it's more concise.

--
Mathieu Lirzin

Re: emacs packages

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> To be concrete about what I'm proposing, attached you find:
>
> - An ELPA package importer
> - An 'emacs-build-system'

Sounds very useful to me!

> There are a lot of Emacs packages. For this reason I would like to
> propose to prefix them with 'emacs-' as we do with Python, ...

Sounds good.  I think I prefer ‘emacs-’ to ‘el-’ because that’s probably
what people expect.

> To make those packages automatically available in Emacs without the
> need for any code in the user '.emacs' file, I would suggest to
> include in our Emacs package site initialization file some custom code
> (to activate our ELPA emacs packages) such as the one in the attached
> 'guix-elpa.el' file.

guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
so that should be enough.

Some random comments:

> ;; Path relative to 'out' where we install ELPA packages.  We avoid the
> ;; ".../elpa" path as Emacs expects to find the ELPA repository
> ;; 'archive-contents' file and the archive signature.
> (define guix-elpa-packages-path "/share/emacs/site-lisp/guix.d")

Rather %install-suffix (“path” is used exclusively to refer to search
paths in GNU.)

> (define* (build #:key outputs inputs #:allow-other-keys)
>   "Compile .el files."
>   (let* ((emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
>          (out (assoc-ref outputs "out"))
>          (name-ver (store-dir->elpa-name-version out))
>          (el-dir (string-append out guix-elpa-packages-path "/" name-ver)))

Please align the lets.  :-)

>       (fold (lambda (f s)
>               (and s (zero? (system* emacs "--batch" "-Q" "-L" el-dir
>                                      "-f" "batch-byte-compile" f))))
>             #t (find-files "." "\\.el$")))))

Should probably use ‘emacs-batch-eval’ from (guix build emacs-utils),
and perhaps define ‘emacs-byte-compile’ there.

> (define* (copy #:key outputs #:allow-other-keys)
>   "Copy the package content to the installation directory."

s/copy/install/ ?

> ;; from (guix utils).  Should we put it in (guix build utils)?
> (define (package-name->name+version name)
>   "Given NAME, a package name like \"foo-0.9.1b\", return two values:

Yes, should probably be moved to (guix build utils) eventually.

Thanks for the great initiative!

Ludo’.

Re: emacs packages

[Pjotr Prins]

Awesome developments!

On Tue, Jun 16, 2015 at 06:00:44PM +0200, Ludovic Courtès wrote:
> Federico Beffa <beffa@ieee.org> skribis:
> 
> > To be concrete about what I'm proposing, attached you find:
> >
> > - An ELPA package importer
> > - An 'emacs-build-system'
> 
> Sounds very useful to me!
> 
> > There are a lot of Emacs packages. For this reason I would like to
> > propose to prefix them with 'emacs-' as we do with Python, ...
> 
> Sounds good.  I think I prefer ‘emacs-’ to ‘el-’ because that’s probably
> what people expect.
> 
> > To make those packages automatically available in Emacs without the
> > need for any code in the user '.emacs' file, I would suggest to
> > include in our Emacs package site initialization file some custom code
> > (to activate our ELPA emacs packages) such as the one in the attached
> > 'guix-elpa.el' file.
> 
> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
> so that should be enough.
> 
> Some random comments:
> 
> > ;; Path relative to 'out' where we install ELPA packages.  We avoid the
> > ;; ".../elpa" path as Emacs expects to find the ELPA repository
> > ;; 'archive-contents' file and the archive signature.
> > (define guix-elpa-packages-path "/share/emacs/site-lisp/guix.d")
> 
> Rather %install-suffix (“path” is used exclusively to refer to search
> paths in GNU.)
> 
> > (define* (build #:key outputs inputs #:allow-other-keys)
> >   "Compile .el files."
> >   (let* ((emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
> >          (out (assoc-ref outputs "out"))
> >          (name-ver (store-dir->elpa-name-version out))
> >          (el-dir (string-append out guix-elpa-packages-path "/" name-ver)))
> 
> Please align the lets.  :-)
> 
> >       (fold (lambda (f s)
> >               (and s (zero? (system* emacs "--batch" "-Q" "-L" el-dir
> >                                      "-f" "batch-byte-compile" f))))
> >             #t (find-files "." "\\.el$")))))
> 
> Should probably use ‘emacs-batch-eval’ from (guix build emacs-utils),
> and perhaps define ‘emacs-byte-compile’ there.
> 
> > (define* (copy #:key outputs #:allow-other-keys)
> >   "Copy the package content to the installation directory."
> 
> s/copy/install/ ?
> 
> > ;; from (guix utils).  Should we put it in (guix build utils)?
> > (define (package-name->name+version name)
> >   "Given NAME, a package name like \"foo-0.9.1b\", return two values:
> 
> Yes, should probably be moved to (guix build utils) eventually.
> 
> Thanks for the great initiative!
> 
> Ludo’.
> 

-- 

Re: emacs packages

[Mark H Weaver]

Federico Beffa <beffa@ieee.org> writes:

> (define* (build #:key outputs inputs #:allow-other-keys)
>   "Compile .el files."
>   (let* ((emacs (string-append (assoc-ref inputs "emacs") "/bin/emacs"))
>          (out (assoc-ref outputs "out"))
>          (name-ver (store-dir->elpa-name-version out))
>          (el-dir (string-append out guix-elpa-packages-path "/" name-ver)))
>     (setenv "SHELL" "sh")
>     (with-directory-excursion el-dir
>       (fold (lambda (f s)
>               (and s (zero? (system* emacs "--batch" "-Q" "-L" el-dir
>                                      "-f" "batch-byte-compile" f))))
>             #t (find-files "." "\\.el$")))))

FYI, this pattern of using 'fold' with 'and' and a boolean seed would be
more transparent if handled by 'every' from (srfi srfi-1):

  (every (lambda (f)
           (zero? (system* emacs "--batch" "-Q" "-L" el-dir
                           "-f" "batch-byte-compile" f)))
         (find-files "." "\\.el$"))

However, this won't be needed here if you use 'emacs-batch-eval' or add
'emacs-byte-compile' as suggested by Ludovic, which I agree is the right
approach.

     Thanks!
       Mark

Re: emacs packages

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

On Tue, Jun 16, 2015 at 6:24 PM, Mark H Weaver <mhw@netris.org> wrote:
> FYI, this pattern of using 'fold' with 'and' and a boolean seed would be
> more transparent if handled by 'every' from (srfi srfi-1):
>
>   (every (lambda (f)
>            (zero? (system* emacs "--batch" "-Q" "-L" el-dir
>                            "-f" "batch-byte-compile" f)))
>          (find-files "." "\\.el$"))

Thanks for the comment. I saw the form 'every' in the past, but 'fold'
always come first to my mind.  Probably because it took me more
neurons to understand how to use it :-)

> However, this won't be needed here if you use 'emacs-batch-eval' or add
> 'emacs-byte-compile' as suggested by Ludovic, which I agree is the right
> approach.

What do you think about the attached implementation?

Thanks,
Fede

[-- Attachment #2: 0001-build-emacs-utils-Add-emacs-byte-compile.patch --]
[-- Type: text/x-diff, Size: 1475 bytes --]

From d5c33d5237bb371ab3bea88eb78a9ccac583de1d Mon Sep 17 00:00:00 2001
From: Federico Beffa <beffa@fbengineering.ch>
Date: Tue, 16 Jun 2015 21:09:57 +0200
Subject: [PATCH] build: emacs-utils: Add 'emacs-byte-compile'.

* guix/build/emacs-utils.scm (emacs-byte-compile): New procedure.
---
 guix/build/emacs-utils.scm | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/guix/build/emacs-utils.scm b/guix/build/emacs-utils.scm
index 0cff28b..88eda78 100644
--- a/guix/build/emacs-utils.scm
+++ b/guix/build/emacs-utils.scm
@@ -22,6 +22,7 @@
             emacs-batch-eval
             emacs-batch-edit-file
             emacs-generate-autoloads
+            emacs-byte-compile
             emacs-substitute-sexps
             emacs-substitute-variables))
 
@@ -57,6 +58,17 @@
                   (update-directory-autoloads ,directory))))
     (emacs-batch-eval expr)))
 
+(define (emacs-byte-compile files)
+  "Byte compile FILES, a list of file names."
+  (if (pair? files)
+      (for-each (lambda (f)
+                  (let ((expr `(let ()
+                                 (push ,(dirname f) load-path)
+                                 (byte-compile-file ,f))))
+                    (emacs-batch-eval expr)))
+                files)
+      (error "emacs-byte-compile failed: not a list of files!" files)))
+
 (define-syntax emacs-substitute-sexps
   (syntax-rules ()
     "Substitute the S-expression immediately following the first occurrence of
-- 
2.2.1

Re: emacs packages

[Federico Beffa]

On Tue, Jun 16, 2015 at 6:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> - An ELPA package importer
>> - An 'emacs-build-system'
>
> Sounds very useful to me!

Thanks for the interest!

>> To make those packages automatically available in Emacs without the
>> need for any code in the user '.emacs' file, I would suggest to
>> include in our Emacs package site initialization file some custom code
>> (to activate our ELPA emacs packages) such as the one in the attached
>> 'guix-elpa.el' file.
>
> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
> so that should be enough.

Unfortunately this doesn't work without modification. The reason is
that I follow the emacs package.el strategy to install each ELPA
package in it's own sub-directory. Specifically, I'm installing each
package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
in 'guix.el', however, doesn't look in sub-directories below the
profile's '.../site-lisp'.

Another needed detail is the fact that some package's autoload file
(e.g., the one used by AUCTeX) includes a (require 'XXX) statement,
where XXX is a library file provided by the same package. For this to
work, I need to add the directory with the '.el' files to 'load-path'
before loading the autoload file. Then, some package's autoload file
add themselves to the 'load-path'. Therefore we need to remove the
double entry.

This is (part of) the code that I adapted from Emacs package.el to do that:

(defun guix-package-load-all (dir)
  "Activate all packages below DIR. (Adapted from Emacs
'package-activate-1' in 'package.el'.)"
  (when (file-directory-p dir)
    (dolist (subdir (directory-files dir))
      (let ((pkg-dir (expand-file-name subdir dir)))
        (when (file-directory-p pkg-dir)
          (let* ((pkg-desc (guix-package-load-descriptor pkg-dir))
                 (name (guix-package-desc-name pkg-desc))
                 (pkg-file (expand-file-name
                            (format "%s-autoloads.el" name) pkg-dir)))
            (when (file-exists-p pkg-file)
              ;; For some packages to be properly loaded (e.g.,
              ;; AUCTeX), they must be on the load-path.  However,
              ;; some packages add themselves to the load-path.  Hence
              ;; the check to avoid duplications.
              (push pkg-dir load-path)
              (let ((old-lp load-path))
                (with-demoted-errors (load pkg-file nil t))
                (unless (eq old-lp load-path)
                  (setq load-path old-lp))))))))))

The reason for using separate sub-directory is to avoid name clashes
(README, ...). Would be nice to adapt the code in guix.el to look into
subdirectories as well.

Regards,
Fede

Re: emacs packages

[Alex Kost]

Federico Beffa (2015-06-17 10:42 +0300) wrote:

> On Tue, Jun 16, 2015 at 6:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
[...]
>>> To make those packages automatically available in Emacs without the
>>> need for any code in the user '.emacs' file, I would suggest to
>>> include in our Emacs package site initialization file some custom code
>>> (to activate our ELPA emacs packages) such as the one in the attached
>>> 'guix-elpa.el' file.
>>
>> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
>> so that should be enough.
>
> Unfortunately this doesn't work without modification. The reason is
> that I follow the emacs package.el strategy to install each ELPA
> package in it's own sub-directory. Specifically, I'm installing each
> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
> in 'guix.el', however, doesn't look in sub-directories below the
> profile's '.../site-lisp'.
>
> Another needed detail is the fact that some package's autoload file
> (e.g., the one used by AUCTeX) includes a (require 'XXX) statement,

Ouch, what nasty things people do!

> where XXX is a library file provided by the same package. For this to
> work, I need to add the directory with the '.el' files to 'load-path'
> before loading the autoload file. Then, some package's autoload file
> add themselves to the 'load-path'. Therefore we need to remove the
> double entry.

I think double entry will do no harm, so I'm not sure if it's worth
removing.

> This is (part of) the code that I adapted from Emacs package.el to do that:
>
> (defun guix-package-load-all (dir)
>   "Activate all packages below DIR. (Adapted from Emacs
> 'package-activate-1' in 'package.el'.)"
>   (when (file-directory-p dir)
>     (dolist (subdir (directory-files dir))
>       (let ((pkg-dir (expand-file-name subdir dir)))
>         (when (file-directory-p pkg-dir)
>           (let* ((pkg-desc (guix-package-load-descriptor pkg-dir))
>                  (name (guix-package-desc-name pkg-desc))
>                  (pkg-file (expand-file-name
>                             (format "%s-autoloads.el" name) pkg-dir)))
>             (when (file-exists-p pkg-file)
>               ;; For some packages to be properly loaded (e.g.,
>               ;; AUCTeX), they must be on the load-path.  However,
>               ;; some packages add themselves to the load-path.  Hence
>               ;; the check to avoid duplications.
>               (push pkg-dir load-path)
>               (let ((old-lp load-path))
>                 (with-demoted-errors (load pkg-file nil t))
>                 (unless (eq old-lp load-path)
>                   (setq load-path old-lp))))))))))
>
> The reason for using separate sub-directory is to avoid name clashes
> (README, ...). Would be nice to adapt the code in guix.el to look into
> subdirectories as well.

The code for loading "…-autoloads.el" files is placed in
"guix-emacs.el", so perhaps it would be enough to adjust:

- 'guix-emacs-find-autoloads' to make it look at subdirs;

- 'guix-emacs-load-autoloads' to add the subdir to 'load-path' before
  loading "…-autoloads.el".

--------
You are working on a big feature (thank you very much for that!), so
what about making "wip-emacs" (or whatever) branch for a more convenient
working on it?

-- 
Alex

Re: emacs packages

[Mark H Weaver]

Federico Beffa <beffa@ieee.org> writes:

> What do you think about the attached implementation?

[...]

> +(define (emacs-byte-compile files)
> +  "Byte compile FILES, a list of file names."
> +  (if (pair? files)
> +      (for-each (lambda (f)
> +                  (let ((expr `(let ()
> +                                 (push ,(dirname f) load-path)
> +                                 (byte-compile-file ,f))))
> +                    (emacs-batch-eval expr)))
> +                files)
> +      (error "emacs-byte-compile failed: not a list of files!" files)))
> +

I would prefer to let an emacs expert (Alex?) comment on this, but here
are some preliminary comments:

* I wouldn't bother checking the type of 'files', since 'for-each' will
  do that.  Also, () is a list but not a pair.

* Maybe use 'progn' instead of 'let ()' ?

* Instead of launching a separate emacs process for each file, how about
  passing the list of filenames into emacs and do the loop within emacs
  itself?

      Mark

Re: emacs packages

[Alex Kost]

Mark H Weaver (2015-06-17 21:42 +0300) wrote:

> Federico Beffa <beffa@ieee.org> writes:
>
>> What do you think about the attached implementation?
>
> [...]
>
>> +(define (emacs-byte-compile files)
>> +  "Byte compile FILES, a list of file names."
>> +  (if (pair? files)
>> +      (for-each (lambda (f)
>> +                  (let ((expr `(let ()
>> +                                 (push ,(dirname f) load-path)
>> +                                 (byte-compile-file ,f))))
>> +                    (emacs-batch-eval expr)))
>> +                files)
>> +      (error "emacs-byte-compile failed: not a list of files!" files)))
>> +
>
> I would prefer to let an emacs expert (Alex?) comment on this, but here

(I don't consider myself an expert, I'm definitely much less experienced
with Emacs than you or Ludovic.)

> are some preliminary comments:
>
> * I wouldn't bother checking the type of 'files', since 'for-each' will
>   do that.  Also, () is a list but not a pair.
>
> * Maybe use 'progn' instead of 'let ()' ?
>
> * Instead of launching a separate emacs process for each file, how about
>   passing the list of filenames into emacs and do the loop within emacs
>   itself?

I totally agree with these comments.  However since the goal is to
compile all elisp files in some directory, perhaps we may just use
'byte-recompile-directory' instead of looping 'byte-compile-file'.

Also I think there is a problem.  An emacs package can require other
packages, so for a successful compiling, 'load-path' should be augmented
not only with the directory of the original package, but with the
directories of the required packages as well.

-- 
Alex

Re: emacs packages

[Federico Beffa]

On Wed, Jun 17, 2015 at 10:00 PM, Alex Kost <alezost@gmail.com> wrote:
> Mark H Weaver (2015-06-17 21:42 +0300) wrote:
>
>> Federico Beffa <beffa@ieee.org> writes:
>>
>>> What do you think about the attached implementation?
>>
>> [...]
>>
>>> +(define (emacs-byte-compile files)
>>> +  "Byte compile FILES, a list of file names."
>>> +  (if (pair? files)
>>> +      (for-each (lambda (f)
>>> +                  (let ((expr `(let ()
>>> +                                 (push ,(dirname f) load-path)
>>> +                                 (byte-compile-file ,f))))
>>> +                    (emacs-batch-eval expr)))
>>> +                files)
>>> +      (error "emacs-byte-compile failed: not a list of files!" files)))
>>> +
>>
>> I would prefer to let an emacs expert (Alex?) comment on this, but here
>
> (I don't consider myself an expert, I'm definitely much less experienced
> with Emacs than you or Ludovic.)
>
>> are some preliminary comments:
>>
>> * I wouldn't bother checking the type of 'files', since 'for-each' will
>>   do that.  Also, () is a list but not a pair.
>>
>> * Maybe use 'progn' instead of 'let ()' ?
>>
>> * Instead of launching a separate emacs process for each file, how about
>>   passing the list of filenames into emacs and do the loop within emacs
>>   itself?
>
> I totally agree with these comments.  However since the goal is to
> compile all elisp files in some directory, perhaps we may just use
> 'byte-recompile-directory' instead of looping 'byte-compile-file'.
>
> Also I think there is a problem.  An emacs package can require other
> packages, so for a successful compiling, 'load-path' should be augmented
> not only with the directory of the original package, but with the
> directories of the required packages as well.

Thanks for all the good suggestions. I will try to address them.

Regards,
Fede

Re: emacs packages

[Federico Beffa]

On Wed, Jun 17, 2015 at 8:21 PM, Alex Kost <alezost@gmail.com> wrote:
> The code for loading "…-autoloads.el" files is placed in
> "guix-emacs.el", so perhaps it would be enough to adjust:
>
> - 'guix-emacs-find-autoloads' to make it look at subdirs;
>
> - 'guix-emacs-load-autoloads' to add the subdir to 'load-path' before
>   loading "…-autoloads.el".

If you feel like wanting to adapt it yourself, please go on. I have no
experience with elisp apart from trivial things in my ./emacs. file.

Otherwise, I will look into it.

>
> --------
> You are working on a big feature (thank you very much for that!), so
> what about making "wip-emacs" (or whatever) branch for a more convenient
> working on it?

Thanks for your interest! I'm working in a local wip-emacs branch, but
I will have to rearrange commits a little bit. For this reason, I
would prefer to do some more work before sharing it.

Regards,
Fede

Re: emacs packages

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

Federico Beffa (2015-06-18 21:32 +0300) wrote:

> On Wed, Jun 17, 2015 at 8:21 PM, Alex Kost <alezost@gmail.com> wrote:
>> The code for loading "…-autoloads.el" files is placed in
>> "guix-emacs.el", so perhaps it would be enough to adjust:
>>
>> - 'guix-emacs-find-autoloads' to make it look at subdirs;
>>
>> - 'guix-emacs-load-autoloads' to add the subdir to 'load-path' before
>>   loading "…-autoloads.el".
>
> If you feel like wanting to adapt it yourself, please go on. I have no
> experience with elisp apart from trivial things in my ./emacs. file.
>
> Otherwise, I will look into it.

Thanks, I would like to adapt it myself.  I think the attached patch
should be enough to make the packages installed in
"~/.guix-profile/share/emacs/site-lisp/guix.d" be properly autoloaded.
I'll push this commit into "wip-emacs" when it appears, if you don't
mind.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-emacs-Find-autoloads-in-guix.d-subdirectories.patch --]
[-- Type: text/x-diff, Size: 3138 bytes --]

From 7110d5d4a795f91259af37b48a2909ad9451a42d Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Fri, 19 Jun 2015 12:31:59 +0300
Subject: [PATCH] emacs: Find autoloads in "guix.d" subdirectories.

Co-authored-by: Federico Beffa <beffa@fbengineering.ch>.

* emacs/guix-emacs.el (guix-emacs-find-autoloads-in-directory,
  guix-emacs-subdirs): New functions.
  (guix-emacs-find-autoloads): Search for autoloads in "guix.d"
  subdirectories.
  (guix-emacs-load-autoloads): Add subdirectories to 'load-path'.
---
 emacs/guix-emacs.el | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/emacs/guix-emacs.el b/emacs/guix-emacs.el
index 512a2e2..8be2f36 100644
--- a/emacs/guix-emacs.el
+++ b/emacs/guix-emacs.el
@@ -42,13 +42,33 @@ If PROFILE is nil, use `guix-user-profile'."
   (expand-file-name "share/emacs/site-lisp"
                     (or profile guix-user-profile)))
 
+(defun guix-emacs-find-autoloads-in-directory (directory)
+  "Return list of Emacs 'autoloads' files in DIRECTORY."
+  (directory-files directory 'full-name "-autoloads\\.el\\'" 'no-sort))
+
+(defun guix-emacs-subdirs (directory)
+  "Return list of DIRECTORY subdirectories."
+  (cl-remove-if (lambda (file)
+                  (or (string-match-p (rx "/." string-end) file)
+                      (string-match-p (rx "/.." string-end) file)
+                      (not (file-directory-p file))))
+                (directory-files directory 'full-name nil 'no-sort)))
+
 (defun guix-emacs-find-autoloads (&optional profile)
   "Return list of autoloads of Emacs packages installed in PROFILE.
 If PROFILE is nil, use `guix-user-profile'.
 Return nil if there are no emacs packages installed in PROFILE."
-  (let ((dir (guix-emacs-directory profile)))
-    (if (file-directory-p dir)
-        (directory-files dir 'full-name "-autoloads\\.el\\'")
+  (let ((elisp-root-dir (guix-emacs-directory profile)))
+    (if (file-directory-p elisp-root-dir)
+        (let ((elisp-pkgs-dir (expand-file-name "guix.d" elisp-root-dir))
+              (root-autoloads (guix-emacs-find-autoloads-in-directory
+                               elisp-root-dir)))
+          (if (file-directory-p elisp-pkgs-dir)
+              (let ((pkgs-autoloads
+                     (cl-mapcan #'guix-emacs-find-autoloads-in-directory
+                                (guix-emacs-subdirs elisp-pkgs-dir))))
+                (append root-autoloads pkgs-autoloads))
+            root-autoloads))
       (message "Directory '%s' does not exist." dir)
       nil)))
 
@@ -63,7 +83,10 @@ installed in `guix-user-profile'."
          (files (if all
                     autoloads
                   (cl-nset-difference autoloads guix-emacs-autoloads
-                                      :test #'string=))))
+                                      :test #'string=)))
+         (dirs (mapcar #'file-name-directory files))
+         (dirs (cl-remove-duplicates dirs :test #'string=)))
+    (setq load-path (append dirs load-path))
     (dolist (file files)
       (load file 'noerror))
     (setq guix-emacs-autoloads autoloads)))
-- 
2.4.2

Re: emacs packages

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> On Tue, Jun 16, 2015 at 6:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>>> To make those packages automatically available in Emacs without the
>>> need for any code in the user '.emacs' file, I would suggest to
>>> include in our Emacs package site initialization file some custom code
>>> (to activate our ELPA emacs packages) such as the one in the attached
>>> 'guix-elpa.el' file.
>>
>> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
>> so that should be enough.
>
> Unfortunately this doesn't work without modification. The reason is
> that I follow the emacs package.el strategy to install each ELPA
> package in it's own sub-directory. Specifically, I'm installing each
> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
> in 'guix.el', however, doesn't look in sub-directories below the
> profile's '.../site-lisp'.

What does it bring us to follow package.el’s strategy?

My impression is that we could simply follow what guix.el already does,
and thus avoid that guix.d/PACKAGE-VERSION sub-directory.  Of course we
can adjust guix.el as we see fit, but package.el is a completely
separate beast anyway.  Am I missing something?

Thanks,
Ludo’.

Re: emacs packages

[Federico Beffa]

On Fri, Jun 19, 2015 at 2:13 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
>>> so that should be enough.
>>
>> Unfortunately this doesn't work without modification. The reason is
>> that I follow the emacs package.el strategy to install each ELPA
>> package in it's own sub-directory. Specifically, I'm installing each
>> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
>> in 'guix.el', however, doesn't look in sub-directories below the
>> profile's '.../site-lisp'.
>
> What does it bring us to follow package.el’s strategy?
>
> My impression is that we could simply follow what guix.el already does,
> and thus avoid that guix.d/PACKAGE-VERSION sub-directory.  Of course we
> can adjust guix.el as we see fit, but package.el is a completely
> separate beast anyway.  Am I missing something?

Hi Ludo,

the reason for using separate sub-directories is that many packages
include files, such as README, ChangeLog, ..., that are likely to
clash. Even if we would delete all non ".el" files (which probably is
not safe), with more than 2500 packages on MELPA, it is possible that
we would still experience some name clashes. I can imagine that
someone preparing a package may be unaware of the existence of some
other package, possibly not very popular in his circle.

Regards,
Fede

Re: emacs packages

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> On Fri, Jun 19, 2015 at 2:13 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>>> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
>>>> so that should be enough.
>>>
>>> Unfortunately this doesn't work without modification. The reason is
>>> that I follow the emacs package.el strategy to install each ELPA
>>> package in it's own sub-directory. Specifically, I'm installing each
>>> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
>>> in 'guix.el', however, doesn't look in sub-directories below the
>>> profile's '.../site-lisp'.
>>
>> What does it bring us to follow package.el’s strategy?
>>
>> My impression is that we could simply follow what guix.el already does,
>> and thus avoid that guix.d/PACKAGE-VERSION sub-directory.  Of course we
>> can adjust guix.el as we see fit, but package.el is a completely
>> separate beast anyway.  Am I missing something?
>
> Hi Ludo,
>
> the reason for using separate sub-directories is that many packages
> include files, such as README, ChangeLog, ..., that are likely to
> clash. Even if we would delete all non ".el" files (which probably is
> not safe), with more than 2500 packages on MELPA, it is possible that
> we would still experience some name clashes. I can imagine that
> someone preparing a package may be unaware of the existence of some
> other package, possibly not very popular in his circle.

What about copying all the .el files to .../site-lisp, and copy the
other files elsewhere (for instance, ‘README’ and ‘ChangeLog’ to
share/doc/$PACKAGE, and .info files to share/info)?

Note that name clashes in profiles are annoying, but not fatal.

Thanks,
Ludo’.

Re: emacs packages

[Federico Beffa]

On Sun, Jun 21, 2015 at 11:12 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@ieee.org> skribis:
>
>> On Fri, Jun 19, 2015 at 2:13 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>>>> guix.el already takes care of that (info "(guix) Emacs Initial Setup"),
>>>>> so that should be enough.
>>>>
>>>> Unfortunately this doesn't work without modification. The reason is
>>>> that I follow the emacs package.el strategy to install each ELPA
>>>> package in it's own sub-directory. Specifically, I'm installing each
>>>> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
>>>> in 'guix.el', however, doesn't look in sub-directories below the
>>>> profile's '.../site-lisp'.
>>>
>>> What does it bring us to follow package.el’s strategy?
>>>
>>> My impression is that we could simply follow what guix.el already does,
>>> and thus avoid that guix.d/PACKAGE-VERSION sub-directory.  Of course we
>>> can adjust guix.el as we see fit, but package.el is a completely
>>> separate beast anyway.  Am I missing something?
>>
>> Hi Ludo,
>>
>> the reason for using separate sub-directories is that many packages
>> include files, such as README, ChangeLog, ..., that are likely to
>> clash. Even if we would delete all non ".el" files (which probably is
>> not safe), with more than 2500 packages on MELPA, it is possible that
>> we would still experience some name clashes. I can imagine that
>> someone preparing a package may be unaware of the existence of some
>> other package, possibly not very popular in his circle.
>
> What about copying all the .el files to .../site-lisp, and copy the
> other files elsewhere (for instance, ‘README’ and ‘ChangeLog’ to
> share/doc/$PACKAGE, and .info files to share/info)?

I am copying .info files to share/info. I'm not copying README files
to share/doc because these usually do not provide useful documentation
for the user and ChangeLog are usually not up-to-date relict. But if
somebody feels strongly about it, I can change that.

>
> Note that name clashes in profiles are annoying, but not fatal.

For .el files they are.

I also do not think that it is very sane ending up with a flat
directory including hundreds of files. Some hierarchy makes the
organization much more apparent and clean.

Regards,
Fede

Re: emacs packages

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> On Sun, Jun 21, 2015 at 11:12 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Federico Beffa <beffa@ieee.org> skribis:

[...]

>>>>> Unfortunately this doesn't work without modification. The reason is
>>>>> that I follow the emacs package.el strategy to install each ELPA
>>>>> package in it's own sub-directory. Specifically, I'm installing each
>>>>> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
>>>>> in 'guix.el', however, doesn't look in sub-directories below the
>>>>> profile's '.../site-lisp'.

[...]

>>> the reason for using separate sub-directories is that many packages
>>> include files, such as README, ChangeLog, ..., that are likely to
>>> clash. Even if we would delete all non ".el" files (which probably is
>>> not safe), with more than 2500 packages on MELPA, it is possible that
>>> we would still experience some name clashes. I can imagine that
>>> someone preparing a package may be unaware of the existence of some
>>> other package, possibly not very popular in his circle.
>>
>> What about copying all the .el files to .../site-lisp, and copy the
>> other files elsewhere (for instance, ‘README’ and ‘ChangeLog’ to
>> share/doc/$PACKAGE, and .info files to share/info)?
>
> I am copying .info files to share/info.

Ah OK, perfect then!

> I'm not copying README files to share/doc because these usually do not
> provide useful documentation for the user and ChangeLog are usually
> not up-to-date relict. But if somebody feels strongly about it, I can
> change that.

No, that’s fine.

>> Note that name clashes in profiles are annoying, but not fatal.
>
> For .el files they are.

They are fatal but rare, no?  My impression is that people prefix their
.el file names with the package name.  In my profile I have emms, bbdb,
emacs-w3m, magit & deps, geiser, cflow, etc. and there are zero clashes.
I don’t see any clash in the dozen of packages I still have in
~/.emacs.d/elpa/ either.

> I also do not think that it is very sane ending up with a flat
> directory including hundreds of files. Some hierarchy makes the
> organization much more apparent and clean.

The problem is that, unlike Guile modules, elisp module names are
inherently flat, hence the PACKAGE-foo.el convention that people seem to
follow.

But perhaps that convention is not strictly followed, which would
explain why package.el took this route?

I don’t feel strongly against what you suggest.  My main concern would
be the introduction of extra complexity that’s not strictly needed, but
you seem to be saying that it *is* needed.

Regardless, what matters most to me is that guix.el and
‘emacs-build-system’ work consistently.

Thank you!

Ludo’.

Re: emacs packages

[Federico Beffa]

On Mon, Jun 22, 2015 at 9:43 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@ieee.org> skribis:
>
>> On Sun, Jun 21, 2015 at 11:12 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Federico Beffa <beffa@ieee.org> skribis:
>
> [...]
>
>>>>>> Unfortunately this doesn't work without modification. The reason is
>>>>>> that I follow the emacs package.el strategy to install each ELPA
>>>>>> package in it's own sub-directory. Specifically, I'm installing each
>>>>>> package into ".../site-lisp/guix.d/PACKAGE-NAME-VERSION/".  The code
>>>>>> in 'guix.el', however, doesn't look in sub-directories below the
>>>>>> profile's '.../site-lisp'.
>
> [...]
>
>>>> the reason for using separate sub-directories is that many packages
>>>> include files, such as README, ChangeLog, ..., that are likely to
>>>> clash. Even if we would delete all non ".el" files (which probably is
>>>> not safe), with more than 2500 packages on MELPA, it is possible that
>>>> we would still experience some name clashes. I can imagine that
>>>> someone preparing a package may be unaware of the existence of some
>>>> other package, possibly not very popular in his circle.
>>>
>>> What about copying all the .el files to .../site-lisp, and copy the
>>> other files elsewhere (for instance, ‘README’ and ‘ChangeLog’ to
>>> share/doc/$PACKAGE, and .info files to share/info)?
>>
>> I am copying .info files to share/info.
>
> Ah OK, perfect then!
>
>> I'm not copying README files to share/doc because these usually do not
>> provide useful documentation for the user and ChangeLog are usually
>> not up-to-date relict. But if somebody feels strongly about it, I can
>> change that.
>
> No, that’s fine.
>
>>> Note that name clashes in profiles are annoying, but not fatal.
>>
>> For .el files they are.
>
> They are fatal but rare, no?  My impression is that people prefix their
> .el file names with the package name.  In my profile I have emms, bbdb,
> emacs-w3m, magit & deps, geiser, cflow, etc. and there are zero clashes.
> I don’t see any clash in the dozen of packages I still have in
> ~/.emacs.d/elpa/ either.
>
>> I also do not think that it is very sane ending up with a flat
>> directory including hundreds of files. Some hierarchy makes the
>> organization much more apparent and clean.
>
> The problem is that, unlike Guile modules, elisp module names are
> inherently flat, hence the PACKAGE-foo.el convention that people seem to
> follow.
>
> But perhaps that convention is not strictly followed, which would
> explain why package.el took this route?
>
> I don’t feel strongly against what you suggest.  My main concern would
> be the introduction of extra complexity that’s not strictly needed, but
> you seem to be saying that it *is* needed.

I'm just saying: If we use separate directories as package.el does, we
avoid all name clashes with probability 1. If we use a flat structure
such a claim can not be done.

Even if the clash "only" happens with, say, a README file, the state
of the system (user profile) would depend on the order of
installation. To me that is very bad, especially if there is an easy
solution to prevent that as in this case.

>
> Regardless, what matters most to me is that guix.el and
> ‘emacs-build-system’ work consistently.

I fully agree and Alex last Friday already posted a small patch to
make guix.el look into sub-directories.

https://lists.gnu.org/archive/html/guix-devel/2015-06/msg00382.html

Regards,
Fede

Re: emacs packages

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> I'm just saying: If we use separate directories as package.el does, we
> avoid all name clashes with probability 1. If we use a flat structure
> such a claim can not be done.
>
> Even if the clash "only" happens with, say, a README file, the state
> of the system (user profile) would depend on the order of
> installation. To me that is very bad, especially if there is an easy
> solution to prevent that as in this case.
>
>>
>> Regardless, what matters most to me is that guix.el and
>> ‘emacs-build-system’ work consistently.
>
> I fully agree and Alex last Friday already posted a small patch to
> make guix.el look into sub-directories.
>
> https://lists.gnu.org/archive/html/guix-devel/2015-06/msg00382.html

OK, you’ve convinced me, let’s follow the hierarchical approach you
propose.

Thanks for bearing with me!

Ludo’.