bug#43093: emacs-ess is broken by emacs 27.1

bug#43093: emacs-ess is broken by emacs 27.1

[divoplade]

Hello all,

Right now emacs-ess is broken so I cannot upgrade to emacs 27. The
failing test is "test-ess-roxy-literate".

Thank you anyway, I cannot wait to have emacs 27!

divoplade

bug#43093: Is Emacs 27.1 that breaks emacs-ess?

[zimoun]

Dear,

On Fri, 28 Aug 2020 at 22:07, divoplade <d@divoplade.fr> wrote:

> Right now emacs-ess is broken so I cannot upgrade to emacs 27. The
> failing test is "test-ess-roxy-literate".

I confirm.

--8<---------------cut here---------------start------------->8---
guix time-machine --commit=36a09d1853 \
     -- build emacs-ess -K
--8<---------------cut here---------------end--------------->8---

leads to "build of
/gnu/store/xm0lh7c0xfq3l0gfq34hxp1kjcnm6dri-emacs-ess-18.10.2.drv
failed" where the log reads:

--8<---------------cut here---------------start------------->8---
1 unexpected results:
   FAILED  test-ess-roxy-literate

2 skipped results:
  SKIPPED  runner-R-3\.2\.1-buffer-name
  SKIPPED  runner-R-3\.2\.1-defined
--8<---------------cut here---------------end--------------->8---

But the commit 36a09d1853 introducing Emacs 27 had been pushed on Fri Aug 28
00:13:51 2020 -0400 and the CI reported failures before this offending
commit.

https://ci.guix.gnu.org/search?query=spec%3Aguix-master+system%3Ax86_64-linux+emacs-ess

For other examples, see:

https://data.guix.gnu.org/repository/1/branch/master/package/emacs-ess/output-history

reporting issues both on Berlin and Bayfront and before the push of Emacs 27.


Note that the parent commit c05d2cfcbd builds successfully on my
machine.  Therefore, I do not know from where the failure comes from.


All the best,
simon

bug#43093: Is Emacs 27.1 that breaks emacs-ess?

[divoplade]

Hello,
Le vendredi 04 septembre 2020 à 16:44 +0200, zimoun a écrit :
> But the commit 36a09d1853 introducing Emacs 27 had been pushed on Fri
> Aug 28
> 00:13:51 2020 -0400 and the CI reported failures before this
> offending
> commit.
> 
> https://ci.guix.gnu.org/search?query=spec%3Aguix-master+system%3Ax86_64-linux+emacs-ess
> 
> For other examples, see:
> 
> https://data.guix.gnu.org/repository/1/branch/master/package/emacs-ess/output-history
> 
> reporting issues both on Berlin and Bayfront and before the push of
> Emacs 27.
> 

Well, you must be right. I did not think it out long enough. It broke
at the same time I saw the emacs update, so I inferred it incorrectly.
Is it possible to edit the title?

> 
> Note that the parent commit c05d2cfcbd builds successfully on my
> machine.  Therefore, I do not know from where the failure comes from.
I can't find such a commit ID. It does not seem to exist in the git
repo. Do you know the date, author or committer?

bug#43093: Is Emacs 27.1 that breaks emacs-ess?

[Mark H Weaver]

divoplade <d@divoplade.fr> writes:
> Is it possible to edit the title?

Yes.  To do so, send an email to 'control@debbugs.gnu.org' with the
following two-line body:

--8<---------------cut here---------------start------------->8---
retitle 43093 THE_NEW_BUG_TITLE_GOES_HERE
thanks
--8<---------------cut here---------------end--------------->8---

For more details on how to use the control server, see:

  https://debbugs.gnu.org/server-control.html

which is linked from the main debbugs help page, here:

  https://debbugs.gnu.org

      Mark

bug#43093: Is Emacs 27.1 that breaks emacs-ess?

[zimoun]

On Fri, 4 Sep 2020 at 17:26, divoplade <d@divoplade.fr> wrote:

> Well, you must be right. I did not think it out long enough. It broke
> at the same time I saw the emacs update, so I inferred it incorrectly.

Nonetheless, it is still annoying. :-)

> > Note that the parent commit c05d2cfcbd builds successfully on my
> > machine.  Therefore, I do not know from where the failure comes from.
> I can't find such a commit ID. It does not seem to exist in the git
> repo. Do you know the date, author or committer?

   git log --format="%p" -1 36a09d1853

and the answer is c05d2cfcbe.  I do not know how I did the mistake
when copy/pasting.

All the best,
simon

bug#43093: emacs-ess is broken by emacs 27.1

[Tim Howes via Bug reports for GNU Guix]

On 8/28/2020 1:07 PM, divoplade wrote:
> Hello all,
> 
> Right now emacs-ess is broken so I cannot upgrade to emacs 27. The
> failing test is "test-ess-roxy-literate".
> 
> Thank you anyway, I cannot wait to have emacs 27!
> 
> divoplade
> 
> 
> 
> 

Hello,

I think, instead of disabling the test or trying to make it work with 
emacs 27, I'd like to update emacs-ess to a more recent version. Right 
now it's using the latest tagged release (18.10.2), which was from 2018, 
but there's been active development on the github repo since then. I can 
take the latest commit and give it a version number based on the commit 
time, similar to how it's done on melpa (20200903.1516). I'll try making 
a patch to update it to that version.

-Tim

bug#43093: [PATCH] gnu: emacs-ess: Update to 20200903.1516.

[Tim Howes via Bug reports for GNU Guix]

* gnu/packages/statistics.scm (emacs-ess): Update to 20200903.1516.
[version]: Use latest commit, assign version based on commit date.
[source]: Fix snippet for removing julia-mode.  Remove snippet to modify
roxy-preview-Rd-test.  Add snippet to modify r-help-mode test.  Add
snippet to fix install target to install files to correct directories.
[arguments]: Add flag to specify INFODIR.  Remove patch modifying SHELL.
---
This updates emacs-ess to the current version on github and resolves
bugs #43093 and #42259.

 gnu/packages/statistics.scm | 43 ++++++++++++++++---------------------
 1 file changed, 19 insertions(+), 24 deletions(-)

diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
index c1cefa8c41..aa14e2887f 100644
--- a/gnu/packages/statistics.scm
+++ b/gnu/packages/statistics.scm
@@ -5805,29 +5805,24 @@ Java package that provides routines for various statistical distributions.")
 (define-public emacs-ess
   (package
     (name "emacs-ess")
-    (version "18.10.2")
+    ;; Use version number based on commit date and time (UTC), similar to melpa.
+    (version "20200903.1516")
     (source (origin
               (method git-fetch)
               (uri (git-reference
                     (url "https://github.com/emacs-ess/ESS")
-                    (commit (string-append "v" version))))
+                    (commit "09c1adecf8406d85426d14e5efe2aaba6c59f224")))
               (sha256
                (base32
-                "1yq41l2bicwjrc0b731iic20cpcnz6ppigri1jn621qv2qv22vy3"))
+                "0hl8nhf8w3rfpfqb2bfzcbh4lpsk51p83w2j5fc2knd4da4sb1s5"))
               (file-name (git-file-name name version))
               (modules '((guix build utils)))
               (snippet
                '(begin
                   ;; Stop ESS from trying to bundle an external julia-mode.el.
                   (substitute* "lisp/Makefile"
-                    (("^ess-julia.elc: julia-mode.elc") "")
-                    (("^all: julia-mode.el")
-                     "all:"))
-                  ;; Include *.el files in install target.
-                  (substitute* "lisp/Makefile"
-                    (("\t\\$\\(INSTALL) \\$\\(ELC\\) \\$\\(LISPDIR\\)" elc)
-                     (string-append "\t$(INSTALL) $(ELS) ess-autoloads.el "
-                                    "$(LISPDIR)\n" elc)))
+                    ((" \\$\\(JULIAS)") "")
+                    (("\ttest.*julia-mode.*\\.el") ""))
                   ;; Only build docs in info format.
                   (substitute* "doc/Makefile"
                     (("all  : info text")
@@ -5836,12 +5831,16 @@ Java package that provides routines for various statistical distributions.")
                      "install: install-info"))
                   ;; Stop install-info from trying to update the info directory.
                   (substitute* "doc/Makefile"
-                    ((".*\\$\\(INFODIR\\)/dir.*") ""))
-                  ;; Fix roxygen preview test.
-                  (substitute* "test/ess-r-tests.el"
-                               (("Add together two numbers.\n")
-                                "Add together two numbers. ")
-                               (("##' add\\(10, 1\\)") "add(10, 1)"))
+                    ((".*/dir.*") ""))
+                  ;; Fix r-help-mode test.
+                  (substitute* "test/ess-test-r.el"
+                    (("\\(equal ess-help-object \"plot.default\")") "t"))
+                  ;; Install to correct directories.
+                  (substitute* "Makefile"
+                    (("mkdir -p \\$\\(ESSDESTDIR)")
+                     "$(MAKE) -C lisp install; $(MAKE) -C doc install")
+                    (("\\$\\(INSTALL) -R \\./\\* \\$\\(ESSDESTDIR)/")
+                     "$(MAKE) -C etc install"))
                   #t))))
     (build-system gnu-build-system)
     (arguments
@@ -5850,16 +5849,12 @@ Java package that provides routines for various statistical distributions.")
                             (string-append "ETCDIR=" %output
                                            ,base-directory "/etc")
                             (string-append "LISPDIR=" %output
-                                           ,base-directory))
+                                           ,base-directory)
+                            (string-append "INFODIR=" %output
+                                           "/share/info"))
          #:phases
          (modify-phases %standard-phases
            (delete 'configure)
-           (add-before 'build 'more-shebang-patching
-             (lambda* (#:key inputs #:allow-other-keys)
-               (substitute* "Makeconf"
-                 (("SHELL = /bin/sh")
-                  (string-append "SHELL = " (which "sh"))))
-               #t))
            (replace 'check
              (lambda _
                (invoke "make" "test")))))))
-- 
2.28.0

bug#43093: emacs-ess is broken by emacs 27.1

[divoplade]

[-- Attachment #1: Type: text/plain, Size: 2922 bytes --]

Hello Tim,

Le vendredi 04 septembre 2020 à 11:25 -0700, Tim Howes a écrit :
> I think, instead of disabling the test or trying to make it work
> with 
> emacs 27, I'd like to update emacs-ess to a more recent version.
> Right 
> now it's using the latest tagged release (18.10.2), which was from
> 2018, 
> but there's been active development on the github repo since then. I
> can 
> take the latest commit and give it a version number based on the
> commit 
> time, similar to how it's done on melpa (20200903.1516). I'll try
> making 
> a patch to update it to that version.

Right, emacs-ess is already a tag. Let's take the latest commit,
82cd308ae54a6b918bbceb235e6bf02f53e48e19

If I run guix hash -xr within the code, I get
0zw6j8jzrdmy41g6313js7c0xlmc2wmiazx4d4wm6hdvykn8q39k

The version would be 20200905.1022

If I do a drop-in replacement, remove the "Fix roxygen preview test."
modification and change the commands to disable julia, I get one other
test that fails:

Test ess-test-r-help-mode condition:
    (ert-test-failed
     ((should
       (or
	(equal ess-help-object "plot.default")
	(equal ess-help-object "plot")))
      :form
      (or
       (equal ess-help-object "plot.default")
       (equal ess-help-object "plot"))
      :value nil))

The test is in test/ess-test-r.el, line 628.

(ert-deftest ess-test-r-help-mode ()
  (with-r-running nil
    (let ((ess-pop-to-buffer t))
      (ess-display-help-on-object "plot")
      (should (equal ess-help-object "plot"))
      (should (derived-mode-p 'ess-r-help-mode))
      ;; Ensure help buffers after button presses are also in
      ;; `ess-r-help-mode', Bug#836
      (forward-button 2)
      (push-button)
      (should (or (equal ess-help-object "plot.default")
                  (equal ess-help-object "plot"))) ;; Badaboom
                                                ;;(the value is "NULL")
      (should (derived-mode-p 'ess-r-help-mode)))))

Looking at its friend, ess-test-r-index-mode, line 643:

(ert-deftest ess-test-r-index-mode ()
  (skip-unless (not noninteractive)) ;; negation overflow ;p
  (with-r-running nil
    (let ((ess-pop-to-buffer t))
      (ess-display-package-index "stats")
      (should (equal ess-help-object "stats"))
      (should (derived-mode-p 'ess-r-help-mode))
      ;; Ensure help buffers after button presses are also in
      ;; `ess-r-help-mode', Bug#836
      (forward-button 2)
      (push-button)
      (should (equal ess-help-object "plot.default"))
      (should (derived-mode-p 'ess-r-help-mode)))))

it seems that these tests should only run if (not (not (not (not
interactive)))) (i.e. interactive), and ess-test-r-help-mode has missed
a copy-paste.

Also, the license has changed: it is now gpl3+!

Anyway, here is a version that's working (I tested it with "emacs-ess-
next" as a name, but replaced it for this message).

Sorry, I don't know how to make a patch.

Best regards,

divoplade

[-- Attachment #2: modified-emacs.scm --]
[-- Type: text/x-scheme, Size: 3489 bytes --]

(define-public emacs-ess
  (package
    (name "emacs-ess")
    (version "20200905.1022")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "https://github.com/emacs-ess/ESS")
                    (commit "82cd308ae54a6b918bbceb235e6bf02f53e48e19")))
              (sha256
               (base32
                "0zw6j8jzrdmy41g6313js7c0xlmc2wmiazx4d4wm6hdvykn8q39k"))
              (file-name (git-file-name name version))
              (modules '((guix build utils)))
              (snippet
               '(begin
                  ;; Stop ESS from trying to bundle an external julia-mode.el.
                  (substitute* "lisp/Makefile"
                    (("JULIAS := julia-mode.el julia-mode-latexsubs.el") "JULIAS ="))
                  (substitute* "Makefile"
                    (("$(MAKE) julia-mode.el") ""))
                  (substitute* "test/Makefile"
			       (("lisp: julia") "lisp:"))
		  ;; It looks like this test misses a paste
                  (substitute* "test/ess-test-r.el"
			       (("\\(ert-deftest ess-test-r-help-mode \\(\\)") "(ert-deftest ess-test-r-help-mode () (skip-unless (not noninteractive))"))
                  ;; Include *.el files in install target.
                  (substitute* "lisp/Makefile"
                    (("\t\\$\\(INSTALL) \\$\\(ELC\\) \\$\\(LISPDIR\\)" elc)
                     (string-append "\t$(INSTALL) $(ELS) ess-autoloads.el "
                                    "$(LISPDIR)\n" elc)))
                  ;; Only build docs in info format.
                  (substitute* "doc/Makefile"
                    (("all  : info text")
                     "all  : info")
                    (("install: install-info install-other-docs")
                     "install: install-info"))
                  ;; Stop install-info from trying to update the info directory.
                  (substitute* "doc/Makefile"
                    ((".*\\$\\(INFODIR\\)/dir.*") ""))
                  #t))))
    (build-system gnu-build-system)
    (arguments
     (let ((base-directory "/share/emacs/site-lisp"))
       `(#:make-flags (list (string-append "PREFIX=" %output)
                            (string-append "ETCDIR=" %output
                                           ,base-directory "/etc")
                            (string-append "LISPDIR=" %output
                                           ,base-directory))
         #:phases
         (modify-phases %standard-phases
           (delete 'configure)
           (add-before 'build 'more-shebang-patching
             (lambda* (#:key inputs #:allow-other-keys)
               (substitute* "Makeconf"
                 (("SHELL = /bin/sh")
                  (string-append "SHELL = " (which "sh"))))
               #t))
           (replace 'check
             (lambda _
               (invoke "make" "test")))))))
    (inputs
     `(("emacs" ,emacs-minimal)
       ("r-minimal" ,r-minimal)))
    (native-inputs
     `(("perl" ,perl)
       ("r-roxygen2" ,r-roxygen2)
       ("texinfo" ,texinfo)))
    (propagated-inputs
     `(("emacs-julia-mode" ,emacs-julia-mode)))
    (home-page "https://ess.r-project.org/")
    (synopsis "Emacs mode for statistical analysis programs")
    (description "Emacs Speaks Statistics (ESS) is an add-on package for GNU
Emacs.  It is designed to support editing of scripts and interaction with
various statistical analysis programs such as R, Julia, and JAGS.")
    (license license:gpl3+)))

bug#43093: emacs-ess is broken by emacs 27.1

[divoplade]

[-- Attachment #1: Type: text/plain, Size: 140 bytes --]

Le lundi 07 septembre 2020 à 23:19 +0200, divoplade a écrit :
> Sorry, I don't know how to make a patch.

Trying: does it work like that?

[-- Attachment #2: 0001-Update-emacs-ess-to-a-more-recent-snapshot.patch --]
[-- Type: text/x-patch, Size: 3937 bytes --]

From 06cb64632af8982054eb85ab4bbe5c43e86ab653 Mon Sep 17 00:00:00 2001
From: divoplade <d@divoplade.fr>
Date: Mon, 7 Sep 2020 23:33:32 +0200
Subject: [PATCH] Update emacs-ess to a more recent snapshot

Hello guix!

There are a few things to note in that new version:
1. The license has changed, it is now GPLv3+;
2. The previous fix for roxygen is not needed anymore, as far as I can tell;
3. The makefiles seem to have changed, so removing julia is done a little
   differently;
4. The failing test =ess-test-r-help-mode= looks suspiciously like the next
   test (this one passes), =ess-test-r-index-mode=, which has the following line:

  (skip-unless (not noninteractive))

So I figured out that the previous one needed it too, it just got forgotten.

Best regards,

divoplade
---
 gnu/packages/statistics.scm | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
index 70a85e8b60..6585ce54b0 100644
--- a/gnu/packages/statistics.scm
+++ b/gnu/packages/statistics.scm
@@ -5805,24 +5805,29 @@ Java package that provides routines for various statistical distributions.")
 (define-public emacs-ess
   (package
     (name "emacs-ess")
-    (version "18.10.2")
+    (version "20200905.1022")
     (source (origin
               (method git-fetch)
               (uri (git-reference
                     (url "https://github.com/emacs-ess/ESS")
-                    (commit (string-append "v" version))))
+                    (commit "82cd308ae54a6b918bbceb235e6bf02f53e48e19")))
               (sha256
                (base32
-                "1yq41l2bicwjrc0b731iic20cpcnz6ppigri1jn621qv2qv22vy3"))
+                "0zw6j8jzrdmy41g6313js7c0xlmc2wmiazx4d4wm6hdvykn8q39k"))
               (file-name (git-file-name name version))
               (modules '((guix build utils)))
               (snippet
                '(begin
                   ;; Stop ESS from trying to bundle an external julia-mode.el.
                   (substitute* "lisp/Makefile"
-                    (("^ess-julia.elc: julia-mode.elc") "")
-                    (("^all: julia-mode.el")
-                     "all:"))
+                    (("JULIAS := julia-mode.el julia-mode-latexsubs.el") "JULIAS ="))
+                  (substitute* "Makefile"
+                    (("$(MAKE) julia-mode.el") ""))
+                  (substitute* "test/Makefile"
+			       (("lisp: julia") "lisp:"))
+		  ;; It looks like this test misses a paste
+                  (substitute* "test/ess-test-r.el"
+			       (("\\(ert-deftest ess-test-r-help-mode \\(\\)") "(ert-deftest ess-test-r-help-mode () (skip-unless (not noninteractive))"))
                   ;; Include *.el files in install target.
                   (substitute* "lisp/Makefile"
                     (("\t\\$\\(INSTALL) \\$\\(ELC\\) \\$\\(LISPDIR\\)" elc)
@@ -5837,11 +5842,6 @@ Java package that provides routines for various statistical distributions.")
                   ;; Stop install-info from trying to update the info directory.
                   (substitute* "doc/Makefile"
                     ((".*\\$\\(INFODIR\\)/dir.*") ""))
-                  ;; Fix roxygen preview test.
-                  (substitute* "test/ess-r-tests.el"
-                               (("Add together two numbers.\n")
-                                "Add together two numbers. ")
-                               (("##' add\\(10, 1\\)") "add(10, 1)"))
                   #t))))
     (build-system gnu-build-system)
     (arguments
@@ -5877,4 +5877,4 @@ Java package that provides routines for various statistical distributions.")
     (description "Emacs Speaks Statistics (ESS) is an add-on package for GNU
 Emacs.  It is designed to support editing of scripts and interaction with
 various statistical analysis programs such as R, Julia, and JAGS.")
-    (license license:gpl2+)))
+    (license license:gpl3+)))
-- 
2.28.0

bug#43093: emacs-ess is broken by emacs 27.1

[divoplade]

Le lundi 07 septembre 2020 à 23:19 +0200, divoplade a écrit :
> Sorry, I don't know how to make a patch.
> 
By trying to make one, I noticed that you have already sent a patch. I
don't know why I did not recieve it. I am still clueless about a lot of
things here.

Anyway, you forgot to update the license. It is now GPLv3+.

Best regards,

divoplade

bug#43093: emacs-ess is broken by emacs 27.1

[Tim Howes via Bug reports for GNU Guix]

On 9/7/2020 2:52 PM, divoplade wrote:
> Le lundi 07 septembre 2020 à 23:19 +0200, divoplade a écrit :
>> Sorry, I don't know how to make a patch.
>>
> By trying to make one, I noticed that you have already sent a patch. I
> don't know why I did not recieve it. I am still clueless about a lot of
> things here.
> 
> Anyway, you forgot to update the license. It is now GPLv3+.
> 
> Best regards,
> 
> divoplade
> 

That's alright, I should have cc'd you with the patch, but forgot since 
I was using git send-email.

I started a separate thread here for this patch. There still seems to be 
a problem with it, as the build sometimes fails due to a file locking 
issue. And it was suggested that I should version it as a revision of 
18.02.2 instead of this versioning scheme.

https://issues.guix.gnu.org/43208

Thanks also for the note about the license. I'll update that accordingly.
-Tim

bug#43093: [PATCH] gnu: emacs-ess: Update to 20200903.1516.

[Maxim Cournoyer]

Hello!

Tim Howes <timhowes@lavabit.com> writes:

> * gnu/packages/statistics.scm (emacs-ess): Update to 20200903.1516.
> [version]: Use latest commit, assign version based on commit date.
> [source]: Fix snippet for removing julia-mode.  Remove snippet to modify
> roxy-preview-Rd-test.  Add snippet to modify r-help-mode test.  Add
> snippet to fix install target to install files to correct directories.
> [arguments]: Add flag to specify INFODIR.  Remove patch modifying SHELL.
> ---
> This updates emacs-ess to the current version on github and resolves
> bugs #43093 and #42259.

I'm closing this bug, since a more current patch was merged to master
[0] about two days ago, which resolves the issue.

Thank you!

Maxim

[0]  https://issues.guix.gnu.org/43208#11