bug#59232: 27.2; vc-annotate on SVN does not process all lines

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 6152 bytes --]

- Start 'Emacs -Q' to visit a 'big' source code file that is revisionned
  on Subversion 1.6.17 (r11280011).
- Use 'C-x v g' to run vc-annotate to annotate the file.
- When the file size is over a (around) limit of 64KiB, only the lines
  corresponding to about 64KiB characters are annotated as they should.
  The lines after that limit are not annotated.  The command does not
  fail and does not report any problem.  It's just that it does not
  annotate the complete set of lines.

If I try 'svn blame' on that file inside a bash shell running in the
Linux terminal, all lines are annotated.  If I start a shell inside
Emacs (with M-x shell) and I run the 'svn blame' command inside that
shell, then all lines are annotated.

If the file is small, all lines are annotated by vc-annotate.
But when the file is large (over about 64KiB) then the end of the files
are never annotated.

This happens on Subversion.
No crash.
-----------------------------------------------

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    'bt full' and 'xbacktrace'.
For information about debugging Emacs, please read the file
/usr/local/share/emacs/27.2/etc/DEBUG.

In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2022-02-07 built on roup-ubuntu16-4
System Description: Ubuntu 16.04.7 LTS

Recent messages:
Marking matching files...
0 matching files marked
Hid 0 dotfiles.
Marking matching files...
1 matching file marked
Hid 1 dotfile.
Annotating...
Redisplaying annotation...done (Spanned from 14865.2 to 4045.3 days old)
Annotating... done
Mark set

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY GNUTLS
FREETYPE HARFBUZZ XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES
THREADS PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: C++//la

Minor modes in effect:
  smart-dash-c-mode: t
  smart-dash-mode: t
  ggtags-mode: t
  display-time-mode: t
  ido-everywhere: t
  winner-mode: t
  key-chord-mode: t
  global-anzu-mode: t
  anzu-mode: t
  flyspell-mode: t
  superword-mode: t
  show-paren-mode: t
  recentf-mode: t
  which-key-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/home/rouleaup/.emacs.d/utils/sr-speedbar hides
/home/rouleaup/.emacs.d/elpa/sr-speedbar-20161025.831/sr-speedbar
/home/rouleaup/.emacs.d/elpa/lispy-20220203.1437/elpa hides
/home/rouleaup/.emacs.d/elpa/ivy-20211231.1730/elpa

Features:
(shadow sort mail-extr emacsbug message rmc puny format-spec rfc822 mml
mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils smex windmove vc-annotate
pel-vc vc-dir vc vc-dispatcher vc-svn emacros pel-skels-cpp pel-skels-c
pel-list pel-uuid pel-cc pel-cc-find pel-ini pel-ffind-inpath pel-ffind
pel-file smart-dash ggtags ewoc cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs ace-link time
ido-completing-read+ memoize cus-edit minibuf-eldef pel-skels-generic
counsel xdg compile comint ansi-color swiper cl-extra ivy flx ivy-faces
ivy-overlay colir color ido-grid ido pel-completion pel-ido pel-seq
indent-tools yafolding s indent-tools-indentation-of winner pel-xref
pel-text-transform pel-read pel-navigate pel-scroll key-seq
pel-key-chord key-chord anzu dired-aux dired-hide-dotfiles dired-x dired
dired-loaddefs term/xterm xterm tempo pel-skels-elisp pel-text-insert
pel-syntax pel--syntax-macros pel-window pel-tempo pel-skels lispy pcase
delsel lispy-inline thingatpt avy noutline outline easy-mmode etags
fileloop generator xref project edebug backtrace help-fns radix-tree
help-mode lispy-tags mode-local advice find-func pel__hydra hydra ring
lv pel-lispy pel-spell-iedit flyspell ispell pel-spell pel-prompt
cap-words superword subword imenu+ pel-imenu imenu paren pel_keys vc-p4
p4-lowlevel recentf tree-widget wid-edit two-column speedbar sb-image
ezimage dframe ls-lisp delight pel-autoload pel--keys-macros
pel--options pel--macros pel--base pel finder-inf which-key cus-start
cus-load info edmacro kmacro package easymenu browse-url url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 517926 111494)
 (symbols 48 31419 1)
 (strings 32 154937 21608)
 (string-bytes 1 4413034)
 (vectors 16 42875)
 (vector-slots 8 549940 132422)
 (floats 8 292 2069)
 (intervals 56 5802 1445)
 (buffers 1000 17)
 (heap 1024 32345 10264))

[-- Attachment #2: Type: text/html, Size: 6779 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Dmitry Gutov]

On 13.11.2022 02:14, Pierre Rouleau wrote:
> If the file is small, all lines are annotated by vc-annotate.
> But when the file is large (over about 64KiB) then the end of the files
> are never annotated.

Hi!

When you say they are not annotated, do you mean that the contents of 
the file appear truncated (cut off) in the annotate buffer, or that the 
colors is missing, something like that?

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

By not annotated, I mean that the file appears cut off.   All lines visible
in the buffer are colored, including the last line.  But the real file
might have 3000 lines and only 1000 lines are shown in the buffer.

On Sat, Nov 12, 2022 at 9:29 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 13.11.2022 02:14, Pierre Rouleau wrote:
> > If the file is small, all lines are annotated by vc-annotate.
> > But when the file is large (over about 64KiB) then the end of the files
> > are never annotated.
>
> Hi!
>
> When you say they are not annotated, do you mean that the contents of
> the file appear truncated (cut off) in the annotate buffer, or that the
> colors is missing, something like that?
>


-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 1102 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Dmitry Gutov]

It would also be great if you could first re-test with a more recent 
version of Emacs. For instance, 28.1 which was released this year.

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

On Sat, Nov 12, 2022 at 9:30 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> It would also be great if you could first re-test with a more recent
> version of Emacs. For instance, 28.1 which was released this year.
>
Agreed.  Unfortunately the environment I'm in is a corporate one.  I'll
have to build the latest Emacs inside that old version of Linux (Ubuntu
16.04).  And that's inside a VM.  I can try that but that will take some
time (due to some outside constraints).

I'll see what I can do to try in other ways.

-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 935 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Eli Zaretskii]

> From: Pierre Rouleau <prouleau001@gmail.com>
> Date: Sat, 12 Nov 2022 19:14:59 -0500
> 
> - Start 'Emacs -Q' to visit a 'big' source code file that is revisionned
>   on Subversion 1.6.17 (r11280011).

Can you suggest a publicly-accessible SVN repository that can be used
for reproducing this problem and testing possible solutions?  It is
hard to find repositories that fit your conditions these days.

Thanks.

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

On Sun, Nov 13, 2022 at 1:36 AM Eli Zaretskii <eliz@gnu.org> wrote:

> Can you suggest a publicly-accessible SVN repository that can be used
> for reproducing this problem and testing possible solutions?  It is
> hard to find repositories that fit your conditions these days.
>
> As you probably guessed, the repo I'm using is a corporate one and the
environment I'm using is an old version of Linux *because* of the corporate
product requirement.  One way would be to just create a dummy repo with
subversion and commit a large file inside it.  Aside from the current
corporate settings, I had never used Subversion.  Do you know where I could
push a test depot on a public repo site?


-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 1101 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

I just did another test with a different vc backend: the Mercurial
backend.
I created a local Mercurial repo and submitted a file that is not
completely rendered by vc-annotate with the SVN backend.
I modified the Mercurial hosted file, committing my changes in the
Mercurial repo.  Then I used vc-annotate on that Mercurial hosted file and
every line was annotated as it should be.

So it does seem to be specific to the SVN backend.


-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 603 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Andreas Schwab]

On Nov 13 2022, Pierre Rouleau wrote:

> So it does seem to be specific to the SVN backend.

It may be specific to the access method.  Does this also happen if the
SVN server is accessed locally?

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Eli Zaretskii]

> From: Pierre Rouleau <prouleau001@gmail.com>
> Date: Sun, 13 Nov 2022 09:51:22 -0500
> Cc: 59232@debbugs.gnu.org
> 
> On Sun, Nov 13, 2022 at 1:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
> 
>  Can you suggest a publicly-accessible SVN repository that can be used
>  for reproducing this problem and testing possible solutions?  It is
>  hard to find repositories that fit your conditions these days.
> 
> As you probably guessed, the repo I'm using is a corporate one and the environment I'm using is an old
> version of Linux *because* of the corporate product requirement.  One way would be to just create a dummy
> repo with subversion and commit a large file inside it.  Aside from the current corporate settings, I had never
> used Subversion.  Do you know where I could push a test depot on a public repo site?

No, I don't.  I hoped you could know of an already existing
repository.  To reproduce the problem, one doesn't need to commit
anything, one just needs to use an existing repository in a read-only
fashion.

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

On Sun, Nov 13, 2022 at 11:40 AM Eli Zaretskii <eliz@gnu.org> wrote:

> >
> > On Sun, Nov 13, 2022 at 1:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> >  Can you suggest a publicly-accessible SVN repository that can be used
> >  for reproducing this problem and testing possible solutions?  It is
> >  hard to find repositories that fit your conditions these days.
> >
> > As you probably guessed, the repo I'm using is a corporate one and the
> environment I'm using is an old
> > version of Linux *because* of the corporate product requirement.  One
> way would be to just create a dummy
> > repo with subversion and commit a large file inside it.  Aside from the
> current corporate settings, I had never
> > used Subversion.  Do you know where I could push a test depot on a
> public repo site?
>
> No, I don't.  I hoped you could know of an already existing
> repository.  To reproduce the problem, one doesn't need to commit
> anything, one just needs to use an existing repository in a read-only
> fashion.
>
I don't either, unfortunately.  Like most I normally do not use
Subversion.  I'm only using it because of this corporate requirement and if
it wasn't for Emacs I would royally dislike it.  Fortunately Emacs makes it
palatable.  I have seen mentions of https://www.springloops.io/ and
https://riouxsvn.com/ but I've never used those and before I did I'd like
to know if anyone has used them before.

-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 2051 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Stephen Berman]

On Sun, 13 Nov 2022 13:05:09 -0500 Pierre Rouleau <prouleau001@gmail.com> wrote:

> On Sun, Nov 13, 2022 at 11:40 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>  >
>  > On Sun, Nov 13, 2022 at 1:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
>  >
>  >  Can you suggest a publicly-accessible SVN repository that can be used
>  >  for reproducing this problem and testing possible solutions?  It is
>  >  hard to find repositories that fit your conditions these days.
>  >
>  > As you probably guessed, the repo I'm using is a corporate one and
>  > the environment I'm using is an old version of Linux *because* of
>  > the corporate product requirement.  One way would be to just create
>  > a dummy repo with subversion and commit a large file inside it.
>  > Aside from the current corporate settings, I had never used
>  > Subversion.  Do you know where I could push a test depot on a
>  > public repo site?
>
>  No, I don't.  I hoped you could know of an already existing
>  repository.  To reproduce the problem, one doesn't need to commit
>  anything, one just needs to use an existing repository in a read-only
>  fashion.
>
> I don't either, unfortunately.  Like most I normally do not use
> Subversion.  I'm only using it because of this corporate requirement
> and if it wasn't for Emacs I would royally dislike it.  Fortunately
> Emacs makes it palatable.  I have seen mentions of
> https://www.springloops.io/ and https://riouxsvn.com/ but I've never
> used those and before I did I'd like to know if anyone has used them
> before.

The R development source code is in this publicly accessible SVN
repository: https://svn.r-project.org/R/trunk/ but I cannot reproduce
the reported problem with `C-x v g' on a file checked out from that
repository with more than 6000 lines (> 195 kB) with any version of
Emacs from 26 through current master.

Steve Berman

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Sun, Nov 13, 2022 at 6:54 PM Stephen Berman <stephen.berman@gmx.net>
wrote:

> On Sun, 13 Nov 2022 13:05:09 -0500 Pierre Rouleau <prouleau001@gmail.com>
> wrote:
>
> > On Sun, Nov 13, 2022 at 11:40 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> >  > On Sun, Nov 13, 2022 at 1:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >  >
> >  >  Can you suggest a publicly-accessible SVN repository that can be used
> >  >  for reproducing this problem and testing possible solutions?  It is
> >  >  hard to find repositories that fit your conditions these days.
> >  >
>
> The R development source code is in this publicly accessible SVN
> repository: https://svn.r-project.org/R/trunk/ but I cannot reproduce
> the reported problem with `C-x v g' on a file checked out from that
> repository with more than 6000 lines (> 195 kB) with any version of
> Emacs from 26 through current master.
>
> Thanks Steve.
I created a working directory of that repo inside the same
environment (save version of SVN), same Emacs, same file system, everything)
that I use.

With that R repo I was able to annotate /trunk/configure.  All lines are
annotated.
I did not detect any problem with that repo.

But I still have the problem I reported with files inside the corporate
repo I use.

The difference is the underlying protocol used to access the remote repo.
- The R SVN repo is accessed via a https URL.
- I access the corporate repo using a svn+ssh:// URL.

Is it possible a slow access timing out or a low-level protocol problem
might
be the reason behind the failure?

Is there some setting I could change to test?

Thanks

-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 2574 bytes --]

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Andreas Schwab]

On Nov 12 2022, Pierre Rouleau wrote:

> If the file is small, all lines are annotated by vc-annotate.
> But when the file is large (over about 64KiB) then the end of the files
> are never annotated.

What does "never annotated" mean?  How does is look like instead?

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#59232: 27.2; vc-annotate on SVN does not process all lines

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 873 bytes --]

On Sun, Nov 13, 2022 at 3:12 AM Andreas Schwab <schwab@linux-m68k.org>
wrote:

>
> > If the file is small, all lines are annotated by vc-annotate.
> > But when the file is large (over about 64KiB) then the end of the files
> > are never annotated.
>
> What does "never annotated" mean?  How does is look like instead?
>

If the file stored in the repo has 3000 lines and you issue a svn blame on
that file on the command line, the output is 3000 lines.
Inside Emacs if you run (vc-annotate) on that file it would normally create
a buffer with 3000 lines inside it all of them being colored.  What I see
is for that 3000 line file the buffer contains something like 1000 lines
only: the first 1000 lines of the 3000 lines I was expecting to see.  And
those 1000 lines are all colored properly.  I just don't see any
information about the remaining 2000 lines.


-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 1330 bytes --]

bug#59232:

[Urban Engberg]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

I have the same problem in Emacs 29.1. I believe it is due to the svn
backend being called asynchronously, with large file annotations thereby
being cut off before they are done.

The following change solves the problem for me:

--- /home/ue/tmp/vc-svn.el.orig 2023-12-20 13:36:06.427678047 +0100
+++ /home/ue/tmp/vc-svn.el 2023-12-20 13:33:43.881904459 +0100
@@ -770,7 +770,7 @@
 ;; Support for `svn annotate'

 (defun vc-svn-annotate-command (file buf &optional rev)
-  (apply #'vc-svn-command buf 'async file "annotate"
+  (apply #'vc-svn-command buf nil file "annotate"
  (append (vc-switches 'svn 'annotate)
  (if rev (list (concat "-r" rev))))))



-- 
urban@engbergs.dk, 5679+MHJ Århus
<https://www.google.com/maps?q=9F8G5679%2BMHJ>

[-- Attachment #2: Type: text/html, Size: 1293 bytes --]

bug#59232:

[Dmitry Gutov]

On 20/12/2023 14:41, Urban Engberg wrote:
> I have the same problem in Emacs 29.1. I believe it is due to the svn 
> backend being called asynchronously, with large file annotations thereby 
> being cut off before they are done.
> 
> The following change solves the problem for me:
> 
>     --- /home/ue/tmp/vc-svn.el.orig 2023-12-20 13:36:06.427678047 +0100
>     +++ /home/ue/tmp/vc-svn.el 2023-12-20 13:33:43.881904459 +0100
>     @@ -770,7 +770,7 @@
>       ;; Support for `svn annotate'
> 
>       (defun vc-svn-annotate-command (file buf &optional rev)
>     -  (apply #'vc-svn-command buf 'async file "annotate"
>     +  (apply #'vc-svn-command buf nil file "annotate"
>        (append (vc-switches 'svn 'annotate)
>        (if rev (list (concat "-r" rev))))))

This kind of change is bound to make worse the apparent performance of 
vc-annotate with SVN. So it would be great to understand the issue of 
the problem first.

Is that a network error? Do you see the same problem with your corporate 
repo when calling the respective command in the command line?

Worst case, we could create a user option, to only be used in affected 
projects.

bug#59232:

[Urban Engberg]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

On Wed, 20 Dec 2023 at 16:50, Dmitry Gutov <dmitry@gutov.dev> wrote:

> This kind of change is bound to make worse the apparent performance of
> vc-annotate with SVN. So it would be great to understand the issue of
> the problem first.
>
> Is that a network error? Do you see the same problem with your corporate
> repo when calling the respective command in the command line?
>
> Worst case, we could create a user option, to only be used in affected
> projects.
>

I agree, absolutely. This was just how far I had been able to do my
analysis – and I thought it might be helpful to others diagnosing the
problem.

I have done some more debugging am able to get as far as this snippet (from
the function vc-do-command with arguments unfolded, when okstatus is
'async):

(let ((process-connection-type nil))
  (start-file-process
   "svn"
   (get-buffer "*Annotate Program.java (rev 23283)*")
   "svn"
   "--non-interactive" "annotate" "/home/.../Program.java"))

When I run this command directly in Emacs (using eval-defun), around the
first 100 lines of the svn output is written to the *annotate* buffer,
after which I get the output


Process svn exited abnormally with code 1

written to the buffer. If I set process-conection-type to t instead, the
process finishes without problems.

The svn command is run against an SSH-server inhouse and the
annotate-command takes around 1 second when run from my shell, and I
haven't seen it fail.

I run svn version 1.14.2.

If you have any ideas of what I should do from here, I'd be happy to assist.

-- 
urban@engbergs.dk, 5679+MHJ Århus
<https://www.google.com/maps?q=9F8G5679%2BMHJ>

[-- Attachment #2: Type: text/html, Size: 2952 bytes --]

bug#59232:

[Dmitry Gutov]

On 20/12/2023 19:38, Urban Engberg wrote:
> When I run this command directly in Emacs (using eval-defun), around the 
> first 100 lines of the svn output is written to the *annotate* buffer, 
> after which I get the output
> 
> 
>     Process svn exited abnormally with code 1
> 
> written to the buffer. If I set process-conection-type to t instead, the 
> process finishes without problems.
> 
> The svn command is run against an SSH-server inhouse and the 
> annotate-command takes around 1 second when run from my shell, and I 
> haven't seen it fail.

Maybe the command might fails when run in a non-interactive terminal?

You can try emulating the same circumstances in the shell by doing 
something like this:

   echo "" | git blame README | cat

(Of course, by replacing the command in the middle with the SVN command 
that actually runs.)

bug#59232:

[Urban Engberg]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

On Wed, 20 Dec 2023 at 19:35, Dmitry Gutov <dmitry@gutov.dev> wrote:

> Maybe the command might fails when run in a non-interactive terminal?
>
> You can try emulating the same circumstances in the shell by doing
> something like this:
>
>    echo "" | git blame README | cat
>
> (Of course, by replacing the command in the middle with the SVN command
> that actually runs.)
>

I thought about that too, namely as it does not fail when setting
process-conection-type to t. But no,

echo "" | svn --non-interactive annotate Program.java | cat

works just fine.

I shortened the failing statement down to

(let ((process-connection-type nil))
  (start-process
   "xxx"
   "*Test*"
   "svn"
   "annotate" "FILE"))

where FILE contains more than 100 lines – that also seems to be
significant, and using --non-interactive is not. But I am still not able to
figure out if it is the svn command itself that crashes, or it has
something to do with the process communication and Emacs. If it's the
first, it should be possible to find a way that this crashes as well when
run from the shell.

-- 
urban@engbergs.dk, 5679+MHJ Århus
<https://www.google.com/maps?q=9F8G5679%2BMHJ>

[-- Attachment #2: Type: text/html, Size: 2253 bytes --]

bug#59232:

[Dmitry Gutov]

On 20/12/2023 23:36, Urban Engberg wrote:
> I thought about that too, namely as it does not fail when setting  
> process-conection-type to t. But no,
> 
>     echo "" | svn --non-interactive annotate Program.java | cat
> 
> works just fine.

Oh well.

> I shortened the failing statement down to
> 
>     (let ((process-connection-type nil))
>        (start-process
>         "xxx"
>         "*Test*"
>         "svn"
>         "annotate" "FILE"))
> 
> where FILE contains more than 100 lines – that also seems to be 
> significant, and using --non-interactive is not. But I am still not able 
> to figure out if it is the svn command itself that crashes, or it has 
> something to do with the process communication and Emacs. If it's the 
> first, it should be possible to find a way that this crashes as well 
> when run from the shell.

When this ends in failure, is there something at the end of the buffer 
*Test* that looks like stderr output? The process might not just stop 
and return status 1, but it could print something usable as well.

Also, just to be able to separate stderr output more easily, you could 
use 'make-process' instead of 'start-process' because it allows to 
specify a separate buffer for errors.

There are also some options outlined for trying to get more verbose 
output of it here -- 
https://stackoverflow.com/questions/8416989/is-it-possible-to-get-svn-client-debug-output 
-- but it seems like this might only work with some client versions. And 
most answers are 5-10 years old.

bug#59232:

[Urban Engberg]

[-- Attachment #1: Type: text/plain, Size: 1881 bytes --]

On Wed, 20 Dec 2023 at 23:03, Dmitry Gutov <dmitry@gutov.dev> wrote:

> When this ends in failure, is there something at the end of the buffer
> *Test* that looks like stderr output? The process might not just stop
> and return status 1, but it could print something usable as well.
>

Unfortunately no, the end just looks like this:

  23153         fm
Process xxx<3> exited abnormally with code 1



> Also, just to be able to separate stderr output more easily, you could
> use 'make-process' instead of 'start-process' because it allows to
> specify a separate buffer for errors.
>

Using make-process:

(let ((process-connection-type nil))
  (make-process
   :name "xxx"
   :buffer "*Test*"
   :command (list "svn" "annotate" "FILE")))


This fails, just like before. Interestingly, adding

:stderr "*Stderr*"


to the argument list makes the command **not** fail and *Stderr* thus just
contains "Process xxx stderr finished"

There are also some options outlined for trying to get more verbose
> output of it here --
>
> https://stackoverflow.com/questions/8416989/is-it-possible-to-get-svn-client-debug-output
> -- but it seems like this might only work with some client versions. And
> most answers are 5-10 years old.
>

No, I don't get much more from that. But perhaps interesting as well, I
gave the "svn annotate" a "-v" to generate more verbose output. It seems
this makes it output the full date on each line of the output. With this
option, the process is terminated after just 56 lines, or around 4900
characters – close to what we got before. Could it in some way be that the
pipe into the output buffer is closed down prematurely? As again, it
doesn't seem that the svn process itself fails, when run in any other way?

-- 
urban@engbergs.dk, 5679+MHJ Århus
<https://www.google.com/maps?q=9F8G5679%2BMHJ>

[-- Attachment #2: Type: text/html, Size: 3910 bytes --]

bug#59232:

[Dmitry Gutov]

On 21/12/2023 00:50, Urban Engberg wrote:

> Using make-process:
> 
>     (let ((process-connection-type nil))
>        (make-process
>         :name "xxx"
>         :buffer "*Test*"
>         :command (list "svn" "annotate" "FILE")))
> 
> This fails, just like before. Interestingly, adding
> 
>     :stderr "*Stderr*"
> 
> 
> to the argument list makes the command */not*/ fail and *Stderr* thus 
> just contains "Process xxx stderr finished"

Hmm, then perhaps it might make sense to test this full patch. Ideally, 
though, someone knowledgeable about our subprocess system would chime in 
about the whole situation: are there programs like this, and should we 
work around that.

diff --git a/lisp/vc/vc-dispatcher.el b/lisp/vc/vc-dispatcher.el
index fd5f655a0f6..18ba317242b 100644
--- a/lisp/vc/vc-dispatcher.el
+++ b/lisp/vc/vc-dispatcher.el
@@ -379,9 +379,12 @@ vc-do-command
  	    (if (eq okstatus 'async)
  	        ;; Run asynchronously.
  	        (let ((proc
-		       (let ((process-connection-type nil))
-		         (apply #'start-file-process command
-                                (current-buffer) command squeezed))))
+                       (make-process
+                        :name "vc"
+                        :command (cons command squeezed)
+                        :connection-type 'pipe
+                        :buffer (current-buffer)
+                        :stderr " *vc-errors*")))
  		  (when vc-command-messages
  		    (let ((inhibit-message vc-inhibit-message))
  		      (message "Running in background: %s"


>     There are also some options outlined for trying to get more verbose
>     output of it here --
>     https://stackoverflow.com/questions/8416989/is-it-possible-to-get-svn-client-debug-output <https://stackoverflow.com/questions/8416989/is-it-possible-to-get-svn-client-debug-output>
>     -- but it seems like this might only work with some client versions.
>     And
>     most answers are 5-10 years old.
> 
> 
> No, I don't get much more from that. But perhaps interesting as well, I 
> gave the "svn annotate" a "-v" to generate more verbose output. It seems 
> this makes it output the full date on each line of the output. With this 
> option, the process is terminated after just 56 lines, or around 4900 
> characters – close to what we got before. Could it in some way be that 
> the pipe into the output buffer is closed down prematurely?

That the pipe is closed prematurely, but only with SVN and not any other 
VCS client? That seems odd, it likely involved some particular factors. 
Well, aside from the fact that 'svn' inevitably accesses the network.

> As again, it 
> doesn't seem that the svn process itself fails, when run in any other way?

Sure, it is weird.

Other ways you could try to run it are:

- From Emacs's 'M-x shell' buffer.
- Through a shell wrapper that redirects stderr somewhere, but then 
appends it at the end of the output when the main program finishes.
- Through the ':term' terminal emulator in Vim 8.1+? Just being 
thorough, I have no idea how it is implemented.