Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[T. V. Raman]

In vc-hooks.el, vc-handled-backends
is presently declared to handle a large number of backends:

(defcustom vc-handled-backends '(RCS CVS SVN SCCS Arch MCVS)

The last two --arch and MCVS have a particularly bad performance
penalty on systems that have NFS mounts in places other than /net
or /afs --- se variable vc-ignore-dir-regexp

Unless one discovers the above variable (vc-ignore-dir-regexp is
declared as a defvar, and not as defcustom)
nfs based systems take a heavy performance hit --- since function
vc-find-root recursively ascends the directory tree in attempting
to check if files are under version control.

Suggestions:

A) Possibly make vc-handled-backends more conservative by pruning
        it to have RCS CVS and SVN on it by default.

B)      Make vc-ignore-dir-regexp a customizable option via
defcustom,
        and add a pointer to it in the docstring for
        vc-handled-backends so that users discover its purpose

        In my case I chased it down by examining the source code
        after observing that emacs was making excessive nfs
        accesses during file load and save.

-- 
Best Regards,
--raman

      
Email:  raman@users.sf.net
WWW:    http://emacspeak.sf.net/raman/
AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
Google: tv+raman 
IRC:    irc://irc.freenode.net/#emacs

Re: Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[Stefan Monnier]

> The last two --arch and MCVS have a particularly bad performance
> penalty on systems that have NFS mounts in places other than /net
> or /afs --- se variable vc-ignore-dir-regexp

> Unless one discovers the above variable (vc-ignore-dir-regexp is
> declared as a defvar, and not as defcustom)
> nfs based systems take a heavy performance hit --- since function
> vc-find-root recursively ascends the directory tree in attempting
> to check if files are under version control.

> Suggestions:

> A) Possibly make vc-handled-backends more conservative by pruning
>         it to have RCS CVS and SVN on it by default.

> B)      Make vc-ignore-dir-regexp a customizable option via
> defcustom,
>         and add a pointer to it in the docstring for
>         vc-handled-backends so that users discover its purpose

I think B is a good idea in general.  Especially since vc-ignore-dir-regexp
is useful not just for NFS mounts but for any other circumstance where the
user wants to disable VC in some part of his file-system.

I think A might be a good idea as well.  MetaCVS is a rather obscure
revision control system.  GNU Arch is less so, but it's not that popular
either.  And there are several more still unsupported but whose support
should hopefully come at some point (Mercury, bzr, git, DaRCS, monotone,
codeville, OpenCM, you-name-it).  So we most likely will not want *all*
supported systems to be "active".  Maybe now is not the worst time to start
down that road.

>         In my case I chased it down by examining the source code after
>         observing that emacs was making excessive nfs accesses during file
>         load and save.

Just to make sure that it can only be fixed with something like
vc-ignore-dir-regexp, could you describe the particular circumstance where
this happens, as well as describe as precisely as possible what
actually happens?


        Stefan

Re: Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[T. V. Raman]

Stephane --
Here is what caused me to chase down the problem.

Consider an environment where your home dir (and all users home
dirs) come from nfs and are visible as /home/user

Also assume that stat calls on nfs dirs  is slow.

I found that everytime I opened a file or saved a buffer I saw
slowness. If the nfs filers were having a bad day then things in
emacs got really really bad.

I also observed by tailing /var/log/messages that the machine was
waiting on nfs responses for stat calls on /home/MCVS and
/home/{arch} --- neither of which existed --- but that still
meant you waited for the nfs system to say so.

After grepping around for {arch} and MCVS in the elisp sources I
saw that they showed up in VC -- I then toggled debug on quit
(which is what I should have done int he first place)--
saw where emacs was blocked, and the rest was easy --- since the
call in vc-find-root was the one that was initiating the stat
call on /home/MCVS and /home{arch} and since the code indicated
that the cleanest way to turn it off (clean == closest to point
of problem) was vc-ignore-dir-regexp that was the variable I
ended up setting.

Hope this helps--

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:
    >> The last two --arch and MCVS have a particularly bad
    >> performance penalty on systems that have NFS mounts in
    >> places other than /net or /afs --- se variable
    >> vc-ignore-dir-regexp
    Stefan> 
    >> Unless one discovers the above variable
    >> (vc-ignore-dir-regexp is declared as a defvar, and not as
    >> defcustom) nfs based systems take a heavy performance hit
    >> --- since function vc-find-root recursively ascends the
    >> directory tree in attempting to check if files are under
    >> version control.
    Stefan> 
    >> Suggestions:
    Stefan> 
    >> A) Possibly make vc-handled-backends more conservative by
    >> pruning it to have RCS CVS and SVN on it by default.
    Stefan> 
    >> B) Make vc-ignore-dir-regexp a customizable option via
    >> defcustom, and add a pointer to it in the docstring for
    >> vc-handled-backends so that users discover its purpose
    Stefan> 
    Stefan> I think B is a good idea in general.  Especially
    Stefan> since vc-ignore-dir-regexp is useful not just for NFS
    Stefan> mounts but for any other circumstance where the user
    Stefan> wants to disable VC in some part of his file-system.
    Stefan> 
    Stefan> I think A might be a good idea as well.  MetaCVS is a
    Stefan> rather obscure revision control system.  GNU Arch is
    Stefan> less so, but it's not that popular either.  And there
    Stefan> are several more still unsupported but whose support
    Stefan> should hopefully come at some point (Mercury, bzr,
    Stefan> git, DaRCS, monotone, codeville, OpenCM,
    Stefan> you-name-it).  So we most likely will not want *all*
    Stefan> supported systems to be "active".  Maybe now is not
    Stefan> the worst time to start down that road.
    Stefan> 
    >> In my case I chased it down by examining the source code
    >> after observing that emacs was making excessive nfs
    >> accesses during file load and save.
    Stefan> 
    Stefan> Just to make sure that it can only be fixed with
    Stefan> something like vc-ignore-dir-regexp, could you
    Stefan> describe the particular circumstance where this
    Stefan> happens, as well as describe as precisely as possible
    Stefan> what actually happens?
    Stefan> 
    Stefan> 
    Stefan>         Stefan

-- 
Best Regards,
--raman

      
Email:  raman@users.sf.net
WWW:    http://emacspeak.sf.net/raman/
AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
Google: tv+raman 
IRC:    irc://irc.freenode.net/#emacs

Re: Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[Stefan Monnier]

> Consider an environment where your home dir (and all users home
> dirs) come from nfs and are visible as /home/user

Hmm... now that I look at the code I see that the optimization I had put for
this common case is actually missing.  Can you try the patch below, see if
it helps?


        Stefan


--- orig/lisp/vc-hooks.el
+++ mod/lisp/vc-hooks.el
@@ -52,11 +52,13 @@
 (defvar vc-header-alist ())
 (make-obsolete-variable 'vc-header-alist 'vc-BACKEND-header)
 
-(defvar vc-ignore-dir-regexp "\\`\\([\\/][\\/]\\|/net/\\|/afs/\\)\\'"
- "Regexp matching directory names that are not under VC's control.
+(defcustom vc-ignore-dir-regexp "\\`\\([\\/][\\/]\\|/net/\\|/afs/\\)\\'"
+  "Regexp matching directory names that are not under VC's control.
 The default regexp prevents fruitless and time-consuming attempts
 to determine the VC status in directories in which filenames are
-interpreted as hostnames.")
+interpreted as hostnames."
+  :type 'regexp
+  :group 'vc)
 
 (defcustom vc-handled-backends '(RCS CVS SVN SCCS Arch MCVS)
   ;; Arch and MCVS come last because they are per-tree rather than per-dir.
@@ -308,6 +310,9 @@
   "Find the root of a checked out project.
 The function walks up the directory tree from FILE looking for WITNESS.
 If WITNESS if not found, return nil, otherwise return the root."
+  ;; Represent /home/luser/foo as ~/foo so that we don't try to look for
+  ;; witnesses in /home or in /.
+  (setq file (abbreviate-file-name file))
   (let ((root nil))
     (while (not (or root
                    (equal file (setq file (file-name-directory file)))

Re: Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[T. V. Raman]

Your patch appears to work. However it is still more efficient in
my environment to prune out arch and mcvs --- 

>>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:
    >> Consider an environment where your home dir (and all users
    >> home dirs) come from nfs and are visible as /home/user
    Stefan> 
    Stefan> Hmm... now that I look at the code I see that the
    Stefan> optimization I had put for this common case is
    Stefan> actually missing.  Can you try the patch below, see
    Stefan> if it helps?
    Stefan> 
    Stefan> 
    Stefan>         Stefan
    Stefan> 
    Stefan> 
    Stefan> --- orig/lisp/vc-hooks.el +++ mod/lisp/vc-hooks.el @@
    Stefan> -52,11 +52,13 @@ (defvar vc-header-alist ())
    Stefan> (make-obsolete-variable 'vc-header-alist
    Stefan> 'vc-BACKEND-header)
    Stefan>  
    Stefan> -(defvar vc-ignore-dir-regexp
    Stefan> "\\`\\([\\/][\\/]\\|/net/\\|/afs/\\)\\'" - "Regexp
    Stefan> matching directory names that are not under VC's
    Stefan> control.  +(defcustom vc-ignore-dir-regexp
    Stefan> "\\`\\([\\/][\\/]\\|/net/\\|/afs/\\)\\'" + "Regexp
    Stefan> matching directory names that are not under VC's
    Stefan> control.  The default regexp prevents fruitless and
    Stefan> time-consuming attempts to determine the VC status in
    Stefan> directories in which filenames are -interpreted as
    Stefan> hostnames.")  +interpreted as hostnames."  + :type
    Stefan> 'regexp + :group 'vc)
    Stefan>  
    Stefan>  (defcustom vc-handled-backends '(RCS CVS SVN SCCS
    Stefan> Arch MCVS) ;; Arch and MCVS come last because they
    Stefan> are per-tree rather than per-dir.  @@ -308,6 +310,9
    Stefan> @@ "Find the root of a checked out project.  The
    Stefan> function walks up the directory tree from FILE
    Stefan> looking for WITNESS.  If WITNESS if not found, return
    Stefan> nil, otherwise return the root."  + ;; Represent
    Stefan> /home/luser/foo as ~/foo so that we don't try to look
    Stefan> for + ;; witnesses in /home or in /.  + (setq file
    Stefan> (abbreviate-file-name file)) (let ((root nil)) (while
    Stefan> (not (or root (equal file (setq file
    Stefan> (file-name-directory file)))
    Stefan> 
    Stefan> 
    Stefan> _______________________________________________
    Stefan> Emacs-devel mailing list Emacs-devel@gnu.org
    Stefan> http://lists.gnu.org/mailman/listinfo/emacs-devel

-- 
Best Regards,
--raman

      
Email:  raman@users.sf.net
WWW:    http://emacspeak.sf.net/raman/
AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
Google: tv+raman 
IRC:    irc://irc.freenode.net/#emacs

Re: Suggestion: vc-hooks.el: Make vc-handled-backends more conservative?

[Stefan Monnier]

> Your patch appears to work.

Thanks for testing it.  Installed.

> However it is still more efficient in
> my environment to prune out arch and mcvs ---

Of course: I was not suggesting it as an alternative, just as a bugfix
needed e.g. for those people who need Arch to be activated.


        Stefan