dired-move-to-filename-regexp

dired-move-to-filename-regexp

[Lars Hansen]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

Can you explain the reason for this change? (lisp/ChangeLog.8):

1999-02-16  Paul Eggert  <eggert@twinsun.com>
* dired.el (dired-move-to-filename-regexp): Prepend .* so that we
find the last match if there are multiple matches.

It has the consequence that file names starting with a date, as in the 
attached example, are not correctly identified. Removing .* fixes the 
problem, but I don't want to remove it when I don't know why it was added.

[-- Attachment #2: dired --]
[-- Type: text/plain, Size: 426 bytes --]

  drwxr--r--   2 lh       root        16384 Jul 19 19:02 v0103
  -rwxr--r--   1 lh       root        27648 Jul 19 19:00 2004-08-01 Forbedringsregnskab.xls
  -rwxr--r--   1 lh       root        38912 Jul 31 11:39 2004-08-01 Salgsaftale.doc
  -rwxr--r--   1 lh       root        34304 Jun 29 13:41 2004-08-01 Tillæg til Salgsaftale.doc
  -rwxr--r--   1 lh       root        63401 Jul 11 18:45 El-installations eftersyn.pdf

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: dired-move-to-filename-regexp

[Paul Eggert]

Lars Hansen <larsh@math.ku.dk> writes:

> Can you explain the reason for this change? (lisp/ChangeLog.8):
>
> 1999-02-16  Paul Eggert  <eggert@twinsun.com>
> * dired.el (dired-move-to-filename-regexp): Prepend .* so that we
> find the last match if there are multiple matches.

It was because some combinations of link-count, user and group name
and/or number, and sizes looked like dates, and this broke Emacs
dired.

> It has the consequence that file names starting with a date, as in the
> attached example, are not correctly identified.

Yes.  There's no way in general to solve the problem if you stick with
ordinary "ls" output, as it is textually ambiguous if you are trying
to parse all the "ls" versions out there.  The best we can do is to
use heuristics.  (Currently one of the heuristics is, "Please don't
use file names that start with strings that look like dates and are
followed by spaces."  :-)

I thought RMS had recently implemented support for GNU "ls -lD" in CVS
Emacs.  That should solve the problem in general when GNU "ls" is
being used, since "ls -lD" exactly identifies where each file name is.
The problems you're running into would then occur only with non-GNU
"ls", for which the obvious solution is to install GNU "ls".

Re: dired-move-to-filename-regexp

[Richard Stallman]

    I thought RMS had recently implemented support for GNU "ls -lD" in CVS
    Emacs.  That should solve the problem in general when GNU "ls" is
    being used, since "ls -lD" exactly identifies where each file name is.
    The problems you're running into would then occur only with non-GNU
    "ls", for which the obvious solution is to install GNU "ls".

Yes, the latest Dired code knows how to use --dired.
However, it does this only on the GNU system:

  (defvar dired-use-ls-dired (not (not (string-match "gnu" system-configuration)))

If there is a reliable, trouble-free and quick way to check
whether the ls program supports --dired, that could be used instead.

Re: dired-move-to-filename-regexp

[Paul Eggert]

Richard Stallman <rms@gnu.org> writes:

> If there is a reliable, trouble-free and quick way to check
> whether the ls program supports --dired, that could be used instead.

Why not have Emacs try "ls -l --dired" first, and, if that doesn't
work, fall back on "ls -l" without --dired?  The --dired usage would
be considered to "work" only if the output ended in the required
//DIRED// and //DIRED-OPTIONS// lines in the proper forms.

This procedure should be reliable in practice.

Re: dired-move-to-filename-regexp

[Miles Bader]

On Sun, Sep 19, 2004 at 09:49:15PM -0700, Paul Eggert wrote:
> > If there is a reliable, trouble-free and quick way to check
> > whether the ls program supports --dired, that could be used instead.
> 
> Why not have Emacs try "ls -l --dired" first, and, if that doesn't
> work, fall back on "ls -l" without --dired?  The --dired usage would
> be considered to "work" only if the output ended in the required
> //DIRED// and //DIRED-OPTIONS// lines in the proper forms.
> 
> This procedure should be reliable in practice.

BTW, an annoying thing asbout recent versions of ls with emacs is that when
you rename a file in dired, it calls `ls' with just that file to regenerate
its dired line, but since ls only seems the oneo entry, it shrinks all the
numeric/name fields to their minimum size, so the updated line doesn't match
the rest of the output.

One way this might be fixed would be to have --dired output a "spec" for the
output it generates (e.g., a list of field widths it used), and then also
have an option to ls to use a given "spec" to set the fields widths (which
emacs would then pass in from the previous --dired output).

-Miles
-- 
Ich bin ein Virus. Mach' mit und kopiere mich in Deine .signature.

Re: dired-move-to-filename-regexp

[Lars Hansen]

Paul Eggert wrote:

>It was because some combinations of link-count, user and group name
>and/or number, and sizes looked like dates, and this broke Emacs
>dired.
>  
>
Yes, I now see it is documented with an example in a comment in 
dired-move-to-filename-regexp.
But doesn't the ".*" make the "[0-9][kKMGTPEZY]?" superfluous?

Re: dired-move-to-filename-regexp

[Paul Eggert]

Lars Hansen <larsh@math.ku.dk> writes:

> Yes, I now see it is documented with an example in a comment in
> dired-move-to-filename-regexp.
> But doesn't the ".*" make the "[0-9][kKMGTPEZY]?" superfluous?

No, because the "[0-9][kKMGTPEZY]?" forces the date to be preceded by
something that looks like a size.  Without it, there would be more
false matches of dates within file names, e.g., the file name
"foo bar 1 2000 baz" would contain a "date" that looks like "bar 1 2000".

Re: dired-move-to-filename-regexp

[Richard Stallman]

    Why not have Emacs try "ls -l --dired" first, and, if that doesn't
    work, fall back on "ls -l" without --dired?

Running ls can take substantial time, so a failing call to ls is
somewhat unpleasant.  If it happens just the first time, it could be
ok.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    BTW, an annoying thing asbout recent versions of ls with emacs is that when
    you rename a file in dired, it calls `ls' with just that file to regenerate
    its dired line, but since ls only seems the oneo entry, it shrinks all the
    numeric/name fields to their minimum size, so the updated line doesn't match
    the rest of the output.

I recall we discussed this, but I don't recall if we solved the
problem.  Did we decide on any solution?

Re: dired-move-to-filename-regexp

[Paul Eggert]

Richard Stallman <rms@gnu.org> writes:

> I recall we discussed this, but I don't recall if we solved the
> problem.  Did we decide on any solution?

I don't recall anyone designing anything specific, but here are some
thoughts.  Current "ls -l --dired" outputs something like this:

     drwxr-xr-x  22 root root 4096 2004-03-10 13:07 .
   //DIRED// 49 50
   //DIRED-OPTIONS// --quoting-style=literal

We could design a new ls option called "--listing=FORMAT", where
FORMAT specifies the layout of each "ls" line, much as a strftime
format specifies the layout of the output of the "date" command.
Combinations of existing -l, --time-style etc. options would be
equivalent to some particular FORMAT.  The FORMAT could specify the
exact number of columns to use for each datum.  //DIRED-OPTIONS//
would specify the --listing option that would cause "ls" to generate
lines in the same format (including the same column widths).  Existing
versions of "ls" do not put this stuff into the //DIRED-OPTIONS// line
and Emacs would therefore know not to use a --listing option when
doing incremental redisplay unless "ls" is sufficiently new.

There are some issues I haven't thought through.  What if the updated
listing needs wider columns than the old?  How can arbitrary
characters be encoded into FORMAT, and how do they get expressed in
the //DIRED-OPTIONS// line?  To some extent these are Emacs questions
more than "ls" questions.  There's also the issue of how --listing
interacts with the zoo of other "ls" options; this is mostly an "ls"
question.

PS.  I'm CC'ing this to bug-coreutils as it's "ls"-relevant.
bug-coreutils readers: you're joining a thread that began at:
http://lists.gnu.org/archive/html/emacs-devel/2004-09/msg00632.html

Re: dired-move-to-filename-regexp

[Richard Stallman]

I think a better solution is if we can turn off the new "feature"
of adapting the column widths to the data.  It is simple and clean.
Both this and your suggestion would only work with GNU ls, so in that
regard neither one has an advantage.

Meyering, what do you think?

Re: dired-move-to-filename-regexp

[Stefan Monnier]

> I think a better solution is if we can turn off the new "feature"
> of adapting the column widths to the data.  It is simple and clean.
> Both this and your suggestion would only work with GNU ls, so in that
> regard neither one has an advantage.

Another solution is to assume that the current buffer and the one new line
have used the same ls options and contain the same number of space-separated
fields.  Based on this, you can add spaces so as to align the fields in the
new line to the field of the other lines.


        Stefan

Re: dired-move-to-filename-regexp

[Paul Eggert]

Richard Stallman <rms@gnu.org> writes:

> I think a better solution is if we can turn off the new "feature"
> of adapting the column widths to the data.

One way would be to extend --format to allow fairly arbitrary formats
that can specify column widths.  For example, Emacs could do this:

  ls --format='%11m %2l %8o %8g %7s %12d %f'

to have "ls" list the mode, link, owner, group, size, date, and file
name in the traditional widths for "ls -l".  This would resemble the
way "ls --time-style" already handles date formats.

Another possibility is to have a new option to disable the new
feature, e.g. "ls --traditional-column-widths".  This would be a bit
easier to implement, but would be less general and might have problems
of its own (e.g., users and implementations don't agree on what the
traditional widths are).

I understand that some non-GNU ls implementations also adapt column
widths to data, so the Emacs problem would remain with these
implementations, regardless of how we change GNU ls.

Re: dired-move-to-filename-regexp

[Lars Hansen]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Richard Stallman wrote:

>Yes, the latest Dired code knows how to use --dired.
>However, it does this only on the GNU system:
>  
>
If one uses ls-lisp, the following patch should solve the problems.
Do you agree?


[-- Attachment #2: ls-lisp.diff --]
[-- Type: text/plain, Size: 545 bytes --]

*** ls-lisp.el.~1.53.~	Sat Dec 27 19:12:20 2003
--- ls-lisp.el	Thu Sep 23 20:46:33 2004
***************
*** 534,540 ****
  	    " "
  	    (ls-lisp-format-time file-attr time-index now)
  	    " "
! 	    file-name
  	    (if (stringp file-type)	; is a symbolic link
  		(concat " -> " file-type))
  	    "\n"
--- 534,540 ----
  	    " "
  	    (ls-lisp-format-time file-attr time-index now)
  	    " "
! 	    (propertize file-name 'dired-filename t)
  	    (if (stringp file-type)	; is a symbolic link
  		(concat " -> " file-type))
  	    "\n"

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: dired-move-to-filename-regexp

[Miles Bader]

On Thu, Sep 23, 2004 at 12:45:31PM -0400, Richard Stallman wrote:
> I think a better solution is if we can turn off the new "feature"
> of adapting the column widths to the data.  It is simple and clean.

It maybe a bit less work, but it's certainly not "better" -- without the
auto-adjustment feature, it "simply" would use absurdly large amounts of
whitespace _just in case_ a file happens to be hundreds of gigabytes in size
and have thousands of links.  I'd rather reserve the limited space available
in my dired windows for filenames (which overflow often enough even with
column compression).

-Miles
-- 
Come now, if we were really planning to harm you, would we be waiting here, 
 beside the path, in the very darkest part of the forest?

Re: dired-move-to-filename-regexp

[Jim Meyering]

> I think a better solution is if we can turn off the new "feature"
> of adapting the column widths to the data.  It is simple and clean.
> Both this and your suggestion would only work with GNU ls, so in that
> regard neither one has an advantage.
>
> Meyering, what do you think?

Ideally ls would accept a new --format=FMT option that would
work like find's printf format string.  Then it'd be easy to
add a new option to make ls use a format string with fixed
widths to restore the old behavior.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    Another solution is to assume that the current buffer and the one new line
    have used the same ls options and contain the same number of space-separated
    fields.  Based on this, you can add spaces so as to align the fields in the
    new line to the field of the other lines.

That would be rather unreliable, since it would need to figure out the
number of spaces between fields in both the new output and the old
buffer.  The gains that we got from --dired would be lost.
I won't accept this approach.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    > I think a better solution is if we can turn off the new "feature"
    > of adapting the column widths to the data.

    One way would be to extend --format to allow fairly arbitrary formats
    that can specify column widths.  For example, Emacs could do this:

      ls --format='%11m %2l %8o %8g %7s %12d %f'

Sure, this would do the job, but I think it would be extra work and
add no real benefit.  However, if Meyering wants to do it, I won't
object to it.

    Another possibility is to have a new option to disable the new
    feature, e.g. "ls --traditional-column-widths".  This would be a bit
    easier to implement, but would be less general

We don't need extra generality just for generality's sake.

						   and might have problems
    of its own (e.g., users and implementations don't agree on what the
    traditional widths are).

This problem is not real.  For Dired to work right when GNU ls is
installed, we need only that GNU ls be consistent with itself when
--traditional-column-widths is used.

    I understand that some non-GNU ls implementations also adapt column
    widths to data, so the Emacs problem would remain with these
    implementations, regardless of how we change GNU ls.

We can suggest installing GNU ls.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    If one uses ls-lisp, the following patch should solve the problems.
    Do you agree?

It looks right to me.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    It maybe a bit less work, but it's certainly not "better" -- without the
    auto-adjustment feature, it "simply" would use absurdly large amounts of
    whitespace _just in case_ a file happens to be hundreds of gigabytes in size
    and have thousands of links.

If ls does what ls used to do, it won't be worse than it was.

Re: dired-move-to-filename-regexp

[Stefan]

>     Another solution is to assume that the current buffer and the one new
>     line have used the same ls options and contain the same number of
>     space-separated fields.  Based on this, you can add spaces so as to
>     align the fields in the new line to the field of the other lines.

> That would be rather unreliable, since it would need to figure out the
> number of spaces between fields in both the new output and the old
> buffer.  The gains that we got from --dired would be lost.
> I won't accept this approach.

Of course, the filename part would be determined just as it is now (using
the --dired data if available, for example).


        Stefan

Re: dired-move-to-filename-regexp

[Richard Stallman]

    Ideally ls would accept a new --format=FMT option that would
    work like find's printf format string.  Then it'd be easy to
    add a new option to make ls use a format string with fixed
    widths to restore the old behavior.

Would you like to implement that?
If that's the solution you prefer, it will certainly work.

Re: dired-move-to-filename-regexp

[Richard Stallman]

    > That would be rather unreliable, since it would need to figure out the
    > number of spaces between fields in both the new output and the old
    > buffer.  The gains that we got from --dired would be lost.
    > I won't accept this approach.

    Of course, the filename part would be determined just as it is now (using
    the --dired data if available, for example).

Ok, you're right about that part.  When using --dired, we would
still reliably find the filename.  But finding where to insert extra
spaces would not be entirely reliable, just as finding the file name
was not entirely reliable in the past.

Re: dired-move-to-filename-regexp

[Stefan]

>> That would be rather unreliable, since it would need to figure out the
>> number of spaces between fields in both the new output and the old
>> buffer.  The gains that we got from --dired would be lost.
>> I won't accept this approach.

>     Of course, the filename part would be determined just as it is now (using
>     the --dired data if available, for example).

> Ok, you're right about that part.  When using --dired, we would
> still reliably find the filename.  But finding where to insert extra
> spaces would not be entirely reliable, just as finding the file name
> was not entirely reliable in the past.

See below a sample patch that "worked for me".
Indeed, finding the places where to insert spaces is not entirely reliable,
but adding spaces as is done below shouldn't make things worse.
There are 2 cases I can think of where the code could pose problem:
1 - the dired-move-to-filename-regexp needs a specific number of spaces
    at some spot.  This is a rare problem but can happen when we don't
    use --dired.  Examples are the nunber of spaces between day and month,
    which is typically either one or 2 spaces, never more.
2 - If dired-move-to-filename initially gets it wrong (and goes too far),
    we may end up adding spaces in the actual file name.
    This is not as bad as it sounds:
    - It's extremely unlikely because we only add spaces "to align
      filenames" (once the filename is aligned to neighboring filenames, we
      don't add any spaces any more), so if dired-move-to-filename goes too
      far, it's very unlikely that we'll add spaces because it's most likely
      going to be already further than neighboring filenames.
    - It's a situation in which even without adding spaces, dired is all
      confused, so it's not clear we'll make things worse.

Number 1 is actually solved in the code by being paranoid and checking each
time we insert spaces that the outcome of dired-move-to-filename has not
been affected (and reverting the change if it did).

Number 2 could be reduced by trying to make sure that such misidentifications
don't happen.  This could be done e.g. by checking
(file-exists-p (dired-get-filename)).  The code below does not bother to do
that, but it can be easily added.


        Stefan


PS: I rarely use dired myself, so this is very lightly tested.


--- orig/lisp/dired.el
+++ mod/lisp/dired.el
@@ -798,6 +798,98 @@
 	(dired-insert-directory dir dired-actual-switches
 				file-list (not file-list) t)))))
 
+(defun dired-align-file (beg end)
+  "Align the fields of a file to the ones of surrounding lines.
+BEG..END is the line where the file info is located."
+  ;; Some versions of ls try to adjust the size of each field so as to just
+  ;; hold the largest element ("largest" in the current invocation, of
+  ;; course).  So when a single line is output, the size of each field is
+  ;; just big enough for that one output.  Thus when dired refreshes one
+  ;; line, the alignment if this line w.r.t the rest is messed up because
+  ;; the fields of that one line will generally be smaller.
+  ;;
+  ;; To work around this problem, we here add spaces to try and re-align the
+  ;; fields as needed.  Since this is purely aesthetic, it is of utmost
+  ;; importance that it doesn't mess up anything like
+  ;; `dired-move-to-filename'.  To this end, we limit ourselves to adding
+  ;; spaces only, and to only add them at places where there was already at
+  ;; least one space.  This way, as long as `dired-move-to-filename-regexp'
+  ;; always matches spaces with "*" or "+", we know we haven't made anything
+  ;; worse.  There is one spot where the exact number of spaces is
+  ;; important, which is just before the actual filename, so we refrain from
+  ;; adding spaces there (and within the filename as well, of course).
+  (save-excursion
+    (let (file file-col other other-col)
+      ;; Check the there is indeed a file, and that there is anoter adjacent
+      ;; file with which to align, and that additional spaces are needed to
+      ;; align the filenames.
+      (when (and (setq file (progn (goto-char beg)
+				   (dired-move-to-filename nil end)))
+		 (setq file-col (current-column))
+		 (setq other
+		       (or (progn (goto-char beg) (forward-line -1)
+				  (dired-move-to-filename))
+			   (progn (goto-char beg) (forward-line 1)
+				  (dired-move-to-filename))))
+		 (setq other-col (current-column))
+		 ;; Make sure there is some work left to do.
+		 (> other-col file-col))
+	;; Keep positions uptodate when we insert stuff.
+	(if (> other file) (setq other (copy-marker other)))
+	(setq file (copy-marker file))
+	;; Main loop.
+	(goto-char beg)
+	(while (and (> other-col file-col)
+		    (skip-chars-forward "^ ")
+		    ;; Skip the spaces, and make sure there's at least one.
+		    (> (skip-chars-forward " ") 0)
+		    ;; Don't touch anything just before (and after) the
+		    ;; beginning of the filename.
+		    (> file (point)))
+	  ;; We're now just in front of a field, with a space behind us.
+	  (let* ((curcol (current-column))
+		 ;; Nums are right-aligned.
+		 (num-align (looking-at "[0-9]"))
+		 ;; Let's look at the other line, in the same column: we
+		 ;; should be either near the end of the previous field, or
+		 ;; in the space between that field and the next.
+		 ;; [ Of course, it's also possible that we're already within
+		 ;; the next field or even past it, but that's highly
+		 ;; unlikely since other-col > file-col. ]
+		 ;; Let's find the distance to the alignment-point (either
+		 ;; the beginning or the end of the next field, depending on
+		 ;; whether this field is left or right aligned).
+		 (align-pt-offset
+		  (save-excursion
+		    (goto-char other)
+		    (move-to-column curcol)
+		    (when (looking-at
+			   (concat
+			    (if (eq (char-before) ?\ ) " *" "[^ ]* *")
+			    (if num-align "[0-9][^ ]*")))
+		      (- (match-end 0) (match-beginning 0)))))
+		 ;; Now, the number of spaces to insert is align-pt-offset
+		 ;; minus the distance to the equivalent point on the
+		 ;; current line.
+		 (spaces
+		  (if (not num-align)
+		      align-pt-offset
+		    (and align-pt-offset
+			 (save-excursion
+			   (skip-chars-forward "^ ")
+			   (- align-pt-offset (- (current-column) curcol)))))))
+	    (when (and spaces (> spaces 0))
+	      (setq file-col (+ spaces file-col))
+	      (if (> file-col other-col)
+		  (setq spaces (- spaces (- file-col other-col))))
+	      (insert-char ?\s spaces)
+	      ;; Let's just make really sure we did not mess up.
+	      (save-excursion
+		(unless (and (dired-move-to-filename) (= (point) file))
+		  ;; Damn!  We messed up: let's revert the change.
+		  (delete-char (- spaces)))))))))))
+			 
+
 (defun dired-insert-directory (dir switches &optional file-list wildcard hdr)
   "Insert a directory listing of DIR, Dired style.
 Use SWITCHES to make the listings.
@@ -816,7 +909,10 @@
     ;; with the new value of dired-move-to-filename-regexp.
     (if file-list
 	(dolist (f file-list)
-	  (insert-directory f switches nil nil))
+	  (let ((beg (point)))
+	    (insert-directory f switches nil nil)
+	    ;; Re-align fields, if necessary.
+	    (dired-align-file beg (point))))
       (insert-directory dir switches wildcard (not wildcard)))
     ;; Quote certain characters, unless ls quoted them for us.
     (if (not (string-match "b" dired-actual-switches))

Re: dired-move-to-filename-regexp

[Richard Stallman]

Ok, you've convinced me.  If no one else sees a problem,
please install your patch.

Re: dired-move-to-filename-regexp

[Miles Bader]

Richard Stallman <rms@gnu.org> writes:
>     It maybe a bit less work, but it's certainly not "better" -- without the
>     auto-adjustment feature, it "simply" would use absurdly large amounts of
>     whitespace _just in case_ a file happens to be hundreds of gigabytes in size
>     and have thousands of links.
>
> If ls does what ls used to do, it won't be worse than it was.

Actually I think it's been getting worse recently because of the advent
of large file systems, etc -- you can no longer assume that 8 or 9
characters is enough for the file's size.

Morever, it was never that good: the excess whitespace caused annoyance
for me even in the old days.  The new flexibility of ls is generally a
_win_ from a user's point of view, it's only slightly unusual
applications of it such as emacs' dired updating-after-rename in which
it's not.  I like dired, and I'd like it to be as user-friendly as
possible.

However, the argument seems moot, as Stefan's patch would appear to
solve the problem.  Yay!  Thanks Stefan...

-Miles
-- 
((lambda (x) (list x x)) (lambda (x) (list x x)))

Re: dired-move-to-filename-regexp

[Stefan]

> Ok, you've convinced me.  If no one else sees a problem,
> please install your patch.

I've installed it.  But as mentioned, this is lightly tested.
Please people who use dired, beat on it (whether or not your `ls' does
funny realignment: it's important that the new code shouldn't make things
worse in *any* case).
The code is used only when refreshing a few specific lines (typically just
one), so it's not used that often.


        Stefan