* Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) @ 2009-07-05 21:01 Mathias Dahl 2009-07-05 22:58 ` Chong Yidong 2009-07-06 3:06 ` Eli Zaretskii 0 siblings, 2 replies; 49+ messages in thread From: Mathias Dahl @ 2009-07-05 21:01 UTC (permalink / raw) To: emacs-devel I understand that the release of 23.1 is not far away and I am a bit worried that this bug won't be solved before that. Is there anything I can do to help? I use a lot of UNC file names / paths in my work and the current slowness is very annoying (both opening files and listing them in Dired). As this was a problem in the past too it isn't unreasonable to believe that the bug has reappeared by mistake when fixing some other thing (could it be related to TRAMP?). I am willing to go bug hunting as long as it's in elisp-land and would like to get some hints on where to start digging. For details see: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=3542 Thanks! /Mathias ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-05 21:01 Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) Mathias Dahl @ 2009-07-05 22:58 ` Chong Yidong 2009-07-06 14:30 ` Mathias Dahl 2009-07-06 3:06 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Chong Yidong @ 2009-07-05 22:58 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel Mathias Dahl <mathias.dahl@gmail.com> writes: > I understand that the release of 23.1 is not far away and I am a bit > worried that this bug won't be solved before that. Is there anything I > can do to help? I use a lot of UNC file names / paths in my work and > the current slowness is very annoying (both opening files and listing > them in Dired). As this was a problem in the past too it isn't > unreasonable to believe that the bug has reappeared by mistake when > fixing some other thing (could it be related to TRAMP?). If you have the time, try reverting to previous versions (e.g. `cvs up -Pd -D 2009-01-01') and find out which checkin introduced the problem. A binary search should narrow it down quite quickly. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-05 22:58 ` Chong Yidong @ 2009-07-06 14:30 ` Mathias Dahl 2009-07-06 14:55 ` Chong Yidong 0 siblings, 1 reply; 49+ messages in thread From: Mathias Dahl @ 2009-07-06 14:30 UTC (permalink / raw) To: Chong Yidong; +Cc: emacs-devel > If you have the time, try reverting to previous versions (e.g. `cvs up > -Pd -D 2009-01-01') and find out which checkin introduced the problem. > A binary search should narrow it down quite quickly. I don't compile it myself, instead I rely on the precompiled pretests over at alpha.gnu.org. But maybe I will find the time to download them all and test, to narrow it down at least somewhat. Considering the results of my Dired experiments, does it suggest the problem is in some low level file listing function someplace? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-06 14:30 ` Mathias Dahl @ 2009-07-06 14:55 ` Chong Yidong 2009-07-07 11:00 ` Mathias Dahl 0 siblings, 1 reply; 49+ messages in thread From: Chong Yidong @ 2009-07-06 14:55 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel Mathias Dahl <mathias.dahl@gmail.com> writes: > I don't compile it myself, instead I rely on the precompiled pretests > over at alpha.gnu.org. But maybe I will find the time to download them > all and test, to narrow it down at least somewhat. Please use CVS. That lets you track down the problem with better granularity, and it's more convenient. See http://savannah.gnu.org/cvs/?group=emacs for a description of how to check out the cvs repository. After that, as I said, the command to revert to earlier revisions is (e.g.) `cvs up -Pd -D 2009-01-01'. > Considering the results of my Dired experiments, does it suggest the > problem is in some low level file listing function someplace? No idea, sorry. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-06 14:55 ` Chong Yidong @ 2009-07-07 11:00 ` Mathias Dahl 2009-07-07 11:14 ` Miles Bader 0 siblings, 1 reply; 49+ messages in thread From: Mathias Dahl @ 2009-07-07 11:00 UTC (permalink / raw) To: Chong Yidong; +Cc: emacs-devel > Please use CVS. That lets you track down the problem with better > granularity, and it's more convenient. See > http://savannah.gnu.org/cvs/?group=emacs for a description of how to > check out the cvs repository. After that, as I said, the command to > revert to earlier revisions is (e.g.) `cvs up -Pd -D 2009-01-01'. Just so that I understand you correctly, are you suggesting I download and compile CVS emacs and do the "cvs up" trick and repeat this process until I get a compiled version that works? If so, it might take a while since I have never compiled Emacs myself under w32. Maybe I will try to follow the guide Lennart has provided with me, we'll see. Debugging elisp had been much more fun :) Is there some low level primitive built in C that is always involved in file listings etc that could be the culprit? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-07 11:00 ` Mathias Dahl @ 2009-07-07 11:14 ` Miles Bader 0 siblings, 0 replies; 49+ messages in thread From: Miles Bader @ 2009-07-07 11:14 UTC (permalink / raw) To: Mathias Dahl; +Cc: Chong Yidong, emacs-devel Mathias Dahl <mathias.dahl@gmail.com> writes: > Just so that I understand you correctly, are you suggesting I download > and compile CVS emacs and do the "cvs up" trick and repeat this > process until I get a compiled version that works? If so, it might > take a while since I have never compiled Emacs myself under w32. Maybe > I will try to follow the guide Lennart has provided with me, we'll > see. Debugging elisp had been much more fun :) It should be much easier (and faster) if you use the git mirror and "git bisect". [e.g., "git clone git://git.sv.gnu.org/emacs.git"] -Miles -- Monday, n. In Christian countries, the day after the baseball game. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-05 21:01 Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) Mathias Dahl 2009-07-05 22:58 ` Chong Yidong @ 2009-07-06 3:06 ` Eli Zaretskii 2009-07-06 7:38 ` Mathias Dahl 1 sibling, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-06 3:06 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel > From: Mathias Dahl <mathias.dahl@gmail.com> > Date: Sun, 5 Jul 2009 23:01:53 +0200 > > I understand that the release of 23.1 is not far away and I am a bit > worried that this bug won't be solved before that. Is there anything I > can do to help? I use a lot of UNC file names / paths in my work and > the current slowness is very annoying (both opening files and listing > them in Dired). As this was a problem in the past too it isn't > unreasonable to believe that the bug has reappeared by mistake when > fixing some other thing (could it be related to TRAMP?). > > I am willing to go bug hunting as long as it's in elisp-land and would > like to get some hints on where to start digging. Please post some measurements (the original report says "a few seconds", which is quite vague) and the basic performance data of your machine. Also, please tell if this happens with every file, even one visited in Fundamental mode, or only with some files. I will then try to reproduce this. Best I remember, I didn't have such problems on my machine, but maybe I misremember. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-06 3:06 ` Eli Zaretskii @ 2009-07-06 7:38 ` Mathias Dahl 2009-07-06 20:01 ` Eli Zaretskii 0 siblings, 1 reply; 49+ messages in thread From: Mathias Dahl @ 2009-07-06 7:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel > Please post some measurements (the original report says "a few > seconds", which is quite vague) and the basic performance data of your > machine. Also, please tell if this happens with every file, even one > visited in Fundamental mode, or only with some files. I will then try > to reproduce this. Best I remember, I didn't have such problems on my > machine, but maybe I misremember. I did all tests below in emacs -Q. //gbgfs1/archive75 is a share that contains 260 directories and three files. The server is located in our office. C-x d //gbgfs1/archive75 RET Emacs 22.1.1: 1 s Emacs 23.0.95.1: 95 s So, that is about 95 times slower... Opening a subfolder in dired in the directory listing I got in the above operation. The subfolder opened contains only one folder. Emacs 22.1.1: immediate (under 1 s) Emacs 23.0.95.1: 3 s Opening a file, 2470 bytes, using RET in dired and in a directory further down in the structure above: Emacs 22.1.1: immediate (under 1 s) Emacs 23.0.95.1: 3 s I opened a file that was about 20 times larger than the above, from the same folder, and it took about the same time, so it seems that the actual reading of the file contents is not affected. Opening a folder with about 270 files even further down in the same structure: Emacs 22.1.1: 2-3 s Emacs 23.0.95.1: 86 s This seems to happen for any type of file I open. My PC has a Pentium 4 CPU, running at 3 GHz, with 1.5 GBG of RAM, according to My Computer -> RMB -> Properties -> General and I am using Windows XP Professional, Version 2002, Service Pack 2. Any tests in particular you want me to run? /Mathias ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-06 7:38 ` Mathias Dahl @ 2009-07-06 20:01 ` Eli Zaretskii 2009-07-08 15:01 ` Eli Zaretskii 0 siblings, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-06 20:01 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel > From: Mathias Dahl <mathias.dahl@gmail.com> > Date: Mon, 6 Jul 2009 09:38:07 +0200 > Cc: emacs-devel@gnu.org > > //gbgfs1/archive75 is a share that contains 260 directories and three > files. The server is located in our office. > > C-x d //gbgfs1/archive75 RET > > Emacs 22.1.1: 1 s > Emacs 23.0.95.1: 95 s > > So, that is about 95 times slower... Ouch! I'm quite sure I would have noticed such a terrible slowdown. Let me time similar commands tomorrow in my office. > Any tests in particular you want me to run? Not yet. But perhaps install StraceNT (http://stracent.en.softonic.com/) and see what are the differences between Emacs 22 and Emacs 23. Maybe you will even be able to see the system call(s) that take most of this extra time. TIA ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-06 20:01 ` Eli Zaretskii @ 2009-07-08 15:01 ` Eli Zaretskii 2009-07-08 20:47 ` Mathias Dahl 0 siblings, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-08 15:01 UTC (permalink / raw) To: mathias.dahl, emacs-devel > Date: Mon, 06 Jul 2009 23:01:19 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: emacs-devel@gnu.org > > > From: Mathias Dahl <mathias.dahl@gmail.com> > > Date: Mon, 6 Jul 2009 09:38:07 +0200 > > Cc: emacs-devel@gnu.org > > > > //gbgfs1/archive75 is a share that contains 260 directories and three > > files. The server is located in our office. > > > > C-x d //gbgfs1/archive75 RET > > > > Emacs 22.1.1: 1 s > > Emacs 23.0.95.1: 95 s > > > > So, that is about 95 times slower... > > Ouch! I'm quite sure I would have noticed such a terrible slowdown. > > Let me time similar commands tomorrow in my office. Sorry for a long delay. I finally found a few minutes to look into this. First, I don't see a ~100-fold slowdown, I see only a 10-fold slowdown (vs Emacs 22.3). This is with a directory that has ~130 subdirectories. I have a hypothesis for the reason of this slowdown, but I need a few more facts from you to make sure the slowdown I see is caused by the same factors as what you see: . Do you see significant difference in speed with cold and hot cache? That is, if you run Dired on the above directory, then kill the Dired buffer and immediately run Dired again on the same directory, what times do you see then in Emacs 23? . Does it help to set w32-get-true-file-attributes to nil? If the previous test shows a significant speedup with hot cache, please try this with a cold cache, which may mean rebooting the machine where you are trying this, or maybe waiting long enough for the cache to "cool down" and in addition restarting Emacs (to get rid of Emacs's internal caching). . If w32-get-true-file-attributes does have a significant effect, please tell how many different user names and group names you see in the listing presented by "C-x d". Thanks in advance. > > Any tests in particular you want me to run? > > Not yet. But perhaps install StraceNT > (http://stracent.en.softonic.com/) and see what are the differences > between Emacs 22 and Emacs 23. Maybe you will even be able to see the > system call(s) that take most of this extra time. Did you have a chance to do this? If so, can you post the results? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-08 15:01 ` Eli Zaretskii @ 2009-07-08 20:47 ` Mathias Dahl 2009-07-09 11:37 ` Jason Rumney 2009-07-09 12:56 ` Eli Zaretskii 0 siblings, 2 replies; 49+ messages in thread From: Mathias Dahl @ 2009-07-08 20:47 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel > First, I don't see a ~100-fold slowdown, I see only a 10-fold slowdown > (vs Emacs 22.3). This is with a directory that has ~130 > subdirectories. The more files there are, the bigger the slowdown seems to be, in my tests. > . Do you see significant difference in speed with cold and hot cache? > That is, if you run Dired on the above directory, then kill the > Dired buffer and immediately run Dired again on the same directory, > what times do you see then in Emacs 23? Hehe, it got even worse the second time :) > . Does it help to set w32-get-true-file-attributes to nil? Woooo! Um, yes... It helped all right. From 85 (first try) and 115 (second try as per reuest above), down to 1,4 s. Shall we call that as success? :) > previous test shows a significant speedup with hot cache, please > try this with a cold cache, which may mean rebooting the machine > where you are trying this, or maybe waiting long enough for the > cache to "cool down" and in addition restarting Emacs (to get rid > of Emacs's internal caching). I assume I don't need to test this now. If I do, ask me again. > . If w32-get-true-file-attributes does have a significant effect, > please tell how many different user names and group names you see > in the listing presented by "C-x d". It looks like this: //gbgfs1/archive75: total used in directory 8728 available 8198544 drwxrwxrwx 1 Administrators Domain Users 0 1970-01-01 . dr-xr-xr-x 1 mdahse Domain Users 0 1970-01-01 .. drwxrwxrwx 1 demoruntime Domain Users 0 2008-12-05 _ConfigBuild drwxrwxrwx 1 demoruntime Domain Users 0 2008-06-09 acadc ... and then all other lines after that has the same user and group names. The one that differs above (mdahse) is my own username. >> Not yet. But perhaps install StraceNT >> (http://stracent.en.softonic.com/) and see what are the differences >> between Emacs 22 and Emacs 23. Maybe you will even be able to see the >> system call(s) that take most of this extra time. > > Did you have a chance to do this? If so, can you post the results? Not yet, no. Is it still interesting? /Mathias ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-08 20:47 ` Mathias Dahl @ 2009-07-09 11:37 ` Jason Rumney 2009-07-09 11:53 ` Mathias Dahl 2009-07-09 12:56 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Jason Rumney @ 2009-07-09 11:37 UTC (permalink / raw) To: Mathias Dahl; +Cc: Eli Zaretskii, emacs-devel Mathias Dahl wrote: >> . Does it help to set w32-get-true-file-attributes to nil? >> > > Woooo! Um, yes... It helped all right. From 85 (first try) and 115 > (second try as per reuest above), down to 1,4 s. Shall we call that as > success? :) > Did you have it set to t in your .emacs, or does the default of 'local not work for you? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 11:37 ` Jason Rumney @ 2009-07-09 11:53 ` Mathias Dahl 2009-07-09 16:11 ` Drew Adams 2009-07-09 19:13 ` Eli Zaretskii 0 siblings, 2 replies; 49+ messages in thread From: Mathias Dahl @ 2009-07-09 11:53 UTC (permalink / raw) To: Jason Rumney; +Cc: Eli Zaretskii, emacs-devel >>> . Does it help to set w32-get-true-file-attributes to nil? >>> >> >> Woooo! Um, yes... It helped all right. From 85 (first try) and 115 >> (second try as per reuest above), down to 1,4 s. Shall we call that as >> success? :) >> > > Did you have it set to t in your .emacs, or does the default of 'local not work for you? I did not have it set so it had the value 'local before I changed it to nil now. I googled for the variable now and saw this: "If the variable w32-get-true-file-attributes is non-nil (the default), Emacs tries to determine the accurate link counts for files. This option is only useful on NTFS volumes, and it *considerably slows down Dired and other features*, so use it only on fast machines." Hmm... There is a suggestion to use it only on fast machines at the same time it is turned on by default. Shouldn't it be the other way around? Btw, I consider my machine to be quite fast even though it is a couple of years old. I suggest we change the default value of this variable to nil (or make the code faster). /Mathias ^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 11:53 ` Mathias Dahl @ 2009-07-09 16:11 ` Drew Adams 2009-07-09 16:25 ` Jason Rumney 2009-07-09 19:11 ` Eli Zaretskii 2009-07-09 19:13 ` Eli Zaretskii 1 sibling, 2 replies; 49+ messages in thread From: Drew Adams @ 2009-07-09 16:11 UTC (permalink / raw) To: 'Mathias Dahl', 'Jason Rumney' Cc: 'Eli Zaretskii', emacs-devel > "If the variable w32-get-true-file-attributes is non-nil (the > default), Emacs tries to determine the accurate link counts for files. > This option is only useful on NTFS volumes, and it *considerably slows > down Dired and other features*, so use it only on fast machines." > > Hmm... There is a suggestion to use it only on fast machines at the > same time it is turned on by default. Shouldn't it be the other way > around? Btw, I consider my machine to be quite fast even though it is > a couple of years old. > > I suggest we change the default value of this variable to nil (or make > the code faster). Dumb question: Is there a way for Emacs to know whether the format is NTFS or FAT(32)? If so, then Emacs could use nil for FAT volumes. (If not, it seems like nil would be the best default value.) ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 16:11 ` Drew Adams @ 2009-07-09 16:25 ` Jason Rumney 2009-07-09 17:03 ` Drew Adams 2009-07-09 18:47 ` Eli Zaretskii 2009-07-09 19:11 ` Eli Zaretskii 1 sibling, 2 replies; 49+ messages in thread From: Jason Rumney @ 2009-07-09 16:25 UTC (permalink / raw) To: Drew Adams; +Cc: 'Eli Zaretskii', emacs-devel, 'Mathias Dahl' Drew Adams wrote: >> Hmm... There is a suggestion to use it only on fast machines at the >> same time it is turned on by default. Shouldn't it be the other way >> around? Btw, I consider my machine to be quite fast even though it is >> a couple of years old. >> >> I suggest we change the default value of this variable to nil (or make >> the code faster). >> > > Dumb question: Is there a way for Emacs to know whether the format is NTFS or > FAT(32)? If so, then Emacs could use nil for FAT volumes. > That suggestion dates back to the days when "fast machines" were running at 200MHz or so. The problem here is not the speed of the machine, but the network. The default value of w32-get-true-file-attributes is 'local, which means nil when going across the network, but apparently that is not being recognized in all places - which I think is what Eli is investigating. ^ permalink raw reply [flat|nested] 49+ messages in thread
* RE: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 16:25 ` Jason Rumney @ 2009-07-09 17:03 ` Drew Adams 2009-07-09 18:59 ` Eli Zaretskii 2009-07-09 18:47 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Drew Adams @ 2009-07-09 17:03 UTC (permalink / raw) To: 'Jason Rumney' Cc: 'Eli Zaretskii', emacs-devel, 'Mathias Dahl' > > Dumb question: Is there a way for Emacs to know whether the > > format is NTFS or FAT(32)? If so, then Emacs could use nil > > for FAT volumes. > > That suggestion dates back to the days when "fast machines" > were running at 200MHz or so. The problem here is not the > speed of the machine, but the network. The default value of > w32-get-true-file-attributes is 'local, which means nil when > going across the network, but apparently that is not being > recognized in all places - which I think is what Eli > is investigating. I see (I think). I thought that the doc was saying that it should be non-nil only for NTFS, which should be independent of whether the file system is local or over a network. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 17:03 ` Drew Adams @ 2009-07-09 18:59 ` Eli Zaretskii 0 siblings, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-09 18:59 UTC (permalink / raw) To: Drew Adams; +Cc: mathias.dahl, emacs-devel, jasonr > From: "Drew Adams" <drew.adams@oracle.com> > Cc: "'Mathias Dahl'" <mathias.dahl@gmail.com>, > "'Eli Zaretskii'" <eliz@gnu.org>, <emacs-devel@gnu.org> > Date: Thu, 9 Jul 2009 10:03:42 -0700 > > > > Dumb question: Is there a way for Emacs to know whether the > > > format is NTFS or FAT(32)? If so, then Emacs could use nil > > > for FAT volumes. > > > > That suggestion dates back to the days when "fast machines" > > were running at 200MHz or so. The problem here is not the > > speed of the machine, but the network. The default value of > > w32-get-true-file-attributes is 'local, which means nil when > > going across the network, but apparently that is not being > > recognized in all places - which I think is what Eli > > is investigating. > > I see (I think). I thought that the doc was saying that it should be non-nil > only for NTFS, which should be independent of whether the file system is local > or over a network. Network drives can be NTFS volumes as well, and usually are if the server is a Windows machine. But accessing all of the features we now support in file-attributes on Windows can be slow over the network, especially if the network is busy. I made the doc string less categorical, thanks for pointing that out. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 16:25 ` Jason Rumney 2009-07-09 17:03 ` Drew Adams @ 2009-07-09 18:47 ` Eli Zaretskii 2009-07-09 21:33 ` Chong Yidong 2009-07-11 19:17 ` Stefan Monnier 1 sibling, 2 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-09 18:47 UTC (permalink / raw) To: Jason Rumney; +Cc: emacs-devel, drew.adams, mathias.dahl > Date: Fri, 10 Jul 2009 00:25:25 +0800 > From: Jason Rumney <jasonr@gnu.org> > CC: 'Mathias Dahl' <mathias.dahl@gmail.com>, > 'Eli Zaretskii' <eliz@gnu.org>, > emacs-devel@gnu.org > > Drew Adams wrote: > >> Hmm... There is a suggestion to use it only on fast machines at the > >> same time it is turned on by default. Shouldn't it be the other way > >> around? Btw, I consider my machine to be quite fast even though it is > >> a couple of years old. > >> > >> I suggest we change the default value of this variable to nil (or make > >> the code faster). > >> > > > > Dumb question: Is there a way for Emacs to know whether the format is NTFS or > > FAT(32)? If so, then Emacs could use nil for FAT volumes. > > > That suggestion dates back to the days when "fast machines" were running > at 200MHz or so. The problem here is not the speed of the machine, but > the network. The default value of w32-get-true-file-attributes is > 'local, which means nil when going across the network, but apparently > that is not being recognized in all places - which I think is what Eli > is investigating. Right. We didn't treat UNC file names as remote, it's as simple as that. I installed the patch below on the trunk. Stefan and Yidong, is it okay to install on the release branch as well? 2009-07-09 Eli Zaretskii <eliz@gnu.org> * w32.c (stat): Treat UNC file names as residing on remote drives. (Bug#3542) --- src/w32.c.orig 2009-06-21 10:38:18.000000000 +0300 +++ src/w32.c 2009-07-09 16:31:51.250000000 +0300 @@ -3154,11 +3154,13 @@ } } - /* GetDriveType needs the root directory of NAME's drive. */ - if (!(strlen (name) >= 2 && IS_DEVICE_SEP (name[1]))) - devtype = GetDriveType (NULL); /* use root of current diectory */ + if (IS_DIRECTORY_SEP (name[0]) && IS_DIRECTORY_SEP (name[1])) + devtype = DRIVE_REMOTE; /* assume UNC name is remote */ + else if (!(strlen (name) >= 2 && IS_DEVICE_SEP (name[1]))) + devtype = GetDriveType (NULL); /* use root of current drive */ else { + /* GetDriveType needs the root directory of NAME's drive. */ strncpy (drive_root, name, 3); drive_root[3] = '\0'; devtype = GetDriveType (drive_root); ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 18:47 ` Eli Zaretskii @ 2009-07-09 21:33 ` Chong Yidong 2009-07-10 8:54 ` Eli Zaretskii 2009-07-11 19:17 ` Stefan Monnier 1 sibling, 1 reply; 49+ messages in thread From: Chong Yidong @ 2009-07-09 21:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: mathias.dahl, emacs-devel, drew.adams, Jason Rumney Eli Zaretskii <eliz@gnu.org> writes: > Right. We didn't treat UNC file names as remote, it's as simple as > that. > > I installed the patch below on the trunk. > > Stefan and Yidong, is it okay to install on the release branch as > well? > > 2009-07-09 Eli Zaretskii <eliz@gnu.org> > > * w32.c (stat): Treat UNC file names as residing on remote > drives. (Bug#3542) Looks OK to me. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 21:33 ` Chong Yidong @ 2009-07-10 8:54 ` Eli Zaretskii 0 siblings, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-10 8:54 UTC (permalink / raw) To: Chong Yidong; +Cc: mathias.dahl, emacs-devel, drew.adams, jasonr > From: Chong Yidong <cyd@stupidchicken.com> > Cc: Jason Rumney <jasonr@gnu.org>, emacs-devel@gnu.org, drew.adams@oracle.com, > mathias.dahl@gmail.com > Date: Thu, 09 Jul 2009 17:33:51 -0400 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Right. We didn't treat UNC file names as remote, it's as simple as > > that. > > > > I installed the patch below on the trunk. > > > > Stefan and Yidong, is it okay to install on the release branch as > > well? > > > > 2009-07-09 Eli Zaretskii <eliz@gnu.org> > > > > * w32.c (stat): Treat UNC file names as residing on remote > > drives. (Bug#3542) > > Looks OK to me. Thanks, committed to the branch as well. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 18:47 ` Eli Zaretskii 2009-07-09 21:33 ` Chong Yidong @ 2009-07-11 19:17 ` Stefan Monnier 2009-07-11 20:22 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Stefan Monnier @ 2009-07-11 19:17 UTC (permalink / raw) To: Eli Zaretskii; +Cc: mathias.dahl, emacs-devel, drew.adams, Jason Rumney > Stefan and Yidong, is it okay to install on the release branch as > well? Fine by me as well. BTW, could you factor out your "remoteness test" and make it available to file-remote-p? Stefan ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-11 19:17 ` Stefan Monnier @ 2009-07-11 20:22 ` Eli Zaretskii 2009-07-13 12:17 ` Stefan Monnier 2009-07-13 13:57 ` Andreas Schwab 0 siblings, 2 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-11 20:22 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel, jasonr, drew.adams, mathias.dahl > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Jason Rumney <jasonr@gnu.org>, emacs-devel@gnu.org, drew.adams@oracle.com, mathias.dahl@gmail.com > Date: Sat, 11 Jul 2009 15:17:29 -0400 > > > Stefan and Yidong, is it okay to install on the release branch as > > well? > > Fine by me as well. Thanks. > BTW, could you factor out your "remoteness test" and make it available > to file-remote-p? Already done by today's commits to the trunk. The function is called is_slow_fs, and returns non-zero if its argument resides on a filesystem deemed slow. I'm not sure this is what you want for file-remote-p. Perhaps you only want files on remote (a.k.a. networked) filesystems. There's code to do that as well (in w32.c:logon_network_drive), so if you want, I can make it a separate routine. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-11 20:22 ` Eli Zaretskii @ 2009-07-13 12:17 ` Stefan Monnier 2009-07-13 13:38 ` Andreas Schwab 2009-07-13 18:56 ` Eli Zaretskii 2009-07-13 13:57 ` Andreas Schwab 1 sibling, 2 replies; 49+ messages in thread From: Stefan Monnier @ 2009-07-13 12:17 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, jasonr, drew.adams, mathias.dahl > Already done by today's commits to the trunk. The function is called > is_slow_fs, and returns non-zero if its argument resides on a > filesystem deemed slow. > I'm not sure this is what you want for file-remote-p. Perhaps you > only want files on remote (a.k.a. networked) filesystems. There's I'm not sure I understand the difference. Could you give an example of a filesystem that's slow but not "remote (a.k.a. networked)"? Stefan PS: the criterion for file-remote-p is (C-h f file-remote-p): "...A file is considered "remote" if accessing it is likely to be slower or less reliable than accessing local files..." ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 12:17 ` Stefan Monnier @ 2009-07-13 13:38 ` Andreas Schwab 2009-07-13 19:00 ` Eli Zaretskii 2009-07-13 18:56 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Andreas Schwab @ 2009-07-13 13:38 UTC (permalink / raw) To: Stefan Monnier Cc: mathias.dahl, Eli Zaretskii, jasonr, drew.adams, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > I'm not sure I understand the difference. Could you give an example of > a filesystem that's slow but not "remote (a.k.a. networked)"? Filesystems aren't slow, devices are. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 13:38 ` Andreas Schwab @ 2009-07-13 19:00 ` Eli Zaretskii 0 siblings, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-13 19:00 UTC (permalink / raw) To: Andreas Schwab; +Cc: mathias.dahl, jasonr, monnier, drew.adams, emacs-devel > From: Andreas Schwab <schwab@linux-m68k.org> > Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org, jasonr@gnu.org, > drew.adams@oracle.com, mathias.dahl@gmail.com > Date: Mon, 13 Jul 2009 15:38:02 +0200 > > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > > I'm not sure I understand the difference. Could you give an example of > > a filesystem that's slow but not "remote (a.k.a. networked)"? > > Filesystems aren't slow, devices are. Well, filesystems _shouldn't_ be slow, though some are ;-) But yes, I agree that ``filesystem'' is a kind of misnomer, although some filesystems are associated with very specific devices. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 12:17 ` Stefan Monnier 2009-07-13 13:38 ` Andreas Schwab @ 2009-07-13 18:56 ` Eli Zaretskii 2009-07-14 0:51 ` Stefan Monnier 1 sibling, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-13 18:56 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel, jasonr, drew.adams, mathias.dahl > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: mathias.dahl@gmail.com, emacs-devel@gnu.org, drew.adams@oracle.com, jasonr@gnu.org > Date: Mon, 13 Jul 2009 08:17:08 -0400 > > > Already done by today's commits to the trunk. The function is called > > is_slow_fs, and returns non-zero if its argument resides on a > > filesystem deemed slow. > > I'm not sure this is what you want for file-remote-p. Perhaps you > > only want files on remote (a.k.a. networked) filesystems. There's > > I'm not sure I understand the difference. Could you give an example of > a filesystem that's slow but not "remote (a.k.a. networked)"? The code in is_slow_fs considers only fixed disks and RAM disks not ``slow''. Other types, which include CDs and removable media such as USB memory sticks, are considered ``slow''. > PS: the criterion for file-remote-p is (C-h f file-remote-p): > "...A file is considered "remote" if accessing it is likely to be slower or > less reliable than accessing local files..." Right, but I remember a prolonged discussion about that which IIRC ended in controversy. That's why I asked. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 18:56 ` Eli Zaretskii @ 2009-07-14 0:51 ` Stefan Monnier 0 siblings, 0 replies; 49+ messages in thread From: Stefan Monnier @ 2009-07-14 0:51 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, jasonr, drew.adams, mathias.dahl >> > Already done by today's commits to the trunk. The function is called >> > is_slow_fs, and returns non-zero if its argument resides on a >> > filesystem deemed slow. >> > I'm not sure this is what you want for file-remote-p. Perhaps you >> > only want files on remote (a.k.a. networked) filesystems. There's >> >> I'm not sure I understand the difference. Could you give an example of >> a filesystem that's slow but not "remote (a.k.a. networked)"? > The code in is_slow_fs considers only fixed disks and RAM disks not > ``slow''. Other types, which include CDs and removable media such as > USB memory sticks, are considered ``slow''. Hmm... removable media... I think it should still be considered as non-remote, although accessing a USB stick after you removed it (but without unmounting it) may hang just like accessing a dead NFS server. Stefan ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-11 20:22 ` Eli Zaretskii 2009-07-13 12:17 ` Stefan Monnier @ 2009-07-13 13:57 ` Andreas Schwab 2009-07-13 19:02 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Andreas Schwab @ 2009-07-13 13:57 UTC (permalink / raw) To: Eli Zaretskii Cc: mathias.dahl, jasonr, Stefan Monnier, drew.adams, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Already done by today's commits to the trunk. The function is called > is_slow_fs, and returns non-zero if its argument resides on a > filesystem deemed slow. Btw, Lisp strings are guaranteed to be NUL terminated, no need to create a copy to use is as a C string. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 13:57 ` Andreas Schwab @ 2009-07-13 19:02 ` Eli Zaretskii 2009-07-13 19:39 ` Andreas Schwab 0 siblings, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-13 19:02 UTC (permalink / raw) To: Andreas Schwab; +Cc: emacs-devel > From: Andreas Schwab <schwab@linux-m68k.org> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org, > jasonr@gnu.org, drew.adams@oracle.com, mathias.dahl@gmail.com > Date: Mon, 13 Jul 2009 15:57:48 +0200 > > Btw, Lisp strings are guaranteed to be NUL terminated, no need to create > a copy to use is as a C string. Really? I will have to look at the code. Did I dream or at some point in the past they weren't guaranteed to be null-terminated? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 19:02 ` Eli Zaretskii @ 2009-07-13 19:39 ` Andreas Schwab 2009-07-13 20:13 ` Eli Zaretskii 0 siblings, 1 reply; 49+ messages in thread From: Andreas Schwab @ 2009-07-13 19:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andreas Schwab <schwab@linux-m68k.org> >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org, >> jasonr@gnu.org, drew.adams@oracle.com, mathias.dahl@gmail.com >> Date: Mon, 13 Jul 2009 15:57:48 +0200 >> >> Btw, Lisp strings are guaranteed to be NUL terminated, no need to create >> a copy to use is as a C string. > > Really? I will have to look at the code. See allocate_string_data. > Did I dream or at some point in the past they weren't guaranteed to > be null-terminated? Not in the last 18 years. :-) Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 19:39 ` Andreas Schwab @ 2009-07-13 20:13 ` Eli Zaretskii 2009-07-13 21:04 ` Andreas Schwab ` (2 more replies) 0 siblings, 3 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-13 20:13 UTC (permalink / raw) To: Andreas Schwab; +Cc: emacs-devel > From: Andreas Schwab <schwab@linux-m68k.org> > Cc: emacs-devel@gnu.org > Date: Mon, 13 Jul 2009 21:39:20 +0200 > > See allocate_string_data. Aha. But it sounds like it's not just me who is confused. Here's just two examples: From doc.c: strp = SDATA (string); while (strp < SDATA (string) + SBYTES (string)) (why not "while *strp"?) From fileio.c: nm = (unsigned char *) alloca (SBYTES (filename) + 1); bcopy (SDATA (filename), nm, SBYTES (filename) + 1); (why +1? it potentially accesses memory beyond end of `filename's contents) Anyway, thanks. I will fix the code I committed. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 20:13 ` Eli Zaretskii @ 2009-07-13 21:04 ` Andreas Schwab 2009-07-13 23:29 ` Chong Yidong 2009-07-14 0:31 ` YAMAMOTO Mitsuharu 2009-07-14 0:54 ` Stefan Monnier 2 siblings, 1 reply; 49+ messages in thread From: Andreas Schwab @ 2009-07-13 21:04 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Aha. But it sounds like it's not just me who is confused. Here's > just two examples: > > From doc.c: > > strp = SDATA (string); > while (strp < SDATA (string) + SBYTES (string)) > > (why not "while *strp"?) Just because strings are always NUL terminated does not mean that every NUL terminates a string. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 21:04 ` Andreas Schwab @ 2009-07-13 23:29 ` Chong Yidong 0 siblings, 0 replies; 49+ messages in thread From: Chong Yidong @ 2009-07-13 23:29 UTC (permalink / raw) To: Andreas Schwab; +Cc: Eli Zaretskii, emacs-devel Andreas Schwab <schwab@linux-m68k.org> writes: > Just because strings are always NUL terminated does not mean that every > NUL terminates a string. Yes, and in fact this struck recently---see Bug#3772. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 20:13 ` Eli Zaretskii 2009-07-13 21:04 ` Andreas Schwab @ 2009-07-14 0:31 ` YAMAMOTO Mitsuharu 2009-07-14 0:54 ` Stefan Monnier 2 siblings, 0 replies; 49+ messages in thread From: YAMAMOTO Mitsuharu @ 2009-07-14 0:31 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Andreas Schwab, emacs-devel >>>>> On Mon, 13 Jul 2009 23:13:20 +0300, Eli Zaretskii <eliz@gnu.org> said: > Aha. But it sounds like it's not just me who is confused. Here's > just two examples: > From fileio.c: > nm = (unsigned char *) alloca (SBYTES (filename) + 1); > bcopy (SDATA (filename), nm, SBYTES (filename) + 1); > (why +1? it potentially accesses memory beyond end of `filename's > contents) It is necessary because SBYTES does not count the extra NUL character. E.g., SBYTES (empty_unibyte_string) == 0, and *(SDATA (empty_unibyte_string)) == '\0'. YAMAMOTO Mitsuharu mituharu@math.s.chiba-u.ac.jp ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-13 20:13 ` Eli Zaretskii 2009-07-13 21:04 ` Andreas Schwab 2009-07-14 0:31 ` YAMAMOTO Mitsuharu @ 2009-07-14 0:54 ` Stefan Monnier 2009-07-14 3:18 ` Eli Zaretskii 2 siblings, 1 reply; 49+ messages in thread From: Stefan Monnier @ 2009-07-14 0:54 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Andreas Schwab, emacs-devel > Aha. But it sounds like it's not just me who is confused. Here's > just two examples: > From doc.c: > strp = SDATA (string); > while (strp < SDATA (string) + SBYTES (string)) > (why not "while *strp"?) As said Andreas, this would stop at the first NUL, which may appear within the string. > From fileio.c: > nm = (unsigned char *) alloca (SBYTES (filename) + 1); > bcopy (SDATA (filename), nm, SBYTES (filename) + 1); > (why +1? it potentially accesses memory beyond end of `filename's > contents) The +1 is precisely used to make sure we copy the terminating NUL. Stefan ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 0:54 ` Stefan Monnier @ 2009-07-14 3:18 ` Eli Zaretskii 2009-07-14 4:28 ` Miles Bader ` (2 more replies) 0 siblings, 3 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-14 3:18 UTC (permalink / raw) To: Stefan Monnier; +Cc: schwab, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Andreas Schwab <schwab@linux-m68k.org>, emacs-devel@gnu.org > Date: Mon, 13 Jul 2009 20:54:14 -0400 > > > Aha. But it sounds like it's not just me who is confused. Here's > > just two examples: > > > From doc.c: > > > strp = SDATA (string); > > while (strp < SDATA (string) + SBYTES (string)) > > > (why not "while *strp"?) > > As said Andreas, this would stop at the first NUL, which may appear > within the string. But then a gazillion other places are buggy: we _do_ use SDATA(str) as a C string, and pass it to functions that will stop examining the string on the first null. A random example: d = opendir (SDATA (Fdirectory_file_name (encoded_dir))); What am I missing? > > From fileio.c: > > nm = (unsigned char *) alloca (SBYTES (filename) + 1); > > bcopy (SDATA (filename), nm, SBYTES (filename) + 1); > > (why +1? it potentially accesses memory beyond end of `filename's > > contents) > > The +1 is precisely used to make sure we copy the terminating NUL. That's not my reading of allocate_string_data. Are you sure? Anyway, if that's true, then again we have bugs in other places. Like this one: directory_nbytes = SBYTES (directory); if (directory_nbytes == 0 || !IS_ANY_SEP (SREF (directory, directory_nbytes - 1))) needsep = 1; [...] int nbytes = len + directory_nbytes + needsep; fullname = make_uninit_multibyte_string (nbytes, nbytes); bcopy (SDATA (directory), SDATA (fullname), directory_nbytes); ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 3:18 ` Eli Zaretskii @ 2009-07-14 4:28 ` Miles Bader 2009-07-14 19:14 ` Eli Zaretskii 2009-07-14 4:31 ` Haojun Bao 2009-07-14 18:18 ` Stefan Monnier 2 siblings, 1 reply; 49+ messages in thread From: Miles Bader @ 2009-07-14 4:28 UTC (permalink / raw) To: Eli Zaretskii; +Cc: schwab, Stefan Monnier, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> As said Andreas, this would stop at the first NUL, which may appear >> within the string. > > But then a gazillion other places are buggy: we _do_ use SDATA(str) as > a C string, and pass it to functions that will stop examining the > string on the first null. A random example: > > d = opendir (SDATA (Fdirectory_file_name (encoded_dir))); That's a limitation of opendir (and posix filenames in general), and there's absolutely nothing we can do it (so there's no point in worrying about that case), but we shouldn't introduce new limitations, since for many other uses, NUL-containing strings are fine. -Miles -- Scriptures, n. The sacred books of our holy religion, as distinguished from the false and profane writings on which all other faiths are based. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 4:28 ` Miles Bader @ 2009-07-14 19:14 ` Eli Zaretskii 2009-07-14 19:32 ` Davis Herring 0 siblings, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-14 19:14 UTC (permalink / raw) To: Miles Bader; +Cc: emacs-devel > From: Miles Bader <miles@gnu.org> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, schwab@linux-m68k.org, > emacs-devel@gnu.org > Date: Tue, 14 Jul 2009 13:28:19 +0900 > > > d = opendir (SDATA (Fdirectory_file_name (encoded_dir))); > > That's a limitation of opendir (and posix filenames in general), and > there's absolutely nothing we can do it (so there's no point in worrying > about that case) That's not what I meant. What I meant is the problem that could happen if encoded_dir's value was foo^@bar, and there was also a directory called "foo". Won't we then opendir "foo"? And what if opendir was replaced by unlink? ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 19:14 ` Eli Zaretskii @ 2009-07-14 19:32 ` Davis Herring 2009-07-14 20:03 ` Eli Zaretskii 2009-07-15 9:19 ` David Kastrup 0 siblings, 2 replies; 49+ messages in thread From: Davis Herring @ 2009-07-14 19:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, Miles Bader > That's not what I meant. What I meant is the problem that could > happen if encoded_dir's value was foo^@bar, and there was also a > directory called "foo". Won't we then opendir "foo"? Of course. But is the equivalence between "foo" and "foo^@bar" fundamentally worse than the (almost: consider symlinks) equivalence between "foo" and "foo/"? Maybe it just doesn't matter much. > And what if opendir was replaced by unlink? Nothing. You can't unlink directories. ;) Davis -- This product is sold by volume, not by mass. If it appears too dense or too sparse, it is because mass-energy conversion has occurred during shipping. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 19:32 ` Davis Herring @ 2009-07-14 20:03 ` Eli Zaretskii 2009-07-14 20:27 ` Miles Bader 2009-07-15 9:19 ` David Kastrup 1 sibling, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2009-07-14 20:03 UTC (permalink / raw) To: herring; +Cc: emacs-devel, miles > Date: Tue, 14 Jul 2009 12:32:18 -0700 (PDT) > From: "Davis Herring" <herring@lanl.gov> > Cc: "Miles Bader" <miles@gnu.org>, emacs-devel@gnu.org > > > That's not what I meant. What I meant is the problem that could > > happen if encoded_dir's value was foo^@bar, and there was also a > > directory called "foo". Won't we then opendir "foo"? > > Of course. But is the equivalence between "foo" and "foo^@bar" > fundamentally worse than the (almost: consider symlinks) equivalence > between "foo" and "foo/"? Yes, it is (IMO): directory "foo^@bar" does not exist, so you should have got an error. Instead, you silently open a directory by another name, as if "foo^@bar" existed. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 20:03 ` Eli Zaretskii @ 2009-07-14 20:27 ` Miles Bader 2009-07-14 21:05 ` Eli Zaretskii 0 siblings, 1 reply; 49+ messages in thread From: Miles Bader @ 2009-07-14 20:27 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Yes, it is (IMO): directory "foo^@bar" does not exist, so you should > have got an error. Instead, you silently open a directory by another > name, as if "foo^@bar" existed. So you're suggesting that each use of a system function with such limitations be preceded by "if (contains_null(string)) error ();" or something? I suppose, though it seems such an obscure issue that I guess I don't really care. -Miles -- Next to fried food, the South has suffered most from oratory. -- Walter Hines Page ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 20:27 ` Miles Bader @ 2009-07-14 21:05 ` Eli Zaretskii 0 siblings, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-14 21:05 UTC (permalink / raw) To: Miles Bader; +Cc: emacs-devel > From: Miles Bader <miles@gnu.org> > Cc: herring@lanl.gov, emacs-devel@gnu.org > Date: Wed, 15 Jul 2009 05:27:34 +0900 > > Eli Zaretskii <eliz@gnu.org> writes: > > Yes, it is (IMO): directory "foo^@bar" does not exist, so you should > > have got an error. Instead, you silently open a directory by another > > name, as if "foo^@bar" existed. > > So you're suggesting that each use of a system function with such > limitations be preceded by "if (contains_null(string)) error ();" or something? Yes, something like that. We always do a CHECK_STRING in those cases, at least for the arguments of the primitives; maybe we should have CHECK_FILENAME or something. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 19:32 ` Davis Herring 2009-07-14 20:03 ` Eli Zaretskii @ 2009-07-15 9:19 ` David Kastrup 1 sibling, 0 replies; 49+ messages in thread From: David Kastrup @ 2009-07-15 9:19 UTC (permalink / raw) To: emacs-devel "Davis Herring" <herring@lanl.gov> writes: >> That's not what I meant. What I meant is the problem that could >> happen if encoded_dir's value was foo^@bar, and there was also a >> directory called "foo". Won't we then opendir "foo"? > > Of course. But is the equivalence between "foo" and "foo^@bar" > fundamentally worse than the (almost: consider symlinks) equivalence > between "foo" and "foo/"? Maybe it just doesn't matter much. > >> And what if opendir was replaced by unlink? > > Nothing. You can't unlink directories. ;) As root under Solaris, you can. This has been a nasty cause of surprise for decades. I can't vouch that it has not changed in _very_ recent years, but there certainly will systems be around where this is the case. -- David Kastrup ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 3:18 ` Eli Zaretskii 2009-07-14 4:28 ` Miles Bader @ 2009-07-14 4:31 ` Haojun Bao 2009-07-14 18:18 ` Stefan Monnier 2 siblings, 0 replies; 49+ messages in thread From: Haojun Bao @ 2009-07-14 4:31 UTC (permalink / raw) To: emacs-devel Eli Zaretskii <...> writes: >> From: Stefan Monnier <monnier@iro.umontreal.ca> >> Cc: Andreas Schwab <schwab@linux-m68k.org>, emacs-devel@gnu.org >> Date: Mon, 13 Jul 2009 20:54:14 -0400 >> >> > Aha. But it sounds like it's not just me who is confused. Here's >> > just two examples: >> >> > From doc.c: >> >> > strp = SDATA (string); >> > while (strp < SDATA (string) + SBYTES (string)) >> >> > (why not "while *strp"?) >> >> As said Andreas, this would stop at the first NUL, which may appear >> within the string. > > But then a gazillion other places are buggy: we _do_ use SDATA(str) as > a C string, and pass it to functions that will stop examining the > string on the first null. A random example: > > d = opendir (SDATA (Fdirectory_file_name (encoded_dir))); > > What am I missing? NUL can not be in a pathname, so it should be safe here to treat it as a classical C string. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 3:18 ` Eli Zaretskii 2009-07-14 4:28 ` Miles Bader 2009-07-14 4:31 ` Haojun Bao @ 2009-07-14 18:18 ` Stefan Monnier 2009-07-14 19:57 ` Eli Zaretskii 2 siblings, 1 reply; 49+ messages in thread From: Stefan Monnier @ 2009-07-14 18:18 UTC (permalink / raw) To: Eli Zaretskii; +Cc: schwab, emacs-devel >> > From fileio.c: >> > nm = (unsigned char *) alloca (SBYTES (filename) + 1); >> > bcopy (SDATA (filename), nm, SBYTES (filename) + 1); >> > (why +1? it potentially accesses memory beyond end of `filename's >> > contents) >> The +1 is precisely used to make sure we copy the terminating NUL. > That's not my reading of allocate_string_data. Are you sure? I'm not sure about any piece of code, no. Bugs are common. But my reading of allocate_string_data starts by: /* Set up Lisp_String S for holding NCHARS characters, NBYTES bytes, plus a NUL byte at the end. Allocate an sdata structure for S, and > Anyway, if that's true, then again we have bugs in other places. Like > this one: > directory_nbytes = SBYTES (directory); > if (directory_nbytes == 0 > || !IS_ANY_SEP (SREF (directory, directory_nbytes - 1))) > needsep = 1; > [...] > int nbytes = len + directory_nbytes + needsep; > fullname = make_uninit_multibyte_string (nbytes, nbytes); > bcopy (SDATA (directory), SDATA (fullname), > directory_nbytes); make_uninit_multibyte_string calls allocate_string_data which does STRING_DATA (s)[nbytes] = '\0'; so the destination of the `bcopy' already has the terminating NUL. So, I'm overall pretty sure our strings *should* always have a NUL right after the last byte. OTOH I'd be surprised if there isn't a bug somewhere that makes it possible for some strings to occasionally fail to have this terminating NUL (most likely, any bug that leads to a crash could also lead to such a situation). Stefan ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-14 18:18 ` Stefan Monnier @ 2009-07-14 19:57 ` Eli Zaretskii 0 siblings, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-14 19:57 UTC (permalink / raw) To: Stefan Monnier; +Cc: emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: schwab@linux-m68k.org, emacs-devel@gnu.org > Date: Tue, 14 Jul 2009 14:18:53 -0400 > > > directory_nbytes = SBYTES (directory); > > if (directory_nbytes == 0 > > || !IS_ANY_SEP (SREF (directory, directory_nbytes - 1))) > > needsep = 1; > > [...] > > int nbytes = len + directory_nbytes + needsep; > > fullname = make_uninit_multibyte_string (nbytes, nbytes); > > bcopy (SDATA (directory), SDATA (fullname), > > directory_nbytes); > > make_uninit_multibyte_string calls allocate_string_data which does > > STRING_DATA (s)[nbytes] = '\0'; > > so the destination of the `bcopy' already has the terminating NUL. Perhaps most of the places where we use these paradigms are okay due to all these subtle corners that together make everything work. But IMHO it's inherently unsafe to use character arrays that are not true C strings as if they were C strings. For one, it violates the mental model each C programmer has about strings, and that can easily lead to misunderstanding, confusion, and bugs. For example (from dbusbind.c): char x[DBUS_MAXIMUM_SIGNATURE_LENGTH]; [...] strcpy (x, SDATA (CAR_SAFE (XD_NEXT_VALUE (elt)))); [...] sprintf (signature, "%c%s", dtype, x); or case DBUS_TYPE_SIGNATURE: { char *val = SDATA (Fstring_make_unibyte (object)); XD_DEBUG_MESSAGE ("%c %s", dtype, val); How can one convince herself that this code is safe without knowing too much about the Lisp strings whose data gets handled here as C strings? Can they have embedded nulls or cannot they? Same here (editfns.c): if (SBYTES (val) > message_length) { message_length = SBYTES (val); message_text = (char *)xrealloc (message_text, message_length); } bcopy (SDATA (val), message_text, SBYTES (val)); message2 (message_text, SBYTES (val), STRING_MULTIBYTE (val)); message_text[] is not a C string here, because it's not null-terminated (and doesn't have enough space to be terminated). Without looking at the implementation of message2, whose 1st arg is a `char *', how can one know that there's no bug here? Or here (search.c): raw_pattern_size = SCHARS (string); raw_pattern_size_byte = SCHARS (string); raw_pattern = (unsigned char *) alloca (raw_pattern_size + 1); copy_text (SDATA (string), raw_pattern, SBYTES (string), 1, 0); raw_pattern[] is not null-terminated, and we then use it, directly and indirectly, in many places. Without studying each use, there's no way you can determine that there cannot be a bug here. Etc., etc. -- I see other places where maybe it works, maybe it doesn't. One needs to study the code very carefully and look at many functions up and down the call stack, just to determine if a few lines don't constitute a bug. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 16:11 ` Drew Adams 2009-07-09 16:25 ` Jason Rumney @ 2009-07-09 19:11 ` Eli Zaretskii 1 sibling, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-09 19:11 UTC (permalink / raw) To: Drew Adams; +Cc: jasonr, emacs-devel, mathias.dahl > From: "Drew Adams" <drew.adams@oracle.com> > Cc: "'Eli Zaretskii'" <eliz@gnu.org>, <emacs-devel@gnu.org> > Date: Thu, 9 Jul 2009 09:11:10 -0700 > > Dumb question: Is there a way for Emacs to know whether the format is NTFS or > FAT(32)? Yes (and it does). > If so, then Emacs could use nil for FAT volumes. Doing that by default is too radical, IMO. Having this option non-nil on FAT/FAT32/XFAT has its merits, although less so than on NTFS. And the speedup for local drives would be minimal. > (If not, it seems like nil would be the best default value.) I prefer to carry out a plan I have for speeding up the emulated `stat' so that it works faster without sacrificing features. Those features are important for important use-cases, such as support for files and directories that are private to the user (inaccessible by other Windows users). Note that the slow-down due to additional functionality is only acutely visible when we call file-attributes on long lists of files, like when it is called from directory-files-and-attributes. A single call to file-attributes for a single file is very fast even for networked drives. I think nil should be the last resort, for those pathological cases where disk access is dog slow no matter what you do. Setting the option to nil by default would punish too many use-cases for now good reason. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-09 11:53 ` Mathias Dahl 2009-07-09 16:11 ` Drew Adams @ 2009-07-09 19:13 ` Eli Zaretskii 1 sibling, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-09 19:13 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel, jasonr > From: Mathias Dahl <mathias.dahl@gmail.com> > Date: Thu, 9 Jul 2009 13:53:17 +0200 > Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org > > I suggest we change the default value of this variable to nil (or make > the code faster). The code is written to effectively behave as if the variable was nil for remote filesystems. It just didn't handle UNC file names as remote. Now it does. Making the code faster (even beyond this bugfix) is the plan. Changing the default to nil isn't. Thanks for your help in finding the bug and fixing it. ^ permalink raw reply [flat|nested] 49+ messages in thread
* Re: Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) 2009-07-08 20:47 ` Mathias Dahl 2009-07-09 11:37 ` Jason Rumney @ 2009-07-09 12:56 ` Eli Zaretskii 1 sibling, 0 replies; 49+ messages in thread From: Eli Zaretskii @ 2009-07-09 12:56 UTC (permalink / raw) To: Mathias Dahl; +Cc: emacs-devel > From: Mathias Dahl <mathias.dahl@gmail.com> > Date: Wed, 8 Jul 2009 22:47:37 +0200 > Cc: emacs-devel@gnu.org > > > First, I don't see a ~100-fold slowdown, I see only a 10-fold slowdown > > (vs Emacs 22.3). This is with a directory that has ~130 > > subdirectories. To clarify, I see 1 sec with Emacs 22.3, and 10 to 11 seconds with Emacs 23.0.95. > The more files there are, the bigger the slowdown seems to be, in my tests. Can you see if it's approximately linear in the number of files? > > . Do you see significant difference in speed with cold and hot cache? > > That is, if you run Dired on the above directory, then kill the > > Dired buffer and immediately run Dired again on the same directory, > > what times do you see then in Emacs 23? > > Hehe, it got even worse the second time :) That's somewhat strange. For me, the second time is very fast: 3-4 seconds instead of 11. > > . Does it help to set w32-get-true-file-attributes to nil? > > Woooo! Um, yes... It helped all right. From 85 (first try) and 115 > (second try as per reuest above), down to 1,4 s. Shall we call that as > success? :) Yes, probably. I will send a patch today or tomorrow for you to try. Thanks. ^ permalink raw reply [flat|nested] 49+ messages in thread
end of thread, other threads:[~2009-07-15 9:19 UTC | newest] Thread overview: 49+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-07-05 21:01 Willing to debug bug #3542 (23.0.94; File access via UNC path slow again under Windows) Mathias Dahl 2009-07-05 22:58 ` Chong Yidong 2009-07-06 14:30 ` Mathias Dahl 2009-07-06 14:55 ` Chong Yidong 2009-07-07 11:00 ` Mathias Dahl 2009-07-07 11:14 ` Miles Bader 2009-07-06 3:06 ` Eli Zaretskii 2009-07-06 7:38 ` Mathias Dahl 2009-07-06 20:01 ` Eli Zaretskii 2009-07-08 15:01 ` Eli Zaretskii 2009-07-08 20:47 ` Mathias Dahl 2009-07-09 11:37 ` Jason Rumney 2009-07-09 11:53 ` Mathias Dahl 2009-07-09 16:11 ` Drew Adams 2009-07-09 16:25 ` Jason Rumney 2009-07-09 17:03 ` Drew Adams 2009-07-09 18:59 ` Eli Zaretskii 2009-07-09 18:47 ` Eli Zaretskii 2009-07-09 21:33 ` Chong Yidong 2009-07-10 8:54 ` Eli Zaretskii 2009-07-11 19:17 ` Stefan Monnier 2009-07-11 20:22 ` Eli Zaretskii 2009-07-13 12:17 ` Stefan Monnier 2009-07-13 13:38 ` Andreas Schwab 2009-07-13 19:00 ` Eli Zaretskii 2009-07-13 18:56 ` Eli Zaretskii 2009-07-14 0:51 ` Stefan Monnier 2009-07-13 13:57 ` Andreas Schwab 2009-07-13 19:02 ` Eli Zaretskii 2009-07-13 19:39 ` Andreas Schwab 2009-07-13 20:13 ` Eli Zaretskii 2009-07-13 21:04 ` Andreas Schwab 2009-07-13 23:29 ` Chong Yidong 2009-07-14 0:31 ` YAMAMOTO Mitsuharu 2009-07-14 0:54 ` Stefan Monnier 2009-07-14 3:18 ` Eli Zaretskii 2009-07-14 4:28 ` Miles Bader 2009-07-14 19:14 ` Eli Zaretskii 2009-07-14 19:32 ` Davis Herring 2009-07-14 20:03 ` Eli Zaretskii 2009-07-14 20:27 ` Miles Bader 2009-07-14 21:05 ` Eli Zaretskii 2009-07-15 9:19 ` David Kastrup 2009-07-14 4:31 ` Haojun Bao 2009-07-14 18:18 ` Stefan Monnier 2009-07-14 19:57 ` Eli Zaretskii 2009-07-09 19:11 ` Eli Zaretskii 2009-07-09 19:13 ` Eli Zaretskii 2009-07-09 12:56 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).