From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#22534: File notify broken on Windows Date: Sat, 06 Feb 2016 18:53:04 +0200 Message-ID: <83wpqhsrv3.fsf@gnu.org> References: <83h9hrytom.fsf@gnu.org> <87lh72xln0.fsf@gmx.de> <838u30wbx5.fsf@gnu.org> <83fux7v56c.fsf@gnu.org> <871t8rm5x8.fsf@gmx.de> <83si17t02p.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1454777664 1008 80.91.229.3 (6 Feb 2016 16:54:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 6 Feb 2016 16:54:24 +0000 (UTC) Cc: 22534@debbugs.gnu.org, fabrice.popineau@gmail.com To: michael.albinus@gmx.de Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Feb 06 17:54:11 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aS67e-0001Ax-Uf for geb-bug-gnu-emacs@m.gmane.org; Sat, 06 Feb 2016 17:54:11 +0100 Original-Received: from localhost ([::1]:55060 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aS67e-0006Lx-F2 for geb-bug-gnu-emacs@m.gmane.org; Sat, 06 Feb 2016 11:54:10 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51600) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aS67Z-0006LL-Uu for bug-gnu-emacs@gnu.org; Sat, 06 Feb 2016 11:54:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aS67W-0001jn-O6 for bug-gnu-emacs@gnu.org; Sat, 06 Feb 2016 11:54:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:55672) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aS67W-0001jX-Jn for bug-gnu-emacs@gnu.org; Sat, 06 Feb 2016 11:54:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84) (envelope-from ) id 1aS67W-00068E-FC for bug-gnu-emacs@gnu.org; Sat, 06 Feb 2016 11:54:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 06 Feb 2016 16:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22534 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 22534-submit@debbugs.gnu.org id=B22534.145477761923539 (code B ref 22534); Sat, 06 Feb 2016 16:54:02 +0000 Original-Received: (at 22534) by debbugs.gnu.org; 6 Feb 2016 16:53:39 +0000 Original-Received: from localhost ([127.0.0.1]:36028 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aS678-00067a-Pa for submit@debbugs.gnu.org; Sat, 06 Feb 2016 11:53:39 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:60531) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aS677-00067P-U4 for 22534@debbugs.gnu.org; Sat, 06 Feb 2016 11:53:38 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aS671-0001W2-N5 for 22534@debbugs.gnu.org; Sat, 06 Feb 2016 11:53:32 -0500 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40311) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aS66x-0001V4-Uq; Sat, 06 Feb 2016 11:53:27 -0500 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3034 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aS66x-0002YN-2t; Sat, 06 Feb 2016 11:53:27 -0500 In-reply-to: <83si17t02p.fsf@gnu.org> (message from Eli Zaretskii on Fri, 05 Feb 2016 21:43:26 +0200) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:112567 Archived-At: > Date: Fri, 05 Feb 2016 21:43:26 +0200 > From: Eli Zaretskii > Cc: 22534@debbugs.gnu.org, fabrice.popineau@gmail.com > > > From: Michael Albinus > > Cc: Eli Zaretskii , 22534@debbugs.gnu.org > > Date: Fri, 05 Feb 2016 18:18:59 +0100 > > > > > I have tried with different values of file-notify--test-timeout but it > > > doesn't seem to make any difference. I also boosted the size of the > > > file_notifications[] array (* 16) without any difference. I have > > > tested with sit-for between each rename operation and it does not work > > > either above 260 files > > > > Is it worth to continue to look for the problem in w32notify? I would simply > > set the loop limit there to 250, that's it. > > It would be a waste of your time to look into that. Go ahead and > lower the number; I will look into the problem when I have time and > see what come up with. We can then pick up where we left off. I've looked into this. The simple patch below fixes the test for me. The problem was that all the 1000 renames were run in a single long series. On MS-Windows this generates 2000 events, because the target file exists and needs to be deleted first. However, w32notify has a static limit of about 1000 events it can collect without losing some (that's on a 32-bit system, probably less on a 64-bit OS). The last part of the riddle is that in batch mode file notifications are not read on MS-Windows unless someone calls read-event or a similar function that calls 'pselect' to check for input. If they are not read, they keep accumulating, and are then reported all together. I also suspect that the Windows API use by w32notify has its own limit of events it can accumulate, and beyond that limit simply drops the older events on the floor. The solution is to call read-event while renaming the files. Then I could go back to 1000 renames without failures. Michael, is there a reason not to issue those additional calls to read-event on other platforms? If necessary, those calls could be made conditional on MS-Windows. Here's the proposed patch. Fabrice, please see if this works for you (the full test runs for about 7 minutes, so be patient). Btw, we could make the test run faster by lowering the 0.1 argument to the 2 read-event calls while we create the 2000 files at the beginning of the test, I see no need for waiting that long for a single file creation. diff --git a/test/automated/file-notify-tests.el b/test/automated/file-notify-tests.el index 629d85b..5fc4ff8 100644 --- a/test/automated/file-notify-tests.el +++ b/test/automated/file-notify-tests.el @@ -66,7 +66,7 @@ file-notify--test-timeout "Timeout to wait for arriving events, in seconds." (cond ((file-remote-p temporary-file-directory) 6) - ((string-equal (file-notify--test-library) "w32notify") 20) + ((string-equal (file-notify--test-library) "w32notify") 10) ((eq system-type 'cygwin) 10) (t 3))) @@ -797,10 +797,7 @@ file-notify--test-with-events file-notify--test-tmpfile '(change) 'file-notify--test-event-handler))) (unwind-protect - ;; In case of w32notify, the upper limit of events to handle - ;; seems to be 260. Reason unknown. - (let ((n (if (string-equal (file-notify--test-library) "w32notify") - 250 1000)) + (let ((n 1000) source-file-list target-file-list (default-directory file-notify--test-tmpfile)) (dotimes (i n) @@ -832,10 +829,11 @@ file-notify--test-with-events (let ((source-file-list source-file-list) (target-file-list target-file-list)) (while (and source-file-list target-file-list) - (rename-file (pop source-file-list) (pop target-file-list) t)))) + (rename-file (pop source-file-list) (pop target-file-list) t) + (read-event nil nil 0.02)))) (file-notify--test-with-events (make-list n 'deleted) (dolist (file target-file-list) - (delete-file file)))) + (prog1 (delete-file file) (read-event nil nil 0.02))))) (file-notify--test-cleanup))) (file-notify--deftest-remote file-notify-test06-many-events