From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: David Edmondson Newsgroups: gmane.emacs.bugs Subject: bug#40280: gnus-cloud doesn't do anything useful Date: Sun, 29 Mar 2020 18:15:28 +0100 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="50918"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) To: 40280@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Mar 29 19:16:17 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jIbXl-000D8g-1Z for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 29 Mar 2020 19:16:17 +0200 Original-Received: from localhost ([::1]:39762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jIbXj-0001uQ-OL for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 29 Mar 2020 13:16:15 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:50190) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jIbXZ-0001uE-2M for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2020 13:16:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jIbXX-0007OU-8n for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2020 13:16:04 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:48111) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jIbXW-0007Ne-Tc for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2020 13:16:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jIbXW-0007pN-Lc; Sun, 29 Mar 2020 13:16:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: David Edmondson Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, bugs@gnus.org Resent-Date: Sun, 29 Mar 2020 17:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 40280 X-GNU-PR-Package: emacs,gnus X-Debbugs-Original-To: submit@debbugs.gnu.org (The Gnus Bugfixing Girls + Boys) Original-Received: via spool by submit@debbugs.gnu.org id=B.158550213930052 (code B ref -1); Sun, 29 Mar 2020 17:16:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 29 Mar 2020 17:15:39 +0000 Original-Received: from localhost ([127.0.0.1]:59656 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jIbX9-0007oe-6g for submit@debbugs.gnu.org; Sun, 29 Mar 2020 13:15:39 -0400 Original-Received: from mail-wm1-f67.google.com ([209.85.128.67]:50643) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jIbX6-0007oP-W6 for submit@debbugs.gnu.org; Sun, 29 Mar 2020 13:15:37 -0400 Original-Received: by mail-wm1-f67.google.com with SMTP id d198so17193947wmd.0 for ; Sun, 29 Mar 2020 10:15:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dme-org.20150623.gappssmtp.com; s=20150623; h=to:subject:from:date:message-id:user-agent:mime-version; bh=2TkMCmKEJZSdwSVPTDZdEY3rDszkok7LlZKQOwi+ThA=; b=AjNv0USYFCtWYvTnYcFeXB3E4BozYyUhNmwCkths4RpTTYF3ayVatlH/z5E6BryXhL GfT+X27VKIjdv0ibnakCUtyzYNV/BnI+T/BeNMVjLRbRSbLdNo7ZET5RkV9z6MxsRE4I qzEV/VA/JSgBVPRxUN7M49MGPZaEVLlMTCVhYmPCSaHwtS9kZb+ucXadtchE81lenXBc MedhMIeOYCIovbuhy06IccEqfmT68gr1JYYgGOIr+1C4N9tu8hCQTgWmjswX5za9E1S9 3xPPxl1zKxhXVtd8Xfafv8YdrmsXMGV1P2Echoid8G7Xr7vko8NTZzoT17wv3RSrlbMV e+bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:subject:from:date:message-id:user-agent :mime-version; bh=2TkMCmKEJZSdwSVPTDZdEY3rDszkok7LlZKQOwi+ThA=; b=tVZNosUuK5CuaD9keYdDE6dTxCzlRVcFkOa7TJQpojzxm+idHPiHVlcNRHRcv6n7T8 KbPqXrcVndI7SWDzG2ZpUA2BCnrNfIgwV/Q5/at1eG0qDo+qep+gVGbmC6KX7TpLMEy2 p9DN1CxOT6bIgWCeZfbfW2tGE+/o+t5OCptC7FB7ijxCUgZ1CtvmsQzBsAHfcnId5gAY nUz590SHLfRgOsFwHDuapw/nGVUqVvNizmJHT2J5CPq5Q9rzt6xEN1JcR4t1f0BEy2ps NYhYbwHBLS3hzxFBe0puIKwnCEorrZ1Kl/IutevzTZW0gMBXW1UPLwZMUNfFDAyvKFrV UFEA== X-Gm-Message-State: ANhLgQ1DqwUnd+imvlIAEJJlVGcjvc9OcvLe4vwokLWleDNuTsAJhq70 hywIgYLxzkGzmNxJyRo6V13rrw== X-Google-Smtp-Source: ADFU+vsgpMjIC9hvRbMko/ceK6w+blYZ2sCLb81PN3r0ramckKjl4rS4y99+qE0SeEeh5CtDBI1bcA== X-Received: by 2002:a7b:c004:: with SMTP id c4mr8963030wmb.108.1585502130795; Sun, 29 Mar 2020 10:15:30 -0700 (PDT) Original-Received: from disaster-area.hh.sledj.net (disaster-area.hh.sledj.net. [2001:8b0:bb71:7140:64::1]) by smtp.gmail.com with ESMTPSA id d7sm3743164wrr.77.2020.03.29.10.15.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 29 Mar 2020 10:15:29 -0700 (PDT) Original-Received: from localhost (disaster-area.hh.sledj.net [local]) by disaster-area.hh.sledj.net (OpenSMTPD) with ESMTPA id 0455d20a; Sun, 29 Mar 2020 17:15:28 +0000 (UTC) X-HGTTG: vroomfondel X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:177823 Archived-At: --=-=-= Content-Type: text/plain After some time away, I'm playing with Gnus again. In this instance, gnus-cloud isn't working for me. Specifically, I can add a gnus-cloud storage server (on nnimap) and register another server (nntp) with it. ~u causes messages to be added to Emacs-Cloud and they appear to have the right kind of content. ~d on another machine never does anything useful (e.g. mark some articles as read) because this test: (if (and newer (not force-older)) (gnus-message 3 "Skipping outdated cloud info for group %s, the info is from %s (now is %s)" group date now) always returns true. Looking at how `newer' is calculated, it compares the group timestamp from the gnus-cloud message with the current time. It's not clear to me how that test is ever going to return `nil', unless I have a version of emacs running somewhere in the future that updates gnus-cloud. In general I would have expected to replay any changes with a higher sequence number, irrespective of the date of the changes. That they happened in the past is somewhat the point, isn't it? :-) Another problem is that after a set of Emacs-Cloud messages have been used to replay, the emacs instance doing that replay doesn't persist the last seen sequence value as `gnus-cloud-sequence', so the next time it attempts to replay it will unnecessarily replay the same sequence numbered messages again. Attached is a suggested patch. This makes things work for me in some limited testing. Gnus v5.13 GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.11) of 2019-09-09, modified by Debian 200 news.gmane.io InterNetNews NNRP server INN 2.6.3 ready (posting ok) 100 Legal commands ARTICLE [message-ID|number] AUTHINFO USER name|PASS password|SASL mechanism [initial-response]|GENERIC program [argument ...] BODY [message-ID|number] CAPABILITIES [keyword] COMPRESS DEFLATE DATE GROUP newsgroup HDR header [message-ID|range] HEAD [message-ID|number] HELP IHAVE message-ID LAST LIST [ACTIVE [wildmat]|ACTIVE.TIMES [wildmat]|COUNTS [wildmat]|DISTRIB.PATS|DISTRIBUTIONS|HEADERS [MSGID|RANGE]|MODERATORS|MOTD|NEWSGROUPS [wildmat]|OVERVIEW.FMT|SUBSCRIPTIONS [wildmat]] LISTGROUP [newsgroup [range]] MODE READER NEWGROUPS [yy]yymmdd hhmmss [GMT] NEWNEWS wildmat [yy]yymmdd hhmmss [GMT] NEXT OVER [range] POST QUIT STARTTLS STAT [message-ID|number] XGTITLE [wildmat] XHDR header [message-ID|range] XOVER [range] XPAT header message-ID|range pattern [pattern ...] Report problems to . . 382 Begin TLS negotiation now 100 Legal commands ARTICLE [message-ID|number] AUTHINFO USER name|PASS password|SASL mechanism [initial-response]|GENERIC program [argument ...] BODY [message-ID|number] CAPABILITIES [keyword] COMPRESS DEFLATE DATE GROUP newsgroup HDR header [message-ID|range] HEAD [message-ID|number] HELP IHAVE message-ID LAST LIST [ACTIVE [wildmat]|ACTIVE.TIMES [wildmat]|COUNTS [wildmat]|DISTRIB.PATS|DISTRIBUTIONS|HEADERS [MSGID|RANGE]|MODERATORS|MOTD|NEWSGROUPS [wildmat]|OVERVIEW.FMT|SUBSCRIPTIONS [wildmat]] LISTGROUP [newsgroup [range]] MODE READER NEWGROUPS [yy]yymmdd hhmmss [GMT] NEWNEWS wildmat [yy]yymmdd hhmmss [GMT] NEXT OVER [range] POST QUIT STARTTLS STAT [message-ID|number] XGTITLE [wildmat] XHDR header [message-ID|range] XOVER [range] XPAT header message-ID|range pattern [pattern ...] Report problems to . . --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-gnus-cloud-Improve-cloud-sync.patch >From a3bdd2219855a72e2ee90de1b6f823c9678a470c Mon Sep 17 00:00:00 2001 From: David Edmondson Date: Sat, 28 Mar 2020 19:03:58 +0000 Subject: [PATCH] gnus-cloud: Improve cloud sync After replaying a set of actions downloaded by gnus-cloud, persist the highest sequence number seen as the local `gnus-cloud-sequence' number, in order that a future download will not unnecessarily replay previously seen actions and any future uploads from this emacs instance use a higher sequence number than that downloaded. Remove the test on whether individual newsrc entries are older than the current time, as that is always going to be the case. --- lisp/gnus/gnus-cloud.el | 54 ++++++++++++++++++++--------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/lisp/gnus/gnus-cloud.el b/lisp/gnus/gnus-cloud.el index da6231d73300..7ea691e7220c 100644 --- a/lisp/gnus/gnus-cloud.el +++ b/lisp/gnus/gnus-cloud.el @@ -223,13 +223,10 @@ easy interactive way to set this from the Server buffer." (t (gnus-message 1 "Unknown type %s; ignoring" type)))))) -(defun gnus-cloud-update-newsrc-data (group elem &optional force-older) - "Update the newsrc data for GROUP from ELEM. -Use old data if FORCE-OLDER is not nil." +(defun gnus-cloud-update-newsrc-data (group elem) + "Update the newsrc data for GROUP from ELEM." (let* ((contents (plist-get elem :contents)) (date (or (plist-get elem :timestamp) "0")) - (now (gnus-cloud-timestamp nil)) - (newer (string-lessp date now)) (group-info (gnus-get-info group))) (if (and contents (stringp (nth 0 contents)) @@ -238,15 +235,13 @@ Use old data if FORCE-OLDER is not nil." (if (equal (format "%S" group-info) (format "%S" contents)) (gnus-message 3 "Skipping cloud update of group %s, the info is the same" group) - (if (and newer (not force-older)) - (gnus-message 3 "Skipping outdated cloud info for group %s, the info is from %s (now is %s)" group date now) - (when (or (not gnus-cloud-interactive) - (gnus-y-or-n-p - (format "%s has older different info in the cloud as of %s, update it here? " - group date))) - (gnus-message 2 "Installing cloud update of group %s" group) - (gnus-set-info group contents) - (gnus-group-update-group group)))) + (when (or (not gnus-cloud-interactive) + (gnus-y-or-n-p + (format "%s has different info in the cloud from %s, update it here? " + group date))) + (gnus-message 2 "Installing cloud update of group %s" group) + (gnus-set-info group contents) + (gnus-group-update-group group))) (gnus-error 1 "Sorry, group %s is not subscribed" group)) (gnus-error 1 "Sorry, could not update newsrc for group %s (invalid data %S)" group elem)))) @@ -380,8 +375,9 @@ When FULL is t, upload everything, not just a difference from the last full." (gnus-cloud-files-to-upload full) (gnus-cloud-collect-full-newsrc))) (group (gnus-group-full-name gnus-cloud-group-name gnus-cloud-method))) + (setq gnus-cloud-sequence (1+ (or gnus-cloud-sequence 0))) (insert (format "Subject: (sequence: %s type: %s storage-method: %s)\n" - (or gnus-cloud-sequence "UNKNOWN") + gnus-cloud-sequence (if full :full :partial) gnus-cloud-storage-method)) (insert "From: nobody@gnus.cloud.invalid\n") @@ -390,7 +386,6 @@ When FULL is t, upload everything, not just a difference from the last full." (if (gnus-request-accept-article gnus-cloud-group-name gnus-cloud-method t t) (progn - (setq gnus-cloud-sequence (1+ (or gnus-cloud-sequence 0))) (gnus-cloud-add-timestamps elems) (gnus-message 3 "Uploaded Gnus Cloud data successfully to %s" group) (gnus-group-refresh-group group)) @@ -459,18 +454,21 @@ instead of `gnus-cloud-sequence'. When UPDATE is t, returns the result of calling `gnus-cloud-update-all'. Otherwise, returns the Gnus Cloud data chunks." (let ((articles nil) + (highest-sequence-seen gnus-cloud-sequence) chunks) (dolist (header (gnus-cloud-available-chunks)) - (when (> (gnus-cloud-chunk-sequence (mail-header-subject header)) - (or sequence-override gnus-cloud-sequence -1)) - - (if (string-match (format "storage-method: %s" gnus-cloud-storage-method) - (mail-header-subject header)) - (push (mail-header-number header) articles) - (gnus-message 1 "Skipping article %s because it didn't match the Gnus Cloud method %s: %s" - (mail-header-number header) - gnus-cloud-storage-method - (mail-header-subject header))))) + (let ((this-sequence (gnus-cloud-chunk-sequence (mail-header-subject header)))) + (when (> this-sequence (or sequence-override gnus-cloud-sequence -1)) + + (if (string-match (format "storage-method: %s" gnus-cloud-storage-method) + (mail-header-subject header)) + (progn + (push (mail-header-number header) articles) + (setq highest-sequence-seen (max highest-sequence-seen this-sequence))) + (gnus-message 1 "Skipping article %s because it didn't match the Gnus Cloud method %s: %s" + (mail-header-number header) + gnus-cloud-storage-method + (mail-header-subject header)))))) (when articles (nnimap-request-articles (nreverse articles) gnus-cloud-group-name) (with-current-buffer nntp-server-buffer @@ -480,7 +478,9 @@ Otherwise, returns the Gnus Cloud data chunks." (push (gnus-cloud-parse-chunk) chunks) (forward-line 1)))) (if update - (mapcar #'gnus-cloud-update-all chunks) + (progn + (mapcar #'gnus-cloud-update-all chunks) + (setq gnus-cloud-sequence highest-sequence-seen)) chunks))) (defun gnus-cloud-server-p (server) -- 2.25.1 --=-=-= Content-Type: text/plain dme. -- And the sign said: long haired freaky people need not apply. --=-=-=--