bug#28845: gnus-cloud does not work

bug#28845: gnus-cloud does not work

[Andreas Schwab]

When trying to set up a gnus-cloud I got this error, rendering it unsuable:

nnimap-request-accept-article: Writing to process: Bad file descriptor, *nnimap*

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Sun, 15 Oct 2017 10:38:44 +0200 Andreas Schwab <schwab@linux-m68k.org> wrote: 

AS> When trying to set up a gnus-cloud I got this error, rendering it unsuable:
AS> nnimap-request-accept-article: Writing to process: Bad file descriptor, *nnimap*

Can you provide a stack trace, your settings, or other typically useful
information for resolving a bug?

Thanks
Ted

bug#28845: gnus-cloud does not work

[Andreas Schwab]

How can I undo the mess the cloud setup has left behind?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sun, 15 Oct 2017 10:38:44 +0200 Andreas Schwab <schwab@linux-m68k.org> wrote: 
>
> AS> When trying to set up a gnus-cloud I got this error, rendering it unsuable:
> AS> nnimap-request-accept-article: Writing to process: Bad file descriptor, *nnimap*
>
> Can you provide a stack trace, your settings, or other typically useful
> information for resolving a bug?
>
> Thanks
> Ted

Since I was one of the, apparently, few who used 'gnus-sync' and I've
missed it ever since it disappeared, I thought I'd chime in here. I'd be
glad to provide all the info I can.

For starters I don't have the same problem as Andreas. I've done the
setup as described in the manual. My *Server* buffer looks like this:

     {nnfolder:archive} (opened)
     {nndraft:} (opened)
     {nntp:eternal-september} (opened) (cloud-sync)
     {nnimap:fastmail} (opened) (CLOUD-HOST)
     {nnimap:gmail} (opened)
     {nntp:gmane} (opened) (cloud-sync)

When I try to do '~ u' in the *Group* buffer I get the following
message (after entering and reentering a passphrase):

Failed to upload Gnus Cloud data to nnimap+fastmail:Emacs-Cloud

I've turned on `nnimap-record-commands', and this is what's in the *imap
log* buffer:

16:14:20 [mail.messagingengine.com] 324 SELECT "Emacs-Cloud"^M
16:14:20 [mail.messagingengine.com] 325 CREATE "Emacs-Cloud"^M
16:14:25 [mail.messagingengine.com] 326 APPEND "Emacs-Cloud" {4362}^M

I'm also running with debug-on-error, but that doesn't trigger.

That's about as far as I got. Long ago I tried debugging it, but I got
lost in the maze of nnimap.

If there's some more data I can provide I'd be more than happy to.

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Mon, 11 Dec 2017 22:44:55 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 

AS> How can I undo the mess the cloud setup has left behind?

Could you be more specific?

Ted

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Tue, 12 Dec 2017 16:24:09 +0100 Joakim Jalap <joakim.jalap@fastmail.com> wrote: 

JJ> Since I was one of the, apparently, few who used 'gnus-sync' and I've
JJ> missed it ever since it disappeared, I thought I'd chime in here. I'd be
JJ> glad to provide all the info I can.

Thanks for your help.

JJ> For starters I don't have the same problem as Andreas. I've done the
JJ> setup as described in the manual. My *Server* buffer looks like this:

JJ>      {nnfolder:archive} (opened)
JJ>      {nndraft:} (opened)
JJ>      {nntp:eternal-september} (opened) (cloud-sync)
JJ>      {nnimap:fastmail} (opened) (CLOUD-HOST)
JJ>      {nnimap:gmail} (opened)
JJ>      {nntp:gmane} (opened) (cloud-sync)

JJ> When I try to do '~ u' in the *Group* buffer I get the following
JJ> message (after entering and reentering a passphrase):

JJ> Failed to upload Gnus Cloud data to nnimap+fastmail:Emacs-Cloud

JJ> I've turned on `nnimap-record-commands', and this is what's in the *imap
JJ> log* buffer:

JJ> 16:14:20 [mail.messagingengine.com] 324 SELECT "Emacs-Cloud"^M
JJ> 16:14:20 [mail.messagingengine.com] 325 CREATE "Emacs-Cloud"^M
JJ> 16:14:25 [mail.messagingengine.com] 326 APPEND "Emacs-Cloud" {4362}^M

JJ> I'm also running with debug-on-error, but that doesn't trigger.

I assume there's nothing useful in *Messages*?

Hmm. I just tried it and had no issues on upload or download. Could you
try setting `gnus-cloud-storage-method' to nil to bypass encoding
altogether? Then try the other methods, to see if the default 'epg
storage method is the issue? If it is, we can focus on the EPG
integration. If it's not, we'll keep digging into nnimap itself.

Thanks again
Ted

bug#28845: gnus-cloud does not work

[Andreas Schwab]

On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:

> On Mon, 11 Dec 2017 22:44:55 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 
>
> AS> How can I undo the mess the cloud setup has left behind?
>
> Could you be more specific?

gnus-server-toggle-cloud-method-server is grossly misnamed, it doesn't
toggle anything.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Tue, 12 Dec 2017 20:20:48 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 

AS> On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> On Mon, 11 Dec 2017 22:44:55 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 
>> 
AS> How can I undo the mess the cloud setup has left behind?
>> 
>> Could you be more specific?

AS> gnus-server-toggle-cloud-method-server is grossly misnamed, it doesn't
AS> toggle anything.

You're right. It sets `gnus-cloud-method' but doesn't toggle. You should remove that
customization and the IMAP group it created, and perhaps that will work
for you.

Ted

bug#28845: gnus-cloud does not work

[Andreas Schwab]

On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:

> You're right. It sets `gnus-cloud-method' but doesn't toggle. You should remove that
> customization and the IMAP group it created, and perhaps that will work
> for you.

I don't see any IMAP group that can be removed.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Tue, 12 Dec 2017 20:42:37 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 

AS> On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> You're right. It sets `gnus-cloud-method' but doesn't toggle. You should remove that
>> customization and the IMAP group it created, and perhaps that will work
>> for you.

AS> I don't see any IMAP group that can be removed.

That's great!

Ted

bug#28845: gnus-cloud does not work

[Andreas Schwab]

On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:

> On Tue, 12 Dec 2017 20:42:37 +0100 Andreas Schwab <schwab@linux-m68k.org> wrote: 
>
> AS> On Dez 12 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:
>>> You're right. It sets `gnus-cloud-method' but doesn't toggle. You should remove that
>>> customization and the IMAP group it created, and perhaps that will work
>>> for you.
>
> AS> I don't see any IMAP group that can be removed.
>
> That's great!

Nope.  There is nothing either that makes this working.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> JJ> I'm also running with debug-on-error, but that doesn't trigger.
>
> I assume there's nothing useful in *Messages*?

Unfortunately not.

> Hmm. I just tried it and had no issues on upload or download. Could you
> try setting `gnus-cloud-storage-method' to nil to bypass encoding
> altogether? Then try the other methods, to see if the default 'epg
> storage method is the issue? If it is, we can focus on the EPG
> integration. If it's not, we'll keep digging into nnimap itself.

I tried them all, but the result is the same (except that I'm not asked
for a password of course).

Regards,
Joakim

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Wed, 13 Dec 2017 08:49:22 +0100 Joakim Jalap <joakim.jalap@fastmail.com> wrote: 

JJ> Ted Zlatanov <tzz@lifelogs.com> writes:
JJ> I'm also running with debug-on-error, but that doesn't trigger.
>> 
>> I assume there's nothing useful in *Messages*?

JJ> Unfortunately not.

OK. I just tested with the latest master build of emacs.git and had no
issues. So we know it's something different in either code or IMAP
server support.

To ensure we're testing the same setup, can you try against a gmail IMAP
server? You should be able to `I' set the cloud host from the Server
buffer.

Also see if you can test with the latest checkout of master to remove
the possibility something old is broken.

>> Hmm. I just tried it and had no issues on upload or download. Could you
>> try setting `gnus-cloud-storage-method' to nil to bypass encoding
>> altogether? Then try the other methods, to see if the default 'epg
>> storage method is the issue? If it is, we can focus on the EPG
>> integration. If it's not, we'll keep digging into nnimap itself.

JJ> I tried them all, but the result is the same (except that I'm not asked
JJ> for a password of course).

OK, let's stick to nil for now to make testing easier and remove that
extra complexity.

Thanks
Ted

bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> OK. I just tested with the latest master build of emacs.git and had no
> issues. So we know it's something different in either code or IMAP
> server support.
>
> To ensure we're testing the same setup, can you try against a gmail IMAP
> server? You should be able to `I' set the cloud host from the Server
> buffer.
>
> Also see if you can test with the latest checkout of master to remove
> the possibility something old is broken.

I think I have a pretty recent version, I pulled master on the 11th.

> OK, let's stick to nil for now to make testing easier and remove that
> extra complexity.

Funnily enough, when I changed to gmail, it still doesn't work with nil,
but it seems to work with 'epg!

When I try with nil I get this in *Messages*:

Leaving cloud data plaintext
Message contains NUL characters.  Delete, continue, abort?  (d, c, a, ?): 
Failed to upload Gnus Cloud data to nnimap+gmail:Emacs-Cloud

and this in *imap log*:

16:21:51 [imap.gmail.com] (inhibited)
16:21:53 [imap.gmail.com] 1039 CAPABILITY^M
16:21:57 [imap.gmail.com] 1040 APPEND "Emacs-Cloud" {22926}^M

When I try with 'epg I get this in *Messages*:

Opening connection to imap.gmail.com via tls...
Opening connection to imap.gmail.com...done
Uploaded Gnus Cloud data successfully to nnimap+gmail:Emacs-Cloud

and this in *imap log*:

16:23:03 [imap.gmail.com] (inhibited)
16:23:07 [imap.gmail.com] 1042 CAPABILITY^M
16:23:09 [imap.gmail.com] 1043 APPEND "Emacs-Cloud" {9544}^M


When I do '~ d' I get this in *Messages*:

nnimap read 0k from imap.gmail.com [2 times]

and this in *imap log*:

16:27:06 [imap.gmail.com] 1044 EXAMINE "Emacs-Cloud"^M
16:27:06 [imap.gmail.com] 1045 UID FETCH 1:* FLAGS^M
16:27:07 [imap.gmail.com] 1046 SELECT "Emacs-Cloud"^M
16:27:07 [imap.gmail.com] 1047 UID FETCH 0:1 (UID RFC822.SIZE X-GM-LABELS BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref X-Diary-Time-Zone X-Diary-Dow X-Diary-Year X-Diary-Month X-Diary-Dom X-Diary-Hour X-Diary-Minute To Newsgroups Cc)])^M


So... I think it seems to work with gmail (not sure how I can verify it
actually works... I will try from home tonight) but not with fastmail.
And for me it doesn't work with nil, but 'epg works.


Something which might be of importance is that I created the folder (is
it called that?) 'Emacs-Cloud' manually in fastmail. That was the last
time I tried to debug this. Before I did that I got some other error.

Reagards,
Joakim

bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> To ensure we're testing the same setup, can you try against a gmail IMAP
> server? You should be able to `I' set the cloud host from the Server
> buffer.

I tried downloading from my home computer, but nothing happens, I get
this in *Messages*

nnimap read 0k from imap.gmail.com

and this in *imap log*:

19:42:25 [imap.gmail.com] 81 SELECT "Emacs-Cloud"^M
19:42:25 [imap.gmail.com] 82 SELECT "Emacs-Cloud"^M
19:42:26 [imap.gmail.com] 83 UID FETCH  (UID RFC822.SIZE X-GM-LABELS BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups Cc)])^M

But nothing changes in my *Group* buffer, which I assumed it would?

Regards,
Joakim

bug#28845: gnus-cloud does not work, bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Wed, 13 Dec 2017 16:45:30 +0100 Joakim Jalap <joakim.jalap@fastmail.com> wrote: 

JJ> Funnily enough, when I changed to gmail, it still doesn't work with nil,
JJ> but it seems to work with 'epg!

Oh. Try 'base64 or even better, use the Customize interface. Sorry.

JJ> When I do '~ d' I get this in *Messages*:

JJ> nnimap read 0k from imap.gmail.com [2 times]

JJ> and this in *imap log*:

JJ> 16:27:06 [imap.gmail.com] 1044 EXAMINE "Emacs-Cloud"^M
JJ> 16:27:06 [imap.gmail.com] 1045 UID FETCH 1:* FLAGS^M
JJ> 16:27:07 [imap.gmail.com] 1046 SELECT "Emacs-Cloud"^M
JJ> 16:27:07 [imap.gmail.com] 1047 UID FETCH 0:1 (UID RFC822.SIZE X-GM-LABELS
JJ> BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References
JJ> In-Reply-To Xref X-Diary-Time-Zone X-Diary-Dow X-Diary-Year X-Diary-Month
JJ> X-Diary-Dom X-Diary-Hour X-Diary-Minute To Newsgroups Cc)])^M

OK, you need to either set `gnus-cloud-sequence' to 0 or run

M-x (gnus-cloud-download-data nil (- gnus-cloud-sequence 3)) RET

It won't download anything before or equal to `gnus-cloud-sequence' but
the above command will force it (without updating anything on disk or in
the newsrc).

JJ> So... I think it seems to work with gmail (not sure how I can verify it
JJ> actually works... I will try from home tonight) but not with fastmail.
JJ> And for me it doesn't work with nil, but 'epg works.

OK, let's leave fastmail out as a potential factor for now and make sure
we have a known good case. Almost there :)

JJ> Something which might be of importance is that I created the folder (is
JJ> it called that?) 'Emacs-Cloud' manually in fastmail. That was the last
JJ> time I tried to debug this. Before I did that I got some other error.

That shouldn't matter, but it's good to know and maybe it can be automated.

JJ> I tried downloading from my home computer, but nothing happens, I get
JJ> this in *Messages*
...
JJ> 19:42:26 [imap.gmail.com] 83 UID FETCH  (UID RFC822.SIZE X-GM-LABELS BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups Cc)])^M

JJ> But nothing changes in my *Group* buffer, which I assumed it would?

Same as above, do the M-x manually :)

Thanks
Ted

bug#28845: gnus-cloud does not work, bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 13 Dec 2017 16:45:30 +0100 Joakim Jalap <joakim.jalap@fastmail.com> wrote: 
>
> JJ> Funnily enough, when I changed to gmail, it still doesn't work with nil,
> JJ> but it seems to work with 'epg!
>
> Oh. Try 'base64 or even better, use the Customize interface. Sorry.

I think I was unclear, I meant I tried all the different
storage-methods. I set the via the customize interface. I tried with
Base64 but it didn't work either.

> OK, you need to either set `gnus-cloud-sequence' to 0 or run
>
> M-x (gnus-cloud-download-data nil (- gnus-cloud-sequence 3)) RET
>
> It won't download anything before or equal to `gnus-cloud-sequence' but
> the above command will force it (without updating anything on disk or in
> the newsrc).

(I assume you meant M-:)

Yes that seems to work! I returns a... big blob of something. :)

> JJ> I tried downloading from my home computer, but nothing happens, I get
> JJ> this in *Messages*
> ...
> JJ> 19:42:26 [imap.gmail.com] 83 UID FETCH  (UID RFC822.SIZE X-GM-LABELS BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups Cc)])^M
>
> JJ> But nothing changes in my *Group* buffer, which I assumed it would?
>
> Same as above, do the M-x manually :)

I will try to remember to do it tonight!

Regards,
Joakim

bug#28845: gnus-cloud does not work

[Ted Zlatanov]

On Thu, 14 Dec 2017 12:29:01 +0100 Joakim Jalap <joakim.jalap@fastmail.com> wrote: 

JJ> I think I was unclear, I meant I tried all the different
JJ> storage-methods. I set the via the customize interface. I tried with
JJ> Base64 but it didn't work either.

OK.

>> OK, you need to either set `gnus-cloud-sequence' to 0 or run
>> 
>> M-: (gnus-cloud-download-data nil (- gnus-cloud-sequence 3)) RET

JJ> Yes that seems to work! I returns a... big blob of something. :)

Right, that's the actual data. So if you set gnus-cloud-sequence lower
than the biggest one in the IMAP group, that data will be downloaded and
interpreted.

Thanks for your help.

Ted

bug#28845: gnus-cloud does not work

[Joakim Jalap]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Right, that's the actual data. So if you set gnus-cloud-sequence lower
> than the biggest one in the IMAP group, that data will be downloaded and
> interpreted.
>
> Thanks for your help.
>
I'm terribly sorry, but I just can't seem to get it to work. Perhaps you
could show me the exact commands needed to 'reset' everything after our
experiments? Right now it just doesn't work, I can download the
gnus-cloud data but it refuses to apply it because of something with the
timestamps: "Skipping outdated cloud info for group...". I don't
understand what this means... do I have to download the data the first
thing I do when starting gnus, before pressing 'g'?

Anyway, what I wan't to do is the following: I have a computer 'box1'
where I run gnus in screen, and I ssh to it to read my news.

I have another computer 'box2' (my desktop) and I'd like to read the
news there instead, so I want to get the state of gnus to be exactly the
same on the two boxes (where right now box2 is far behind, showing as
unread stuff I read months ago on box1).

So could you please tell me the commands to use on box1 and box2 in
order to get themn synced? Uploading data to gmail seems to work now, so
I guess we can skip that step. What I tried doing so far is this:

I deleted all the mails in the Emacs-Cloud folder on gmail, and then:

On box1:
(setq gnus-cloud-sequence 0)
~-u

On box2:
(setq gnus-cloud-sequence 0)
~-d

However, the *Group* buffer on box2 seems unchanged, and I got the
messages about "Skipping outdated cloud info". So is there something
more I need to reset?

Regards,
Joakim