OfflineIMAP and Python

OfflineIMAP and Python

[Paul van der Walt]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

Hello Guix,

I wanted to try using offlineimap from Guix (i already used to use it
From my regular package manager), but i'm running into what is probably
a PATH problem.

------------8<--------------------------
$ ./pre-inst-env guix package -i offlineimap
The following package will be installed:
   offlineimap	6.5.5	/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5

19 packages in profile

$ which offlineimap
/home/paul/.guix-profile/bin/offlineimap

$ offlineimap
Traceback (most recent call last):
  File "/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5/bin/.offlineimap-real", line 20, in <module>
    from offlineimap import OfflineImap
  File "/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5/lib/python2.7/site-packages/offlineimap/__init__.py", line 19, in <module>
    from offlineimap.init import OfflineImap
  File "/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5/lib/python2.7/site-packages/offlineimap/init.py", line 20, in <module>
    import threading
  File "/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/threading.py", line 13, in <module>
    from collections import deque as _deque
  File "/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/collections.py", line 9, in <module>
    from operator import itemgetter as _itemgetter, eq as _eq
ImportError: /gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so: undefined symbol: _PyUnicodeUCS2_AsDefaultEncodedString
------------8<--------------------------

I didn't get any warnings about my PATH when installing the package, so
before i go digging around too much, i was wondering, who else uses
offlineimap and how did they get it to work?

Kindest regards,
p.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: OfflineIMAP and Python

[Ricardo Wurmus]

Paul van der Walt writes:

> ImportError: /gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so: undefined symbol: _PyUnicodeUCS2_AsDefaultEncodedString

This usually happens when you use the wrong python.  What does `which
python` tell you?  I get this error when I accidentally upgrade my Guix
python-2.7.6 to python-3.x and the only "python" executable in my PATH
is the system python at /usr/bin/python.

OfflineIMAP works fine for me when I make sure that python-2.7.6 is in
fact installed and the default "python" executable in my PATH.

~~ Ricardo

Re: OfflineIMAP and Python

[Paul van der Walt]

On 2015-04-21 at 13:03, quoth Ricardo Wurmus:
> OfflineIMAP works fine for me when I make sure that python-2.7.6 is in
> fact installed and the default "python" executable in my PATH.

Weird, it seemed to help, but then gave another error (which i cannot
reproduce now, it was something about the index / state or something),
and now, it's back to giving the first error.

$ which python
/home/paul/.guix-profile/bin/python
$ ls -lah `which python`
lrwxrwxrwx 4 root guix-builder 67 Jan  1  1970 /home/paul/.guix-profile/bin/python -> /gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/bin/python

...so that should be fine... 

Re: OfflineIMAP and Python

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Paul van der Walt writes:
>
>> ImportError: /gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so: undefined symbol: _PyUnicodeUCS2_AsDefaultEncodedString
>
> This usually happens when you use the wrong python.

The ‘python’ executable is necessarily the right one, because it’s
written in the shebang, and this operator.so is the right one too.

It could be that the wrong libpython was loaded, though.

Could you run ‘strace -o log offlineimap’ and then:

  grep 'open.*\.so.*= [0-9]' log

?

Ludo’.

Re: OfflineIMAP and Python

[Ricardo Wurmus]

Ludovic Courtès writes:

>>> ImportError: /gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so: undefined symbol: _PyUnicodeUCS2_AsDefaultEncodedString
>>
>> This usually happens when you use the wrong python.
>
> The ‘python’ executable is necessarily the right one, because it’s
> written in the shebang, and this operator.so is the right one too.

True.  The reason why it happens in my setup is that my ~/.offlineimaprc
contains the settings:

    [general]
    pythonfile = ~/.offlineimap.py
    ...

This Python script has a shebang "#!/usr/bin/env python", so when
offlineimap runs the script the wrong python executable might be
loaded (dependent on PATH).

(I realise that this probably isn't a common setup, so my suggestion to
check the python executable wasn't very helpful.)

~~ Ricardo

Re: OfflineIMAP and Python

[Paul van der Walt]

On 2015-04-21 at 17:37, quoth Ludovic Courtès:
> The ‘python’ executable is necessarily the right one, because it’s
> written in the shebang, and this operator.so is the right one too.
>
> It could be that the wrong libpython was loaded, though.

Okay, i had some more time to look into this. First, i applied Ricardo's
change -- i also use the 'pythonfile' option. I've changed that file's
first line to read

#!/usr/bin/env python

I still get errors though.

> Could you run ‘strace -o log offlineimap’ and then:
>
>   grep 'open.*\.so.*= [0-9]' log

That gives me

-------------8<-----------------------
open("/usr/lib/libreadline.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libhistory.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libncursesw.so.5", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libpython2.7.so.1.0", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
open("/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/_collections.so", O_RDONLY) = 7
open("/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/_collections.so", O_RDONLY|O_CLOEXEC) = 8
open("/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so", O_RDONLY) = 7
open("/gnu/store/zla3af321lkf2n80330fw81mxgbz1gj3-python-2.7.6/lib/python2.7/lib-dynload/operator.so", O_RDONLY|O_CLOEXEC) = 8
-------------8<-----------------------

..so i think you're right Ludo. What can i do about that?

Re: OfflineIMAP and Python

[Paul van der Walt]

On 2015-04-23 at 12:27, quoth Paul van der Walt:
> ..so i think you're right Ludo. What can i do about that?

Running

$ LD_LIBRARY_PATH=.guix-profile/lib/ offlineimap

seems to get me further, although now the error has changed to

[multiple repetitions, once per folder]
ERROR: ERROR in syncfolder for foo folder bla: Traceback (most recent call last):
  File "/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5/lib/python2.7/site-packages/offlineimap/accounts.py", line 407, in syncfolder
    statusfolder.cachemessagelist()
  File "/gnu/store/innphl8ph0iakzarp61bvg4v144p3hbc-offlineimap-6.5.5/lib/python2.7/site-packages/offlineimap/folder/LocalStatus.py", line 72, in cachemessagelist
    assert(line == magicline)
AssertionError

I'm now not sure if that's still a library issue, or perhaps something
offlineimap is doing wrong?

Re: OfflineIMAP and Python

[Ludovic Courtès]

Paul van der Walt <paul@denknerd.org> skribis:

> On 2015-04-21 at 17:37, quoth Ludovic Courtès:
>> The ‘python’ executable is necessarily the right one, because it’s
>> written in the shebang, and this operator.so is the right one too.
>>
>> It could be that the wrong libpython was loaded, though.
>
> Okay, i had some more time to look into this. First, i applied Ricardo's
> change -- i also use the 'pythonfile' option. I've changed that file's
> first line to read
>
> #!/usr/bin/env python

But I suspect that gives you /usr/bin/python instead of
/home/paul/.guix-profile/bin/python no?  Changing the shebang to
explicitly use the latter would be safer.

>> Could you run ‘strace -o log offlineimap’ and then:
>>
>>   grep 'open.*\.so.*= [0-9]' log
>
> That gives me
>
> -------------8<-----------------------
> open("/usr/lib/libreadline.so.6", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/lib/libhistory.so.6", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/lib/libncursesw.so.5", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3

The ‘execve’ before would probably show us that /usr/bin/python is
running I suppose.

Ludo’.

Re: OfflineIMAP and Python

[Ludovic Courtès]

Paul van der Walt <paul@denknerd.org> skribis:

> On 2015-04-23 at 12:27, quoth Paul van der Walt:
>> ..so i think you're right Ludo. What can i do about that?
>
> Running
>
> $ LD_LIBRARY_PATH=.guix-profile/lib/ offlineimap

Possibly this is doing the wrong thing: still running /usr/bin/python,
but running it against Guix’s libraries, no?

Ludo’.

Re: OfflineIMAP and Python

[Paul van der Walt]

On 2015-04-23 at 21:04, quoth Ludovic Courtès:
> But I suspect that gives you /usr/bin/python instead of
> /home/paul/.guix-profile/bin/python no?  Changing the shebang to
> explicitly use the latter would be safer.

Ah i hadn't understood that. Okay, i've put the literal Guix python path
there, but it still gives me the "assert" errors. This does not happen
if i `guix package -r offlineimap`. Therefore, i don't think that was
the problem. What do you mean about 'execve'? (Interestingly enough,
running the strace command gives the old missing symbol error, just
running `offlineimap` gives me the assert errors....)

Re: OfflineIMAP and Python

[Ludovic Courtès]

Paul van der Walt <paul@denknerd.org> skribis:

> On 2015-04-23 at 21:04, quoth Ludovic Courtès:
>> But I suspect that gives you /usr/bin/python instead of
>> /home/paul/.guix-profile/bin/python no?  Changing the shebang to
>> explicitly use the latter would be safer.
>
> Ah i hadn't understood that. Okay, i've put the literal Guix python path
> there, but it still gives me the "assert" errors. This does not happen
> if i `guix package -r offlineimap`.

It may be a genuine error, and the other distro’s offlineimap (or its
dependencies) might be different?

Ludo’.

Re: OfflineIMAP and Python

[Ricardo Wurmus]

Paul van der Walt writes:

> On 2015-04-23 at 21:04, quoth Ludovic Courtès:
>> But I suspect that gives you /usr/bin/python instead of
>> /home/paul/.guix-profile/bin/python no?  Changing the shebang to
>> explicitly use the latter would be safer.
>
> Ah i hadn't understood that. Okay, i've put the literal Guix python path
> there, but it still gives me the "assert" errors. This does not happen
> if i `guix package -r offlineimap`. Therefore, i don't think that was
> the problem.

Could you try without the "pythonfile" line in your ~/.offlineimaprc and
let us know if there's any different behaviour?

I'm using offlineimap (installed from Guix on Fedora) without any
problems.