Need help fixing testcases with time-difference

Need help fixing testcases with time-difference

[Hartmut Goebel]

Hi,

I'm currently working on django, the web application framework.
Unfortunalty some tests fail. These are all testing time- and timezone
calculations.

Failures are like this:

AssertionError: datetime.timedelta(0, 3600, 16) not less than
datetime.timedelta(0, 2)

which means the returned time difference is ca. 1 hour, but allowed are
only 2 minutes. The testcase os this one
https://github.com/django/django/blob/master/tests/file_storage/tests.py#L239>

I already added tzdata to native-inputs, but this does not solve the issue.

Any hints?

Below please find the stripped down package definition.

(define-module (gnu packages django)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix build-system python)
  #:use-module (gnu packages base)
  #:use-module (gnu packages python))

(define-public python-django
  (package
    (name "python-django")
    (version "1.10")
    (source (origin
              (method url-fetch)
              (uri (pypi-uri "Django" version))
              (sha256
               (base32
                "01bh5yra6zyxcpqacahbwfbn0y4ivw07j2jsw3crvmjzivb6if26"))))
    (build-system python-build-system)
    (arguments
     '(#:phases
       (modify-phases %standard-phases
         (replace
             'check
           (lambda* _
           (let* ((old-path (getenv "PYTHONPATH")))
             (chdir "tests")
             (setenv "PYTHONPATH"
                     (string-append ".." (if old-path
                                             (string-append ":" old-path))))
             (zero? (system* "python" "runtests.py"))
           ))))))
    (inputs
     ; Django uses pkg_resources (which is part of setuptools) to
     ; locate templates at run-time.
     `(("python-setuptools" ,python-setuptools)))
    (native-inputs
     `(("tzdata", tzdata)
       ("python-docutils" ,python-docutils)
       ("python-jinja2" ,python-jinja2) ; >= 2.7
       ("python-numpy" ,python-numpy)
       ("python-pillow" ,python-pillow)
       ("python-pyyaml" ,python-pyyaml)
       ("python-pytz" ,python-pytz)
       ("python-sqlparse" ,python-sqlparse)
       ("python-tblib" ,python-tblib)
       ;; for Python 2: enum34 and mock
       ("python-enum34" ,python-enum34)
       ("python-mock" ,python-mock)
     ))
    (home-page "http://www.djangoproject.com/")
    (synopsis "High-level Python Web framework")
    (description "")
    (license license:bsd-3)))


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Need help fixing testcases with time-difference

[Vincent Legoll]

Hello,

On Thu, Aug 18, 2016 at 10:05 AM, Hartmut Goebel
<h.goebel@crazy-compilers.com> wrote:
> Hi,
>
> I'm currently working on django, the web application framework.
> Unfortunalty some tests fail. These are all testing time- and timezone
> calculations.
>
> Failures are like this:
>
> AssertionError: datetime.timedelta(0, 3600, 16) not less than
> datetime.timedelta(0, 2)
>
> which means the returned time difference is ca. 1 hour, but allowed are
> only 2 minutes. The testcase os this one
> https://github.com/django/django/blob/master/tests/file_storage/tests.py#L239>
>
> I already added tzdata to native-inputs, but this does not solve the issue.
>
> Any hints?

That looks like a Daylight Saving Time mismatch, could that be possible ?

DST is 1h delta, which would be datetime.timedelta(0, 3600), thus if we remove
that, the test assertion becomes true...

>>> assert datetime.timedelta(0, 3600, 16) < datetime.timedelta(0, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError
>>> assert (datetime.timedelta(0, 3600, 16) - datetime.timedelta(0, 3600)) < datetime.timedelta(0, 2)

WDYT?

-- 
Vincent Legoll

Re: Need help fixing testcases with time-difference

[Hartmut Goebel]

Am 18.08.2016 um 10:59 schrieb Vincent Legoll:
> That looks like a Daylight Saving Time mismatch, could that be possible ?

That's been my first thought, too. But the test is using timezone
"Africa/Algiers" (see
<https://github.com/django/django/blob/master/tests/file_storage/tests.py#L152>)
which AFAIK does not have DST.

Maybe the timezone data is not available when running the test?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Need help fixing testcases with time-difference

[Vincent Legoll]

>> That looks like a Daylight Saving Time mismatch, could that be possible ?
>
> That's been my first thought, too. But the test is using timezone

I've not looked too hard, but that test seems to use the storage to check access
time on a file, could it be that your FS use the "noatime" option or
does not produce
correctly updated file access timestamps ?

-- 
Vincent Legoll

Re: Need help fixing testcases with time-difference

[Hartmut Goebel]

Am 18.08.2016 um 13:10 schrieb Vincent Legoll:
> I've not looked too hard, but that test seems to use the storage to check access
> time on a file, could it be that your FS use the "noatime" option or
> does not produce
> correctly updated file access timestamps ?

Good spot! But my filesystems are mounted with "relatime", so this
should not matter. https://blog.confirm.ch/mount-options-atime-vs-relatime/>

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Need help fixing testcases with time-difference

[Vincent Legoll]

>> I've not looked too hard, but that test seems to use the storage to check access
>> time on a file, could it be that your FS use the "noatime" option or
>> does not produce correctly updated file access timestamps ?
>
> Good spot! But my filesystems are mounted with "relatime", so this
> should not matter. https://blog.confirm.ch/mount-options-atime-vs-relatime/>

Yep the relatime bugs should have been ironed out a few years ago.

D'oh ! That would have been an easy way out...

Are you in a DST-enabled timezone or not ?

-- 
Vincent Legoll

Re: Need help fixing testcases with time-difference

[Vincent Legoll]

Probably a long shot, but... Maybe you can try the following:

change line 158 of tests/file_storage/tests.py:

now_in_algiers = timezone.make_aware(datetime.now())

into

now_in_algiers = timezone.make_aware(datetime.now(), is_dst=False)

and see if this changes the outcome...

-- 
Vincent Legoll

Re: Need help fixing testcases with time-difference

[Marius Bakke]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Hi,
>
> I'm currently working on django, the web application framework.
> Unfortunalty some tests fail. These are all testing time- and timezone
> calculations.
>
> Failures are like this:
>
> AssertionError: datetime.timedelta(0, 3600, 16) not less than
> datetime.timedelta(0, 2)
>
> which means the returned time difference is ca. 1 hour, but allowed are
> only 2 minutes. The testcase os this one
> https://github.com/django/django/blob/master/tests/file_storage/tests.py#L239>
>
> I already added tzdata to native-inputs, but this does not solve the issue.

I think you need to tell it where to find timezone data as well. Try
adding a phase like this:

          (add-before 'check 'set-tzdir
            (lambda* (#:key inputs #:allow-other-keys)
              (setenv "TZDIR"
                      (string-append (assoc-ref inputs "tzdata")
                                     "/share/zoneinfo"))
              #t))

Cheers,
Marius

Re: Need help fixing testcases with time-difference

[Hartmut Goebel]

Hi,

> I think you need to tell it where to find timezone data as well. Try
> adding a phase like this:
>
>           (add-before 'check 'set-tzdir
>             (lambda* (#:key inputs #:allow-other-keys)
>               (setenv "TZDIR"
>                       (string-append (assoc-ref inputs "tzdata")
>                                      "/share/zoneinfo"))
>               #t))

This was what I've been searching for and this did the trick! Thanks a
lot! You are great!

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |