Pretest note

Pretest note

[Chong Yidong]

We're now heading into the final stages of the pretest.  In about a week
(exact date will be given later), prior to the next pretest, we plan
switch to a "regressions only" policy for commits, i.e. fixes only for
bugs that occur in trunk but not in 23.4.

So if you know of a non-regression bug which you think really needs to
be fixed before 24.1, please point it out now.

If you see a regression against 23.4 (which also probably means it needs
to be fixed before 24.1), please go ahead and issue it a severity of
Important or higher in the bug tracker.  (Actually, there are some
Important bugs which won't make it for 24.1, but the list is short
enough that we should be able to sort it out.)

Re: Pretest note

[Jambunathan K]

Chong Yidong <cyd@gnu.org> writes:

> So if you know of a non-regression bug which you think really needs to
> be fixed before 24.1, please point it out now.

I don't know whether this qualifies as non-regression bug. I would like
to see Bug#9914 cleared for Emacs-24.1 release.

See footnote in (info "(org) Literal examples in ODT export").
-- 

Re: Pretest note

[Leo]

I wonder if some Gnus debugging output should be turned off such as:
gnus-backend-trace or nnimap-log-command?

Leo

Re: Pretest note

[Lars Magne Ingebrigtsen]

Leo <sdl.web@gmail.com> writes:

> I wonder if some Gnus debugging output should be turned off such as:
> gnus-backend-trace or nnimap-log-command?

The latter has been switched off already, but I forgot about the first
one.  I'll fix that...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Pretest note

[Leo]

On 2012-03-12 19:12 +0800, Lars Magne Ingebrigtsen wrote:
> The latter has been switched off already, but I forgot about the first
> one.  I'll fix that...

Thanks.

Leo

Re: Pretest note

[Carsten Mattner]

On Mon, Mar 12, 2012 at 4:13 AM, Chong Yidong <cyd@gnu.org> wrote:
> We're now heading into the final stages of the pretest.  In about a week
> (exact date will be given later), prior to the next pretest, we plan
> switch to a "regressions only" policy for commits, i.e. fixes only for
> bugs that occur in trunk but not in 23.4.
>
> So if you know of a non-regression bug which you think really needs to
> be fixed before 24.1, please point it out now.
>
> If you see a regression against 23.4 (which also probably means it needs
> to be fixed before 24.1), please go ahead and issue it a severity of
> Important or higher in the bug tracker.  (Actually, there are some
> Important bugs which won't make it for 24.1, but the list is short
> enough that we should be able to sort it out.)

Don't we want to do something about the leak(s) or at least verify
down that they're correct "caching" or "reuse" behavior?

Re: Pretest note

[John Yates]

On Sun, Mar 11, 2012 at 11:13 PM, Chong Yidong <cyd@gnu.org> wrote:
> We're now heading into the final stages of the pretest.  In about a week
> (exact date will be given later), prior to the next pretest, we plan
> switch to a "regressions only" policy for commits, i.e. fixes only for
> bugs that occur in trunk but not in 23.4.
>
> So if you know of a non-regression bug which you think really needs to
> be fixed before 24.1, please point it out now.
>
> If you see a regression against 23.4 (which also probably means it needs
> to be fixed before 24.1), please go ahead and issue it a severity of
> Important or higher in the bug tracker.  (Actually, there are some
> Important bugs which won't make it for 24.1, but the list is short
> enough that we should be able to sort it out.)

Where do cc-mode performance regressions fall under this policy?

/john

Re: Pretest note

[David De La Harpe Golden]

On 12/03/12 03:13, Chong Yidong wrote:

> So if you know of a non-regression bug which you think really needs to
> be fixed before 24.1, please point it out now.


Well, perhaps an ill-timed can of worms, but the selections area still 
has some irritating known problems AFAICS.  I haven't personally had 
much time to look into them for the past, uh, year.

I think the two of most concern for usability are probably "selection 
resurrection" (#8996) and "surprising scrolled selection" (#9623), 
though YMMV.  Both are still present in the 24.0.94 pretest (tested just 
now), but #9623 might be considered to count as a regression, with the 
appropriate squinting.

#8996 probably doesn't though - obviously select-active-regions didn't 
default to on in 23, but if you do turn it on the same behaviour is 
observed, so not a regression, same bug in 23 and 24.

#8996 has a working patch, but it is ugly and probably not trunkworthy,
there has to be a neater way to do it...

Re: Pretest note

[Chong Yidong]

Carsten Mattner <carstenmattner@googlemail.com> writes:

> Don't we want to do something about the leak(s) or at least verify
> down that they're correct "caching" or "reuse" behavior?

AFAICT, this is Jan's report that memory is not returned to the system
on Mac OS?  Obviously, if someone can help pin this down, that will be
appreciated.  Yamamoto Mitsuharu gave one suggestion here:

http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.

Also, I'd like to know whether it really involves memory not being
returned at all, independently of gnutls and networking.  For example,
here is some Elisp code that visits a file and kills the buffer a few
thousand times:

(dotimes (n 50000)
  (with-temp-buffer
    (insert-file-contents "/path/to/emacs/etc/NEWS")))

If Emacs on Mac OS really never returns memory, this should chew up all
the memory on your system.  Does it?  Check also if Emacs 23 is
affected.

Re: Pretest note

[Chong Yidong]

John Yates <john@yates-sheets.org> writes:

> Where do cc-mode performance regressions fall under this policy?

They are counted as regressions.  There are a number of issues that Alan
is actively working, I believe.

Re: Pretest note

[Carsten Mattner]

On Tue, Mar 13, 2012 at 7:25 AM, Chong Yidong <cyd@gnu.org> wrote:
> Carsten Mattner <carstenmattner@googlemail.com> writes:
>
>> Don't we want to do something about the leak(s) or at least verify
>> down that they're correct "caching" or "reuse" behavior?
>
> AFAICT, this is Jan's report that memory is not returned to the system
> on Mac OS?  Obviously, if someone can help pin this down, that will be
> appreciated.  Yamamoto Mitsuharu gave one suggestion here:

While Jan was able to fix some obvious Cocoa issues and crashes,
this is not a Mac OS specific problem. I see the same extensive
non-reclaimed memory growth on Linux in both the X and terminal
client.

> http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.
>
> Also, I'd like to know whether it really involves memory not being
> returned at all, independently of gnutls and networking.  For example,

No network or gnutls use in any of my Linux and OS X builds.

config options on Linux:
--without-selinux --without-tiff \
    --without-sound --without-dbus --without-gconf \
    --without-gsettings --without-xft --without-gsettings \
    --without-jpeg --without-gif --without-gpm \
    --with-x-toolkit=no --without-gnutls"

config options on Mac OS (to force building a 32-bit binary):
CFLAGS=$CFLAGS CC='gcc -arch i386' ./configure --with-ns --without-gnutls

Maybe I should also disable xml2 and png as those might be enabled.

> here is some Elisp code that visits a file and kills the buffer a few
> thousand times:
>
> (dotimes (n 50000)
>  (with-temp-buffer
>    (insert-file-contents "/path/to/emacs/etc/NEWS")))
>
> If Emacs on Mac OS really never returns memory, this should chew up all
> the memory on your system.  Does it?  Check also if Emacs 23 is
> affected.

If and only if that usage pattern is at fault, yes. I'll give it a try.

Re: Pretest note

[Carsten Mattner]

On Tue, Mar 13, 2012 at 10:02 AM, Carsten Mattner
<carstenmattner@googlemail.com> wrote:
> On Tue, Mar 13, 2012 at 7:25 AM, Chong Yidong <cyd@gnu.org> wrote:
>> Carsten Mattner <carstenmattner@googlemail.com> writes:
>>
>>> Don't we want to do something about the leak(s) or at least verify
>>> down that they're correct "caching" or "reuse" behavior?
>>
>> AFAICT, this is Jan's report that memory is not returned to the system
>> on Mac OS?  Obviously, if someone can help pin this down, that will be
>> appreciated.  Yamamoto Mitsuharu gave one suggestion here:
>
> While Jan was able to fix some obvious Cocoa issues and crashes,
> this is not a Mac OS specific problem. I see the same extensive
> non-reclaimed memory growth on Linux in both the X and terminal
> client.
>
>> http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.
>>
>> Also, I'd like to know whether it really involves memory not being
>> returned at all, independently of gnutls and networking.  For example,
>
> No network or gnutls use in any of my Linux and OS X builds.
>
> config options on Linux:
> --without-selinux --without-tiff \
>    --without-sound --without-dbus --without-gconf \
>    --without-gsettings --without-xft --without-gsettings \
>    --without-jpeg --without-gif --without-gpm \
>    --with-x-toolkit=no --without-gnutls"
>
> config options on Mac OS (to force building a 32-bit binary):
> CFLAGS=$CFLAGS CC='gcc -arch i386' ./configure --with-ns --without-gnutls
>
> Maybe I should also disable xml2 and png as those might be enabled.
>
>> here is some Elisp code that visits a file and kills the buffer a few
>> thousand times:
>>
>> (dotimes (n 50000)
>>  (with-temp-buffer
>>    (insert-file-contents "/path/to/emacs/etc/NEWS")))
>>
>> If Emacs on Mac OS really never returns memory, this should chew up all
>> the memory on your system.  Does it?  Check also if Emacs 23 is
>> affected.
>
> If and only if that usage pattern is at fault, yes. I'll give it a try.

Let's not forget that I'd like to use --daemon, but that's not practical
as long as I don't have more physical memory or the leak is
plugged.

<seriously>
If we don't find the problem, I'm not insisting on delaying the release
for me personally. I'm fine tracking bzr and restarting Emacs on an
hourly basis to try one of the two daily rebuilds. I'm used to it.
</seriously>

<rant>
A shorter release cycle would remove the pressure, especially
as most users seem to have enough physical memory to not
care about the leak or fast enough machines to live with the
cc-mode performance regressions. More users should ideally
equal more testers and more useful feedback.
</rant>

Chong, thanks for taking this seriously and writing it off as ghosts
seen by a couple users.

Re: Pretest note

[Carsten Mattner]

On Tue, Mar 13, 2012 at 10:16 AM, Carsten Mattner
<carstenmattner@googlemail.com> wrote:
> Chong, thanks for taking this seriously and writing it off as ghosts

* and _not_ writing it off

> seen by a couple users.

Re: Pretest note

[Tassilo Horn]

Carsten Mattner <carstenmattner@googlemail.com> writes:

Hi Carsten,

>> No network or gnutls use in any of my Linux and OS X builds.
>>
>> config options on Linux:
>> --without-selinux --without-tiff \
>>    --without-sound --without-dbus --without-gconf \
>>    --without-gsettings --without-xft --without-gsettings \
>>    --without-jpeg --without-gif --without-gpm \
>>    --with-x-toolkit=no --without-gnutls"
>>
>> config options on Mac OS (to force building a 32-bit binary):
>> CFLAGS=$CFLAGS CC='gcc -arch i386' ./configure --with-ns --without-gnutls
>>
>> Maybe I should also disable xml2 and png as those might be enabled.

If you have ImageMagick installed, then I think emacs will build with
support for it.  So you might also want to say --without-imagemagick.
Ditto for --without-rsvg.

That said, if you don't open images/SVGs, neither one should be the
cause of the leak.

Bye,
Tassilo

Re: Pretest note

[Jan Djärv]

Hello.

13 mar 2012 kl. 07:25 skrev Chong Yidong <cyd@gnu.org>:

> Carsten Mattner <carstenmattner@googlemail.com> writes:
> 
>> Don't we want to do something about the leak(s) or at least verify
>> down that they're correct "caching" or "reuse" behavior?
> 
> AFAICT, this is Jan's report that memory is not returned to the system
> on Mac OS?  

I can not be sure that it is never returned. It is not returned as much as it is on GNU/Linux. 


> , if someone can help pin this down, that will be
> appreciated.  Yamamoto Mitsuharu gave one suggestion here:
> 
> http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.
> 
> Also, I'd like to know whether it really involves memory not being
> returned at all, independently of gnutls and networking.  For example,
> here is some Elisp code that visits a file and kills the buffer a few
> thousand times:
> 
> (dotimes (n 50000)
>  (with-temp-buffer
>    (insert-file-contents "/path/to/emacs/etc/NEWS")))
> 
> If Emacs on Mac OS really never returns memory, this should chew up all
> the memory on your system.  Does it?  Check also if Emacs 23 is
> affected.

     Jan D. 

Re: Pretest note

[Carsten Mattner]

On Tue, Mar 13, 2012 at 10:33 AM, Tassilo Horn <tassilo@member.fsf.org> wrote:
> If you have ImageMagick installed, then I think emacs will build with
> support for it.  So you might also want to say --without-imagemagick.
> Ditto for --without-rsvg.

Thanks Tassilo.

I don't have imagemagick or rsvg installed, but I'll add it to the list
just in case I ever happen to build on a system with.

Updated and cleaned up linux opts:
--without-selinux  --without-dbus --without-gconf \
    --without-gsettings --without-sound --without-xft --without-gpm \
    --without-tiff --without-jpeg --without-gif --without-png \
    --with-x-toolkit=no --without-gnutls --without-xml2 \
    --without-rsvg --without-imagemagick

Added --without-libxml2 to the Mac configure settings:
CFLAGS=$CFLAGS CC='gcc -arch i386' ./configure --with-ns \
    --without-gnutls --without-xml2

> That said, if you don't open images/SVGs, neither one should be the
> cause of the leak.

Makes sense.
I do wonder how --with-ns is able to make use of XPMs without linking libXpm
according to the autoconf results reports.
Should I add the same --without options to the --with-ns build?
autoconf doesn't indicate to add support for those libs but apparently
Cocoa has other means to indireclty make use of them via ImageKit or
whatever the library is called.

Re: Pretest note

[Jan Djärv]

13 mar 2012 kl. 07:25 skrev Chong Yidong:

> Carsten Mattner <carstenmattner@googlemail.com> writes:
> 
>> Don't we want to do something about the leak(s) or at least verify
>> down that they're correct "caching" or "reuse" behavior?
> 
> AFAICT, this is Jan's report that memory is not returned to the system
> on Mac OS?  Obviously, if someone can help pin this down, that will be
> appreciated.  Yamamoto Mitsuharu gave one suggestion here:
> 
> http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.
> 
> Also, I'd like to know whether it really involves memory not being
> returned at all, independently of gnutls and networking.  For example,
> here is some Elisp code that visits a file and kills the buffer a few
> thousand times:
> 
> (dotimes (n 50000)
>  (with-temp-buffer
>    (insert-file-contents "/path/to/emacs/etc/NEWS")))
> 
> If Emacs on Mac OS really never returns memory, this should chew up all
> the memory on your system.  Does it?  Check also if Emacs 23 is
> affected.

I don't see much difference.
After executing, Emacs 23 uses 33.9 Mbyte real memory, 41.2 virtual.
Emacs trunk: 35.4/42.8

Garbage-collect after the run has no effect.

	Jan D.

Re: Pretest note

[Carsten Mattner]

shared libs used by my --with-ns emacs build:
$ otool -L Emacs:
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
(compatibility version 45.0.0, current version 1138.32.0)

/usr/lib/libresolv.9.dylib
(compatibility version 1.0.0, current version 46.1.0)

/usr/lib/libncurses.5.4.dylib
(compatibility version 5.4.0, current version 5.4.0)

/usr/lib/libSystem.B.dylib
(compatibility version 1.0.0, current version 159.1.0)

/usr/lib/libobjc.A.dylib
(compatibility version 1.0.0, current version 228.0.0)

/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
(compatibility version 1.0.0, current version 53.0.0)

/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
(compatibility version 150.0.0, current version 635.19.0)

/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
(compatibility version 1.0.0, current version 41.0.0)

/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
(compatibility version 300.0.0, current version 833.24.0)

Re: Pretest note

[Stefan Monnier]

> #8996 has a working patch, but it is ugly and probably not trunkworthy,
> there has to be a neater way to do it...

I didn't like it at first, but I'm beginning to find it acceptable (tho
the name `skip-active-region-selection' doesn't sound that good).
Maybe another way to do it is to compare selected-window before and
after running the command.


        Stefan

Re: Pretest note

[Chong Yidong]

Jan Djärv <jan.h.d@swipnet.se> writes:

>> (dotimes (n 50000)
>>  (with-temp-buffer
>>    (insert-file-contents "/path/to/emacs/etc/NEWS")))
>> 
>> If Emacs on Mac OS really never returns memory, this should chew up
>> all the memory on your system.  Does it?  Check also if Emacs 23 is
>> affected.
>
> I don't see much difference.
> After executing, Emacs 23 uses 33.9 Mbyte real memory, 41.2 virtual.
> Emacs trunk: 35.4/42.8
>
> Garbage-collect after the run has no effect.

Then memory can be returned, at least in some circumstances.  Otherwise
50,000 copies of NEWS would take up 3GB of memory.

Re: Pretest note

[Carsten Mattner]

On Tue, Mar 13, 2012 at 7:25 AM, Chong Yidong <cyd@gnu.org> wrote:
> Carsten Mattner <carstenmattner@googlemail.com> writes:
>
>> Don't we want to do something about the leak(s) or at least verify
>> down that they're correct "caching" or "reuse" behavior?
>
> AFAICT, this is Jan's report that memory is not returned to the system
> on Mac OS?  Obviously, if someone can help pin this down, that will be
> appreciated.  Yamamoto Mitsuharu gave one suggestion here:
>
> http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00358.html.
>
> Also, I'd like to know whether it really involves memory not being
> returned at all, independently of gnutls and networking.  For example,
> here is some Elisp code that visits a file and kills the buffer a few
> thousand times:
>
> (dotimes (n 50000)
>  (with-temp-buffer
>    (insert-file-contents "/path/to/emacs/etc/NEWS")))

This keeps emacs 99% busy for a couple minutes and increases
the resident size by 1 or 2 MB.

What's more interesting is that today's bzr build
even with -Q and -Q -nw seems to use more than 1.5x times memory
after startup than it used to.

To rule out any issues caused by --without-xml2 I have rebuilt and
it still start Emacs.app at 42MB while it used to start at 25MB.

What did change?

> If Emacs on Mac OS really never returns memory, this should chew up all
> the memory on your system.  Does it?  Check also if Emacs 23 is
> affected.

Re: Pretest note

[Angelo Graziosi]

Chong Yidong wrote:

> If you see a regression against 23.4

I flagged a few times this:

http://lists.gnu.org/archive/html/emacs-devel/2011-07/msg01118.html
http://lists.gnu.org/archive/html/bug-gnu-emacs/2011-07/msg02023.html
http://lists.gnu.org/archive/html/emacs-devel/2011-11/msg00108.html

The issue is still there.

Verified on Fedora 14, 16; Kubuntu 11.4, 11.10; Ubuntu 11.10; Cygwin 
1.7.9, 1.7.10, 1.7.11 : in all cases GTK build (also gtk3)

  Angelo