Plone › Core Developers

Severe memory leak in zope.i18nmessageid fixed

Classic

List

Threaded

9 messages Options

Hanno Schlichting-4

Severe memory leak in zope.i18nmessageid fixed

Hi.

In my series of "goodies from Zope" I found a nice ticket related to
zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
patch waiting since end of 2009. The problem is found in all Zope
versions since 2.9, which corresponds to Plone 2.5. In a Plone context
this causes some small memory leaks to occur on every page view. So
the more traffic you have, the more severe this problem hits you.

The fix is now released in
http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
Seaver. This will make it into the next Zope 2.12.5 release and into
the next Plone 4.0 beta 3.

The 3.5.1 release is compatible with Zope 2.10 and later, so I
installed it into two larger Plone sites of ours; both of which had
seen major memory problems. We have employed the usual workaround of
scheduling daily to weekly automatic restarts for the Zope processes
so far.

The monitoring data for the last 30 hours looks like this has indeed
fixed the most severe memory problems. After the initial caches are
filled, the memory stabilizes and doesn't seem to grow at all anymore.
So far we had seen memory growth near one gigabyte per day.

If you have any Plone sites which show similar memory problems, I
suggest you try out the new zope.i18nmessageid release. I've run some
local tests and there's certainly more memory leaks in Plone, but none
as severe as this one.

Hanno

[1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
[2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

Remove Ads

JonStahl

Re: Severe memory leak in zope.i18nmessageid fixed

Nice work, Hanno!

Is this a significant enough improvement to merit a 3.3.6 release that
incorporates it?

:jon

On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:

> Hi.
>
> In my series of "goodies from Zope" I found a nice ticket related to
> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
> patch waiting since end of 2009. The problem is found in all Zope
> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
> this causes some small memory leaks to occur on every page view. So
> the more traffic you have, the more severe this problem hits you.
>
> The fix is now released in
> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
> Seaver. This will make it into the next Zope 2.12.5 release and into
> the next Plone 4.0 beta 3.
>
> The 3.5.1 release is compatible with Zope 2.10 and later, so I
> installed it into two larger Plone sites of ours; both of which had
> seen major memory problems. We have employed the usual workaround of
> scheduling daily to weekly automatic restarts for the Zope processes
> so far.
>
> The monitoring data for the last 30 hours looks like this has indeed
> fixed the most severe memory problems. After the initial caches are
> filled, the memory stabilizes and doesn't seem to grow at all anymore.
> So far we had seen memory growth near one gigabyte per day.
>
> If you have any Plone sites which show similar memory problems, I
> suggest you try out the new zope.i18nmessageid release. I've run some
> local tests and there's certainly more memory leaks in Plone, but none
> as severe as this one.
>
> Hanno
>
> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

aclark

Re: Severe memory leak in zope.i18nmessageid fixed

On 2010-04-17, Jon Stahl <[hidden email]> wrote:
> Nice work, Hanno!

Ditto! *loud clapping*

>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?
>
>:jon
>
> On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
>> Hi.
>>
>> In my series of "goodies from Zope" I found a nice ticket related to
>> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
>> patch waiting since end of 2009. The problem is found in all Zope
>> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
>> this causes some small memory leaks to occur on every page view. So
>> the more traffic you have, the more severe this problem hits you.
>>
>> The fix is now released in
>> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
>> Seaver. This will make it into the next Zope 2.12.5 release and into
>> the next Plone 4.0 beta 3.
>>
>> The 3.5.1 release is compatible with Zope 2.10 and later, so I
>> installed it into two larger Plone sites of ours; both of which had
>> seen major memory problems. We have employed the usual workaround of
>> scheduling daily to weekly automatic restarts for the Zope processes
>> so far.
>>
>> The monitoring data for the last 30 hours looks like this has indeed
>> fixed the most severe memory problems. After the initial caches are
>> filled, the memory stabilizes and doesn't seem to grow at all anymore.
>> So far we had seen memory growth near one gigabyte per day.
>>
>> If you have any Plone sites which show similar memory problems, I
>> suggest you try out the new zope.i18nmessageid release. I've run some
>> local tests and there's certainly more memory leaks in Plone, but none
>> as severe as this one.
>>
>> Hanno
>>
>> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
>> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Plone-developers mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/plone-developers
>>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev

--
Alex Clark · http://aclark.net
Author of Plone 3.3 Site Administration · http://aclark.net/plone-site-admin

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

Hans-Peter Locher

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by JonStahl

Hey, great job!

Am 17.04.2010 um 19:07 schrieb Jon Stahl:

> Nice work, Hanno!
>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?

+1

>
> :jon
>
> On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
>> Hi.
>>
>> In my series of "goodies from Zope" I found a nice ticket related to
>> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
>> patch waiting since end of 2009. The problem is found in all Zope
>> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
>> this causes some small memory leaks to occur on every page view. So
>> the more traffic you have, the more severe this problem hits you.
>>
>> The fix is now released in
>> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
>> Seaver. This will make it into the next Zope 2.12.5 release and into
>> the next Plone 4.0 beta 3.
>>
>> The 3.5.1 release is compatible with Zope 2.10 and later, so I
>> installed it into two larger Plone sites of ours; both of which had
>> seen major memory problems. We have employed the usual workaround of
>> scheduling daily to weekly automatic restarts for the Zope processes
>> so far.
>>
>> The monitoring data for the last 30 hours looks like this has indeed
>> fixed the most severe memory problems. After the initial caches are
>> filled, the memory stabilizes and doesn't seem to grow at all anymore.
>> So far we had seen memory growth near one gigabyte per day.
>>
>> If you have any Plone sites which show similar memory problems, I
>> suggest you try out the new zope.i18nmessageid release. I've run some
>> local tests and there's certainly more memory leaks in Plone, but none
>> as severe as this one.
>>
>> Hanno
>>
>> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
>> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Plone-developers mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/plone-developers
>>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

Alexander Limi

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by JonStahl

I'd say it is, but that's ultimately Wichert's call. :)

Thanks for fixing this, and thanks to the people that submitted the patches!

On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]> wrote:

Nice work, Hanno!

Is this a significant enough improvement to merit a 3.3.6 release that
incorporates it?

:jon

On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
> Hi.
>
> In my series of "goodies from Zope" I found a nice ticket related to
> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
> patch waiting since end of 2009. The problem is found in all Zope
> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
> this causes some small memory leaks to occur on every page view. So
> the more traffic you have, the more severe this problem hits you.
>
> The fix is now released in
> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
> Seaver. This will make it into the next Zope 2.12.5 release and into
> the next Plone 4.0 beta 3.
>
> The 3.5.1 release is compatible with Zope 2.10 and later, so I
> installed it into two larger Plone sites of ours; both of which had
> seen major memory problems. We have employed the usual workaround of
> scheduling daily to weekly automatic restarts for the Zope processes
> so far.
>
> The monitoring data for the last 30 hours looks like this has indeed
> fixed the most severe memory problems. After the initial caches are
> filled, the memory stabilizes and doesn't seem to grow at all anymore.
> So far we had seen memory growth near one gigabyte per day.
>
> If you have any Plone sites which show similar memory problems, I
> suggest you try out the new zope.i18nmessageid release. I've run some
> local tests and there's certainly more memory leaks in Plone, but none
> as severe as this one.
>
> Hanno
>
> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

--
Alexander Limi · http://limi.net

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

Alexander Limi · http://limi.net

Wichert Akkerman

Re: Severe memory leak in zope.i18nmessageid fixed

On 4/19/10 08:39 , Alexander Limi wrote:

> I'd say it is, but that's ultimately Wichert's call. :)
>
> Thanks for fixing this, and thanks to the people that submitted the patches!
>
> On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Nice work, Hanno!
>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?

Lets not overdo our top-quoting..

It's up to Hanno to make a new Zope 2.10 release I'ld say.

Wichert.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

Alexander Limi

Re: Severe memory leak in zope.i18nmessageid fixed

On Mon, Apr 19, 2010 at 12:04 AM, Wichert Akkerman <[hidden email]> wrote:

On 4/19/10 08:39 , Alexander Limi wrote:

I'd say it is, but that's ultimately Wichert's call. :)

Thanks for fixing this, and thanks to the people that submitted the patches!

On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
<mailto:[hidden email]>> wrote:

Nice work, Hanno!

Is this a significant enough improvement to merit a 3.3.6 release that
incorporates it?

Lets not overdo our top-quoting..

Heh. When people start top-quoting, I'm too lazy to fix their formatting.

It's up to Hanno to make a new Zope 2.10 release I'ld say.

Ah, true.

Alexander Limi · http://limi.net

Hanno Schlichting-4

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by Wichert Akkerman

On Mon, Apr 19, 2010 at 9:04 AM, Wichert Akkerman <[hidden email]> wrote:
> On 4/19/10 08:39 , Alexander Limi wrote:
>> Thanks for fixing this, and thanks to the people that submitted the patches!

Yep, I didn't do anything, except call this to your attention :)

> It's up to Hanno to make a new Zope 2.10 release I'ld say.

Huh? There's no a release manager for Zope 2.11 and 2.10 and until
someone steps up, they are unmaintained ever since Andreas stepped
down. And I'm definitely not going to step up for these old releases.

So someone else needs to backport these fixes to the appropriate
zope.i18nmessageid branch and release a new Zope 2.10.

Hanno

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

Yuri

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by Alexander Limi

Alexander Limi ha scritto:
> I'd say it is, but that's ultimately Wichert's call. :)
>
> Thanks for fixing this, and thanks to the people that submitted the
> patches!

There'are several software that can detect memleaks in C code. Any
chance someone used to it, to do it for C parts in Plone? :)

>
> On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Nice work, Hanno!
>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?
>
> :jon
>
> On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting
> <[hidden email] <mailto:[hidden email]>> wrote:
> > Hi.
> >
> > In my series of "goodies from Zope" I found a nice ticket related to
> > zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
> > patch waiting since end of 2009. The problem is found in all Zope
> > versions since 2.9, which corresponds to Plone 2.5. In a Plone
> context
> > this causes some small memory leaks to occur on every page view. So
> > the more traffic you have, the more severe this problem hits you.
> >
> > The fix is now released in
> > http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
> > Seaver. This will make it into the next Zope 2.12.5 release and into
> > the next Plone 4.0 beta 3.
> >
> > The 3.5.1 release is compatible with Zope 2.10 and later, so I
> > installed it into two larger Plone sites of ours; both of which had
> > seen major memory problems. We have employed the usual workaround of
> > scheduling daily to weekly automatic restarts for the Zope processes
> > so far.
> >
> > The monitoring data for the last 30 hours looks like this has indeed
> > fixed the most severe memory problems. After the initial caches are
> > filled, the memory stabilizes and doesn't seem to grow at all
> anymore.
> > So far we had seen memory growth near one gigabyte per day.
> >
> > If you have any Plone sites which show similar memory problems, I
> > suggest you try out the new zope.i18nmessageid release. I've run
> some
> > local tests and there's certainly more memory leaks in Plone,
> but none
> > as severe as this one.
> >
> > Hanno
> >
> > [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
> > [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
> >
> >
> ------------------------------------------------------------------------------
> > Download Intel® Parallel Studio Eval
> > Try the new software tools for yourself. Speed compiling, find bugs
> > proactively, and fine-tune applications for parallel performance.
> > See why Intel Parallel Studio got high marks during beta.
> > http://p.sf.net/sfu/intel-sw-dev
> > _______________________________________________
> > Plone-developers mailing list
> > [hidden email]
> <mailto:[hidden email]>
> > https://lists.sourceforge.net/lists/listinfo/plone-developers
> >
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>
>
>
>
> --
> Alexander Limi · http://limi.net
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ------------------------------------------------------------------------
>
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>