Quantcast

Severe memory leak in zope.i18nmessageid fixed

classic Classic list List threaded Threaded
9 messages Options
Hanno Schlichting-4 Hanno Schlichting-4
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Severe memory leak in zope.i18nmessageid fixed

Hi.

In my series of "goodies from Zope" I found a nice ticket related to
zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
patch waiting since end of 2009. The problem is found in all Zope
versions since 2.9, which corresponds to Plone 2.5. In a Plone context
this causes some small memory leaks to occur on every page view. So
the more traffic you have, the more severe this problem hits you.

The fix is now released in
http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
Seaver. This will make it into the next Zope 2.12.5 release and into
the next Plone 4.0 beta 3.

The 3.5.1 release is compatible with Zope 2.10 and later, so I
installed it into two larger Plone sites of ours; both of which had
seen major memory problems. We have employed the usual workaround of
scheduling daily to weekly automatic restarts for the Zope processes
so far.

The monitoring data for the last 30 hours looks like this has indeed
fixed the most severe memory problems. After the initial caches are
filled, the memory stabilizes and doesn't seem to grow at all anymore.
So far we had seen memory growth near one gigabyte per day.

If you have any Plone sites which show similar memory problems, I
suggest you try out the new zope.i18nmessageid release. I've run some
local tests and there's certainly more memory leaks in Plone, but none
as severe as this one.

Hanno

[1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
[2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
JonStahl JonStahl
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

Nice work, Hanno!

Is this a significant enough improvement to merit a 3.3.6 release that
incorporates it?

:jon

On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:

> Hi.
>
> In my series of "goodies from Zope" I found a nice ticket related to
> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
> patch waiting since end of 2009. The problem is found in all Zope
> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
> this causes some small memory leaks to occur on every page view. So
> the more traffic you have, the more severe this problem hits you.
>
> The fix is now released in
> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
> Seaver. This will make it into the next Zope 2.12.5 release and into
> the next Plone 4.0 beta 3.
>
> The 3.5.1 release is compatible with Zope 2.10 and later, so I
> installed it into two larger Plone sites of ours; both of which had
> seen major memory problems. We have employed the usual workaround of
> scheduling daily to weekly automatic restarts for the Zope processes
> so far.
>
> The monitoring data for the last 30 hours looks like this has indeed
> fixed the most severe memory problems. After the initial caches are
> filled, the memory stabilizes and doesn't seem to grow at all anymore.
> So far we had seen memory growth near one gigabyte per day.
>
> If you have any Plone sites which show similar memory problems, I
> suggest you try out the new zope.i18nmessageid release. I've run some
> local tests and there's certainly more memory leaks in Plone, but none
> as severe as this one.
>
> Hanno
>
> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
aclark aclark
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

On 2010-04-17, Jon Stahl <[hidden email]> wrote:
> Nice work, Hanno!

Ditto! *loud clapping*

>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?
>
>:jon
>
> On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
>> Hi.
>>
>> In my series of "goodies from Zope" I found a nice ticket related to
>> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
>> patch waiting since end of 2009. The problem is found in all Zope
>> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
>> this causes some small memory leaks to occur on every page view. So
>> the more traffic you have, the more severe this problem hits you.
>>
>> The fix is now released in
>> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
>> Seaver. This will make it into the next Zope 2.12.5 release and into
>> the next Plone 4.0 beta 3.
>>
>> The 3.5.1 release is compatible with Zope 2.10 and later, so I
>> installed it into two larger Plone sites of ours; both of which had
>> seen major memory problems. We have employed the usual workaround of
>> scheduling daily to weekly automatic restarts for the Zope processes
>> so far.
>>
>> The monitoring data for the last 30 hours looks like this has indeed
>> fixed the most severe memory problems. After the initial caches are
>> filled, the memory stabilizes and doesn't seem to grow at all anymore.
>> So far we had seen memory growth near one gigabyte per day.
>>
>> If you have any Plone sites which show similar memory problems, I
>> suggest you try out the new zope.i18nmessageid release. I've run some
>> local tests and there's certainly more memory leaks in Plone, but none
>> as severe as this one.
>>
>> Hanno
>>
>> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
>> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>>
>> ------------------------------------------------------------------------------
>> Download Intel&#174; Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Plone-developers mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/plone-developers
>>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev


--
Alex Clark · http://aclark.net
Author of Plone 3.3 Site Administration · http://aclark.net/plone-site-admin


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Hans-Peter Locher Hans-Peter Locher
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by JonStahl
Hey, great job!

Am 17.04.2010 um 19:07 schrieb Jon Stahl:

> Nice work, Hanno!
>
> Is this a significant enough improvement to merit a 3.3.6 release that
> incorporates it?

+1

>
> :jon
>
> On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
>> Hi.
>>
>> In my series of "goodies from Zope" I found a nice ticket related to
>> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
>> patch waiting since end of 2009. The problem is found in all Zope
>> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
>> this causes some small memory leaks to occur on every page view. So
>> the more traffic you have, the more severe this problem hits you.
>>
>> The fix is now released in
>> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
>> Seaver. This will make it into the next Zope 2.12.5 release and into
>> the next Plone 4.0 beta 3.
>>
>> The 3.5.1 release is compatible with Zope 2.10 and later, so I
>> installed it into two larger Plone sites of ours; both of which had
>> seen major memory problems. We have employed the usual workaround of
>> scheduling daily to weekly automatic restarts for the Zope processes
>> so far.
>>
>> The monitoring data for the last 30 hours looks like this has indeed
>> fixed the most severe memory problems. After the initial caches are
>> filled, the memory stabilizes and doesn't seem to grow at all anymore.
>> So far we had seen memory growth near one gigabyte per day.
>>
>> If you have any Plone sites which show similar memory problems, I
>> suggest you try out the new zope.i18nmessageid release. I've run some
>> local tests and there's certainly more memory leaks in Plone, but none
>> as severe as this one.
>>
>> Hanno
>>
>> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
>> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>>
>> ------------------------------------------------------------------------------
>> Download Intel&#174; Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Plone-developers mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/plone-developers
>>
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>



------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Alexander Limi Alexander Limi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by JonStahl
I'd say it is, but that's ultimately Wichert's call. :)

Thanks for fixing this, and thanks to the people that submitted the patches!

On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]> wrote:
Nice work, Hanno!

Is this a significant enough improvement to merit a 3.3.6 release that
incorporates it?

:jon

On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting <[hidden email]> wrote:
> Hi.
>
> In my series of "goodies from Zope" I found a nice ticket related to
> zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
> patch waiting since end of 2009. The problem is found in all Zope
> versions since 2.9, which corresponds to Plone 2.5. In a Plone context
> this causes some small memory leaks to occur on every page view. So
> the more traffic you have, the more severe this problem hits you.
>
> The fix is now released in
> http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
> Seaver. This will make it into the next Zope 2.12.5 release and into
> the next Plone 4.0 beta 3.
>
> The 3.5.1 release is compatible with Zope 2.10 and later, so I
> installed it into two larger Plone sites of ours; both of which had
> seen major memory problems. We have employed the usual workaround of
> scheduling daily to weekly automatic restarts for the Zope processes
> so far.
>
> The monitoring data for the last 30 hours looks like this has indeed
> fixed the most severe memory problems. After the initial caches are
> filled, the memory stabilizes and doesn't seem to grow at all anymore.
> So far we had seen memory growth near one gigabyte per day.
>
> If you have any Plone sites which show similar memory problems, I
> suggest you try out the new zope.i18nmessageid release. I've run some
> local tests and there's certainly more memory leaks in Plone, but none
> as severe as this one.
>
> Hanno
>
> [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
> [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers



--
Alexander Limi · http://limi.net

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Alexander Limi · http://limi.net

Wichert Akkerman Wichert Akkerman
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

On 4/19/10 08:39 , Alexander Limi wrote:

> I'd say it is, but that's ultimately Wichert's call. :)
>
> Thanks for fixing this, and thanks to the people that submitted the patches!
>
> On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Nice work, Hanno!
>
>     Is this a significant enough improvement to merit a 3.3.6 release that
>     incorporates it?

Lets not overdo our top-quoting..

It's up to Hanno to make a new Zope 2.10 release I'ld say.

Wichert.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Alexander Limi Alexander Limi
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

On Mon, Apr 19, 2010 at 12:04 AM, Wichert Akkerman <[hidden email]> wrote:
On 4/19/10 08:39 , Alexander Limi wrote:
I'd say it is, but that's ultimately Wichert's call. :)

Thanks for fixing this, and thanks to the people that submitted the patches!

On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
<mailto:[hidden email]>> wrote:

   Nice work, Hanno!

   Is this a significant enough improvement to merit a 3.3.6 release that
   incorporates it?

Lets not overdo our top-quoting..
 
Heh. When people start top-quoting, I'm too lazy to fix their formatting.

It's up to Hanno to make a new Zope 2.10 release I'ld say.

Ah, true.

--
Alexander Limi · http://limi.net

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Alexander Limi · http://limi.net

Hanno Schlichting-4 Hanno Schlichting-4
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by Wichert Akkerman
On Mon, Apr 19, 2010 at 9:04 AM, Wichert Akkerman <[hidden email]> wrote:
> On 4/19/10 08:39 , Alexander Limi wrote:
>> Thanks for fixing this, and thanks to the people that submitted the patches!

Yep, I didn't do anything, except call this to your attention :)

> It's up to Hanno to make a new Zope 2.10 release I'ld say.

Huh? There's no a release manager for Zope 2.11 and 2.10 and until
someone steps up, they are unmaintained ever since Andreas stepped
down. And I'm definitely not going to step up for these old releases.

So someone else needs to backport these fixes to the appropriate
zope.i18nmessageid branch and release a new Zope 2.10.

Hanno

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
yuri-2 yuri-2
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Severe memory leak in zope.i18nmessageid fixed

In reply to this post by Alexander Limi
Alexander Limi ha scritto:
> I'd say it is, but that's ultimately Wichert's call. :)
>
> Thanks for fixing this, and thanks to the people that submitted the
> patches!

There'are several software that can detect memleaks in C code. Any
chance someone used to it, to do it for C parts in Plone? :)

>
> On Sat, Apr 17, 2010 at 10:07 AM, Jon Stahl <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Nice work, Hanno!
>
>     Is this a significant enough improvement to merit a 3.3.6 release that
>     incorporates it?
>
>     :jon
>
>     On Sat, Apr 17, 2010 at 7:35 AM, Hanno Schlichting
>     <[hidden email] <mailto:[hidden email]>> wrote:
>     > Hi.
>     >
>     > In my series of "goodies from Zope" I found a nice ticket related to
>     > zope.i18nmessageid ([1], [2]). Originally reported in 2008 it had a
>     > patch waiting since end of 2009. The problem is found in all Zope
>     > versions since 2.9, which corresponds to Plone 2.5. In a Plone
>     context
>     > this causes some small memory leaks to occur on every page view. So
>     > the more traffic you have, the more severe this problem hits you.
>     >
>     > The fix is now released in
>     > http://pypi.python.org/pypi/zope.i18nmessageid/3.5.1 thanks to Tres
>     > Seaver. This will make it into the next Zope 2.12.5 release and into
>     > the next Plone 4.0 beta 3.
>     >
>     > The 3.5.1 release is compatible with Zope 2.10 and later, so I
>     > installed it into two larger Plone sites of ours; both of which had
>     > seen major memory problems. We have employed the usual workaround of
>     > scheduling daily to weekly automatic restarts for the Zope processes
>     > so far.
>     >
>     > The monitoring data for the last 30 hours looks like this has indeed
>     > fixed the most severe memory problems. After the initial caches are
>     > filled, the memory stabilizes and doesn't seem to grow at all
>     anymore.
>     > So far we had seen memory growth near one gigabyte per day.
>     >
>     > If you have any Plone sites which show similar memory problems, I
>     > suggest you try out the new zope.i18nmessageid release. I've run
>     some
>     > local tests and there's certainly more memory leaks in Plone,
>     but none
>     > as severe as this one.
>     >
>     > Hanno
>     >
>     > [1] https://bugs.launchpad.net/zopetoolkit-project/+bug/257657
>     > [2] https://bugs.launchpad.net/zopetoolkit-project/+bug/489529
>     >
>     >
>     ------------------------------------------------------------------------------
>     > Download Intel&#174; Parallel Studio Eval
>     > Try the new software tools for yourself. Speed compiling, find bugs
>     > proactively, and fine-tune applications for parallel performance.
>     > See why Intel Parallel Studio got high marks during beta.
>     > http://p.sf.net/sfu/intel-sw-dev
>     > _______________________________________________
>     > Plone-developers mailing list
>     > [hidden email]
>     <mailto:[hidden email]>
>     > https://lists.sourceforge.net/lists/listinfo/plone-developers
>     >
>
>     ------------------------------------------------------------------------------
>     Download Intel&#174; Parallel Studio Eval
>     Try the new software tools for yourself. Speed compiling, find bugs
>     proactively, and fine-tune applications for parallel performance.
>     See why Intel Parallel Studio got high marks during beta.
>     http://p.sf.net/sfu/intel-sw-dev
>     _______________________________________________
>     Plone-developers mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.sourceforge.net/lists/listinfo/plone-developers
>
>
>
>
> --
> Alexander Limi · http://limi.net
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Download Intel&#174; Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ------------------------------------------------------------------------
>
> _______________________________________________
> Plone-developers mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>  


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Loading...