Python recursion limit

[anonymous]

On a system with a lot of components (around 1000), the DefaultCC will cause a Python error (maximum recursion depth exceeded) on the component admin page.

It seems to be related to one of the Genshi functions (render) that DefaultCC uses.

[Ryan J Ollos]

Is there any useful info in the stack trace?

[anonymous]

Replying to rjollos:

Is there any useful info in the stack trace?

Unfortunately it is on a system that I can't cut and paste from.

The Genshi version was 0.6-py2.5, and Trac was 0.11.6-py2.6, and I was running DefaultCC 0.2. Although I also observed the same issue on another system that runs the latest version of Trac/Genshi.

The issue is very easy to reproduce. Like I said you just need to add about 1k components (easiest is to just write a simple sqlite script). After that you will see the error when you go to the component admin panel.

According to the traceback, the original call was _dispatch_request from trac/web/main.py at line 450, which in turned called dispatch in the same file at line 227, and then render_template from trac/web/chrome.py at line 773. Then after a bunch of other calls, the __call__ function in /genshi/filters/transform.py will call itself recursively until it crash.

[Ryan J Ollos]

Here is a workaround that seems to be effective:

trunk/defaultcc/admin.py

@@ -10 +10 @@
 # you should have received as part of this distribution.
 #
 
+from genshi import HTML
 from genshi.builder import tag
 from genshi.filters import Transformer
 from trac.core import Component, implements
@@ -156 +157 @@
                                              '/tbody/tr[%d]' % (i + 1))
                         stream |= filter.append(tag.td(default_cc,
                                                        class_='defaultcc'))
+                        # Avoid maximum recusion depth errors.
+                        if i % 100 == 0:
+                            stream = HTML(stream.render())
                 return stream
 
         return stream

I'm not familiar with the internals of Genshi, so there may be a better way. Maybe someone else will have a better idea.

The page rendering is very slow though, and I suspect we can't do much better with Genshi. A workaround would be to modify the table using JavaScript, or perhaps optionally, not append Default CC to the table.

Are you still in need of a fix, and are you running Trac 0.11.6?

[Jun Omae]

We shouldn't use many calls of Transformer() because each call of Transformer() needs one or more stack. CondFieldsGenshiPlugin has the same issue in #10342.

Instead, we should call Transformer.apply with custom transformation method only once. Otherwise, directly transform without Tranformer() like ​Chrome._strip_accesskeys(). See t12198.diff​.

[Ryan J Ollos]

Thanks for the patch! The page renders very fast now. I'll try fixing #10342 as an exercise to better understand the patch :)

[Ryan J Ollos]

In 14805:

0.3dev: Prevent recursion limit with many components.

Patch by Jun Omae. Refs #12198.

[Ryan J Ollos]

In 14806:

0.4dev: Merged [14805] from 0.3 branch. Refs #12198.

[Ryan J Ollos]

In 14807:

0.4dev: Require Trac 1.0 and later on trunk. Refs #12198.

[Ryan J Ollos]

In 14808:

0.4dev: Place Default CC before Default column. Refs #12198.

[Ryan J Ollos]

In 14809:

0.4dev: Removed unused imports. Refs #12198.