Render documentation for modules packaged in EGG | ZIP files | PEP-302 compatible sources

[Olemis Lang]

I implemented this enhancement when I was setting up the Trac environment I use to manage the development of many plugins (especially ​TracGViz plugin which is meant to embed iGoogle gadgets in wiki pages using WikiFormatting). I'm used to see in action in the same site the plugins I build (yes, I eat my own dog food ;). That time I realized that even when TracGViz plugin was installed and living in sys.path, its documentation wasnt shown, and its path was not included in the index page.

I discovered that this was due to two main reasons:

• For the index page, PyDocPlugin filters sys.path explicitly so that only the plugins found in the file system be considered.
• For the module documentation, the fact is that the implementation relies on imp module to load the modules the documentation will be extracted from. This feature is kind of deprecated after PEP 302 -- New Import Hooks has been aproved.

Therefore I wrote a patch to solve the second issue. You can see it in action to generate the ​API reference documentation for TracGViz plugin.

.py

@@ -169 +169 @@
 
     def load_object(self, fullobject):
         """ Load an arbitrary object from a full dotted path. """
+        self.log.debug("Preparing docs for %s", fullobject)
         fullspec = fullobject.split('.')
         i = 0
         module = mfile = mdescr = None
         mpath = self.syspath
+        self.log.debug("Using PATH = %s", mpath)
         # Find module
+        import sys
         if fullspec[0] in sys.builtin_module_names:
             module = __import__(fullspec[0], None, None)
             i += 1
         else:
-            while i < len(fullspec):
+
+#            I removed this since it is not compatible with PEP 302 
+#            import hooks and it doesn't load the modules packaged 
+#            in zip | egg files.
+
+#            while i < len(fullspec):
+#                try:
+#                    f, p, mdescr = imp.find_module(fullspec[i], mpath)
+#                    if mfile:
+#                        mfile.close()
+#                    mfile, mpath = f, [p]
+#                    i += 1
+#                except ImportError:
+#                    break
+            mname = fullspec[0]
+            import sys              # Ensure loaders cache is updated
+            
+            def importer(path):
                 try:
-                    f, p, mdescr = imp.find_module(fullspec[i], mpath)
-                    if mfile:
-                        mfile.close()
-                    mfile, mpath = f, [p]
-                    i += 1
-                except ImportError:
+                    return sys.path_importer_cache[path]
+                except KeyError: # Not in cache. Try path hooks instead
+                    for imp_type in sys.path_hooks:
+                        try:
+                            return imp_type(path)
+                        except:
+                            pass
+                    else:        # No further choices.
+                        return imp.NullImporter(path)
+            for path in mpath:
+                i = 0
+                _imp = importer(path)
+                if isinstance(_imp, imp.NullImporter):
+                        continue
+                if _imp is None:  # Legacy code. No importer object :(
+                    mpath = [path]
+                    while i < len(fullspec):
+                        try:
+                            f, p, mdescr = imp.find_module(fullspec[i], mpath)
+                            if mfile:
+                                mfile.close()
+                            mfile, mpath = f, [p]
+                            i += 1
+                        except ImportError:
+                            break
+                else:
+                    mname = ''
+                    while i < len(fullspec):
+                        try:
+                            mname+= fullspec[i]
+                            self.log.debug("Trying importer '%s' for object '%s'", 
+                                    _imp, mname)
+                            if _imp.find_module(mname) is None:
+                                break
+                            self.log.debug("Ok")
+                            path = os.path.join(path, fullspec[i])
+                            i += 1
+                            self.log.debug("Setting new path %s", path)
+                            _imp = importer(path)
+                            self.log.debug("New importer %s", _imp)
+                            mname+= '.'
+                        except Exception, exc:
+                            self.log.debug("Error %s message '%s'", \
+                                            exc.__class__,
+                                            str(exc))
+                self.log.debug("Search for '%s' stopped at %d", 
+                            fullobject, i)
+                if i > 0:
                     break
+            else:
+                self.log.debug("Unable to import '%s'", fullobject)
+                raise ImportError, fullobject
             try:
                 mname = ".".join(fullspec[0:i])
-                if sys.modules.has_key(mname):
-                    module =  sys.modules[mname]
-                elif mname and mdescr:
-                    module = imp.load_module(mname, mfile, mpath[0], mdescr)
+                self.log.debug("Module name %s", mname)
+                __import__(mname, None, None)
+                self.log.debug("Retrieving %s from sys.path", mname)
+                module =  sys.modules[mname]
+                self.log.debug("Ok")
             finally:
                 if mfile:
                     mfile.close()
@@ -237 +303 @@
 
     def _makedoc(self, target, visibility):
         """Warning: this helper method returns a `str` object."""
+        self.log.debug('Loading module and object')
         module, object = self.load_object(target)
+        self.log.debug('Module and object loaded')
         try:
             self.makedoc_lock.acquire()
             if visibility == 'private' or \

The first issue still remains open, since that PEP doesnt provide standards for module enumeration.

Best Regards,

Olemis Lang