UnicodeEncodeError: 'ascii' codec can't encode character u'\u2192' in position 868: ordinal not in range(128)

[Ryan J Ollos]

An error results when placing the [[ListHacks]] on WikiStart:

2013-06-06 09:38:22,343 Trac[formatter] ERROR: Macro ListHacks(None) failed:
Traceback (most recent call last):
  File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/wiki/formatter.py", line 765, in _macro_formatter
    return macro.ensure_inline(macro.process(args))
  File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/wiki/formatter.py", line 356, in process
    text = self.processor(text)
  File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Trac-1.0.1-py2.6.egg/trac/wiki/formatter.py", line 343, in _macro_processor
    text)
  File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/TracHacks-2.0-py2.6.egg/trachacks/web_ui.py", line 836, in expand_macro
    output = "%s%s\n" % (output, fieldset)
  File "/srv/trac-hacks.org/pve/lib/python2.6/site-packages/Genshi-0.7-py2.6-linux-i686.egg/genshi/builder.py", line 107, in __str__
    return str(self.generate())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2192' in position 868: ordinal not in range(128)

[Ryan J Ollos]

I pulled down the Trac environment and database and couldn't reproduce the issue at first. That was because I setup the environment with Genshi 0.6, but we are running 0.7 on the server. After I upgraded my development environment to Genshi 0.7, I saw the same error as we see on the server.

I think the issue is essentially the same as #10989. The ListHacks macro is doing string formatting to a plain old string in order to join content together.

[Ryan J Ollos]

In 13270:

Fixes #11145: Resolved a UnicodeEncodeError in Genshi 0.7 by not formatting the markup to a string.

[Ryan J Ollos]

#11133 led to the finding of a similar fix in Trac, ​[t 11683], providing confidence that [13270] is the proper fix. It appears that we could have just appended a u to each string, but using builder.tag seems to result in cleaner code.