Unicode error with tagged non-ascii wiki pages names

[puwang]

How to Reproduce

While doing a GET operation on /tags, Trac issued an internal error.

Request parameters:

{'q': u"'oracle8'"}

User Agent was: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0

System Information

Trac	0.12dev-r7311
Python	2.5.2 (r252:60911, May 28 2008, 08:35:32) [GCC 4.2.4 (Debian 4.2.4-1)]
setuptools	0.6c8
SQLite	3.5.9
pysqlite	2.4.1
Genshi	0.5
Pygments	0.10
Subversion	1.4.6 (r28521)
jQuery:	1.2.3

Python Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7311-py2.5.egg/trac/web/main.py", line 447, in _dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.5/site-packages/Trac-0.12dev_r7311-py2.5.egg/trac/web/main.py", line 206, in dispatch
    resp = chosen_handler.process_request(req)
  File "build/bdist.linux-i686/egg/tractags/web_ui.py", line 99, in process_request
    data['tag_body'] =  macro.expand_macro(formatter, None, query)
  File "build/bdist.linux-i686/egg/tractags/macros.py", line 80, in expand_macro
    key=lambda r: str(r[0].id)):
  File "build/bdist.linux-i686/egg/tractags/macros.py", line 80, in <lambda>
    key=lambda r: str(r[0].id)):
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)

[puwang]

I think I found the reason while make new tags for a wiki page names include non-ascii characters,then this bug Reproduce

[anonymous]

same problem with german umlauts -

svn rev: 3117

trac 0.11

UnicodeEncodeError: ('ascii', u'Latex f\xfcr (absolute) Einsteiger', 7, 8, 'ordinal not in range(128)')

[anonymous]

some more input :)

#   File "/usr/local/lib/python2.6/dist-packages/Trac-0.11.4-py2.6.egg/trac/web/main.py", line 435, in  _dispatch_request
Code fragment:

 430. try:
 431. if not env and env_error:
 432. raise HTTPInternalError(env_error)
 433. try:
 434. dispatcher = RequestDispatcher(env)
 435. dispatcher.dispatch(req)
 436. except RequestDone:
 437. pass
 438. resp = req._response or []
 439.  
 440. except HTTPException, e:

Local variables:

Name	Value
after 	[u' except RequestDone:', u' pass', u' resp = ...
before 	[u' try:', u' if not env and env_error:', u' raise ...
dispatcher 	<trac.web.main.RequestDispatcher object at 0xba21b82c>
e 	UnicodeEncodeError('ascii', u'Latex f\xfcr (absolute) Einsteiger', 7, 8, ...
env 	<trac.env.Environment object at 0xb9f03e8c>
env_error 	None
exc_info 	(<type 'exceptions.UnicodeEncodeError'>, UnicodeEncodeError('ascii', ...
filename 	'/usr/local/lib/python2.6/dist-packages/Trac-0.11.4-py2.6.egg/trac/web/main ...
frames 	[{'function': '_dispatch_request', 'lines_before': [u' try:', u' ...
has_admin 	True
line 	u' dispatcher.dispatch(req)'
lineno 	434
message 	u"UnicodeEncodeError: ('ascii', u'Latex f\\xfcr (absolute) Einsteiger', 7, ...
req 	<Request "GET u'/tags'">
resp 	[]
tb 	<traceback object at 0xba39ba2c>
tb_hide 	None
traceback 	u'Traceback (most recent call last):\n File ...
# File "/usr/local/lib/python2.6/dist-packages/Trac-0.11.4-py2.6.egg/trac/web/main.py", line 205, in dispatch
Code fragment:

 200. req.args.get('__FORM_TOKEN') != req.form_token:
 201. raise HTTPBadRequest('Missing or invalid form token. '
 202. 'Do you have cookies enabled?')
 203.  
 204. # Process the request and render the template
 205. resp = chosen_handler.process_request(req)
 206. if resp:
 207. if len(resp) == 2: # Clearsilver
 208. chrome.populate_hdf(req)
 209. template, content_type = \
 210. self._post_process_request(req, *resp)

Local variables:
Name	Value
chosen_handler 	<tractags.web_ui.TagRequestHandler object at 0xba21bb4c>
chrome 	<trac.web.chrome.Chrome object at 0xba21b44c>
err 	(<type 'exceptions.UnicodeEncodeError'>, UnicodeEncodeError('ascii', ...
handler 	<tractags.web_ui.TagRequestHandler object at 0xba21bb4c>
req 	<Request "GET u'/tags'">
self 	<trac.web.main.RequestDispatcher object at 0xba21b82c>
# File "build/bdist.linux-i686/egg/tractags/web_ui.py", line 99, in process_request
Local variables:
Name	Value
ListTaggedMacro 	<class 'tractags.macros.ListTaggedMacro'>
TagCloudMacro 	<class 'tractags.macros.TagCloudMacro'>
checked_realms 	['wiki', 'ticket', 'downloads']
data 	{'tag_query': u"'Vortrag'", 'tag_realms': [{'checked': True, 'name': ...
formatter 	<trac.wiki.formatter.Formatter object at 0xba3bf60c>
macro 	<tractags.macros.ListTaggedMacro object at 0xba3c0aec>
match 	<_sre.SRE_Match object at 0xba39d160>
p 	<tracdownloads.tags.DownloadsTagProvider object at 0xba220d8c>
query 	u"(realm:wiki or realm:ticket or realm:downloads) ('Vortrag')"
r 	'downloads'
realm 	'downloads'
realms 	['wiki', 'ticket', 'downloads']
req 	<Request "GET u'/tags'">
self 	<tractags.web_ui.TagRequestHandler object at 0xba21bb4c>
# File "build/bdist.linux-i686/egg/tractags/macros.py", line 80, in expand_macro
Local variables:
Name	Value
content 	u"(realm:wiki or realm:ticket or realm:downloads) ('Vortrag')"
formatter 	<trac.wiki.formatter.Formatter object at 0xba3bf60c>
link 	<function link at 0xba399994>
name 	None
query_result 	<generator object query at 0xba0d97d4>
req 	<Request "GET u'/tags'">
self 	<tractags.macros.ListTaggedMacro object at 0xba3c0aec>
ul 	<Element "ul">
# File "build/bdist.linux-i686/egg/tractags/macros.py", line 80, in <lambda> 

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.22pre) Gecko/20090327 Ubuntu/8.04 (hardy) Firefox/2.0.0.22pre
Trac: 	0.11.4
Python: 	2.6.2 (release26-maint, Apr 19 2009, 02:11:59) [GCC 4.3.3]
setuptools: 	0.6c9
SQLite: 	3.6.10
pysqlite: 	2.4.1
Genshi: 	0.5.1
mod_python: 	3.3.1
Subversion: 	1.5.4 (r33841)
jQuery:	1.2.6

[anonymous]

ups... sorry

[anonymous]

crap found something more... two "Vortraege" one named "Linux from Scratch" the other one "Linux fuer Einsteiger" where the 'ue' in 'fuer' is the german umlaut. where the

sorted(query_result,key=lambda r: str(r[0].id )):

in macro.py line 80 fails.

so my line reads now:

for resource, tags in query_result:

which works

[lndr]

Here there is a patch/solution for the encoding problem

​http://trac-hacks.org/attachment/ticket/6908/listTaggedWithUmlaut.patch