ASCII encoding problem with German umlauts

[Immanuel Scheerer]

I can't use German umlauts in the forums. If I type "ü" and try to safe a post, I get the following error:

Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 356, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.3/site-packages/trac/web/main.py", line 224, in dispatch
    resp = chosen_handler.process_request(req)
  File "build/bdist.linux-x86_64/egg/tracdiscussion/core.py", line 70, in process_request
  File "build/bdist.linux-x86_64/egg/tracdiscussion/api.py", line 31, in render_discussion
  File "build/bdist.linux-x86_64/egg/tracdiscussion/api.py", line 643, in _do_action
  File "build/bdist.linux-x86_64/egg/tracdiscussion/api.py", line 957, in add_message
  File "/usr/lib/python2.3/site-packages/trac/db/util.py", line 47, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/usr/lib/python2.3/site-packages/trac/db/util.py", line 47, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/usr/lib/python2.3/site-packages/MySQLdb/cursors.py", line 137, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.3/site-packages/MySQLdb/connections.py", line 33, in defaulterrorhandler
    raise errorclass, errorvalue
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 26: ordinal not in range(128)

I found a ​similar ticket in trac, which was fixed by ​this changeset. But unfortunately I couldn't fix the problem for the forum.

[Radek Bartoň]

It's working for me. What is your configuration in trac.ini's [trac] section? I have "default_charset = iso-8859-15" and my system uses utf-8 as default encoding.

[Marco Pracucci]

I get the same error when I post a message that contains special chars, like "è à ò". I have "default_charset = iso-8859-15" and my system uses utf-8 as default encoding (it_IT.utf8).

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe8' in position 28: ordinal not in range(128)

[Radek Bartoň]

Try it once more with lastest 0.10 branch. I made some changes but it looks like a problem with Trac's util.html module too. If error still occurs, please post me traceback and debug log cut too.

[Marco Pracucci]

I have download the last release and the error still occurs...

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 356, in dispatch_request
    dispatcher.dispatch(req)
  File "/usr/lib/python2.4/site-packages/trac/web/main.py", line 224, in dispatch
    resp = chosen_handler.process_request(req)
  File "build/bdist.linux-i586/egg/tracdiscussion/core.py", line 72, in process_request
  File "build/bdist.linux-i586/egg/tracdiscussion/api.py", line 32, in render_discussion
  File "build/bdist.linux-i586/egg/tracdiscussion/api.py", line 495, in _do_action
  File "build/bdist.linux-i586/egg/tracdiscussion/api.py", line 930, in add_topic
  File "/usr/lib/python2.4/site-packages/trac/db/util.py", line 47, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/usr/lib/python2.4/site-packages/trac/db/util.py", line 47, in execute
    return self.cursor.execute(sql_escape_percent(sql), args)
  File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 148, in execute
    query = query % db.literal(args)
  File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 232, in literal
    return self.escape(o, self.encoders)
  File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 174, in string_literal
    return db.string_literal(obj)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 27-31: ordinal not in range(128)

[Radek Bartoň]

Relevant part of debug log could tell me a lot. Could you please post it too. Thank you.

[Radek Bartoň]

Will be fixed in next changeset...

[Radek Bartoň]

Fixed in changeset 1987.