GraphvizMacro.render_macro UnicodeDecodeError thrown

[Peter Kropf]

Peter Kropf wrote:

The plugin has been tested on Linux x86 system with python 2.3.5 and the 0.9-stable and trunk branches of Trac.

Testing on other platforms would be appreciated.

Hi!

I had a prolbem with Graphviz Plugin v0.6.7 and trac 0.10, I first wanted to create a ticket at tac-hacks, but the spam-filter didn't let me :-) So I paste the text here, sorry for the formatting:

If run with trac 0.10, and Python 2.4 GraphvizMacro.render_macro() throws exceptions. According to trac 0.10 release notes, "everything is unicode" now, and that seems to be the problem.

• sha_key = sha.new(self.processor + self.processor_options +

content).hexdigest() throws UnicodeDecodeError

• I tried to correct this in the form:

sha_key = sha.new((self.processor + self.processor_options + content).encode('latin1', 'replace')).hexdigest()

• this works, but after that there occurs an error in

GraphvizMacro.launch() at p_in.writelines()

So it seems to be a more thorough problem. I also tried to correct the writelines() problem with input.encode(...), it worked in some way, but after that the temporary .png files had 0 length, so the plugin didn't work either.

So, unicode strings in trac 0.10 seem to mess up everything. Unfortunately, I don't know graphviz and the plugin good enough to correct the problem myself. By the way, if you only use ascii characters in the graphviz macro definition, everything works.

Istvan

[Peter Kropf]

Can you attached a sample dot file that causes the problem?

[Peter Kropf]

From Steve:

I didn't get through the spam filter this time either, so here is a sample that should reproduce the problem:

graph G1 { node [shape=box, orientation=0, fontsize=16, color=blue]; edge [fontsize=6 ] hello [label="éáoúuöüä" URL=HelloUrl ];

[Peter Kropf]

When you get a chance, could you try the version under branches/v0.6?

Changeset:1254 contains my initial thoughts on fixing this problem. I've tried it on Trac 0.9.6 and 0.10 running on Ubuntu Linux. If this works, then I'll clean it up a bit and add the ability to specify the encoding in the macro content.

[Peter Kropf]

That should be changeset:1254, case counts in the Trac wiki links ;-)

[anonymous]

Replying to pkropf:

That should be changeset:1254, case counts in the Trac wiki links ;-)

I tried it, this time I got an error like this:

Strangely, I couldn't reproduce it in the interpreter, I tried it with python 2.4.3 and 2.3.5. The re.sub seems to accept every kind of string, so I couldn't figure out what causes the problem.

Traceback (most recent call last):

File "/var/lib/python-support/python2.4/trac/web/main.py", line 335, in dispatch_request

dispatcher.dispatch(req)

File "/var/lib/python-support/python2.4/trac/web/main.py", line 220, in dispatch

resp = chosen_handler.process_request(req)

File "/var/lib/python-support/python2.4/trac/wiki/web_ui.py", line 111, in process_request

self._render_editor(req, db, page, preview=True)

File "/var/lib/python-support/python2.4/trac/wiki/web_ui.py", line 386, in _render_editor

infopage_html? = wiki_to_html(page.text, self.env, req, db)

File "/var/lib/python-support/python2.4/trac/wiki/formatter.py", line 999, in wiki_to_html

Formatter(env, req, absurls, db).format(wikitext, out, escape_newlines)

File "/var/lib/python-support/python2.4/trac/wiki/formatter.py", line 793, in format

self.handle_code_block(line)

File "/var/lib/python-support/python2.4/trac/wiki/formatter.py", line 732, in handle_code_block

self.out.write(to_unicode(self.code_processor.process(

File "/var/lib/python-support/python2.4/trac/wiki/formatter.py", line 112, in process

text = self.processor(req, text)

File "/var/lib/python-support/python2.4/trac/wiki/formatter.py", line 100, in _macro_processor

return self.macro_provider.render_macro(req, self.name, text)

File "build\bdist.win32\egg\graphviz\graphviz.py", line 189, in render_macro File "sre.py", line 142, in sub

return _compile(pattern, 0).sub(repl, string, count)

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 472: ordinal not in range(128)

[anonymous]

i have the same (?) problem here:
i upgraded from 0.9.6 to 0.10, rebuilt all python-related stuff and the eggs, and put them back.
all is good except this one: (please excuse the formatting)

2006-11-07 01:56:42,434 Trac[main] ERROR: 'ascii' codec can't encode character u
'\xe4' in position 1002: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/local/lib/python2.4/site-packages/trac/web/main.py", line 356, in d
ispatch_request
    dispatcher.dispatch(req)
  File "/usr/local/lib/python2.4/site-packages/trac/web/main.py", line 224, in d
ispatch
    resp = chosen_handler.process_request(req)
  File "build/bdist.freebsd-5.5-STABLE-i386/egg/tractags/web_ui.py", line 64, in
 process_request
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/web_ui.py", line 134, i
n process_request
    self._render_view(req, db, page)
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/web_ui.py", line 446, i
n _render_view
    req.hdf['wiki'] = {
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/formatter.py", line 999
, in wiki_to_html
    Formatter(env, req, absurls, db).format(wikitext, out, escape_newlines)
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/formatter.py", line 793
, in format
    self.handle_code_block(line)
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/formatter.py", line 732
, in handle_code_block
    self.out.write(to_unicode(self.code_processor.process(
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/formatter.py", line 112
, in process
    text = self.processor(req, text)
  File "/usr/local/lib/python2.4/site-packages/trac/wiki/formatter.py", line 100
, in _macro_processor
    return self.macro_provider.render_macro(req, self.name, text)
  File "build/bdist.freebsd-5.5-STABLE-i386/egg/graphviz/graphviz.py", line 167,
 in render_macro
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 100
2: ordinal not in range(128)

[anonymous]

I've fixed troubles with UnicodeDecodeError exception when you use Unicode and "URL=" parameter.

Just replace this line

content = re.sub(r'URL="(.*?)"', self.expand_wiki_links, content)

with this one

content = re.sub(r'URL="(.*?)"', self.expand_wiki_links, content.decode(encoding)).encode(encoding)

[Peter Kropf]

Fixed w/ changeset:1731.