Calling function ticket.get() for defined tickets in Agilo return an error.

[anonymous]

Calling function ticket.get() defined tickets in Agilo return an error. Below Traceback output from log:

2011-03-01 10:39:37,852 Trac[web_ui] ERROR: RPC(XML-RPC) Unhandled protocol error
Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/tracrpc/web_ui.py", line 167, in _rpc_process
    protocol.send_rpc_result(req, result)
  File "build/bdist.linux-i686/egg/tracrpc/xml_rpc.py", line 108, in send_rpc_result
    xmlrpclib.dumps(result, methodresponse=True), rpcreq['mimetype'])
  File "/usr/lib/python2.5/xmlrpclib.py", line 1080, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.5/xmlrpclib.py", line 623, in dumps
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 635, in __dump
    f(self, value, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 695, in dump_array
    dump(v, write)
  File "/usr/lib/python2.5/xmlrpclib.py", line 633, in __dump
    raise TypeError, "cannot marshal %s objects" % type(value)
TypeError: cannot marshal <class 'agilo.ticket.model.TicketValueWrapper'> objects

[osimons]

Talk to the Agilo people. I won't support their reworked ticket system via RPC.

[Olemis Lang]

@osimons :

I agree with you, however, what's the recommended approach so that plugins will be able to serialize custom objects via RPC ... (e.g. serialization hooks)

Should we think about adding support for this ? I was wondering, it's not an easy task due to the fact that (+ random thoughts ;)

• Plugin knows what kinds of objects it handles
• Different RPC protocols should define different serialization rules for the same custom type
• so we have a matrix ... where does specific serialization fit ?
• ...
• At least built-in JSON-RPC handler can rely on JSONEncoder to do this ...

Two approaches come to my mind :

• Default object serialization strategy.
• Introduce "serializers", i.e. objects able to convert instances of custom types to a hierarchy of simple objects (e.g. strings, integers, lists, dicts, ...) that may be serialized in headless mode afterwards.

[osimons]

Replying to olemis:

I agree with you, however, what's the recommended approach so that plugins will be able to serialize custom objects via RPC ... (e.g. serialization hooks)

Should we think about adding support for this ?

IF your plugin replaces the Trac ticket system to the point where you instead of regular types use agilo.ticket.model.TicketValueWrapper to encapsulate the data, then I don't see why all plugins should change to accomodate this. It should be their code (or some other plugin) that provides custom logic for dealing with this - like for example:

• They can subclass str, int or whatever or use some mix-in or meta-magic to support their own behaviour while still letting the fields maintain their basic identity and return regular isinstance responses and be serializable as regular types?
• Subclass the RPC Ticket module and add a small conversion layer to each method, or perhaps provide a one-method accessor that for any result will 'unwrap' before returning to RPC?
• Let it just fail if they for some reason don't want to support RPC access to information for their custom ticket system by using external APIs (like RPC)?

Outside RPC scope, IMHO.

[osimons]

Yours too then, I suppose.

[osimons]

Aargh. Again.

[anonymous]

Replying to osimons:

Replying to olemis:

I agree with you, however, what's the recommended approach so that plugins will be able to serialize custom objects via RPC ... (e.g. serialization hooks)

Should we think about adding support for this ?

IF your plugin replaces the Trac ticket system to the point where you instead of regular types use agilo.ticket.model.TicketValueWrapper to encapsulate the data, then I don't see why all plugins should change to accomodate this.

I agree with you 100%. They should be responsible for implementing and manage the impact of changes introduced by that plugin.

I was thinking of other circumstances where these kinds of behaviors may be useful (and may also help'em to implement RPC support ... but that's a side efect ;). For instance, let's suppose component C defines RPC method m and returns some complex information and somewhere ticket data needs to be included, and also there are custom model objects of type CM (e.g. Trac resource) that need to be sent back and forth via RPC, let's say that except some fields (e.g. password, ...).

In that case probably it would be nice to specify how to serialize tickets and instances of CM once, and then include those objects somewhere in the object tree defined by the RPC result object (i.e. the data to serialize and send back to the client) and built-in components will handle the rest of serialization steps.

AFAIK, something like this is already available in .NET .

Outside RPC scope, IMHO.

Maybe you're right, but in the end all RPC flavors rely on (particular) data serialization rules (e.g. Hessian is a binary protocol supporting default generic object serialization + other features ...). So it's always the lowest layer of RPC solutions AFAICS.

It may also be an added complexity, but probably would be nice to have the opportunity of controlling serialization rules for custom types.

[anonymous]

In any case this wouldn't be a critical but a nice to have feature .

[osimons]

Perhaps the Agilo people could chime in if they have an opinion on the matter, but seeing protocols can be arbitrary we need to stick to lowest common denominator for the supported type system.

There is no point in serializing an object and passing it down the wire to a client, if the same object can't be described as XML or JSON in a simple way for other clients requesting that. And there is no point in requesting ticket data and returning some complex type that clients like Mylyn can't ever be expected to make sense of.

That is why the wrapper/type system in use by Agilo should learn how to quack like ducks... A wrapper around a string by subclassing str and adding some __new__() magic, or whatever other way would allow it to behave like the underlying type.

I haven't looked at Agilo code, nor have I any plans to do so. Most of my arguments are of course highly speculative when I don't know what they actually do or why they do it...

[Andrea Tomasini]

Hey there,

sorry for the delay in replying, but I don't seem to see this ticket in "My Tickets" report :-(

We will fix this problem for sure, it has been introduced recently as till version 0.8.* has been working correctly, I mean at least not breaking. The whole problem is that Agilo for Trac, introduces ticket hierarchies and we will have to find a way to allow Mylyn to visualize them. How does the XML-RPC plugin expect to get "list of tickets" or "ids" as result of a property read? For example if we have a "User Story" ticket with linked "Task" tickets and we want the Mylyn plugin to correctly visualize a hierarchy for this, how should we prepare the data so that the XML-RPC plugin correctly generates the stream that Mylyn expects? Is there some example/documentation rather than reading the code? :-)

Thanks ANdreaT

[osimons]

You'd need to call the ticket.query() RPC method with whatever query arguments you like - any string supported by the underlying Trac query system is supported. Method docs:

array ticket.query(string qstr="status!=closed")

Perform a ticket query, returning a list of ticket ID's. All queries will use stored settings for maximum number of results per page and paging options. Use max=n to define number of results to receive, and use page=n to page through larger result sets. Using max=0 will turn off paging and return all results.

I'll leave it to the Mylyn people to have an opinion about how to visualize and use your custom information. However, anyone can create custom fields with same or similar names or meaning as you use, so trying to "auto-detect" Agilo via fields on a ticket may not be a proper way to integrate. I would rather think that you should provide optional RPC support via your plugin, so that you exported various agilo.* methods. Mylyn can then query the available methods and determine if Agilo is supported (and at what version) - and use your custom methods to get and set where the default ticket.* methods don't provide the proper support.

[Adrian Wilkins]

How about this ; TicketValueWrapper is a DictType without any additional fields to serialize apart from the ticket (which you don't want to serialize anyway). Marshaller maintains a list of types it can serialize. So we just put TicketValueWrapper on that list and tell it to use the dump function for the plain DictType .. (see bottom two lines)

class TicketValueWrapper(dict):
# Trac accesses ticket.values directly sometimes (especially in 
# _init_defaults). However we need to intercept type changes to use 
# the right list of ticket fields so we use this wrapper.


    def __init__(self, ticket):
        dict.__init__(self)
        self._ticket = ticket
    
    def __setitem__(self, name, value):
        # FIXME: (AT) This doesn't make sense as the change of type will happen 
        # through the normal setter, Trac will cycle through the values and set
        # the changed ones, in case of type, will generate another reset of the
        # fields.
        if name == Key.TYPE:
            self._ticket._reset_type_fields(value)
        dict.__setitem__(self, name, value)
    
    def setdefault(self, name, value):
        if (name == Key.TYPE) and (Key.TYPE not in self):
            # FIXME: (AT) This is called from init_defaults when the ticket
            # initializes and for the third time we call _reset_type_fields
            self._ticket._reset_type_fields(value)
        dict.setdefault(self, name, value)

# Add this subclass of DictType to the list of permitted types that Marshaller knows
xmlrpclib.Marshaller.dispatch[TicketValueWrapper] = xmlrpclib.Marshaller.dump_struct

Alas, it's not enough to fix all my problems but you can at least fetch a list of tickets into Mylyn (but not submit any ; that's another issue)

[Andrea Tomasini]

Thanks for the help, and thanks for the patience, we are going to plan some fixes on this in the next iteration. We have not been receiving email notification from this site, for a long time. I have realised today that there was a problem with email verification in the Account settings, now it seems to work again...

[Stefano Rago]

Hi, sorry for not updating this thread in a while. This issue has been fixed in Agilo for Trac 0.9.6 (1.3.6 PRO) and in a later version (0.9.8 / 1.3.8 PRO) support for creating new Tasks via Mylyn has been added.