ticket.milestone.update or create with 'due' parameter fail

[takuzo.ohara@…]

I am using python2.5, trac 0.11dev(r7035), XmlRpcPlugin trunk(r3650).

Using following simple script to update due date for a milestone:

import xmlrpclib

testurl = "http://unko:unko@10.0.0.5/proj/test11/login/xmlrpc"

server = xmlrpclib.ServerProxy(testurl)
milestone = server.ticket.milestone.get("testmilestone")
dt = datetime.datetime(1973, 10, 23)
server.ticket.milestone.update("testmilestone", {'due': dt})

Trac fails with following error:

2008-05-11 02:52:02,670 Trac[model] INFO: Updating milestone "testmilestone"
2008-05-11 02:52:02,692 Trac[web_ui] ERROR: unsupported operand type(s) for -: 'instance' and 'datetime.datetime'
2008-05-11 02:52:02,790 Trac[web_ui] ERROR: Traceback (most recent call last):
  File "/home/takuzo/debian/trac/xmlrpcplugin/tracrpc/web_ui.py", line 60, in process_request
    result = XMLRPCSystem(self.env).get_method(method)(req, args)
  File "/home/takuzo/debian/trac/xmlrpcplugin/tracrpc/api.py", line 85, in __call__
    result = self.callable(req, *args)
  File "/home/takuzo/debian/trac/xmlrpcplugin/tracrpc/ticket.py", line 215, in update
    self._updateHelper(name, attributes).update()
  File "/usr/lib/python2.5/site-packages/Trac-0.11dev_r7035-py2.5.egg/trac/ticket/model.py", line 692, in update
    (self.name, to_timestamp(self.due), to_timestamp(self.completed),
  File "/usr/lib/python2.5/site-packages/Trac-0.11dev_r7035-py2.5.egg/trac/util/datefmt.py", line 56, in to_timestamp
    return diff.days * 86400 + diff.seconds
TypeError: unsupported operand type(s) for -: 'instance' and 'datetime.datetime'

It is same with ticket.milestone.create.

It seems like xmlrpclib.DateTime and datetime.datetime can not be subtracted at to_timestamp() in trac/util/datefmt.py.

Am I doing something wrong?

[Alec Thomas]

You should be passing use_datetime=True to xmlrpclib.ServerProxy() if you wish to ​... use ... datetime objects.

If that doesn't fix your problem, feel free to reopen.

[takuzo.ohara@…]

No, I modified the previous test script as follows, but it did not change the result.

-server = xmlrpclib.ServerProxy(testurl)
+server = xmlrpclib.ServerProxy(testurl, use_datetime=True)

[takuzo.ohara@…]

I created a patch to trac to work around this problem.

What is the correct way to fix this?

--- Trac-0.11dev_r7035-py2.5.egg.orig/trac/util/datefmt.py
+++ Trac-0.11dev_r7035-py2.5.egg/trac/util/datefmt.py
@@ -26,6 +26,8 @@ from datetime import tzinfo, timedelta, 
 from trac.core import TracError
 from trac.util.text import to_unicode
 
+import xmlrpclib
+
 # Date/time utilities
 
 # -- conversion
@@ -49,8 +51,16 @@ def to_datetime(t, tzinfo=None):
     raise TypeError('expecting datetime, int, long, float, or None; got %s' %
                     type(t))
 
+def xmlrpc_datetime_to_datetime(data):
+    """Return datetime (in utc) from XMLRPC datetime string (is always utc)"""
+    t = list(time.strptime(data.value, "%Y%m%dT%H:%M:%S")[0:6])
+    return apply(datetime, t + [0, utc])
+
+
 def to_timestamp(dt):
     """Return the corresponding POSIX timestamp"""
+    if isinstance(dt, xmlrpclib.DateTime):
+        dt = xmlrpc_datetime_to_datetime(dt)
     if dt:
         diff = dt - _epoc
         return diff.days * 86400 + diff.seconds

[Thijs Triemstra]

The attached patch​ solves this issue for me. It basically turns the existing to_datetime method into a to_xmlrpc_datetime (because that's what it does) and adds a real to_datetime method that converts a XMLRPC DateTime into a datetime.datetime. This new check for DateTime is currently not applied in all ticket.* methods, but I'll attach an updated patch if that becomes necessary in my application.

[osimons]

Reclosed #434, and will add a patch shortly that largely verifies and completes these fixes. It does not do the renaming of methods. It builds on top of attachment:ticket:1075:ticket_workflow-r3074.diff​.

[osimons]

Datetime field update/create.

[osimons]

Much better idea. Normalize arguments from rpc before it hits any methods.

[osimons]

The updated patch (attachment:t3011-datetime2-r3074.diff​) normalizes arguments before they are processed by a method. Trac 0.11 uses datetime objects in all code and models, so it makes much more sense to convert any input we receive so that methods don't have to be concerned with it.

It would make sense to do the same for output as well, so that methods just return datetime objects that are then converted to xmlrpclib.DateTime on its way out. Left for later...

[osimons]

(In [6046]) XmlRpcPlugin: Changes to datetime handling, by converting any timestamps to Python datetime before arguments are passed to methods. Like Trac 0.11, all code should use and expect regular datetime.datetime objects.

The rework of code to transform input, should also make it easier to solve other issues related to input.

Closes #3011.