Auth/privileges problems with XML-RPC

[igorsantos07@…]

My user has the XML-RPC privilege set in TracAdmin, but it's unable to use the plugin. I've tried using PHP (my original idea) and also followed the python example that is in the plugin page. No success, always the same error: "XML_RPC privileges are required to perform this operation".

My password is correct, but if I change the address to login/rpc or login/xmlrpc, or if I change the username/password to something wrong, it still gives me the privileges error.

How can I fix this problem? I'm somewhat lost =(

[osimons]

Are you using Basic Authentication as provided by Apache (or other web server) or some plugin hack to simulate Basic auth? When using a regular browser in a new session (with no auto-user-password entry) are you challenged for authentication at /login/rpc url? By the browser or by pretty HTML form on the page?

The server needs to challenge for the authentication, otherwise the client won't provide the credentials.

You can likely test this quite easily by adding XML-RPC permission to anonymous and making request to /rpc (and possibly even /login/rpc). If RPC requests then works, the server is not making required challenges to work with your client library.

If you still think all is as it should be at your end, it is time to start pasting some information:

• System Information overview of versions and plugins from "About Trac" (as admin)
• Trac debug log from a single request via RPC just to see what happens

[anonymous]

Well, all four URLs provided (/rpc and /jsonrpc, and /login/rpc /login/jsonrpc) take me to the same examples page, explaining what to do with that plugin, and a list of methods, wiki-style.

I'm starting to think that something is really wrong =( haha Is this a misconfiguration? How can I fix that?

[Olemis Lang]

Replying to anonymous:

Well, all four URLs provided (/rpc and /jsonrpc, and /login/rpc /login/jsonrpc) take me to the same examples page, explaining what to do with that plugin, and a list of methods, wiki-style.

I'm starting to think that something is really wrong =( haha Is this a misconfiguration? How can I fix that?

There's nothing wrong, that's expected behavior, if request content-type header is not set to e.g. application/json for JSON-RPC and accept header contains text/html (AFAICR) , which is standard browser behavior.

Please read the documentation, and please provide system information including version of the plugin in use ... ;)

[anonymous]

Well, sorry for being that dumb. haha
Links with the requests I made (I think they are correct now) and the Trac Information: ​http://www.pastie.org/1741056

My boss is away today, but if needed i'll attach the log by tomorrow or monday.

Ty in advance (:

[anonymous]

​http://www.pastie.org/1783327 Trac debug log
​http://www.pastie.org/1783333 Python script used, as seen in one of the tutorials I read

Any ideas?

[Olemis Lang]

AFAICS it's failing @ this line req.perm.require('XML_RPC') but you specify correct user & password (unless my.user be an invalid user name ?) . The fact is that Trac seems not to identify user credentials , so it's executed like if it was an anonymous call .

This means there's an issue with authentication handling. Nonetheless the fact is that the plugin doesn't care about that and plays a passive role in this matter by delegating these details to other authentication-specific components.

As a workaround (at least to debug a little ...) I suggest you to :

• Grant XML-RPC permission to anonymnous user and see if it works (the problem is still there but at least you'll be able to confirm aforementioned hypothesis)
• Forward this request to trac-users mailing list together with information about plugins installed and server configuration in order to isolate and debug your particular authentication issues .

@osimons : IMO this should a happily closed & invalid ticket. Feel free to pull the trigger ... ;o)

[osimons]

Replying to olemis:

• Grant XML-RPC permission to anonymnous user and see if it works (the problem is still there but at least you'll be able to confirm aforementioned hypothesis)

Typo by the way, make sure you grant the permission to user anonymous. Anyway, could you please confirm if that makes it work or not? I suspect it does.

Looking at your list of plugins, it seems you are using AccountManagerPlugin for authentication (ie. html forms login). You need to read and understand the documentation at wiki:XmlRpcPlugin#Troubleshooting where it says that you then also need HttpAuthPlugin installed and enabled. This plugin hacks a basic authentication format for certain URLs, and you want to make sure you add /login/rpc to the paths configuration for this plugin.

[anonymous]

Ok, we have enabled HttpAuthPlugin and configured it as follows:

[httpauth]  
paths = /xmlrpc, /login/xmlrpc, /rpc, /login/rpc, /jsonrpc, /login/jsonrpc

And it still gives us the same error, saying that the user anonymous doesn't have permissions.
Finally, we can't create an anonymous user, the user page says simply that we can't create it.

[anonymous]

Replying to anonymous:

Finally, we can't create an anonymous user, the user page says simply that we can't create it.

Don't create user anonymous - 'anonymous' is the name assigned to any user that isn't logged in. All you need to do in Permissions admin page is assign XML_RPC permission to user anonymous.

[anonymous]

You can't create anonymous user. It's always there OOTB so there's no need to create it . It's a reserved user name . Just add permissions using trac-admin or admin web interface.

If it works using anonymous RPC requests then this will become a support request (not an issue ;o) so please forward your request to trac-users mailing list.

[original anonymous]

Ok, permissions granted to anonymous user. The script worked, but it keep working as I am the anonymous, not the user I said.

The HttpAuthPlugin is enabled and configured as said, but it keeps like is said in wiki:XmlRpcPlugin#Troubleshooting: "Every access will look like anonymous access."

Anyway, I have already sent a message to the mailing list.

[Franz]

I guess I am facing a similar problem:

When connecting trac by eclipse-mylyn and only an autenticated user has XML_RPC rights, eclipse plugins always says: Insufficient permissions for selected access type. The error won't occur if anonymous has XML_RPC rights.

I guess it might be a server-side configuration problem, but don't know enough where and what to config. I am using apache httpd without basic-auth, because I am using AccountManagerPlugin.

Thanks in advance for any hint.

[Dragon]

I'm having the same problem and don't know how to solve.
After installed HttpAuthPlugin, I received a new exception:
HTTP server returned unexpected status: Unauthorized
Someone helps me, please!

[Franz]

I am getting the following error with Trac 0.13dev-r10991 running on apache httpd (2.8 (WSGIProcessGroup /trac/trac1 WSGIApplicationGroup %{GLOBAL})) and newest XmlRpcPlugin (r11306):

2012-03-22 15:02:09,533 Trac[web_ui] ERROR: RPC(XML-RPC) Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/web_ui.py", line 158, in _rpc_process
    result = (XMLRPCSystem(self.env).get_method(method_name)(req, args))[0]
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/api.py", line 197, in __call__
    result = self.callable(req, *args)
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/ticket.py", line 241, in update
    " ".join([warning for warning in req.chrome['warnings']]))
ServiceException: No permission to edit the ticket description. No permission to change ticket fields. No permissions to add a comment. Sorry, can not save your changes. This ticket has been modified by someone else since you started field gefasoft_section must be set field gefasoft_phase must be set field gefasoft_trigger must be set field gefasoft_impact must be set field gefasoft_target must be set field gefasoft_defecttype must be set field gefasoft_qualifier must be set field gefasoft_source must be set field gefasoft_age must be set
2012-03-22 15:02:09,533 Trac[xml_rpc] ERROR: No permission to edit the ticket description. No permission to change ticket fields. No permissions to add a comment. Sorry, can not save your changes. This ticket has been modified by someone else since you started field gefasoft_section must be set field gefasoft_phase must be set field gefasoft_trigger must be set field gefasoft_impact must be set field gefasoft_target must be set field gefasoft_defecttype must be set field gefasoft_qualifier must be set field gefasoft_source must be set field gefasoft_age must be set
2012-03-22 15:02:09,534 Trac[xml_rpc] ERROR: Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/web_ui.py", line 158, in _rpc_process
    result = (XMLRPCSystem(self.env).get_method(method_name)(req, args))[0]
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/api.py", line 197, in __call__
    result = self.callable(req, *args)
  File "/usr/local/lib/python2.6/dist-packages/TracXMLRPC-1.1.2_r11306-py2.6.egg/tracrpc/ticket.py", line 241, in update
    " ".join([warning for warning in req.chrome['warnings']]))
ServiceException: No permission to edit the ticket description. No permission to change ticket fields. No permissions to add a comment. Sorry, can not save your changes. This ticket has been modified by someone else since you started field gefasoft_section must be set field gefasoft_phase must be set field gefasoft_trigger must be set field gefasoft_impact must be set field gefasoft_target must be set field gefasoft_defecttype must be set field gefasoft_qualifier must be set field gefasoft_source must be set field gefasoft_age must be set

I tried connecting with eclipse mylyn (newest update). User anonymous has permission XML_RPC. eclipse mylyn shows error: "Submit failed: Mid-air collision occurred. Synchronize task and re-submit changes." But synchronizing tasks or query does not help anything.

Might any custom fields cause problems? Did I miss anything to configure (maybe in httpd)?

[osimons]

Replying to framay:

I tried connecting with eclipse mylyn (newest update). User anonymous has permission XML_RPC. eclipse mylyn shows error: "Submit failed: Mid-air collision occurred. Synchronize task and re-submit changes." But synchronizing tasks or query does not help anything.

Might any custom fields cause problems? Did I miss anything to configure (maybe in httpd)?

Does this really look like the same problem? Your error says (broken for clarity):

ServiceException:
No permission to edit the ticket description.
No permission to change ticket fields.
No permissions to add a comment.
Sorry, can not save your changes.
This ticket has been modified by someone else since you started
field gefasoft_section must be set
field gefasoft_phase must be set
field gefasoft_trigger must be set
field gefasoft_impact must be set
field gefasoft_target must be set
field gefasoft_defecttype must be set
field gefasoft_qualifier must be set
field gefasoft_source must be set
field gefasoft_age must be set

• If posting as anonymous, does anonymous also have the necessary privileges to actually update the ticket?
• You seem to have other plugins installed, some which seem to enforce required field?

Perhaps debug logging provides more information.

That said, the plugin fails various tests with Trac 0.13dev - including ticket update tests. 49 tests gives 2 failures and 3 errors when running the tests suite just now with latest Trac tunk. Generally I don't spend much time trying to keep all intermediate Trac trunk changesets working, and you should not run development releases of Trac unless you are prepared to handle and research problems yourself.

[Franz]

log file of test trac system

[Franz]

Replying to osimons:

• If posting as anonymous, does anonymous also have the necessary privileges to actually update the ticket?

I tried it again on another test instance, where less plugins are set active. I double checked that user anonymous has enough permissions (TRAC_ADMIN, XML_RPC).

• You seem to have other plugins installed, some which seem to enforce required field?

I disabled all our self-made plugins, so only 2 plugins are left: TracXMLRPC 1.1.2-r11306 of course and TracAccountManager 0.3dev-r9929. But still the following error occurs (nobody changed anything on that system in the meantime):

ServiceException: Sorry, can not save your changes. This ticket has been modified by someone else since you started

Perhaps debug logging provides more information.

Attached log-file has debug logging (note: all plugins are set active at this point).

That said, the plugin fails various tests with Trac 0.13dev - including ticket update tests. 49 tests gives 2 failures and 3 errors when running the tests suite just now with latest Trac tunk. Generally I don't spend much time trying to keep all intermediate Trac trunk changesets working, and you should not run development releases of Trac unless you are prepared to handle and research problems yourself.

Well, I know that it is not the official Trac release, but as discussed in ​trac-dev current trunk provides features, which serveral collegues has already requested. So that's why we do not use a 2 year-old version 0.12.

[osimons]

Replying to osimons:

Does this really look like the same problem?

Your problem is very different from the issue originally raised by this ticket, so let's stop mixing them. I've created #9921 for your issue. Please follow up there.

[gcarre20@…]

Replying to framay:

We started to have the "Unauthorised" error as well here after switching from tracd to apache as web front end. Our conf is using the HttpAuthPlugin and a specific user was used to change tickets until now.

My apache logs are saying "Digest: client used wrong authentication scheme `Basic': /login/xmlrpc" so I guess something has to be done at the apache configuration

If this can't be done I'm resigned to give XMLRPC rights to anonymous but would like to avoid this...

[anonymous]

Hello everyone,

I still have the problem of creation of a ticket with xmlrpc (with login) although it works without login:

Case 1: I use this piece of source code:

-----------------  code  ------------			
XmlRpcClientConfigImpl conf = new XmlRpcClientConfigImpl();
conf.setConnectionTimeout(60 * 1000);
conf.setBasicUserName("lazhar");conf.setBasicPassword("lazhar");
conf.setServerURL(new URL("'''http://localhost:8000/my-project/xmlrpc'''"));
System.out.println("11111111111111111111");
XmlRpcClient client = new XmlRpcClient();
client.setConfig(conf);
TrackerDynamicProxy proxy = new TrackerDynamicProxy(client);
Ticket ticket1 = (Ticket)proxy.newInstance(Ticket.class);
int id=ticket1.create("testtttttttttttttttt", "test-description");
System.out.println("id=="+id);
-------------------- result  ---------------------
11111111111111111111
id==46

Case 2: I use this piece of source code:

-----------------  code  ------------			
XmlRpcClientConfigImpl conf = new XmlRpcClientConfigImpl();
conf.setConnectionTimeout(60 * 1000);
conf.setBasicUserName("lazhar");conf.setBasicPassword("lazhar");
conf.setServerURL(new URL("'''http://localhost:8000/login/my-project/xmlrpc'''"));
System.out.println("11111111111111111111");
XmlRpcClient client = new XmlRpcClient();
client.setConfig(conf);
TrackerDynamicProxy proxy = new TrackerDynamicProxy(client);
Ticket ticket1 = (Ticket)proxy.newInstance(Ticket.class);
int id=ticket1.create("testtttttttttttttttt", "test-description");
System.out.println("id=="+id);
-------------------- result  ---------------------
11111111111111111111
java.lang.reflect.UndeclaredThrowableException
	at $Proxy0.create(Unknown Source)
	at aa.MainTest.main(MainTest.java:51)
Caused by: org.apache.xmlrpc.client.XmlRpcHttpTransportException: HTTP server returned unexpected status: Unauthorized
	at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.getInputStream(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(Unknown Source)
	at org.apache.xmlrpc.client.XmlRpcClient.execute(Unknown Source)
	at org.lustin.trac.xmlprc.TrackerDynamicProxy$1.invoke(TrackerDynamicProxy.java:125)

[anonymous]

version of Trac and plugin installed

[David.Byrne@…]

I am using the AccountManager plugin and am having a similar issue. If I add XML_RPC priveleges to anonymous, it works. If I only have it on authenticated, it gives me the 403 fault. I've tried adding the environ_auth_overwrite parameter to the account-manager section, but that doesn't seem to help. My account-manager section looks like:

[account-manager]
account_changes_notify_addresses = MYSUPPORT@COMPANY.COM
authentication_url =
environ_auth_overwrite = false
force_passwd_change = true
hash_method = HtPasswdHashMethod
htdigest_realm =
htpasswd_hash_type = crypt
login_attempt_max_count = 0
notify_actions = new,delete
password_file =
password_store = SessionStore
persistent_sessions = false
verify_email = true

[anonymous]

I am facing the same problem,I think perhaps it doesn't support.

[Olemis Lang]

Replying to David.Byrne@…:

I am using the AccountManager plugin and am having a similar issue. If I add XML_RPC priveleges to anonymous, it works. If I only have it on authenticated, it gives me the 403 fault. I've tried adding the environ_auth_overwrite parameter to the account-manager section, but that doesn't seem to help.

[...]

There are many reasons beyond plugin control leading to these sort of errors . Did you submit RPC request by pointing at /login/rpc URL ? Could you please enable DEBUG logging level and post the relevant entries you find when invoking a given RPC method ?

[osimons]

Just for the record: The RPC plugin does not perform any authentication. Nothing at all. Nada. It only uses whatever is forwarded along with the request from Trac / Web server. Be it 'anonymous' or a specific username is all the same to the plugin. It does not even know the difference. The only reason for supporting /login/rpc is to allow a custom URL that the web server can use to challenge for authentication and still forward requests to the plugin correctly afterwards.

If you depend on authenticated users for RPC access, you have to ensure that you use an authentication mechanism that match the capabilities of whatever RPC client library you use. Every library for every protocol for every language will have differences with regards to authentication features and usage. Usually always Basic auth, sometimes Digest, but never custom login forms that depend on username + password + CSRF token being POSTed, and cookies stored and reused for subsequent requests (essentially as provided by regular Trac AccountManagerPlugin).

Filing tickets against the XmlRpcPlugin is not going to help because there just isn't anything we can do about it. If the problem is with some feature of AccountManagerPlugin, HttpAuthPlugin or other plugins that are supposed to help, then file the ticket against those plugins instead to get the needed attention.

[tomasz@…]

Very simply solution:

I moved from Apache Basic auth into AccountManager plugin because of obvious advantages, but lost access to xmlprc. So i simply reset apache conf back, but only for /login/

<Location ~ "/trac/myTrac/">
  AuthUserFile /var/lib/trac/myTrac/trac.htpasswd
  Require valid-user
</Location>

into

<Location ~ "/trac/myTrac/login/">
  AuthUserFile /var/lib/trac/myTrac/trac.htpasswd
  Require valid-user
</Location>

And now all users logs using AccountManager and apps which uses xmlrpc - with good-old basic auth.

So you can kill two birds with one stone :D

[tomasz@…]

My solution of this problem is using curl and cookies. In first step i use /login page - getting cookies (form_token) and setting username and password, then I use /xmlrpc

My php code for this - this is first successful attempt - now I will try not to login everycall - but i have to share with you ASAP.

function do_call($request) {
  $ckfile = tempnam ("/tmp", 'cookiename');
  $url = 'https://example.com/trac/myTRAC/login';
  $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile);
  curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  $output = curl_exec($ch);
  curl_close($ch);
  preg_match('/name="__FORM_TOKEN" value="(.*)" /',$output,$matches);

  $url = "https://example.com/trac/myTRAC/login";
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile);
  curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);

  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt ($ch, CURLOPT_POST, 1);
  curl_setopt($ch, CURLOPT_POSTFIELDS,
           http_build_query(array('username' => $_SESSION['username'],'password' => $_SESSION['password'],'__FORM_TOKEN' => $matches[1],'submit' => 'Login')));

  $output = curl_exec($ch);
  curl_close ($ch);
  $url = "https://example.com/trac/myTRAC/rpc";

  $header[] = "Content-type: application/json";
  $header[] = "Content-length: ".strlen($request);

  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile);
  curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

  curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
  curl_setopt ($ch, CURLOPT_POST, 1);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $request);

  $data = curl_exec($ch);
  if (curl_errno($ch)) {
    print curl_error($ch);
  } else {
    curl_close($ch);
    return $data;
  }
}

$query='{"method":"ticket.update","params":['.$id.',"Ticket accepted",{"status":"accepted"},true]}';
$json = json_decode(do_call($query));

[walty8@…]

I finally worked it out after quite some time of struggling, just some sharing here.

1. I installed the Trac 1.2, AccountManagerPlugin 0.5dev, and XmlRpcPlugin 1.1.7dev

2. The troubleshoot section of XmlRpcPlugin page suggests use configuration of environ_auth_overwrite = false, unfortunately it does not have any effect in my case.

3. I installed HttpAuthPlugin 1.2 as suggested above, and configured the following

[httpauth]
paths = /xmlrpc, /login/xmlrpc, /rpc, /login/rpc

4. Last trick, it turns out the following configuration is needed in apache wsgi configuration (as mentioned in HttpAuthPlugin wiki page).

WSGIPassAuthorization On

Now the snippet using ServerProxy (inside /login/rpc) works out fine.

[Jun Omae]

It would be helpful to respond Authentication information not available if /login/ is anonymously requested.

xmlrpcplugin/trunk/tracrpc/web_ui.py

@@ -140 +140 @@
         rpcreq = req.rpc = {'mimetype': content_type}
         try :
             self.log.debug("RPC(%s) call by '%s'", proto_id, req.authname)
+            if req.path_info.startswith('/login/') and \
+                    req.authname == 'anonymous':
+                raise TracError("Authentication information not available")
             rpcreq = req.rpc = protocol.parse_rpc_request(req, content_type)
             rpcreq['mimetype'] = content_type
 

If Apache and htpasswd/htdigest file are used, it would be simple and easy to authenticate for /login/rpc on Apache instead of AccountManagerPlugin and HttpAuthPlugin.

    # for tracrpc
    <Location "/*/login/*rpc">
        AuthType Basic
        AuthName realm
        AuthUserFile /var/lib/trac/.htpasswd
        Require valid-user
    </Location>

[Ryan J Ollos]

In 17052:

TracXMLRPC 1.1.7dev: Raise TracError for /login path as anonymous

Patch by Jun Omae.

Refs #8644.