TracForms - form details retrieved but forms still empty

[anonymous]

After migrating my Trac instance from SQLite to Postgres and upgrading TracForms from 0.3 to 0.4.1, forms created prior to the migration are not displayed on the wiki page. When the 'Form details' link is clicked, all of the old 'Saved Values' are present and correct, but none of that data shows up on the wiki page itself. I have already run 'trac-admin /path/to/env upgrade', and there are no errors in the logs (PG or Trac).

If I update the same form by adding a value to, for example, a textarea, it saves correctly and is visible upon reload, but it does (erroneously?) create a new form id in the database (in the forms table) instead of using the existing one. After adding a value, if I click on the 'Form details' link again, I'm shown two available records:

Forms in MyFormPage
Available records

Form 69 (subcontext = 'None')
Form 76

Form id 69 has all of the old data in the DB and on the 'Form details' (Saved Values) page, but the actual wiki page shows no form data. Form id 76 only has the one new value in a separate DB entry and 'Form details' page, and the actual wiki page shows only the one value. In the forms table in the database, both entries have nothing in the 'subcontext' column. In fact, both entries are essentially identical except for the form id and the form state (as detailed above).

Since the plugin works perfectly fine for new forms & form data, I'm guessing there is something less conspicuous (such as an index or sequence or something like that) which is causing the issue, but I'm not quite sure what that 'thing' is. The priority is set to 'high' because the plugin partially works (new entries), but all pre-migration data is not displaying (though it is in the DB and the 'Form details' page).

[anonymous]

Figured out the problem. When I migrated the TracForms tables from SQLite to PG, the NULL's didn't get changed to empty strings, and TracForms uses this conditional: AND subcontext=E. As a result, I had to run this to fix things:

trac=> BEGIN;
trac=> UPDATE forms SET subcontext='' WHERE subcontext IS NULL;
trac=> COMMIT;

It would probably be a good idea for TracForms to check whether subcontext is NULL when performing upgrades and set it to the empty string if not.

[anonymous]

Bah, formatting... it uses this conditional: AND subcontext=E''

[Steffen Hoffmann]

Thank you for the report and even more for you further enlightening explanations.

Admittedly testing PostgreSQL compatibility is hard for me, since I don't use it myself at all. So your findings are very valuable to iron out some remaining glitches after the big PostgreSQL centric work that went into the code pushing towards tracforms-0.3.1 .

Still very encouraging, that it seem to run smoothly apart from that. I'll have a look into this for sure.

[anonymous]

No problem. By the way, I noticed that TracForms don't seem to work in tickets anymore. For example, if I have a form with checkboxes, it works in the wiki just fine, but if I use the Include macro to pull that page into a ticket, none of the data is fetched. Instead, I see things like "Note: [textbox] unknown unknown", whereas on the wiki page I'd see "Note: [textbox] 01/02/03 01:02:03 myusername". Additionally, instead of updating the form in the ticket when I click the submit button, it generates a new ticket response (thereby including two copies of the page) which also has none of the data. Updating the form from the ticket does not update the actual form either (loading the wiki page shows no changes when those changes were made from a ticket).

I can't find anything in the Trac logs, even at DEBUG, that would indicate what's going on. I got the "2011-09-16 08:16:24,733 Trac[macros] WARNING: Executing pre-0.11 Wiki macro Include by provider <class 'includemacro.macros.IncludeMacro'>" message, but if I understand #1973 correctly, I don't think that should be causing these problems. I did see this in the Postgres logs:

            SELECT  id,
                    realm,
                    resource_id,
                    subcontext,
                    author,
                    time,
                    keep_history,
                    track_fields
            FROM    forms
            
                WHERE   realm=E'ticket'
                    AND resource_id=E'5555'

I'm not sure why it's looking for realm=E'ticket' though - it should be grabbing realm=E!'wiki' AND resource_id=E!'wikipagename', where wikipagename is the page that was included in the ticket via the Include macro. Any thoughts?

[anonymous]

The PG logs also indicate that the id isn't being passed correctly to get_tracform_fieldinfo:

            SELECT  author, time
            FROM    forms_fields
            WHERE   id=NULL
                AND field=E'wikipagename'

[Steffen Hoffmann]

Replying to anonymous:

No problem. By the way, I noticed that TracForms don't seem to work in tickets anymore. For example, if I have a form with checkboxes, it works in the wiki just fine, but if I use the Include macro to pull that page into a ticket, none of the data is fetched. Instead, I see things like "Note: [textbox] unknown unknown", whereas on the wiki page I'd see "Note: [textbox] 01/02/03 01:02:03 myusername".

Do I understand you right here, that your ticket displays the same form correctly, just without the data shown in the wiki page, that you pull in via Include? Please read the wiki docs again, because this is a feature, not a bug, really. It's meant to work like that for form templating. The context will always be the actual context of the request (for the ticket in this example), not for the wiki page included there.

Additionally, instead of updating the form in the ticket when I click the submit button, it generates a new ticket response (thereby including two copies of the page) which also has none of the data. Updating the form from the ticket does not update the actual form either (loading the wiki page shows no changes when those changes were made from a ticket).

Well the last sentence should be clear now from the explanation above. Data will be saved for the ticket. What you describe for the response on submit is not clear. Duplicating the form somehow? I'm using TracForms with Trac 0.13dev in production with both, tickets and wiki, and never seen behavior similar to your description. You're with Trac 0.11?

I can't find anything in the Trac logs, even at DEBUG, that would indicate what's going on. I got the "2011-09-16 08:16:24,733 Trac[macros] WARNING: Executing pre-0.11 Wiki macro Include by provider <class 'includemacro.macros.IncludeMacro'>" message, but if I understand #1973 correctly, I don't think that should be causing these problems.

Correct.

I did see this in the Postgres logs:

            SELECT  id,
                    realm,
                    resource_id,
                    subcontext,
                    author,
                    time,
                    keep_history,
                    track_fields
            FROM    forms
            
                WHERE   realm=E'ticket'
                    AND resource_id=E'5555'

I'm not sure why it's looking for realm=E'ticket' though - it should be grabbing realm=E!'wiki' AND resource_id=E!'wikipagename', where wikipagename is the page that was included in the ticket via the Include macro.

No, as I told you several times by now. Forms are always executed in the context of the main document that includes the form. Where it is pulling from becomes totally irrelevant after inclusion of the wiki markup (before executing actual TracForms processor instructions).

[Steffen Hoffmann]

My comments should have clarified the situation and you observations, but I got no definitive feedback by now. Any more questions?

Because I can't see a way to help you further on, I'll step this down priority-wise for now, and will close it later on after some more time without more complaint.

[Steffen Hoffmann]

The reason for this ticket was an upgrade issue, caused by a diversion of data representation by different Trac db back-ends. Nice, that the OP found and shared a fix himself.

Evidently comment 4 was caused by fundamental misunderstanding the way TracForms work internally, but no more comments after my attempt to clarify. So nothing left to do here.