AttributeError on db.rollback() in environment_needs_upgrade

[Steffen Hoffmann]

On environment upgrade with trac-admin of Trac 0.13dev I get

  File "/lib/python2.5/site-packages/Trac-0.13dev-py2.5.egg/trac/env.py", line 646, in needs_upgrade
    if participant.environment_needs_upgrade(db):
  File "build/bdist.linux-i686/egg/announcer/api.py", line 493, in environment_needs_upgrade
    db.rollback()
  File "/lib/python2.5/site-packages/Trac-0.13dev-py2.5.egg/trac/db/util.py", line 107, in __getattr__
    raise AttributeError
AttributeError:

This applies to all recent versions of Trac, because the rollback method has been removed from read-only connections. db argument to environment_needs_upgrade IEnvironmentSetupParticipant method is such a read-only connection (see ​T:#10451).

Maybe this is related to other currently open tickets, but then this is rather blurred there, so better have it explicit here.

[Steffen Hoffmann]

I encountered the same issue with TagsPlugin before, so a solution similar to [11041] should work. Or better don't rely on these checks for non-existent db tables.

May I go for workaround (test cursor attributes before) or should I rather try to rework the whole method?

[Ryan J Ollos]

#10230 closed as a duplicate.

[Ryan J Ollos]

comment:18:ticket:9521 and later contains the best solution to this problem, if I've been understanding the progress correctly.

[Ryan J Ollos]

#8475 closed as a duplicate.

[Ryan J Ollos]

comment:14:ticket:5774 seems to have this same problem.

[Steffen Hoffmann]

Taking over now. As outlined in #5774, this is WiP and a major headache not only for this plugin. But due to success with TagsPlugin I'm confident to have a solution rather soon.

In fact most of the code is already in place, but I'm preparing upgrade modules and unit tests for the upgrade process - nothing of that is readily available - right now. This requires a clear mind and considerable SQL foo too. Thanks for your patience so far.

[Steffen Hoffmann]

#9282 is another ticket, that seems related to same/similar issue.

[Steffen Hoffmann]

(In [12296]) TracAnnouncer: Part 1 of 7 - Move towards a versioned db schema for this plugin, refs #5774 and #9742.

Main feature of the code added here:

• schema version tracked in system Trac db table

Now we introduce the common, recommended approach of tracking plugin db schema versions in Trac db table system, that doesn't require any guessing later on.

This changeset requires a database upgrade.

Make sure to backup your db before upgrading, especially if you put faith into recovering user settings later on. While it may become technically possible to recover parts of an older configuration, the value of such a configuration is rather questionable in context of the current feature set, and I guess that doing a conceptual reinitialization by starting from scratch is what many (most?) users should consider anyway.

Stand back and wait, at least until it has survived more rigorous testing by other developers and until enough data migration code has been developed to preserve critical parts of your Trac applications configuration in production.

[Steffen Hoffmann]

(In [12297]) TracAnnouncer: Part 2 of 7 - Restore db-related plugin history, refs #5774 and #9742.

Main feature of the code added here:

• seamless schema upgrade by dedicated upgrade scripts - Trac core style

From the beginnung (see [3015]) this plugin's SQL driven schema check relied on a SELECT raising an exeption for non-existing db table. This has been discussed lately and found to be a flawed approach, that is even known to break installations and interferes with db upgrades for Trac 0.13dev and ultimately Trac 1.0 as well.

After introduction of the db schema version number for this plugin, table existence testing is required one last time, current schema version is registered, and only the registered schema version gets checked further on.

[Steffen Hoffmann]

(In [12298]) TracAnnouncer: Part 3 of 7 - Follow historic footsteps in schema development for this plugin, refs #5774 and #9742.

The schema discovery logic already leaks a bit about my recent research and about the number of required follow-up changes to add an incremental upgrade module for each discovered schema revision as well.

Futhermore, thanks to Jun Omae we utilize a db-specific, non-intrusive query for retrieving the db table list used in any table existence check from now.

[Steffen Hoffmann]

(In [12299]) TracAnnouncer: Part 4 of 7 - Just taste an evolutionary moment, refs #5774 and #9742.

The switch to the current subscription interface was certainly a reasonable and visionary step, but it has still some tasks pending for completion.

[Steffen Hoffmann]

(In [12300]) TracAnnouncer: Part 5 of 7 - Care for some schema adjustments, refs #5774 and #9742.

Early adopters beware: The long-awaited script for data migration from subscriptions to subscription and subscription_attribute db tables is still a pending issue, that requires even more investigation.

Currently the upgrade procedure just drops the old table irrevocably, following corresponding historic schema changes, hereby labeled as revision 4. Make sure to backup your db before upgrading, if you still put faith into recovering some old user settings later on - you've been warned twice now.

[Steffen Hoffmann]

(In [12301]) TracAnnouncer: Part 6 of 7 - The best way great ideas can go: refinement, refs #5774 and #9742.

With a seemingly small, but essential db table change we're approaching at the current db schema now.

[Steffen Hoffmann]

(In [12302]) TracAnnouncer: Part 7 of 7 - Finally: Go from present to future, refs #5774, #7975, #8065, #9742 and #10384.

Now we've bridged the gap and provide an upgrade path for each historic schema revision of this plugin, while data migration is incomplete yet, especially regarding subscription attributes stored in session_attribute (before v3). Due to component name changes the conversion will be rather complicated, and therefore corresponding research and development is postponed to a future date and will largely depend on explicite demand as well.