wiki:MoveOffTrac

Version 23 (modified by btlogy, at 2025-08-21T12:25:37Z) (diff)

--

Move Off Trac

The goal of this page is to cover the phases of a project aiming at moving some critical features from Trac to an other solution (or combination of).

More information about the start of this project can be found in ticket #4095.

Discussions also happened in Nuts&Bolts meetings: see from WeeklyMeeting#April22024.

In addition, and as mentioned in the deliverables, more details can be found in this dedicated ​MoveOffTrac repository.

Scope

Goals and Requirements

  1. MUST replace Trac as currently used for Ticket and Wiki by some alternative(s):
    • MUST look better: current UI looks old which makes the Tahoe-LAFS project looking dead
    • MUST be better maintained: Trac is dying? max 2 contributors per month - ​https://openhub.net/p/trac
    • MUST be easier to maintain: Trac is difficult to update (current = v1.0.13/2016-09-11, latest = v1.6.0/2023-09-23)?
    • MUST allow Self-Registration (Trac requires manual registration via email)
    • COULD support OAuth2 with Github?
    • MUST be (F)OSS
    • MUST be self-hostable
      • note "able": MUST be a story for getting to a self-hosted instance if we want
      • ..but doesn't have to be self-hosted right away
  2. MUST replace the current landing page (start/home page from the Trac/Wiki?):
  3. MUST replace the current binary repository for Tahoe-LAFS releases (​https://tahoe-lafs.org/downloads)
    • by providing at least a similar way to transfer files via ssh
  4. COULD be used to replace:
    • Github code hosting and review (pull request), keeping only a mirrored clone
    • Github Actions (to avoid leaving secrets in environment variables)
    • Circle CI (to avoid giving them too many permissions)

Inclusions

  • Trac users of the Tahoe-LAFS project
  • Trac issues of the Tahoe-LAFS project
  • Trac wiki pages of the Tahoe-LAFS project
  • Trac HTML home page of the Tahoe-LAFS project
  • Hall of Fame HTML page of the Tahoe-LAFS project
  • The related DNS records (mostly: tahoe-lafs.org)

Exclusions

  • Other Trac projects (see DevInfra)
  • Buildbot master instances for some other Trac projects
  • DARCS SCM for some of the other Trac projects
  • Any other services provided by the current server and not yet documented in DevInfra

Deliverables

  1. A VPS (hosted by Hetzner) providing the following features powered by NixOS and Gitea or Forgejo:
    • a tracking system provisioned with the issues migrated from Trac (same numbers)
    • a Wiki system provisioned with the relevant pages migrated from Trac (same names)
    • a static website replacing the landing page from Trac with the code required for CI and CD
    • a blog post for the Hall of Fame page (if sensible - fallback = static page)
    • a (Git) repository defining the VPS it-self and its configuration as code (including the secrets using sops)
    • optionally including the ToFu code allowing to manage the related DNS records (if Gandi supports it)
  2. A detailed migration plan to handle the transition (DNS changes and/or redirections)
    • documented in a this wiki page
    • covering the possible manual steps related to DNS records, HTTP redirections or URL rewriting
    • with a (Git) repository including/referring to the tools used to actually migrate
  3. An high-level migration plan for the features hosted on the VPS described to be later hosted on CodeBerg SaaS (assuming it is possible)
    • documented in this wiki page too

Deliverable 1 - self-hosted server

  1. DONE - Testing/PoC (retired):
  2. IN PROGRESS - Production (details in #4161):
    1. PENDING - Website only: VPS + OS + NGinx + website + redirect/proxy rules to legacy resources
    2. PENDING - All features from the PoC above: issues and wiki hosted by Forgejo
      • ⛔ Extract the data and reconfigure Trac in RO via root access to the legacy server
      • ⏸️ OR asynchronous workaround with Meejah
      • ⏸️ Rework the redirect/proxy rules to use Forgejo instead of Trac

Deliverable 2 - migration plan from Trac to self-hosted Gitea or Forgejo

  • Requirements:
    • Code: ​MoveOffTrac and ​trac2gitea submodule
    • Credentials:
      • SSH key to access Trac data on Linode server
      • API token to write content under the Forgejo org/owner
  • Steps: 1 On Trac server (the export is scripted in helpers/trac_dump.sh in ​MoveOffTrac):
    1. Activate the Python environment if needed
    2. Launch a trac-admin <path/to/project> shell and run the hotcopy <dump_dir> command
    3. Purge session from the hotcopy with trac-admin <dump_dir> and session purge ...
    4. Purge db backup from rm -rf <dump_dir>/db/trac-backup*
    5. Purge log files from the hotcopy rm -f <dump_dir>/log/*
    6. Compact the DB with sqlite3 <dump_dir>/db/trac.db 'VACUUM;'
    7. Copy trac.htdigest to <dump_dir>
    8. Copy trac public_html to <dump_dir>
    9. Archive the Trac data tar -czvf <dump_file> <dump_dir>
  1. On a workstation:
    1. Checkout ​MoveOffTrac and ​trac2giteam submodule (e.g. in ~/MoveOffTrac)
    2. Restore the dump with tar -C trac/project -xzvf <dump_file> --strip-components=1
    3. Adapt or patch the configuration with patch trac/project/conf/trac.ini < helpers/trac.ini.patch if needed
    4. Launch a docker-compose run --rm trac bash -c "trac-admin project" session and run permission add <username_here> admin if needed
    5. Start the server with docker-compose run --rm --service-ports trac
    6. Open a browser on ​http://localhost:8000/project/ and verify the service (including the login if needed)
    7. Edit the WikiStart page to replace link: /~trac/ -> ../chrome/site/ or transform them as attachment if needed
  2. On the Forgejo server:
    1. Checkout ​MoveOffTrac and ​trac2giteam submodule (e.g. in ~/MoveOffTrac)
    2. Create an empty repository and (e.g. trac) and disable unwanted features (e.g. Unit/Code?)
    3. Create an empty Wiki home page in the repository prepared above
    4. Build trac2gitea using the instructions provided (in ​trac2gitea)
    5. Transfer the Trac project directory from the workstation (previous sextion) (e.g.: in ~/MoveOffTrac/trac/project)
    6. Remote: migrate the Trac project to Forgejo: 
        export TOKEN=<token>
  export OWNER=tahoe-lafs
  export REPO=trac
  sudo ./trac2gitea/trac2gitea --app-ini /var/lib/forgejo/custom/conf/app.ini --db-only trac/project /var/lib/forgejo $OWNER $REPO
  sudo ./trac2gitea/trac2gitea --app-ini /var/lib/forgejo/custom/conf/app.ini --wiki-only --wiki-token=$TOKEN --wiki-dir=/tmp/$REPO trac/project /var/lib/forgejo $OWNER $REPO
    7. Remote: systemctl start forgejo.service
  • Caveats:
    • migration works only with sqlite DB!
    • tables are not always correctly converted to MD (e.g. #1170 - possibly wrongly formatted)
    • some original links are still not converted and points to tahoe-lafs.org (e.g. ​http://!?)
    • tickets authors need to be created as user before the migration to link contributions to their profile
    • changes history of wiki pages will be not be linked to their authors

Deliverable 3 - high-level migration plan from self-hosted to an SaaS provider

  1. DONE - Migration of the Forgejo repository/ies:
    • ✅ Plan:
      1. Sign-in to ​Forgejo and create an API token on Forgejo with Read access on repositories own by the ​Tahoe-LAFS org
      2. Sign-in to CodeBerg as a member of the ​Tahoe-LAFS org
      3. Use the ​migration tool on CodeBerg to migrate the repository/ies from ​Forgejo
      4. Archive the migrated repository/ies on ​Forgejo
    • ✅ PoC (examples):
      • Issues and Wiki pages originally hosted on Trac (archived)
        • Outcome: ​https://codeberg.org/Tahoe-LAFS/trac-2024-07-25
        • Caveats:
          • all links to contributors are broken (likely because CodeBerg will not allow this)
          • all links to comments are broken (likely because they use the ID from the source!)
          • all the link to attachment are broken (likely because the migration tool does not cover them - yet)
      • Code, issues, pull requests and CI/CD of the new static website (archived)
  2. IN PROGRESS - Migration of the static website
    • ▶️ Prospects:
      1. ​CodeBerg Pages, which will break legacy links (e.g. from GitHub)
      2. ​StaticHost, which might support redirect/proxy rules and avoid broken links.