Intermittent 502 response from Trac

[btlogy]

As mentioned in #4176, some users have confirmed receiving the following answer from ​Trac intermittently:

502 Bad Gateway
nginx/1.10.3 (Ubuntu)

Now the disk space issue has been fixed, it seems like refreshing the page usually works around this 502 problem.

However, as the checks from the Upttime monitoring seems to show, there are cases where a user could not get a proper answer from the server even after 3 retries!

Here are the details about the incidents reported for yesterday 20th of May:

• ​Wiki is down at 07:27 UTC #424

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki
  Result from test 502 0.028272
  Result from test 502 0.016042
  Result from test 502 0.016079
  

• ​Web is down at 08:06 UTC #425

  Checking https://www.tahoe-lafs.org/
  Result from test 502 1.064882
  Result from test 502 0.13437
  Result from test 502 0.13468
  

• ​Wiki is down at 08:21 UTC #426

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki
  Result from test 502 0.169718
  Result from test 502 0.131224
  Result from test 502 0.094317
  

• ​Wiki is down at 10:22 UTC #427

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TitleIndex
  Result from test 502 0.063878
  Result from test 502 0.063225
  Result from test 502 0.07016
  

• ​Tickets is down at 10:56 UTC #428

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/query
  Result from test 502 0.652738
  Result from test 502 0.068121
  Result from test 502 0.068075
  

• ​Wiki is down at 18:06 UTC #429

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TitleIndex
  Result from test 502 0.05107
  Result from test 502 0.050887
  Result from test 502 0.050857
  

• ​Web is down at 20:21 UTC #430

  Checking https://www.tahoe-lafs.org/
  Result from test 502 1.079979
  Result from test 502 0.031663
  Result from test 502 0.031868
  

• ​Wiki is down at 20:34 UTC #431

  Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TitleIndex
  Result from test 502 0.079088
  Result from test 502 0.065165
  Result from test 502 0.079371
  

• ​Web is down at 21:01 UTC #432

  Checking https://www.tahoe-lafs.org/
  Result from test 502 1.622923
  Result from test 502 0.106893
  Result from test 502 0.160037
  

There are likely more cases which have not raised incident like the following one where one of the retries get the proper answer:

Checking https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TitleIndex
Result from test 502 0.083277
Result from test 502 0.097025
Result from test 200 13.603759

Hopefully those timestamps could help to spot the root cause in the logs of the server.

[btlogy]

The upptime monitoring tool keeps ​reporting intermittent 502 responses and I just had a 500 server error while refreshing this page!

The root cause of this problem is likely to be found in the logs of the server, but only Brian and Meejah have access to it.

[meejah]

Exarkun also has access, AFAIK. Perhaps others.

[btlogy]

I got another bunch of 502 while trying to use Trac today:

GET /trac/tahoe-lafs/chrome/common/trac.ico HTTP/1.1
Host: tahoe-lafs.org
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0
Accept: image/avif,image/webp,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br, zstd
Connection: keep-alive
Referer: https://tahoe-lafs.org/trac/tahoe-lafs/query?status=new&status=reopened&col=id&col=summary&col=keywords&col=status&col=owner&col=type&col=priority&desc=1&order=id
Cookie: trac_form_token=7c1802bec3711d516417977c; trac_auth=841947b09524d27722502e4a5df63098
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-origin

HTTP/1.1 502 Bad Gateway
Server: nginx/1.10.3 (Ubuntu)
Date: Fri, 04 Jul 2025 07:56:21 GMT
Content-Type: text/html
Content-Length: 182
Connection: keep-alive