Torrents slowing to a crawl or stopping variable time after starting and new torrents stuck in "Checking"

[beefcakequad]

hey all - running into an odd problem with my deluge deployment, I can't quite tell where the problem is happening so starting with the application itself.

So I have Deluge deployed in a k3s cluster. It's contained in a deployment with a VPN client (gluetun) that controls its network stack so traffic ONLY goes out via the VPN tunnel and dies if the VPN client/tunnel has problems. Since deployment about a week ago I've noticed that after a variable period of time Deluge just stops working. The webUI and daemon are still running, but in-progress downloads slow to nearly a crawl and newly added torrents don't pass the "Checking" phase. Is this a known issue with Deluge or should I start looking elsewhere?

Some notable information:
- If I tear down the deployment and rebuild it, things work perfectly fine until the error wants to happen again
- I have to include tear down and rebuild of the Deluge config PVC in order to "fully reset" things
- There's no set "timeline" for this to happen; I've seen the same torrent make it to 67% and die, 32%, 81%.. it's all variable
- Within my k3s cluster everything else functions (seemingly) perfectly fine
- My CNI is Cilium; my CSI is Longhorn
- Downloads are written to a PVC provided by Longhorn; they are moved to a network share after download is complete
- I'm seeing no errors in my Gluetun container, no errors in my Deluge container, none in any of my CSI and CNI containers
- The only log entry of concern I DO see is a warning in Deluge:
on_alert_performance: <name of torrent>: performance warning: max outstanding piece requests reached, outstanding_request_limit_reached

[code_glitch]

I can confirm this a thing on Arch and Manjaro as of today

[beefcakequad]

What's your underlying architecture?

[code_glitch]

x86_64, Installed natively from the package manager so deluge 1:2.1.1-4 in both cases.

I launched deluged from a terminal for a couple days, no interesting output

[beefcakequad]

So sounds like a possible Deluge bug then rather than architectural considering we're seeing the same problem on two very different architectures

[code_glitch]

Sadly. I'd love just know how on a certain date it breaks in the same way on what should be two quite different setups. Looking at syslogs it doesn't even look like the underlying system updated itself.

Edit: did some sanity checks and wiping the config, disabling all plugins and setting the log level to info doesn't reveal anything interesting.

The only lead is that when active the caches all read 0. 0 blocks written or read, 0 hits and read cache size is 0. Cache size itself is not 0 and its been told to have 1024 16KiB blocks valid for 60 seconds. Not sure if caches are a symptom of almost no transfer or the lack of caching is causing the symptoms but this is unusable. Thankfully I have another instance with deluge fro like 8 years ago which still runs perfect so I guess I migrate all my use to that until this is fixed

[mhertz]

The caching thing a red hearing im affraid.

I don't know anything about manjaro, but deluge and libtorrent didn't change since 6+ months on arch regardless of testing repo enabled - I don't see this BTW, but granted not a big torrenter either, but regardless.

Anyway, all I can suggest for traffic issues in deluge is try change libtorrent version, v1.2.x preferably and especially when having issues, and you get your cache options/monitoring back as bonus(not used anymore in v2.x) It's in aur and pypa(pip) - edit: Not on pypa though in py3.11 build which needed here.

[code_glitch]

So I tried some more things before I read your reply including installing some old 2.0.5 package binaries I found in an old snapshot and building 2.0.5 fresh, neither of which helped.

A bit of poking around suggested that on arch the libtorrent used by deluge is from libtorrent-rasterbar. Just to see if it was a recent regression I have checked some recent versions and have found that moving from 2.0.9 to 2.0.10 produces zero improvement. Going back down to 2.0.8 or 2.0.7 also doesn't improve things. 2.0.5 failed to compile with the current PKGBUILD but I so it and 2.0.6 are left untested since I got bored of rebuilding packages over and over

I compiled 1.2.19 with a slightly modified PKGBUILD and things seem to be working again so I''m going to shove this thing in the PkgIgnore list and hope I never get to experience this again.

FWIW, Compiling 1.x fought me a little longer than I wouldv'e liked and while this was going on I was running 2.0.7 and I noticed a glimmer of hope for a few minutes where things sped up to a couple dozen mbit/s briefly. No candle to 1.2.19 though where immediately on add transfer rates are a solid few hundred mbit/s every time though. Weird bug.

Edit: Downgraded further to 1.2.11 (older versions seem to not build against modern libboost) and I am blown away. I always thought libtorrent just couldnt keep up with increasingly fast connections. As it happens, old libtorrent can still saturate a connection, it just seems that the newer the libtorrent, the shitter it gets? I can saturate gigabit finally

[code_glitch]

Well I thought it was fixed but now its back without any OS updates. I've re-installed a clean debian 12 and deluge and thats impacted out of the box so it shouldn't be long before someone reproduces this. Versions on debian are 2.0.3 server, libtorrent 2.0.8.0

Edit to add:
Inspecting the systemd logs for the out of the box deluged service debian ships with shows the same errors spammed to high heaven I saw after downgrading the libtorrent for the previous installation.

Code: Select all

Unhandled error in Deferred:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/deluge/core/torrentmanager.py", line 697, in add_async_callback
    torrent = self._add_torrent_obj(
  File "/usr/lib/python3/dist-packages/deluge/core/torrentmanager.py", line 668, in _add_torrent_obj
    log.info(
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1905, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1815, in _cancellableInlineCallbacks
    _inlineCallbacks(None, gen, status)
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "/usr/lib/python3/dist-packages/deluge/log.py", line 69, in info
    yield LoggingLoggerClass.info(self, msg, *args, **kwargs)
  File "/usr/lib/python3.11/logging/__init__.py", line 1489, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.11/logging/__init__.py", line 1622, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info, stacklevel)
builtins.TypeError: Logging.findCaller() takes from 1 to 2 positional arguments but 3 were given
Temporarily disabling observer LegacyLogObserverWrapper(<bound method TwistedLoggingObserver.emit of <deluge.log.TwistedLoggingObserver object at 0x7f98b35a5d50>>) due to exception: [Failure instance: Traceback: <class 'TypeError'>: Logging.findCaller() takes from 1 to 2 positional arguments but 3 were given
/usr/lib/python3/dist-packages/deluge/core/alertmanager.py:140:handle_alerts
/usr/lib/python3/dist-packages/twisted/internet/defer.py:344:__del__
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:190:failure
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:142:emit
--- <exception caught here> ---
/usr/lib/python3/dist-packages/twisted/logger/_observer.py:81:__call__
/usr/lib/python3/dist-packages/twisted/logger/_legacy.py:90:__call__
/usr/lib/python3/dist-packages/deluge/log.py:204:emit
/usr/lib/python3.11/logging/__init__.py:1536:critical
/usr/lib/python3.11/logging/__init__.py:1622:_log
]
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/deluge/core/alertmanager.py", line 140, in handle_alerts
    for attr in dir(alert)
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 344, in __del__
    log.failure(format, self.failResult, debugInfo=debugInfo)
  File "/usr/lib/python3/dist-packages/twisted/logger/_logger.py", line 190, in failure
    self.emit(level, format, log_failure=failure, **kwargs)
  File "/usr/lib/python3/dist-packages/twisted/logger/_logger.py", line 142, in emit
    self.observer(event)
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/logger/_observer.py", line 81, in __call__
    observer(event)
  File "/usr/lib/python3/dist-packages/twisted/logger/_legacy.py", line 90, in __call__
    self.legacyObserver(event)
  File "/usr/lib/python3/dist-packages/deluge/log.py", line 204, in emit
    getattr(LoggingLoggerClass, event_dict['log_level'].name)(
  File "/usr/lib/python3.11/logging/__init__.py", line 1536, in critical
    self._log(CRITICAL, msg, args, **kwargs)
  File "/usr/lib/python3.11/logging/__init__.py", line 1622, in _log
    fn, lno, func, sinfo = self.findCaller(stack_info, stacklevel)
builtins.TypeError: Logging.findCaller() takes from 1 to 2 positional arguments but 3 were given

[mhertz]

Deluge 2.0.3 simply too old for current python/twisted, libtorrent, probably overloading ram with those issues continually printed. I before posted these fixes, but if possible then imho should rather update, pypi, manual, docker, flat etc.