Page 1 of 2

Re: Any way to reduce memory when using Deluge

Posted: Tue Sep 21, 2021 6:46 am
by shamael
Hi,
Their are settings that can help but may reduce the seeding capacity too. On a regular computer the caching of the OS tends to "eat" all the memory, certainly with a Bitorrent client.

You can find some hints with the ltconfig plugin and the built-in profile "low memory".

deluge-gtk using large amounts of resident RAM - memory leaks?

Posted: Thu Sep 23, 2021 11:45 am
by iconoclasthero
shamael wrote: Tue Sep 21, 2021 6:46 am Hi,
Their are settings that can help but may reduce the seeding capacity too. On a regular computer the caching of the OS tends to "eat" all the memory, certainly with a Bitorrent client.

You can find some hints with the ltconfig plugin and the built-in profile "low memory".
Would you please elaborate on this? What does "the OS tends to 'eat' all the memory" mean? Memory- and/or space-leaks? Something else? Is this a feature or a bug? Should it be reported & tracked?

I wake up and take a look at what deluge-gtk is using in terms of gnome-system-manager: memory / top: res and it is enormous: 758 MiB, 799M, respectively...and that's just today, it can be twice that on other mornings. Also, can you point to anything more specific than 'itconfig plugin and [its] built-in low-memory profile?' Are there guides or tutorials for itconfig that would make dealing with this plugin easier? Not for nothing, but there's zero documentation on git about how to use, only "WARNING: Modify settings at your own risk!" and a broken link to the libtorrent manual on the rasterbar website. Given what I understand the power of itconfig to be, perhaps it is a bit irresponsible (wc?) to suggest someone go play with those settings without appropriate caveats?

Further: The itconfig plugin will not install for me. (This might be an issue that requires a separate thread...) I built from source, tried restarting deluge-gtk and installing, restarting deluged and installing, downloading the compiled version and installing in restarted deluge-gtk/deluged environment and nothing happens. I checked `cat /var/log/syslog |grep deluged`, noted where it ended, and tried installing again; no update to syslog. NB: `cat /var/log/syslog |grep itconfig` yields no results.

Version info:
$ deluged --version; deluge-gtk --version
deluged 2.0.3
libtorrent: 1.1.13.0
Python: 3.8.10
OS: Linux Ubuntu 20.04 focal
deluge-gtk 2.0.3
libtorrent: 1.1.13.0
Python: 3.8.10
OS: Linux Ubuntu 20.04 focal

Re: Any way to reduce memory when using Deluge

Posted: Sun Sep 26, 2021 3:07 pm
by iconoclasthero
In looking at this again, if deluge-gtk is what is "eating all the memory" in thin-client mode (TCM) (this is my problem; OP was not specific enough with details to determine), what good is ltconfig going to do? Doesn't ltconfig deal with the server-side part of deluge (i.e., deluged)? If that is correct then ltconfig cannot do anything to ameliorate the memory problems of deluge-gtk.

Re: Any way to reduce memory when using Deluge

Posted: Mon Sep 27, 2021 10:21 am
by shamael
There are two part that can be split about the memory and influence you have on it: The OS cache and the Libtorrent cache.

From my experience the Libtorrent cache tends to cache torrent chunks and has no need to be big. It depends for the device capacities of course but with a 8GB machine for example a 512MB cache is enough. This can be tweaked directly in preferences.
To have more control on Libtorrent settings (set depending the version and compilation) the ltconfig plugin can be added. Inside the plugin almost all settings can be set and the default template will help you to identify the scope expected. The High Performance profile for example will try to give all resource/pool/cache a computer can give but is based on a specific example. Reason why a template is helpful but not always applicable as it (architecture of server specs). The same thing for the low memory profile. Each setting can be reviewed in the libtorrent documentation https://libtorrent.org/reference-Settings.html

The OS cache (free -h will show you the buffering/cache column) is also part of the game if engaged in the configuration. This one tends to eat all the memory but in a smart way as managed by the kernel itself. It has a good view about all memories available and has a better chance to cache anything possible w/o having any OOM kill. This is by the way the option the Libtorrent has embraced: the version 2.X will use exclusively the OS cache and the libtorrent cache option should disappear.

The interface (GTK) is indeed not part of you problem here, all these options are for the backend and not the frontend. I didn't check the last bug reports but a memory leak is always possible.

Re: Any way to reduce memory when using Deluge

Posted: Wed Sep 29, 2021 12:28 pm
by iconoclasthero
Sorry, OP, I hope this helps you some because it looks like I've pretty well hijacked your thread.
shamael wrote: Mon Sep 27, 2021 10:21 am There are two part that can be split about the memory and influence you have on it: The OS cache and the Libtorrent cache.

From my experience the Libtorrent cache tends to cache torrent chunks and has no need to be big. It depends for the device capacities of course but with a 8GB machine for example a 512MB cache is enough. This can be tweaked directly in preferences.
To have more control on Libtorrent settings (set depending the version and compilation) the ltconfig plugin can be added. Inside the plugin almost all settings can be set and the default template will help you to identify the scope expected. The High Performance profile for example will try to give all resource/pool/cache a computer can give but is based on a specific example. Reason why a template is helpful but not always applicable as it (architecture of server specs). The same thing for the low memory profile. Each setting can be reviewed in the libtorrent documentation https://libtorrent.org/reference-Settings.html

The OS cache (free -h will show you the buffering/cache column) is also part of the game if engaged in the configuration. This one tends to eat all the memory but in a smart way as managed by the kernel itself. It has a good view about all memories available and has a better chance to cache anything possible w/o having any OOM kill. This is by the way the option the Libtorrent has embraced: the version 2.X will use exclusively the OS cache and the libtorrent cache option should disappear.

The interface (GTK) is indeed not part of you problem here, all these options are for the backend and not the frontend. I didn't check the last bug reports but a memory leak is always possible.
Ok, I mostly follow that. What's "OOM kill?" [Out of memory kill; I'll read up.] WRT to libtorrent rasterbar 2.x: is that something that is worth my time to compile and upgrade (unrelated to this problem). [I.e., upgrade over libtorrent: 1.1.13.0]


As far as the memory deluged is using (and, presumably, what ittorrent can address): I'm not especially concerned. My overall uptime right now is > 2 days and deluged (presumably up for that long) is using 154 M of physical memory vs. 3 GB for deluge-gtk (htop RES column). Here's two views, both have been filtered on 'delug', one is sorted by RES, one is treed:
Image

Image

Here's the view sorted by Memory in gnome-system-monitor:
Image

You asked about what free -h is reporting in the OS cache column:

Code: Select all

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           23Gi       9.3Gi       795Mi       966Mi        13Gi        12Gi
Swap:         4.0Gi       1.7Gi       2.3Gi
You mentioned bug reports...I guess ultimately what I'm trying to figure out what should I do about deluge-gtk monopolizing RAM:
  • Is this a problem for system performance? I am sure there's some way to get chron to restart deluge-gtk at daily intervals if so.
  • Is this a reportable bug that I should submit or just find a workaround, e.g., as described above?

Re: Any way to reduce memory when using Deluge

Posted: Thu Sep 30, 2021 8:31 am
by shamael
(drop if not relevant, I'm travelling so read it too fast maybe )
Deluge can run as a systemd unit (client-server architecture) so you don't have to let the GTK opened.This is what I do most of the time and realize it's maybe not your case: Deluged as a systemd unit with the GTK connecting locally on it (or remotely).

Would it be not a better option than restarting the GTK if this is well your setup?

Re: Any way to reduce memory when using Deluge

Posted: Thu Sep 30, 2021 1:27 pm
by iconoclasthero
I don't really understand what you're trying to say. As I mentioned before, I do not know what the OP is doing, but for me, I have systemd upstart deluged upon boot. If I want to see what is going on with deluged and/or add/remove torrents, I use deluged-gtk in TCM to connect to deluged. This is on a desktop server so it's always on and I just leave stuff open including deluged-gtk and all my browsers. Deluged-gtk, Google Voice, and Gmail all have crazy memory leaks...though the latter two seem to be getting somewhat better.

WRT to submitting this issue as a ticket: https://dev.deluge-torrent.org/ticket/3306
WRT to all memory issues as tickets: https://dev.deluge-torrent.org/search?q ... on&wiki=on

So, clearly there's tickets with memory open (3306) from exactly two years ago today that shows this as a confirmed issue on linux... Everything else, including periodically restarting deluge-gtk, seems to be to be so much putting lipstick on a pig.

Re: Any way to reduce memory when using Deluge

Posted: Fri Oct 01, 2021 9:26 am
by shamael
Thx for the details, it highlights ho your setup is made and all above make sense.
As is seems not solved since long time the Webui left opened is a -not so good- possible alternative right now or the GTK restart of course.

PS: I guess no specific plugins enabled?

Re: Any way to reduce memory when using Deluge

Posted: Mon Oct 04, 2021 6:56 pm
by iconoclasthero
The only plugins presently enabled:
YaRSS2
Label
ltConfig
I just enabled
AutoAdd (I can only use the directory I had before -- Cannot add a new one!)
BlockList
DefaultTrackers

I had stats enabled but it was causing the known logging problem and, since I didn't use it, I simply got rid of it. I should probably use .

Re: Any way to reduce memory when using Deluge

Posted: Tue Oct 19, 2021 4:32 pm
by mrRobot
Is there a reason people aren't using the thin client setup instead of the gtk? That's what I've always used, and never had any memory issues...
https://dev.deluge-torrent.org/wiki/Use ... ThinClient