Deluge not reporting hash-checks properly

Specific support for Deluge on Microsoft Windows OS
bengalih
Leecher
Leecher
Posts: 56
Joined: Fri Feb 14, 2014 3:31 am

Re: Deluge not reporting hash-checks properly

Post by bengalih »

I'm still unclear here as to the order this is all done.

So - session.state is managed by Deluge and contains the parameters to set libtorrent's operations, yes? While libtorrent's session state information is not unique, it seems Deluge's creation of a file named "session.state" and then using the data in this file to configure the session environment is unique to Deluge.

When you make changes via ltconfig it is clearly modifying the ltconfig.conf file, but is it also modifying session.state?
I'm confused on this point because you say ltconfig knows nothing about session.state, but Deluge also doesn't know anything about ltconfig (as it is a plugin), right?

Can you also specify what specifically would be problematic if one were to delete the session.state as part of Deluge's startup or shutdown process? It would seemingly have to rebuild the ltconfig settings, which I presume it does through what is stored in ltconfig.conf, but does it properly do this on the first restart after session.state is removed? Or is there a race condition where this file would need to be recreated first and then Deluge started again? What other settings (apart from those stored in .conf files) are stored in session.state?

What negative/unexpected effects would occur if no session.state existed upon Deluge launch?
User avatar
ambipro
Moderator
Moderator
Posts: 445
Joined: Thu May 19, 2022 3:33 am
Contact:

Re: Deluge not reporting hash-checks properly

Post by ambipro »

1. This isn't unique to Deluge, libtorrent settings can be configured in qBittorrent and several others (either via direct references to them being libtorrent settings or otherwise) - any libtorrent settings that are configurable via a torrent client "front-end" utilizing libtorrent in any way other than the default settings would use the method Deluge uses to load this information. The files format may not be identical to another, and its behavior with ltConfig does not set defaults properly in this way (since it's a third party plugin) - but this method or one very similar is used because that is how libtorrent is configured.

2. My understanding is that session.state would be saved probably at least when the session is initialized, however I haven't confirmed that the session.state is saved at the regular intervals the fastresume state file is, which IIRC is every 3minutes.

3. If you were to load the libtorrent defaults and apply on startup, it would be my assumption that the session.state file would be updated. After that you could load the high performance seed profile or something else, and it would work as you expect. I'm not sure there are any noticeable adverse effects to deleting it...but without actually testing this in some strange scenarios I can only assume as much. It seems like an incredibly weird thing to have included without purpose, and seeing as you shouldn't be changing your libtorrent config all that often, just loading the previous setting or default seems like a much better solution than interrupting what would otherwise be the normal flow of operation.

4. It would be the equivalent of a fresh install I suppose prior to Deluge setting the few options it does related to connections and network protocols, etc.
mhertz
Moderator
Moderator
Posts: 2216
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: Deluge not reporting hash-checks properly

Post by mhertz »

Ambipro explained fine, so not much to say here, and forgive me for repeating myself with certain stuff just to make sure the full picture realized.

Deluge starts and initializes new libtorrent session and feeds it previously saved session.state if there. Then adds all own libtorrent and otherwise settings, then loads plugins like ltconfig adding possibly its own libtorrent settings(main functionality by ltconfig, so not "possibly" here, but lots plugins do also). Deluge don't know anything about session.state other than signals libtorrent to save it at shutdown and every 200 secs, plus load at startup, that's all, or so I believe atleast. So there's 3 places of settings for libtorrent, meaning a "default" is a term not really possible here. Granted a default hard-coded preset added to ltconfig, but in addition to difference between libtorrent versions completely ignored and not updated for many years, then what about the 50 or alike of them which deluge modifies and lets you modify in its ui. Ltconfig saves current state, to use for reset when unticking option, but if ticking, changing, restarting and unticking, then the "default" gone and void, so cannot be set back properly. So, either have to ignore this, which I propose, as no clean way overcome, or, apply the possibly/possibly-not default preset value here, but could be changed by deluge already, so would imho need check that first, or from other plugin. Same if deleting config setting from ltconfig.conf, as then ltconfig doesn't change that, as have no knowledge of said option's default. It never touches session.state but reads current libtorrent session which as said is a mismatch between session.state(old settings), deluge's current settings and plugins settings, well not mismatch but newest defined taking precedence I mean. With a plugin like ltconfig, which includes possibility of changing _all_ libtorrent settings, then no default state really possible, unless hackishly with a default preset like here.

Again, there's a chance I have misunderstood something, so please also take what I say here somewhat with a grain of salt atleast ;)

Yes as ambipro stated, you can delete session.state fine, e.g scripted as you infer, and not loose anything and just use little more time and computing power. The dht table would be recreated and not restored, but isn't either important and just little less effecient.

When changing in ltconfig, then it doesn't mess with session.state as said, but like with deluge, then the changed stuff gets saved/cached to disk, in addition to just memory, by deluge signaling this saving every 200 sec and at closing. Session.state is just current state of all settings and such, at the time, for effeciency, though skews any "default" notion.

I also thought about saving every setting changed, with an extra 'orig_*' value in config, to use as persistent backup, for use if unticked, instead of current, which only is current session startup, but still problematic imho, as could be changed later by deluge itself, if a deluge setting also, I mean.

It's just the downside of having such a nice powerfully plugin like ltconfig, where everything tunable, while using a client caching all libtorrent settings additionally, which pretty much all do as ambipro stated, as suggested in the tutorial etc. What saved inside can also be configured it eludes too in libtorrent api docs, but we're not gonna make deluge change for this I atleast would presume, and imho nor should, for trading effeciency for a edgecase scenario of single plugin.
bengalih
Leecher
Leecher
Posts: 56
Joined: Fri Feb 14, 2014 3:31 am

Re: Deluge not reporting hash-checks properly

Post by bengalih »

mhertz wrote: Mon Feb 12, 2024 1:14 pm Ltconfig saves current state, to use for reset when unticking option, but if ticking, changing, restarting and unticking, then the "default" gone and void, so cannot be set back properly. So, either have to ignore this, which I propose, as no clean way overcome, or, apply the possibly/possibly-not default preset value here, but could be changed by deluge already, so would imho need check that first, or from other plugin. Same if deleting config setting from ltconfig.conf, as then ltconfig doesn't change that, as have no knowledge of said option's default.
Sorry I'm still having a hard time understanding your explanation of "defaults" being "gone/void".
I assume we are referring to the default behavior of libtorrent? Won't libtorrent just use default settings (like any of those defined https://www.libtorrent.org/reference-Settings.html assuming they are not defined anywhere else?

So the way we define a default value - don't we just remove any hard-coded value that defines the setting thus causing libtorrent to simply use its default value? I don't quite get how it is that hard to guess/assume what a "possibly/possibly-not" default value is. It is either a value set (presumably in session.state) or one that's not set there.

Unless, perhaps you are referring to what ltconfig know of as a default value (or even a valid value?). I'm not sure how ltconfig gets its list of settings, is it hardcoded or enumerated from the libtorrent binary? If the latter, you are saying it is possible for ltconfig to load the tunable settings from libtorrent, but not know what the default settings are? If so then I understand part of the issue there that ltconfig cannot reliably show what the default settings might be. I assume that if I were to delete session.state and my ltconfig.conf file then the settings that ltconfig would show are libtorrent defaults plus any modification made by the standard deluge configuration settings (as defined by the user under the standard deulge settings interface tabs). I assume at least some of these are tuned by Deluge to be something other than the defaults right out the gate.

I'm not as concerned about ltconfig being able to restore/display what "default" settings are as I am about the concept of the ltconfig.conf file not being honored (i.e. bypassed by what is in session.state). I suppose if there were values that could be defined in multiple places...i.e. a setting you can define both in standard Deluge settings and in ltconfig. I don't know which one would take precedent, but there is an argument to be made that if a setting in ltconfig wasn't being honored because the Deluge-specific setting took precedence one might agree that was ok (of course we would hope that the Deluge setting that was taking precedence was also defined in the core.conf or similar in such a way that we could say something like "libtorrent will honor ltconfig.conf settings unless they are over-ridden by a setting defined in core.conf...or the opposite as it were").

Again, I'm still overall a bit confused as I expect I will continue to be unless I want to try to understand the code. As I am only minimally proficient in Python and not at all familiar with the code structure I don't think I can spend the time to do this and therefore will have to continue to rely on both your expertise to determine what is feasible and what isn't. From your explanations so far it sounds like ltConfig is the main culprit. That does not mean that Deluge could not possibly be enhanced to deal with the issue, but I agree implementing a workaround just for one plugin (unless that change allowed more flexibility overall for any plugin) is probably not the way to go. So the question then becomes can the issue be fixed totally within ltConfig. If the answer is "yes" but the addendum is, it isn't worth the time to do then I would simply say "fine" and walk away. I would still disagree with that assessment, but as I am not the one who would be putting in the work, I can't really make more of an argument about that. As long as we acknowledge that the reason the fix isn't being made is it is too difficult or time consuming to fix an error that one might consider an edge-case. If the answer is "no", then I might argue that the reason is because of design which could probably be fixed by a larger overhaul of the plugin - but again I'm not going to press that issue at all because it is not me he is doing the work!

TBH, I think the best solution, especially if you feel that not actually changing anything in the code is feasible or would cause too many issues with other moving parts is simply to provide these caveats more clearly to the end user. As I've said from the beginning, the biggest frustration is that this didn't behave as any rationale user would expect...i.e. ltconfig.conf settings aren't honored (i.e. a missing setting doesn't reset to default). While this issue has been intimated in the ltconfig thread at one time or another, it just isn't clear. I'm assuming that the original author of ltconfig is not going to make any more changes as they have been inactive (at least on GH) for years.

If I could suggest a solution it would be this, based on my assertion that anything documented is considered a "feature" not a bug:

- Update ltconfig to have a message right at the top (either above or below the "Apply settings on startup" message that gives a brief explanation/caveat about default settings.
- Update the output ltconfig.conf file with a comment (if possible). That also mentions something about directly modifying/removing settings.
These should probably mention something about session.state removal to ensure all settings are "reset".

Had either/both of these been in effect (or possible clearly spelled out on the GH page), there would be a much better public understanding of the non-standard behavior of the plugin.

It seems these are both low-impact, quick changes that only serve to better educate. For now, someone having the same concerns as me would be forced to wade through years of forum posts to figure out what's going on.

As for me, I will simply be removing session.state whenever I need to make a change to ltconfig.settings in the future and hope that doesn't have any other adverse effect.

thanks.
User avatar
ambipro
Moderator
Moderator
Posts: 445
Joined: Thu May 19, 2022 3:33 am
Contact:

Re: Deluge not reporting hash-checks properly

Post by ambipro »

Just to clarify, "settings not being honored" is not really an accurate definition. If you uncheck the setting, or remove it from the ltConfig.conf file, it is no longer being set.

It is technically honoring that by not setting anything, and the previous setting remains applied. You can simply use Deluge (or libtorrents) defaults by deleting the session.state - which retains the previous setting you have removed from being set by ltConfig.

I know it's ambiguous and confusing to think of it this way, but this is what the code is doing.
bengalih
Leecher
Leecher
Posts: 56
Joined: Fri Feb 14, 2014 3:31 am

Re: Deluge not reporting hash-checks properly

Post by bengalih »

ambipro wrote: Mon Feb 12, 2024 10:36 pm Just to clarify, "settings not being honored" is not really an accurate definition. If you uncheck the setting, or remove it from the ltConfig.conf file, it is no longer being set.

It is technically honoring that by not setting anything, and the previous setting remains applied. You can simply use Deluge (or libtorrents) defaults by deleting the session.state - which retains the previous setting you have removed from being set by ltConfig.

I know it's ambiguous and confusing to think of it this way, but this is what the code is doing.
I'll take your word on what the code is doing, and whether the fault lies with Deluge, libtorrent, or ltConfig.
However, as I've stated the average user would expect that a setting removed from the config file would mean the setting reverts back to its defaults. This has been true for .conf driven packages for just about ever (or really any software which uses configuration files or registry entries to set their values).

Traditionally, honoring settings that are not there means using a default value for those settings, so no, I don't think it is appropriate to say that ltconfig is honoring settings that aren't there by not changing anything. The plugin (for the various reasons described) simply does not act like one would expect a .conf driven package to behave, seemingly due to the session.state issue.

We can agree to disagree here, but the behavior isn't intuitive because session.state is overriding what is (not) in ltconfig.conf*

Also, I'm not sure I'm reading your statement correctly:
You can simply use Deluge (or libtorrents) defaults by deleting the session.state - which retains the previous setting you have removed from being set by ltConfig.
Unless I'm really not following something here you can also use use the defaults (i.e. explicitly set libtorrent options and/or the default values from omitted settings) set by ltconfig if you just remove session.state. i.e., if you delete session.state then things behave "intuitively" by loading only those things set by ltconfig and/or deluge. If you want to go back to a completely default state (or at least default as defined by your basic Deluge settings), then you can delete both session.state and ltconfig.conf, right?

* Also just want to point out here for sake of comprehensiveness (mostly for others as you probably are aware of this too either from observation or your knowledge of the code), but the ltConfig plugin doesn't even honor its own current settings in this way (taking libtorrent out of the equation). I say this because it seems that ltConfig reads its configuration from either the current session of libtorrent, or directly from session.state itself. If for instance I enable disable_hash_checks in the UI, then stop Deluge and manually remove this line from ltconfig.conf, when Deluge starts up again ltconf will still show this checked under both "setting" and "actual." It would seem to make sense that it might get it's current "actual" from session, but the "setting" should be derived from the actual .conf file - which it is not. IOW there really is a large disconnect from what ltconfig displays vs what is defined in the .conf file because of this issue of "only changing what is explicitly set".
Last edited by bengalih on Tue Feb 13, 2024 3:42 am, edited 1 time in total.
bengalih
Leecher
Leecher
Posts: 56
Joined: Fri Feb 14, 2014 3:31 am

Re: Deluge not reporting hash-checks properly

Post by bengalih »

Actually, I just did a test that to me, seems not to follow how things should operate based on the caveats you guys describe.
Maybe you can recreate my test to see if you get the same thing and if you believe it is expected behavior.
It certainly seems like dangerous behavior to me.

- So I start off with an ltconfig with a couple setting (but not disable_hash_check) and I delete session.state.
- I start up Deluge and load ltConfig and look at the disable_hash_check setting. All checkboxes are unchecked.
- I then go ahead and check to enable that settings and restart Deluge.
- When checking ltconfig settings all checkboxes are now checked. Also the .conf file shows disable_hash_check=True.
(this is all expected behavior)
- I now want to disable it, I try two ways:

(A) I disable the checkbox under the "Name column".
- Doing so prevents me from altering the other columns
- It also removes the line for this setting in its entirety from ltconfig.conf.
- I then restart Deluge and see that the checkbox is still enabled under Setting/Actual
NB: This is really the most misleading out of all of ltConf's configuration. It literally is allowing you to uncheck the setting, removing that setting from .conf file, but then still applying that setting on next boot! This is however the problem we have been discussing - so while it isn't good behavior, we have agreed it is what it is. However it gets even stranger...

(B) instead of disabling the first checkbox "Name" I disable the "setting" option.
- This has the effect of explicitly reversing the setting and you can see this in the .conf file because it will set "disable_hash_checks": false,"
- If I restart Deluge it will show the disable_hash_checks still checked, but the setting/actual checkboxes will be blank (which is correct because the explicitly set value).
(Again, this is "intended" behavior and how you have stated one is supposed to deal with this by always explicitly reversing a setting to what you want it to be).
- Now, I shut down Deluge and manually remove the "disable_hash_checks": false," from .conf file and restart Deluge.
- What we should expect to see is that all checkboxes are disabled but what I actually see is Setting/Actual is enabled again.

*** I've done this B test about 8-10 times and I will admit it does not always result in the same behavior...but it does *sometimes* (really the majority of the time in my tests).
For a quicker test I do like this:
1) Set "disable_hash_checks": true in ltconfig.conf.
2) Start and then Stop Deluge (10 seconds wait)
3) Set "disable_hash_checks": false in ltconfig.conf
4) Start and then Stop Deluge (10 seconds wait)
5) Remove" disable_hash_check" line from ltconfig.conf
6) Start Deluge

What we should expect to see is that disable_hash_check is unchecked completely, but we don't always see this.
Sometimes I've even repeated starting at step #3 and verified Setting/Actual shows as disabled, only to show again as checked after I remove the line completely.

According to the way you guys have describe this is to work I have to explicitly revert the setting to negate it. Which I do, and that works.
However, at that point if I were to remove the "disable_hash_checks" line from my conf file one should then presume that the setting should remain off, since the last value explicitly set was false. But that is not what happens.

Please see if you can recreate for yourselves and then let me know if that behavior is expected (and if so, please explain how to justify it in the context of what we already discussed).

thanks!
mhertz
Moderator
Moderator
Posts: 2216
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: Deluge not reporting hash-checks properly

Post by mhertz »

Hi guys, sorry little slow at times, don't forget though, usually atleast. Thanks for clearing technicalities up ambipro, appreciated, and we're mostly on same page I feel. Btw, I realuze this was actually originally your supporting-role thread, well "your" is little stupidly put, but I forgot that I butted in, and stated would leave it to you to further talk, so i'm sorry about that bro, anyway, just to go over the points of bengalih, and i'll try keep more direct and to the point and not to much repeatings or superfluous wording - my siblings often say I talk(write) to much and they have zero clue what i'm on about lol - technicalities during remote support I mean, but whatever... :)

Anyway, what you describe is a bug if happens, yes. If I understood it right at least. On quick testing I couldn't reproduce, but granted only did quick, not thorough. I will repeat testing this in depth at a later time and if finding an issue or something of interest then will post back of-course. Any setting not in config shouldn't ever be ticked in first column, and setting/actual should usual be same most of time(setting=actual usually, I just meant), but not like you explained is normal, no.

ltconfig and deluge and everything, looks at the libtorrent session for seeing/changing current settings and never in settings.state, but libtorrent just uses that file as a cache, which deluge defines it to self load and save, that's all, nothing else, but as said it is like a default preset of sorts which has the ability to interfer, as libtorrent loads in at startup.

Libtorent knows defaults of-course, but when changed away from said defaults, then it's up to the changer to make note of such, and reverse, and deluge does that for it's own changed libtorrents values, but ltconfig doesn't, because not really possible imho, and only changes what's ticked i.e. in it's config as ambipro stated, nothing else, and only for going the extra mile has added ability to restore to "default" for current run, when unticked option again. If deleting a config-setting manually, then it has no way of knowing if you deleted a setting from the conf, or if it was never set in first place, and hence ignores. Deluge and plugins have a list of default settings to revert from, but ltconfig cannot really do that imho. Sorry repeating here.

What you describe, the actual bug I mean, means that it doesn't write to config-file properly I would presume, though it already writes it after pressing apply I believe, so no time needed between restarts even, so indeed strange, but as said I didn't see here, yet atleast. Or else the session state wasen't saved properly at shutdown or every 200 secs(e.g. closed before 200 secs and failed at shutdown saving properly etc) and hence old values in it persists into next startup, but shouldn't enable first column in ltconfig ever atleast, whcih I believe you said in one of your older posts too.

As said I agree fully in that it's not normal and indeed counter-intuitive, and when I said before "edge-case", then that isn't really accurate neither imho, but I just meant that usually it doesn't cause big issues for people, or confusion, as since never really talked about before, other than you, as ambipro stated, but also I believe it is because it is a hard to notice issue, and e.g. I also only noticed by chance, during testing once, and for most doesn't disrupt there usage and hence not realizing what's going on behind there back.

Imho I agree it should be documented e.g. on it's webpage, but i'm just affraid of adding unneeded confusion where people come back here asking what that means and if they should be concerned or alike, and it cannot be written shortly imho atleast to explain issue, and few would even understand it still lol, so personally I wouldn't add it to the main UI, and probably neither to config, well maybe would be okay as a comment there as you said, not really sure, but atleast it should be documented on webpage imho. However you will have to take that up with the dev as i'm just a fellow forum member like yourself, and granted I forked it for the slight few small changes I made, but i'm not really actively developing honestly, and so I will not make this change myself, but feel you on the proposal atleast is just what I mean. Also i'm sorry I initially talked about we might could do something for it, I was a little too optimistic there and not full picture realized.

Anyway, I understand it's really hard to understand, especially if not studying the code, so feel free ask away again untill satisfied with explenation atleast :)

Edit: Forgot say, that if using the default preset, which is hardcoded, then e.g. as small example resets listen_interfaces(incomming-interface and port) back to 0.0.0.0 with port 6881, and connection_speed(connections per sec) back to 10, despite what you had in deluge already, which is what I mean with not knowing defaults etc, and both these neither actually are defaults(outdated hardcoded preset), neither for libtorrent itself, or deluge, so it's a little of a problem handling such.
rhc
New User
New User
Posts: 2
Joined: Sat Apr 13, 2024 6:55 am

Re: Deluge not reporting hash-checks properly

Post by rhc »

I apologize for necroing this old thread. Here is an *untested* idea:
On construction, a session object is configured by a session_params object. The session_params object notably contain session_settings, the state of the DHT node (e.g. routing table), the session's IP filter as well as the disk I/O back-end and dht storage to use.
Most deluge users likely want to keep state of the DHT node, the session's IP filter, etc. But it is not clear to me why people would want to keep settings already available from config files.

So one workaround is to delete settings from session state before saving to session.state. This can be done by, for example, modifying _save_session_state in core.py as follows:

replace:

Code: Select all

_file.write(lt.bencode(self.session.save_state()))
with:

Code: Select all

state = self.session.save_state()
try:
    del state[b'settings']
except KeyError:
    pass
_file.write(lt.bencode(state))
One might also want to modify _load_session_state similarly.

deluged starts and stops without error after modification in debian 12, but I did not try anything further.
Last edited by rhc on Sat Apr 13, 2024 7:42 am, edited 1 time in total.
User avatar
ambipro
Moderator
Moderator
Posts: 445
Joined: Thu May 19, 2022 3:33 am
Contact:

Re: Deluge not reporting hash-checks properly

Post by ambipro »

This is an interesting thought, and I'm not entirely sure what, if any, consequences may come from this, as I'm not fully sure of the extent in which this is a necessary piece of the puzzle in some other aspect of Deluge or libtorrent.

My first thought is that there _COULD_ be some reason that this is done this way...but it very well could just be an oversight to this edge-case.

You could open a ticket and submit a PR on the github with this, it seems like a very minor change that could solve this problem (assuming it was an oversight) - worst case, we will find out if there's a reason and perhaps another solution will be proposed or something.
Post Reply