seeding_time - what is the ultimate source of truth?

General support for problems installing or using Deluge
Post Reply
basher
Leecher
Leecher
Posts: 91
Joined: Wed Sep 29, 2021 8:42 am
Location: Estonia/Spain

seeding_time - what is the ultimate source of truth?

Post by basher »

Disclaimer: using Deluge 1.3.15 due to seedbox limitatios.

While extending AutoRemovePlus plugin I've come across an oddity around seed time. Looks like force_announce() of a torrent updates the seed_time. Does this mean force_announce() asks for this value from the tracker, as opposed to updating tracker with the values Deluge has? How to access up-to-date seed_time data? Who has it to begin with?

Following code excerpt is taken from the plugin, demonstrating what's happening:

Code: Select all

    def reannounce(self, tid, t):
        # note the first two announce_* arg values are the defaults from https://libtorrent.org/reference-Torrent_Handle.html#force_reannounce()
        announce_seconds = 0    # how many seconds from now to issue the tracker announces; default = 0
        announce_trkr_idx = -1  # specifies which tracker to re-announce. If set to -1 (which is the default), all trackers are re-announced.
        announce_flags = 1  

        try:
            t.handle.force_reannounce(announce_seconds, announce_trkr_idx, announce_flags)  
            log.error("AutoRemovePlus.reannounce(): forced reannounce OK for torrent [%s]", tid)
            return True
        except Exception as e:
            log.error(
                    "AutoRemovePlus.reannounce(): Problems calling libtorrent.torr.force_reannounce(): %s", e
            )

        return False


    def remove_torrent(self, tid, torrent, remove_data):
        seed_time = torrent.get_status(['seeding_time'])['seeding_time']
        seed_time_h = _get_seed_time((tid, torrent))
        log.error("remove_torrent(): pre-announce seed_time: [%s], h: [%s]", seed_time, seed_time_h)

        # update trackers to make sure the latest upload amount & time are reflected
        # prior to nuking torrent.
        if not self.reannounce(tid, torrent):
            log.error(
                "AutoRemovePlus.remove_torrent(): reannounce failed for torrent: [%s]; skipping remove", tid)
            return False

        log.error("Running remove_torrent() for [{}] with remove data = {}".format(tid, remove_data))
        try:
            self.torrentmanager.remove(tid, remove_data=remove_data)

            seed_time = torrent.get_status(['seeding_time'])['seeding_time']
            seed_time_h = _get_seed_time((tid, torrent))
            age_sec = time.time() - torrent.time_added
            # note these 2 total_time_* are in bytes, not time values:
            total_time_uploaded = torrent.get_status(['total_uploaded'])['total_uploaded']  # in deluge's internal status, it's under status.all_time_upload
            total_time_downloaded = torrent.get_status(['all_time_download'])['all_time_download']

            log.error("remove_torrent(): successfully removed torrent: [%s]. seed_time: [%s], h: [%s], ratio: %s, age_sec: [%s], total_time_up: [%s], total_time_down: [%s]",
                      tid, seed_time, seed_time_h, torrent.get_ratio(), age_sec, total_time_uploaded, total_time_downloaded)
        except Exception as e:
            log.warning(
                    "remove_torrent(): AutoRemovePlus: Problems removing torrent [%s]: %s", tid, e
            )
This code produces following log entries:

Code: Select all

remove_torrent(): pre-announce seed_time: [632251], h: [175.6253]
AutoRemovePlus.reannounce(): forced reannounce OK for torrent [78616b03e471bca7d8ce9597b6c55fca00ee42e1]
Running remove_torrent() for [78616b03e471bca7d8ce9597b6c55fca00ee42e1] with remove data = True
remove_torrent(): successfully removed torrent: [78616b03e471bca7d8ce9597b6c55fca00ee42e1]. seed_time: [571045], h: [158.6236], ratio: 0.328677713114, age_sec: [633576.046782], total_time_up: [7246677822], total_time_down: [22068859177]
Note seed_time = 632251s before force_reannounce, and 571045s after.

What's going on here?
mhertz
Moderator
Moderator
Posts: 2195
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: seeding_time - what is the ultimate source of truth?

Post by mhertz »

I'm not sure, but quick look at the specification's tracker section states upload/download sent(e.g. so tracker can calculate ratio etc), and didn't see seeding_time/seed_time etc, and think that's just queried from the libtorrent session locally, but could be wrong though. They can manually calculate and log seed-time themselves I presume from timing the time after getting 'event: completed' and untill geting 'event: stopped'.

I don't have deluge1 to test, atleast not right now, but I took autoremoveplus from tote94, as most like original deluge1 version, and added a force_reannounce() with the override min_interval and queried the seeding_time from libtorrent before and after this, and I setup this upon every interval and gets printed to terminal deluge opened from, just to keep tabs continually, and granted i've not run it more than a few minuttes, but the two repeatedly printed numbers of seeding_time printed to terminal every 6 secs roughly(I used interval of 0.001), where both identical, but as said, didn't run hours like you, and was on deluge2, hence I also added the deluge2 flag needed for non-cached libtorrent queries(update=True), which isn't available/needed on deluge1, as uncached always(one of the performance updates of deluge2).

Code: Select all

martin@arch ~ % deluge "https://cdimage.debian.org/debian-cd/current/amd64/bt-cd/debian-11.1.0-amd64-netinst.iso.torrent"
0.0011111111111111111  
0.0011111111111111111
0.0019444444444444444
0.0019444444444444444
0.0030555555555555557
0.0030555555555555557
[...]
Btw, you query the torrent after having deleted it - I query before deleting it as makes better sense imho, I didn't even know it would work when doing it like that, and when trying here it doesn't i.e. empty returned - could it be an old value from the resume file or something, e.g from last deluge run, and not removed just yet? Probably not though and just thinking out loud. Is it same if querying just after the force_reannounce() like me?

I maybe can get to a deluge1 later, but will then need do it in a VM, as installed deluge1 before on main system for testing something and had to install a bunch deps manually and get old compatible libtorrent from the arch-archive of outdated packages, as the AUR package was broken/outdated somehow seemingly, and only VM I have currently is ubuntu, which i'm not good at honestly(PPAs etc), but of-course shouldn't be that hard i'm sure.

Anyway, good questions, as usual ;)

Edit: Doh, I did get an empty seeding_time returned when quering after removal, but just noticed that it was in a try loop, hence no errrors shown, and I stupidly queried from the 'tid' var, as no 't' there, hence empty result, so I added 't' var to be transfered additionally to remove_torrent() and queried through 't' there, and now not empty, but same matching result as before announcing/deleting in my short test runs. Anyway, just wanted to correct that, and say i'm testing the same on deluge1 shortly, and just finished adding my "debugs" to the original deluge1 plugin - btw, off-topic but if continually messing with a plugin, then good tip I gathered, is instead of 'changing > rebuilding' repeatedly, or script that as I used to do, then extract your plugin into a dir you name whatever and just ending in '.egg', and then place that into plugins dir in deluge profile dir, and then you can edit the files therein and restart deluge for picking new stuff up, untill finished and can rebuild(can also just zip up the two dirs if not wanting bother with setup.py). Will return with new edit shortly about the deluge1 (quick test-)run.
basher
Leecher
Leecher
Posts: 91
Joined: Wed Sep 29, 2021 8:42 am
Location: Estonia/Spain

Re: seeding_time - what is the ultimate source of truth?

Post by basher »

> Btw, you query the torrent after having deleted it - I query before deleting it as makes better sense imho
Holy hell, didn't even notice that! I'll amend the code and let it run again. But ye, good question as to what data was even returned after having deleted the torrent.
> I maybe can get to a deluge1 later
See you're using linux, so recommend grabbing a Docker container. Any debian buster image would install deluge 1.3.15 by default, with all the required deps.
> extract your plugin into a dir you name whatever and just ending in '.egg', and then place that into plugins dir in deluge profile dir, and then you can edit the files therein and restart deluge for picking new stuff up
So if the plugin code was in a directory ~/.config/deluge/plugins/arp.egg/, it'd be picked up by Deluge? Thought it has to be .egg, ie a zip file. Plus at the very least it needs the correct -pyX.Y versioning in the name as well, at least that's what I've read. Good to know, but in this case Deluge itself is running on a remote machine, so I'm just updating the .egg via a git repo.

Thanks for taking an interest.
basher
Leecher
Leecher
Posts: 91
Joined: Wed Sep 29, 2021 8:42 am
Location: Estonia/Spain

Re: seeding_time - what is the ultimate source of truth?

Post by basher »

Okay, think it was a case of a PEBCAK; new log:

Code: Select all

remove_torrent(): pre-announce seed_time: [609490], h: [169.3028]
remove_torrent(): post-announce/pre-remove seed_time: [609490], h: [169.3028]
remove_torrent(): successfully removed torrent: ... seed_time: [607686], h: [168.8017], ...
Ie it's the post-remove statistics that look off. Whatever they are. Case closed, thanks!
mhertz
Moderator
Moderator
Posts: 2195
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: seeding_time - what is the ultimate source of truth?

Post by mhertz »

Cool, and you're welcome mate :) Didn't see a change on deluge1 neither, and thanks for tip about docker, that's much easier indeed.

Yes what you said about egg's is correct that states that, but i'm testing like that(.egg dir) on deluge2 and now on deluge1, with core.py open in vim and just :w whenever made change and restart deluge. Python treats zips(as e.g. in egg's) as dirs usually, and only respects py version if in file-name, hence why I usually remove that part, if remembers, from posted plugins as usually compatible regardless, so people don't need rename - btw, deluge2 was supposed to be python version agnostic for plugins(i.e disabled py-version checking of filename), though the commit doing this, didn't work however and so plugins still needing renaming when mismatched(e.g. as also can be seen by the many posts here and elsewhere about why plugin X isn't seen/doesn't enable), but I just submitted PR yesterday to fix it(and which also can be manually added to deluge1 btw).

Thanks likewise for interesting conversation :)
basher
Leecher
Leecher
Posts: 91
Joined: Wed Sep 29, 2021 8:42 am
Location: Estonia/Spain

Re: seeding_time - what is the ultimate source of truth?

Post by basher »

Bit of an update here. I've been using ARP with "force_reannounce_before_remove": true, causing torrent to be force-reannounced prior to removal.
My experience shows this still doesn't fix the issue where at least some trackers flag you for hit&run because their seed time doesn't match what deluge/autoremoveplus thought it was prior to removing the torrent.

I've come across another solution that pauses&resumes the torrent before nuking it. Haven't tried it -- any guess whether it could help with this problem?
mhertz
Moderator
Moderator
Posts: 2195
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: seeding_time - what is the ultimate source of truth?

Post by mhertz »

I think it's just an easy way to force-reannounce without getting handle etc, like you did(pause/resume forces always I believe), and so only diff might be the pause(sleep here I mean - break), but I'm not sure, sorry.

Sorry to hear didn't work fully BTW :( Strange...

Please others in the know speak up kindly thanks :)

Edit: Sorry basher, I have no idea how libtorrent handles resume, forcing or not, don't know why I assumed it always forced, but if I find anything from glancing the source later, then I'll speak up of-course, but probably not, sorry for noise.
Post Reply