Page 1 of 1

how best to isolate tracking problems? [SOLVED]

Posted: Thu Apr 26, 2018 3:20 pm
by betazoid
I seed regular losses in tracker stats using Deluge. My question is how do I best isolate the issue?

I have a script that compares Deluge stats with the tracker stats and here is a very typical example of its output:


Analysis is complicated since tracker stats sometimes have low resolution and are delayed by a few minutes, but the script compensates as well as possible (the columns are explained below the table)

    Items #10 and #12 have losses of 0.5 hours AND sizable upload credit (286 and 452 MB respectively); for those, the analysis is consistent over many samples and the loss is not a transient effect).
    Items #10 and #10 are RSS triggered torrents. No doubt the first 30 minutes of tracking is lost, which is usually the peak upload interval since seeding competition is the least then.
    Having 10 to 20% affected torrents is typical. I've not seen torrents lose more than one interval. Nor have I seen losses except immediately after creation. Only RSS triggered torrents have been affected in my observations.

I'm hesitant to turn on tracing because I've read that it is rather a fire hose, and I'd have to trace for days for analysis. Are there any suggestions how to more "surgically" isolate the fault here (to Deluge or the tracker or seedbox or to whatever)?

Re: how best to isolate tracking problems? [SOLVED]

Posted: Fri May 04, 2018 3:20 am
by betazoid
I'm answering my own question, but any comments on methodology and/or conclusions are still welcome.

To gather enough info to (fairly well) isolate the problem, I created a script that logged the state of every torrent only when significant changes were noticed. These changes included state changes, 10MB of additional upload, 5% more progress, and announces (presumed when "next_announce" jumps up). This provides the essence of the timeline easily filtered for one torrent w/o going overboard. For a torrent with shortages, here is a screen capture of the log followed by the tracker data:


Then notice:
    at second 195, the torrent completed and 60MB upload
    at second 1995, the 30m announce was made with 350-60=290MB upload; notice that #seeds jumped indicating tracker responded
    at second 3796, the next 30m announce was made with 410-350=60MB upload; again the #seeds jumped indicating tracker responded
    the tracker only credited only 30 minutes and 107MB after reporting the second announce

My suggestion is that the tracker only credited upload from before completion and the second 30 minute announce. And it so happens that shorting the first interval is fairly commonplace (e.g., about 20% of my torrents). It certainly seems to be a tracker issue.

Re: how best to isolate tracking problems? [SOLVED]

Posted: Tue May 15, 2018 3:53 pm
by betazoid
A couple more updates and puzzles (for me) ...

Firstly, to try to avoid being shorted the 1st 30 minute announce, I began automatically pausing and re-starting each torrent a minute or two after completing its download. This almost always resulted in the tracker doubling the amount downloaded (i.e., it was very counterproductive). Any theories?

Secondly, someone who stuck with rTorrent throughout, suggested the mistracking of rTorrent had been fixed. To test, I split my RSS subscriptions between rTorrent and Deluge, and rTorrent indeed now tracks perfectly (for 100s of torrents) while Deluge continues to be shorted many 1st intervals. Again, any theories?