Incorrect Download Reporting RC5

[Ibslice]

Deluge Deluge 1.2.0 RC5 and maybe even before that is reporting 20 to 30% extra download stats on our site .. many many other trackers. We have spent the last few hours investigating with many members. From what I can tell, it was also happening with the 1.9 series. It is doing this for EVERY member using it on EVERY torrent.

Other testers from other sites also wrote

"ive been testing this client out lately (rc5)
i can confirm it does add a little extra on download stats..more like 10% extra in my case but thats on smaller torrents"

I feel it might be to do with it reporting hash fails but I am not sure. (and which I know it a libtorrent thing usually but rtorrent/libtorrent is reporting fine)

We have had to ban Deluge for now until it is fixed, as are other trackers due to matching tests.

Note: ALL other clients reported correct stats (up and down)

[trektorrents]

This is terrible news...

[andar]

I'll bring this up with the libtorrent developer. For the record, libtorrent has nothing to do with rtorrent, we use an entirely different library: www.libtorrent.org

[loki]

I'd hate to say it, and I'm not sure how it's possible, but, is it possible that the site is the problem? I've been using the latest deluge on a private tracker site and it reports basically exact amounts for downloaded (less than 1% extra possibly due to hashfail, completely acceptable in my opinion, btw these are files between 100 and 700MB). The only slight thing I notice is sometimes it has a bit of a delay on reporting dl/ul amounts.

[andar]

I'd hate to say it, and I'm not sure how it's possible, but, is it possible that the site is the problem? I've been using the latest deluge on a private tracker site and it reports basically exact amounts for downloaded (less than 1% extra possibly due to hashfail, completely acceptable in my opinion, btw these are files between 100 and 700MB). The only slight thing I notice is sometimes it has a bit of a delay on reporting dl/ul amounts.

Yes, it could be the tracker. I think that some trackers may not look at the 'left' key sent in the announce and strictly use the value in the 'downloaded' key. This is a problem for clients, such as Deluge, that include the total downloaded (including failed pieces) in their 'downloaded' key. The proper way to do it is to calculate the peers downloaded total by subtracting the 'left' key from the torrent size. This is specified in BEP-3: http://www.bittorrent.org/beps/bep_0003.html

[trektorrents]

This is a problem for clients, such as Deluge, that include the total downloaded (including failed pieces) in their 'downloaded' key.

But this would account for 10-20% of file size in additional reportage consistently?

[jimmysdg]

After running Deluge with debug logging for a while. Hashfails are not the problem. EVERY torrent I download is reported as downloading over torrent_size without any hash fails. There is a major bug somewhere.

[deevee]

I hope this gets worked out. Using another client until then...

[lvm]

I cannot confirm this problem. Marked my dl amount, downloaded a torrent, checked dl again - calculated dl and dl reported by tracker match within 2%. rc5, libtorrent 0.14.6, hardy 64, tracker: thebox.bz

[trektorrents]

One torrent?

You'd need to go through multiple announces to verify the issue. The issue is potentially the reportage of left versus downloaded, so these disparities only mean anything when the download is incomplete.

If you download the entire file within one or two announces, the issue is moot.