Deluge 0.5.9.2 not completing blocks?

[aninhumer]

Since I have updated to 0.5.9.2, (and continuing in .3 so far) Deluge has not been completing the blocks in torrents properly for me. That is, the overall download of a torrent will show I have downloaded say 70%, whereas the info in files, which only shows the amount in complete blocks, will show anywhere between 30% and barely anything.
Presumably when the torrent finished the file would still be completed (EDIT: checked, it does seem to work when they finish), and for single file torrents this not a great issue. But for multi-file series, it means I have to wait a long time just to see the first episode, even though it should have, and possibly has, downloaded nearly all of it.
I have tried rechecking one of the torrents, but all this did was lose all the uncompleted parts, as I expected. Which also shows another problem this causes, in that I have no way to move the torrents out of that particular install of deluge without losing most of the downloaded data.

One other thing I notice is that it seems to finish blocks properly for a few seconds after starting, so when I start it, a whole bunch of blocks finish, and then it goes back to how it was before.

[mivo]

Hmm, I think I can confirm this. The overall completion rate is shown as 43%. If I look at the progress of the individual files (about 200 of them, each 100 MB), the average completion seems to be 25% (which would match the completion percentage when I switched to 0.5.9.2). There are only a few files that are even over 40%.

I'd recheck, but I'm afraid of losing several days of downloading again (that'd be 4 GB of a poorly seeded torrent where I rarely get more than 20k/s). You said that at the end, the files do get completed properly, even if the displayed progress (of the individual files?) is incorrect?

I already lost some 40 GB to the rechecking bug (which, when it happened the first time, caused me to think that my data had gone bad, when it really was a bug).

EDIT: I tested this with a smaller torrent now. Yes, the individual file's progress bar either does not move at all, or only very slowly (0.1% when it should be 1%), and after rechecking, the overall completion percentage reverts back to previous completion rate, so the data downloaded before the rechecking seems lost. I'm not sure if it really is gone or if this is a display bug. Going to let the file finish (individual file's bar stopped again) to see if it completes.

[mivo]

I tested this a bit more. The "blocks" complete at the end, but the speed at which the individual files' progress bars advance is incorrect. It starts very slowly and then accelerates until at the end the torrent and file completion hit 100% at the same time. To illustrate this, here's an example from a 700 MB torrent (one large file, two tiny files in the kb range, so they are irrelevant, size-wise). I had downloaded 17% of the torrent prior to 0.5.9.2, so both the torrent completion and the file completion (of the large file) were at 17%. Here is how this progressed (first number is torrent completion, second number is file completion):

17% - 17% (prior to 0.5.9.2)
60% - 19.4% (with 0.5.9.2 - below numbers as well)
70% - 20.4%
80% - 23.8%
90% - 35.5%
95% - 55.1%
99% - 87.9%
100% - 100%

So this may only be a display/calculation bug, though rechecking the torrent did seem to lose the data not considered/reflected by the individual files' progress bars. This probably needs more testing, but right now it's probably best not to check/recheck any torrents.

(Do the devs read this forum or should we submit a bug ticket?)

[aninhumer]

As I said before, I expect the reason for these problems is that deluge is not completing the checksum blocks of the torrent.
This results in the lower file completions, as these only show confirmed and checksummed completion, not just total data. (You may have noticed the file completion is usually slightly less than the overall, but not this much)
This also results in losing data on a recheck, because it loses any data which does not form completed blocks.
I didn't think of it in my original post, but it also makes it more difficult to seed, as you can only seed completed blocks.

I don't know if this is a new algorithm or something, but as other people are having this problem, I'll submit a bug report. I just wasn't sure if it was something wrong with my setup.

[mivo]

Well, just had a minor power outage, and that's the end of my 17 of 19 GB download with Deluge since I cannot recheck it without losing over a week of data. I'm pretty surprised that there still has not been a bugfix for this problem. It pretty much makes the current version useless for me. It is not stable. I might try the 0.6.0 branch and, failing that, a client that is more reliable. Losing gigs of data is just something a torrent client shouldn't do. (Yes, I'm slightly frustrated right now because these problems have been there for several "stable" versions now.)

[mivo]

Well, yes, all the data is gone for good (even though it's technically there). Other clients won't import it since the checksums are incorrect. On a side note, 0.6.0, the most recently nightly build, suffers from the same problem. Using Deluge for any downloads that take longer than a few hours seems very risky right now, regardless of the version.

[aninhumer]

I submitted a bug now, and hopefully this will be fixed soon: http://dev.deluge-torrent.org/ticket/295

For now, rolling back to 0.5.9.1 does not lose the downloaded data information from .2 and it does complete blocks normally, so until a fix, I'm doing that.

I did lose about a gig when deluge did an unexpected recheck on one torrent, while I was repeatedly restarting deluge before the roll back to complete as many blocks as possible in case it did lose data, but otherwise I've been relatively unscathed.