Deluge keeps downloading stuff I SPECIFICALLY told not to

[DSpider]

So let's say I want to download a tv show, documentary, whatever. And it comes with a sample file. I uncheck the sample (thus SPECIFICALLY telling Deluge NOT to download it), and surprise surprise, it downloads the sample. The worst part is when there's a compilation full of stuff I don't want, I uncheck almost all of them but end up WITH all of them on my HDD !!! And obviously, Deluge says they're only 0.02%, 0.17% completed yet it takes up the full amount of the file(s) because I chose the Full Alocation option.

I read the "Deluge sometimes creates files that I didn't ask it to download?" section from the FAQ.

I guess that's what happens with samples, generally being identical to the complete file. But I still don't understand why it downloads parts of the torrent that have absolutely nothing in common with the other files from the torrent (apart from the extensions).

And by the way, it's not "sometimes". It's actually every, single, FRIGGING time. At first I said "ok, fine, I'll just have to delete them manually". You know what it does then ? IT DOWNLOADS THEM AGAIN.


I believe uTorrent deals with identical parts (ie. samples) by stuffing them into a .dat file inside the download directory. Can't Deluge do something similar but with them somewhere else ? For instance the ~/.deluge/temp/ or something like that ?

Is there a solution for this ? A patch ? A plugin maybe ?


PS: I'm using the latest Deluge 1.3.1 from the Arch Linux repositories. Everything is up to date and the filesystem is ext4.

[Cas]

I understand your frustration but you have misunderstood the basic principles of a torrent.

This has nothing to do with identical parts, it has to do with pieces that every torrent is divided into. What you will find is that in order to download a particular file within a torrent, the pieces needing to be downloaded at the start and end of the file will usually be shared by its two neighbouring files. this ascii art below i hope makes it clearer:

Code: Select all

mytxtfile1|mytxtfile2|mytxtfile3
piece1|piece2|piece3|piece4|piece5

In this example if you wanted to download only mytxtfile2 you would need to download pieces 2-4 and since pieces 2 & 4 are shared by mytxtfile1 and mytxtfile3 they would have to be partially downloaded too, hence the result you are seeing. This also means that if you are attempting to seed the file you will need those pieces that are shared so you cannot delete those files you deem unneeded.

I agree that the way Deluge deals with this currently could be improved and I think feature requests exist in the ticket system.

[johnnyg]

As Cas said, sharing a piece refers to files being consecutive, not having similar content.
Also, ext4 supports sparse files, so it isn't actually using up the full amount.

[DSpider]

So let me get this straight, "mytxtfile2" is complete on my HDD but if I want to seed it I have to keep that 0.02% from "mytxtfile3" (because of piece4).

Ok... But the problem is "mytxtfile3" takes up the full amount of space instead of just 0.02% of the file.


Worst case scenario is if I sparingly select files from a large compilation and Deluge downloads them all.

But wait. It doesn't download them all. It just allocates the full amount of space of the torrent when I only chose about half the files or so from the list.


This is a very fucked up way to do things. Disk compression could probably solve it by compressing something that probably looks like "000000000000000" (dummy sectors) but it's not an option. Mainly because when I download something my pppoe setup raises the CPU bar to the point of crippling the system (I have yet to discover process prioritizing on Linux).

Also, ext4 supports sparse files, so it isn't actually using up the full amount.

Clicking Properties says otherwise.


Edit: Hmmm... Nautilus-elementary sees an improvement in free space when deleting these files. PCManFM does not.

In fact there's a 14.7 GB difference between the two regarding free space. Hmph.

Edit #2: I thought PCManFM was better coded but after running df -a -h seems Nautilus-elementary is right.

Edit #3: My bad. I think PCManFM uses a different reference (probably 1000 vs 1024). Nautilus-elementary, Thunar and "df" see 700 MB files as 700 MB. PCManFM sees them as 734 MB.

So yeah. Clicking Properties says otherwise. Unless using sparse files isn't enabled by default or something.

[Cas]

I found this plugin that may be of some help to you: http://forum.deluge-torrent.org/viewtop ... =9&t=28915

[DSpider]

The plugin you mentioned deletes partial files only when you remove the torrent from the list. But it's useful, no doubt.

A possible solution would be to hide them inside a ".cache" folder for instance or some kind of container (a la uTorrent *.dat files). I already asked the author if he can make one that simply hides them. That is, until a better method comes along... A container sure would be nice, though.

[loki]

YES, if at all possible hide them in a cache file somewhere from the average user's view and maybe then, finally, we won't have people coming in every few days bitching and complaining they have extra files they didn't want taking up all their valuable hdd space. Or Deluge downloading files they specifically marked as do not download.

[johnnyg]

So let me get this straight, "mytxtfile2" is complete on my HDD but if I want to seed it I have to keep that 0.02% from "mytxtfile3" (because of piece4).

Correct. This is unfortunately a limitation of the BitTorrent protocol.

Worst case scenario is if I sparingly select files from a large compilation and Deluge downloads them all.

But wait. It doesn't download them all. It just allocates the full amount of space of the torrent when I only chose about half the files or so from the list.

Full allocation (i.e. allocating space for the entire torrent) is required to be able to do selective downloading.
However with sparse files, it won't actually occupy the entire amount.

I have yet to discover process prioritizing on Linux

Kinda off topic, but:

Code: Select all

man nice

Also, ext4 supports sparse files, so it isn't actually using up the full amount.

Clicking Properties says otherwise.

That probably shows the apparent size, not the actual size.
Try:

Code: Select all

du -shB1 <file>

[DSpider]

YES, if at all possible hide them in a cache file somewhere from the average user's view and maybe then, finally, we won't have people coming in every few days bitching and complaining they have extra files they didn't want taking up all their valuable hdd space. Or Deluge downloading files they specifically marked as do not download.

I call your sarcasm and raise you an "exactly !"

Hope someone makes a plugin that at least hides them. Because, right now... the way I see it... if I were to do it manually (on Linux) I would probably have to rename them by adding something like ".partials/" in front of each file and folder... But renaming every, single, file (and folder) I mark as "Do Not Download" is a real PITA to say the least. I think Deluge definitely needs to handle them better. A container or wrapper of some kind, for example a zip archive set to "store" (ie. compression level 0) renamed "seedinfo.tmp" or similar. That would probably make Deluge absolutely the perfect BitTorrent client, imo.


johnnyg, thanks for the info. Found a really good tutorial on process prioritizing here if anyone's interested.

Interesting results for the "du -shB1 <file>" command, though. If I were to copy the sample to my user folder, it displays the full amount of space. But if I leave it there and run the command it's significantly smaller (I bet closer to that 3.54% that Deluge says it downloaded). Here:

Code: Select all

[dspider@arch ~]$ du -shB1 ~/Sample.avi
56528896        /home/dspider/Sample.avi
[dspider@arch ~]$ du -shB1 '/media/mount_point/download_directory/download_title/Sample/Sample.avi'
2002944 /media/mount_point/download_directory/download_title/Sample/Sample.avi
[dspider@arch ~]$ 

Speaking of which, why is there a difference between the folder and the downloaded file (3.00% vs 3.54%) ?



How is it possible that the file is larger than the folder ? Is this a bug ?



Edit: Actually it's not larger than the folder because they both read 53.9 MB... Just strange, tho.

And apparently, renaming the sample dir from inside Deluge to ".partials/Sample" created a ".partials" folder with a "Sample" folder inside (which was what I wanted) but I still have the original "Sample" folder in the download directory. Turning off the DeletePartials plugin and restarting Deluge fixed it. Maybe I should let the author know about it. (see bellow)

Edit #2: Holy crap. The whole renaming scheme is buggy as hell with or without DetelePartials enabled. Renaming files and folders starting with ".partials/" is NOT an option. It scatters them all over the place making Deluge see them in places they're not. Was this feature just implemented ?

[johnnyg]

Hope someone makes a plugin that at least hides them. Because, right now... the way I see it... if I were to do it manually (on Linux) I would probably have to rename them by adding something like ".partials/" in front of each file and folder... But renaming every, single, file (and folder) I mark as "Do Not Download" is a real PITA to say the least. I think Deluge definitely needs to handle them better. A container or wrapper of some kind, for example a zip archive set to "store" (ie. compression level 0) renamed "seedinfo.tmp" or similar. That would probably make Deluge absolutely the perfect BitTorrent client, imo.

Making a plugin to rename files you don't which to download wouldn't be too hard to do, however if you wanted to write all the unwanted pieces to a single file (à la utorrent) then that would require changes in libtorrent.

Interesting results for the "du -shB1 <file>" command, though. If I were to copy the sample to my user folder, it displays the full amount of space. But if I leave it there and run the command it's significantly smaller (I bet closer to that 3.54% that Deluge says it downloaded). Here:

Code: Select all

[dspider@arch ~]$ du -shB1 ~/Sample.avi
56528896        /home/dspider/Sample.avi
[dspider@arch ~]$ du -shB1 '/media/mount_point/download_directory/download_title/Sample/Sample.avi'
2002944 /media/mount_point/download_directory/download_title/Sample/Sample.avi
[dspider@arch ~]$ 

That looks like whatever you have used to copy the file doesn't support sparse files and has written the entire file to disk (writing zero bytes for the sparse sections).

Speaking of which, why is there a difference between the folder and the downloaded file (3.00% vs 3.54%) ?



How is it possible that the file is larger than the folder ? Is this a bug ?

Edit: Actually it's not larger than the folder because they both read 53.9 MB... Just strange, tho.

I'm guessing that it's the only file in that folder? Could be a bug in how the progress bars are calculated.

And apparently, renaming the sample dir from inside Deluge to ".partials/Sample" created a ".partials" folder with a "Sample" folder inside (which was what I wanted) but I still have the original "Sample" folder in the download directory. Turning off the DeletePartials plugin and restarting Deluge fixed it. Maybe I should let the author know about it. (see bellow)

That has nothing to do with the DeletePartials plugin; I think libtorrent doesn't remove the original folder once you've renamed it.

Edit #2: Holy crap. The whole renaming scheme is buggy as hell with or without DetelePartials enabled. Renaming files and folders starting with ".partials/" is NOT an option. It scatters them all over the place making Deluge see them in places they're not. Was this feature just implemented ?

I'm guessing this is renaming in the files tab?
Could you give me reproducible steps of what you were trying to rename and what you ended up with versus what you were expecting?