Too Many Pieces Error in Large (>1TB) Torrents?

[DarkSniper]

Hi, I'm attempting to download part of a 1TB+ torrent which has nearly 300k pieces in it. However, Deluge refuses to download it, putting the torrent in the "Error" category, stating "too many pieces in torrent:" as the status.
I'm wondering what the current limit is, and if it is possible to (safely) raise it by editing the something in the source code and recompiling deluge and/or libtorrent.
Thanks!

Environment:
OS: Arch Linux
Deluge Version: 1.3.4
libtorrent Version: 0.15.10
Interface: GTKUI/WebUI connecting to Deluged on headless server
Filesystem: XFS
Installation Method: Package
Plugins: AutoAdd, Batch Renamer, Blocklist, DeletePartials, Label, Notifications, Toggle, TotalTraffic
Link to Problematic Torrent: https://sites.google.com/site/tlmcfiles ... edirects=0

[johnnyg]

Hi,

the problem is related to libtorrent not deluge, so no need to recompile deluge.

In particular, the maximum number of pieces in libtorrent is set to ~262k to reduce memory usage.
I've attached a patch which will increase this limit to roughly 1 million pieces but bare in mind this will increase libtorrent's memory usage (not sure by how much).

A better alternative would be to contact whoever created that torrent and make sure they use piece sizes >= 8MiB for torrents >= 1TiB.

[DarkSniper]

Ah, Okay. Thanks for the patch, it did exactly what I needed.
The box I'm running deluge on usually has plenty of free memory, so a little extra usage is fine. So far it seems to be using about the same amount of memory, but slightly more CPU, though this isn't an issue at this point.
I'll suggest it to them, and see if they'd be willing to switch for the next release.

[felly117]

Hi

I don't want to unnecessarily bump old topics but I'm having the exact same issue as OP was.

However I have no idea on how to go about patching libtorrent. Can someone please help me do this.

Any help is much appreciated

Thanks.

[Cas]

If you use libtorrent 0.16 then this should be enabled by default.

[felly117]

I'm using Deluge on Raspbian which I believe uses libtorrent 0.14.9.

Is there a way of updating this?

Thanks

[Cas]

There is always a way just might not be simple for Raspbian as it will require building from source. Note that if you are upgrading to libtorrent 0.16 you should also upgrade Deluge to 1.3.6.