Bad speed fluctuation and random cutouts.

[Darlok]

It seems we're talking apples and oranges unfortunately; unless your home ISP connection is > 100Mbit, cache shouldn't be the slightest of issues with modern hard drives.

First thing I'd do in your case CoteRL, is try another PHYSICAL drive without going through network, etc. Need to get all the variables out of the problem as possible then work from there. Again, if the problem is cropping up on slower ISP's then it's something else entirely. I've never had an issue with anything 100Mbit or under with what we are discussing. When Gbit and 10Gbit servers come into play, the game changes quickly and the bottleneck actually turns into how fast your storage is.

Obviously the mechanical drive(s) (or SSD) has limitations on read/write and general throughput. This is why I prefer to take that out of the equation by testing on a ramdrive setup. If the problem is gone and pipe is staying full and flat, then you know it's not a network, memory, cpu related issue....

I should have stated that my testing is on Deluge 1.3.5 and libtorrent 0.15.11.0, built from source on Deluge squeeze - same hardware as the original thread starter (bhf). I also run rtorrent on that same box and the issue is not as dramatic, but is still there under heavy load. I am still convinced this is an I/O (cache related) issue unless someone can convince me otherwise with something solid. I also think that some of the problem lies within the libtorrent core itself due to the fact that rtorrent (core is another branch of libtorrent) peaks and valleys are much less dramatic all else being equal.

Again, if you are seeing this problem on <100Mbit, then there is definately something else in play which may or may not be related.

Really hope someone else has some other ideas to kick around. CAS? (yeah gonna call you out), thoughts from a Deluge perspective? Known issues?


EDIT: Ohh and Cote, I don't know jack about CIFS stuff, but I'm starting to think your issue maybe QoS related.

[bhf]

In my case, this helped keep active threads open to flush the buffer before it stacked itself up and effectively killing all incoming I/O (what this problem is). I think I also set Deluge cache to about 840 MB give or take (40960, 360 seconds), but not completely sure how much it's actually helping..No clue honestly how libtorrent handles it's own cache.

From what i can tell this seems to be the problem, a torrent that is ~1GB will hit 100+ Mbit/s while any other torrent ~4GB or larger will hit these speeds then crash, and repeat.
I am pretty confident this is not a IO problem, but cant be certain. As i dont know exactly what type of drives OVH uses.
This problem does not seem to happen in rtorrent. It could be a problem with OVH default distros as every server from them has had this problem, including feral hosting who uses ovh.
This has also happened on Debian.
I have tried pretty much everything as well, IO nice, cache settings in deluge, tweaking linux etc but nothing has really seemed to help.

Let me know if anyone finds a solution that is relativity safe.


Thank you for all the replys guys, its good to see im not the only one.

[buggyme]

I found this thread while looking for another issue. The behavior you are describing is caused by the routers, which don't have enough sockets available. Mine, for example, only allows 1024. Your computer must at least answer with SOMETHING (forgot what common network response is required) to each request, even if the Bittorrent daemon is no longer running. Each of these responses uses another socket #. Half-open sockets count. Anything else your computer is doing on the network counts. Anyone else connected to the same router doing something counts.

Either find a setting to change in your router (no idea how likely this is for yours, but mine has no option), or get a better router.
Perhaps it is possible to tell the router to close the sockets more quickly, but I haven't looked into it.

[xencored]

I am also getting this 'Normal speed followed by slow speeds' over and over again, I stopped using deluge for this very reason. Ive just got the new 1.3.6 hopping this was fixed. Saddly I am still getting it. Now I was running bittorrent and I never got this up&down speeds on that but I so want deluge to work for me and get away from utorrent/bittorrent etc...

http://i.imgur.com/XQuWGbt.jpg

(Also on this forums I can't seem to show a img using the [img] tags I get the error 'It was not possible to determine the dimensions of the image.')

I found this thread while looking for another issue. The behavior you are describing is caused by the routers, which don't have enough sockets available. Mine, for example, only allows 1024. Your computer must at least answer with SOMETHING (forgot what common network response is required) to each request, even if the Bittorrent daemon is no longer running. Each of these responses uses another socket #. Half-open sockets count. Anything else your computer is doing on the network counts. Anyone else connected to the same router doing something counts.

Either find a setting to change in your router (no idea how likely this is for yours, but mine has no option), or get a better router.
Perhaps it is possible to tell the router to close the sockets more quickly, but I haven't looked into it.

If this was the case (for me) why does other clients work fine but deluge does not >.>

[cy53]

Getting random slowdowns on DL here as well. Upload speeds seem unaffected. Running 1.3.6 daemon and client on same Win7 host. This behavior exist if I try DL'ing to the local SSD (Crucial M4 which benchmarks as expected) or to my GigE RAID-5 NAS (EXT4 and xfs partitions also benchmark as expected.) Between CoteRL, srt75 and my datapoints, we've effectively ruled out IO as a bottleneck. Router is definitely not the issue here...it's an Asus RT-66 running a properly tuned config of Tomato-USB using stock QoS (ie: haven't messed with default QoS settings) and has handled all my torrenting traffic flawlessly under uTorrent. Same thing for the WNDR3700 that the Asus replaced.

Having abandoned the crapified uTorrent client about a month ago and moved to Deluge, this occasional slowdown issue is the only real niggle I have with Deluge so far. I've also tried stopping the affected torrent in Deluge and then loading same into uTorrent which then DL's just fine at the speeds I expect so external factors such as the swarm do not appear to be the cause.

So far, the behavior can't be tied to one type of torrent versus another. Happens to small-ish ones like say 200MB or to larger ones like >4GB. What IS consistent is that when the DL starts to crawl the speed bounces around from a few KiB/s to max out around 30KiB/s.

Very frustrating because I only started noticing this behavior about a week ago and I soooo want to move ahead and migrating the daemon to the debian NAS and crank up a FlexGet setup. [edit: migration clarification and kib/s->KiB/s]

[cy53]

So far, the behavior can't be tied to one type of torrent versus another. Happens to small-ish ones like say 200MB or to larger ones like >4GB. What IS consistent is that when the DL starts to crawl the speed bounces around from a few kib/s to max out around 30kib/s.

Apologies for quoting my own post but I thought I'd elaborate on the torrent behavior:

The affected torrent can be running alongside other torrents which are performing as expected, ie: DL'ing at the usual high speeds. My suspicion of the UL speeds being unaffected are based on simply watching the affected torrent for a few minutes and because an affected torrent left to DL overnight will rack up an insanely high ratio.

Restarting the client didn't help. The affected torrent(s) still run slow while the unaffected torrents performed fine. Same thing goes for restarting the daemon.

Annyone else have any additional insight or data points?

[bro]

Have you monitored the CPU usage when this happens?

[xencored]

Have you monitored the CPU usage when this happens?

I have, it uses close to zero on all the pcs ive used deluge on.

[cy53]

It's happening over by me right now. CPU hovering between 5 and 10% and using 64MB of memory (4GB total, 1.8GB committed counting all processes.)

This is sooo frustrating because I really, really like the simple elegance of Deluge and the plugins do everything else I could ask for.

Anything we can do to help pinpoint the issue? Debug build available? I'd have to install an IDE but maybe I/we could snag a trace. Fair warning: I'm not a dev, I'm only aware this can be useful from following the early days of mythtv debugging...

Also, I could switch the daemon over from my Win7 lappy to a freshly built Mint 14 box and see if anything changes.

[bro]

This is most probably a libtorrent issue, so maybe a debug build of libtorrent could help.