Deluge Forum

Support Forum for the Deluge BitTorrent Client
https://forum.deluge-torrent.org/

YaRSS2 Subscriptions unable to download torrents

https://forum.deluge-torrent.org/viewtopic.php?t=51899

Page 1 of 2

YaRSS2 Subscriptions unable to download torrents

Posted: Mon Jul 27, 2015 5:20 pm
by jackapple42
I have been using YaRSS2 in Deluge 1.3.11 on my Ubuntu 14.04 server for over a year without issue. I use a console on my windows 7 computer to access deluge. I use a VPN that all traffic goes through. The error when a subscription is run is as follows:
(2015-07-27T11:53:57): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/1608471EBF8 ... llers.ettv'
(2015-07-27T11:53:57): Adding torrent: 'http://torcache.net/torrent/1608471EBF8 ... ent?title=[kat.cr]falling.skies.s05e05.hdtv.x264.killers.ettv' using cookies: ({},)
(2015-07-27T11:53:57): Failed to add torrent 'Falling Skies S05E05 HDTV x264-KILLERS[ettv]' from url 'http://torcache.net/torrent/1608471EBF8 ... ent?title=[kat.cr]falling.skies.s05e05.hdtv.x264.killers.ettv'

I can add a torrent manually as URL same as above and it will download without problem. Can anyone help me diagnose why the automatic subscription download is no longer working?

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Tue Jul 28, 2015 8:04 am
by Alexfax
Hi,
I have the same problem

Code: Select all

(2015-07-28T00:58:56): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=%5Bkat.cr%5Dthe.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv'
(2015-07-28T00:58:56): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=%5Bkat.cr%5Dthe.strain.s02e03.720p.hdtv.x264.killers.rartv'
(2015-07-28T00:58:57): Failed to add torrent 'The Strain S02E03 720p HDTV x264-KILLERS[rartv]' from url 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=[kat.cr]the.strain.s02e03.720p.hdtv.x264.killers.rartv'
(2015-07-28T00:58:57): Adding torrent: 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=[kat.cr]the.strain.s02e03.720p.hdtv.x264.killers.rartv' using cookies: ({},)
(2015-07-28T00:58:57): Failed to add torrent 'The Strain S02E03 Fort Defiance 720p WEB-DL DD5 1 H 264-Juggalotus[rartv]' from url 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=[kat.cr]the.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv'
(2015-07-28T00:58:57): Adding torrent: 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=[kat.cr]the.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv' using cookies: ({},)

it has been working perfectly till last week
the rss in example is http://kickass.to/usearch/720p%20rar%20 ... Atv/?rss=1

Can I setting the plugin to use the magnet link?

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Tue Jul 28, 2015 11:59 pm
by raweezy
I also have this problem. Plugin worked until sometime last week.

Code: Select all

(2015-07-28T18:35:38): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/25946DE80DA52017A99DFD496417683BC6C9D674.torrent?title=%5Bkat.cr%5Djeopardy.s2015e148.2015.07.27.mp4.hdtv.720p.vader85'
(2015-07-28T18:35:38): Adding torrent: 'http://torcache.net/torrent/25946DE80DA52017A99DFD496417683BC6C9D674.torrent?title=[kat.cr]jeopardy.s2015e148.2015.07.27.mp4.hdtv.720p.vader85' using cookies: ({},)
Feed in question is http://kat.cr/usearch/jeopardy%20s2015% ... Atv/?rss=1

Judging by OP and Alexfax's logs, it seems to be limited to kat.cr feeds. Perhaps someone smarter than I am can shed some light on what to do.

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Fri Aug 21, 2015 4:34 pm
by steelsnake
It appears that torcache blocked the user agent used by YaRSS2 :roll: . The fix is simple, but a bit of work:

* Unzip the YaRSS2 egg (if you have an editor or OS which can edit files inside the zip, skip this step)
* Delete the file yarss2/lib/requests/utils.pyc
* Open and edit yarss2/lib/requests/utils.py (near the end around line 540), commenting out the "return" statement and adding a new one:

Code: Select all

    return " Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
    #return " ".join([
    #        'python-requests/%s' % __version__,
    #        '%s/%s' % (_implementation, _implementation_version),
    #        '%s/%s' % (platform.system(), platform.release()),
    #    ])
* Save the edited file
* Zip the YaRSS2 files back up and rename it to .egg, then replace the file on your server. Be careful to make sure that the file structure in the new egg matches the one in the old one (i.e. don't add extra paths)

You can use any "normal" user agent you like. If you need some more user agent strings, search for "user agent strings" with your preferred search engine.

I'm hesitant to provide a fixed egg since I'm not the original dev nor should anyone trust an egg from a random stranger ;)

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Sun Aug 23, 2015 12:13 am
by raweezy
steelsnake , it worked! Thank you so much!

Some detailed documentation for others who may want to try it: I found the RaRSS2 egg in C:\Users\raweezy\AppData\Roaming\deluge\plugins (I'm running Windows 7).

I made sure to back up the .egg file first, which is always a good idea if you're altering a program's configuration file. I saved YaRSS2-1.3.3-py2.6.egg as a zip, opened it directly, navigated to /lib/requests, deleted utils.pyc, extracted utils.py to a different directory, opened it with Notepad++ (it shows line numbers), commented out those lines with the # symbol, and added the "return " Mozilla/5.0" line of code above it. Last, I copied over utils.py, replacing the old one, and renamed YaRSS2-1.3.3-py2.6.zip to .egg.

Thank you again, steelsnake!

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Sun Aug 23, 2015 10:04 am
by steelsnake
Glad to help :)

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Mon Aug 31, 2015 10:02 pm
by malaquiasjp
steelsnake , I registered just to thank you, it worked smoothly. Thanks!

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Thu Sep 10, 2015 1:06 pm
by bro
Hmm...maybe someone used a very low fetch interval which they didn't like. Or maybe they don't like automated RSS feed parsers at all.
I started on implementing support for custom user agent to fix this.

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Thu Sep 10, 2015 3:15 pm
by steelsnake
I'm guessing the same, bro. It also could be the generic user agent - I've seen a few hits from similar agents probing my workplaces servers.

Btw, thank you for the excellent plugin. It's a HUGE time saver.

Re: YaRSS2 Subscriptions unable to download torrents

Posted: Sat Sep 12, 2015 4:15 am
by nkvFzRnRIN13Br5H
FYI, it looks like torcache has started checking a referrer header. My quick fix:

Code: Select all

diff --git a/yarss2/lib/requests/defaults.py b/yarss2/lib/requests/defaults.py
index 91d038b..32db595 100644
--- a/yarss2/lib/requests/defaults.py
+++ b/yarss2/lib/requests/defaults.py
@@ -31,9 +31,10 @@ from .utils import default_user_agent
 defaults = dict()
 
 defaults['base_headers'] = {
-    'User-Agent': default_user_agent(),
+    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36',
     'Accept-Encoding': ', '.join(('gzip', 'deflate', 'compress')),
-    'Accept': '*/*'
+    'Accept': '*/*',
+    'Referer': 'http://torcache.net/'
 }
 
 defaults['verbose'] = None
I wasn't able to track down why YaRSS2 was choking on the kat.cr RSS to grab a magnet link instead though - it's using a weird syntax that I think feedparser can't handle. I think the torrent:magnetURI bit confuses it instead of it being just magnetURI like in the test cases in the egg. Here's an example of the format.

Code: Select all

    <item>
        <title>Z Nation S02E01 HDTV x264-KILLERS[ettv]</title>

        <category>TV</category>
      <author>http://kat.cr/user/ettv/</author>        <link>http://kat.cr/z-nation-s02e01-hdtv-x264-killers-ettv-t11243089.html</link>
        <guid>http://kat.cr/z-nation-s02e01-hdtv-x264-killers-ettv-t11243089.html</guid>
        <pubDate>Sat, 12 Sep 2015 03:10:32 +0000</pubDate>
        <torrent:contentLength>477606441</torrent:contentLength>
        <torrent:infoHash>7064EC0DBD8ADF2851F724851B6812FD9958AAF3</torrent:infoHash>
        <torrent:magnetURI><![CDATA[magnet:?xt=urn:btih:7064EC0DBD8ADF2851F724851B6812FD9958AAF3&dn=z+nation+s02e01+hdtv+x264+killers+ettv&tr=udp%3A%2F%2Ftracker.publicbt.com%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337]]></torrent:magnetURI>
        <torrent:seeds>0</torrent:seeds>
        <torrent:peers>0</torrent:peers>
        <torrent:verified>1</torrent:verified>
        <torrent:fileName>z.nation.s02e01.hdtv.x264.killers.ettv.torrent</torrent:fileName>
        <enclosure url="http://torcache.net/torrent/7064EC0DBD8ADF2851F724851B6812FD9958AAF3.torrent?title=[kat.cr]z.nation.s02e01.hdtv.x264.killers.ettv" length="477606441" type="application/x-bittorrent" />
    </item>
All times are UTC
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited