YaRSS2 Subscriptions unable to download torrents

Suggest, post, or discuss plugins for Deluge
jackapple42

YaRSS2 Subscriptions unable to download torrents

Post by jackapple42 »

I have been using YaRSS2 in Deluge 1.3.11 on my Ubuntu 14.04 server for over a year without issue. I use a console on my windows 7 computer to access deluge. I use a VPN that all traffic goes through. The error when a subscription is run is as follows:
(2015-07-27T11:53:57): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/1608471EBF8 ... llers.ettv'
(2015-07-27T11:53:57): Adding torrent: 'http://torcache.net/torrent/1608471EBF8 ... ent?title=[kat.cr]falling.skies.s05e05.hdtv.x264.killers.ettv' using cookies: ({},)
(2015-07-27T11:53:57): Failed to add torrent 'Falling Skies S05E05 HDTV x264-KILLERS[ettv]' from url 'http://torcache.net/torrent/1608471EBF8 ... ent?title=[kat.cr]falling.skies.s05e05.hdtv.x264.killers.ettv'

I can add a torrent manually as URL same as above and it will download without problem. Can anyone help me diagnose why the automatic subscription download is no longer working?
Alexfax
New User
New User
Posts: 3
Joined: Tue Jul 28, 2015 7:58 am

Re: YaRSS2 Subscriptions unable to download torrents

Post by Alexfax »

Hi,
I have the same problem

Code: Select all

(2015-07-28T00:58:56): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=%5Bkat.cr%5Dthe.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv'
(2015-07-28T00:58:56): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=%5Bkat.cr%5Dthe.strain.s02e03.720p.hdtv.x264.killers.rartv'
(2015-07-28T00:58:57): Failed to add torrent 'The Strain S02E03 720p HDTV x264-KILLERS[rartv]' from url 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=[kat.cr]the.strain.s02e03.720p.hdtv.x264.killers.rartv'
(2015-07-28T00:58:57): Adding torrent: 'http://torcache.net/torrent/F46FEAAA563D82D03596546C4E34BE5B8BF3702F.torrent?title=[kat.cr]the.strain.s02e03.720p.hdtv.x264.killers.rartv' using cookies: ({},)
(2015-07-28T00:58:57): Failed to add torrent 'The Strain S02E03 Fort Defiance 720p WEB-DL DD5 1 H 264-Juggalotus[rartv]' from url 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=[kat.cr]the.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv'
(2015-07-28T00:58:57): Adding torrent: 'http://torcache.net/torrent/4775C2037CEA977EB068B4F6FDB21E60084078B5.torrent?title=[kat.cr]the.strain.s02e03.fort.defiance.720p.web.dl.dd5.1.h.264.juggalotus.rartv' using cookies: ({},)

it has been working perfectly till last week
the rss in example is http://kickass.to/usearch/720p%20rar%20 ... Atv/?rss=1

Can I setting the plugin to use the magnet link?
Server: deluge 1.3.12 (raspberry) Client: Deluge 1.3.12 (windows)
raweezy
New User
New User
Posts: 2
Joined: Tue Jul 28, 2015 11:48 pm

Re: YaRSS2 Subscriptions unable to download torrents

Post by raweezy »

I also have this problem. Plugin worked until sometime last week.

Code: Select all

(2015-07-28T18:35:38): Unable to decode torrent file! (invalid bencoding) URL: 'http://torcache.net/torrent/25946DE80DA52017A99DFD496417683BC6C9D674.torrent?title=%5Bkat.cr%5Djeopardy.s2015e148.2015.07.27.mp4.hdtv.720p.vader85'
(2015-07-28T18:35:38): Adding torrent: 'http://torcache.net/torrent/25946DE80DA52017A99DFD496417683BC6C9D674.torrent?title=[kat.cr]jeopardy.s2015e148.2015.07.27.mp4.hdtv.720p.vader85' using cookies: ({},)
Feed in question is http://kat.cr/usearch/jeopardy%20s2015% ... Atv/?rss=1

Judging by OP and Alexfax's logs, it seems to be limited to kat.cr feeds. Perhaps someone smarter than I am can shed some light on what to do.
steelsnake
New User
New User
Posts: 5
Joined: Fri Aug 21, 2015 4:18 pm

Re: YaRSS2 Subscriptions unable to download torrents

Post by steelsnake »

It appears that torcache blocked the user agent used by YaRSS2 :roll: . The fix is simple, but a bit of work:

* Unzip the YaRSS2 egg (if you have an editor or OS which can edit files inside the zip, skip this step)
* Delete the file yarss2/lib/requests/utils.pyc
* Open and edit yarss2/lib/requests/utils.py (near the end around line 540), commenting out the "return" statement and adding a new one:

Code: Select all

    return " Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
    #return " ".join([
    #        'python-requests/%s' % __version__,
    #        '%s/%s' % (_implementation, _implementation_version),
    #        '%s/%s' % (platform.system(), platform.release()),
    #    ])
* Save the edited file
* Zip the YaRSS2 files back up and rename it to .egg, then replace the file on your server. Be careful to make sure that the file structure in the new egg matches the one in the old one (i.e. don't add extra paths)

You can use any "normal" user agent you like. If you need some more user agent strings, search for "user agent strings" with your preferred search engine.

I'm hesitant to provide a fixed egg since I'm not the original dev nor should anyone trust an egg from a random stranger ;)
raweezy
New User
New User
Posts: 2
Joined: Tue Jul 28, 2015 11:48 pm

Re: YaRSS2 Subscriptions unable to download torrents

Post by raweezy »

steelsnake , it worked! Thank you so much!

Some detailed documentation for others who may want to try it: I found the RaRSS2 egg in C:\Users\raweezy\AppData\Roaming\deluge\plugins (I'm running Windows 7).

I made sure to back up the .egg file first, which is always a good idea if you're altering a program's configuration file. I saved YaRSS2-1.3.3-py2.6.egg as a zip, opened it directly, navigated to /lib/requests, deleted utils.pyc, extracted utils.py to a different directory, opened it with Notepad++ (it shows line numbers), commented out those lines with the # symbol, and added the "return " Mozilla/5.0" line of code above it. Last, I copied over utils.py, replacing the old one, and renamed YaRSS2-1.3.3-py2.6.zip to .egg.

Thank you again, steelsnake!
steelsnake
New User
New User
Posts: 5
Joined: Fri Aug 21, 2015 4:18 pm

Re: YaRSS2 Subscriptions unable to download torrents

Post by steelsnake »

Glad to help :)
malaquiasjp

Re: YaRSS2 Subscriptions unable to download torrents

Post by malaquiasjp »

steelsnake , I registered just to thank you, it worked smoothly. Thanks!
bro
Top Bloke
Top Bloke
Posts: 364
Joined: Sun Aug 28, 2011 6:46 pm
Location: Norway

Re: YaRSS2 Subscriptions unable to download torrents

Post by bro »

Hmm...maybe someone used a very low fetch interval which they didn't like. Or maybe they don't like automated RSS feed parsers at all.
I started on implementing support for custom user agent to fix this.
When reporting issues, please include any relevant information such as OS (and version), python version (for Windows users this depends on which Deluge installer was used), Deluge version and plugin version.
steelsnake
New User
New User
Posts: 5
Joined: Fri Aug 21, 2015 4:18 pm

Re: YaRSS2 Subscriptions unable to download torrents

Post by steelsnake »

I'm guessing the same, bro. It also could be the generic user agent - I've seen a few hits from similar agents probing my workplaces servers.

Btw, thank you for the excellent plugin. It's a HUGE time saver.
nkvFzRnRIN13Br5H

Re: YaRSS2 Subscriptions unable to download torrents

Post by nkvFzRnRIN13Br5H »

FYI, it looks like torcache has started checking a referrer header. My quick fix:

Code: Select all

diff --git a/yarss2/lib/requests/defaults.py b/yarss2/lib/requests/defaults.py
index 91d038b..32db595 100644
--- a/yarss2/lib/requests/defaults.py
+++ b/yarss2/lib/requests/defaults.py
@@ -31,9 +31,10 @@ from .utils import default_user_agent
 defaults = dict()
 
 defaults['base_headers'] = {
-    'User-Agent': default_user_agent(),
+    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36',
     'Accept-Encoding': ', '.join(('gzip', 'deflate', 'compress')),
-    'Accept': '*/*'
+    'Accept': '*/*',
+    'Referer': 'http://torcache.net/'
 }
 
 defaults['verbose'] = None
I wasn't able to track down why YaRSS2 was choking on the kat.cr RSS to grab a magnet link instead though - it's using a weird syntax that I think feedparser can't handle. I think the torrent:magnetURI bit confuses it instead of it being just magnetURI like in the test cases in the egg. Here's an example of the format.

Code: Select all

    <item>
        <title>Z Nation S02E01 HDTV x264-KILLERS[ettv]</title>

        <category>TV</category>
      <author>http://kat.cr/user/ettv/</author>        <link>http://kat.cr/z-nation-s02e01-hdtv-x264-killers-ettv-t11243089.html</link>
        <guid>http://kat.cr/z-nation-s02e01-hdtv-x264-killers-ettv-t11243089.html</guid>
        <pubDate>Sat, 12 Sep 2015 03:10:32 +0000</pubDate>
        <torrent:contentLength>477606441</torrent:contentLength>
        <torrent:infoHash>7064EC0DBD8ADF2851F724851B6812FD9958AAF3</torrent:infoHash>
        <torrent:magnetURI><![CDATA[magnet:?xt=urn:btih:7064EC0DBD8ADF2851F724851B6812FD9958AAF3&dn=z+nation+s02e01+hdtv+x264+killers+ettv&tr=udp%3A%2F%2Ftracker.publicbt.com%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337]]></torrent:magnetURI>
        <torrent:seeds>0</torrent:seeds>
        <torrent:peers>0</torrent:peers>
        <torrent:verified>1</torrent:verified>
        <torrent:fileName>z.nation.s02e01.hdtv.x264.killers.ettv.torrent</torrent:fileName>
        <enclosure url="http://torcache.net/torrent/7064EC0DBD8ADF2851F724851B6812FD9958AAF3.torrent?title=[kat.cr]z.nation.s02e01.hdtv.x264.killers.ettv" length="477606441" type="application/x-bittorrent" />
    </item>
Post Reply