FlexRSS plugin

[daddy]

I just wrote another RSS plugin. SatNav's is good, but I wanted something a bit more powerful, and don't mind if it is harder to use. I hope this isn't seen as stepping on SatNav's toes--I think these two plugins compliment one another nicely, and I'm guessing most people will prefer his plugin. That said, I should probably start explaining mine.

Most stuff is pretty self-explanatory, but the filters are where things get interesting. FlexRSS allows for multiple patterns per filter, and each pattern is regular expression.

There are (currently) three types for filters: TV Show, TV Show (dated), and Generic. Generic filters will simply check to see if a torrent has already been downloaded by looking at the URL. The other two types check the filter's history based on named patterns in the regular expressions. Recognized names for TV shows are 's', and 'e' ('season'/'series' and 'episode', respectively). Names for dated TV shows are Y, y, m, and d, which correspond to a 4-digit and 2-digit representations of the year for Y and y (respectively), month for m, and day for d. Additionally, the patterns are pre-processed so you can use %s, %e, %Y, %y, %m, and %d for aliases for the appropriate regular expressions.

If there is any interest I'll write up some proper documentation. For now, any feedback (bug reports, feature requests, whatever) would be appreciated. Thanks to everyone who has already provided feedback!

I'll keep this thread updated as newer versions are released.

Updates

• 23 Jul 2007 (0.0.2): Lots of smaller tweaks, but changes worthy of mention include...
  • Significantly improved automatic filter creator (right click on an item in the feeds tab, 'Create filter')
  • Output directory, with a small patch to deluge.
  • Feeds an filters lists are now sorted.
  • Works on non-SVN
  • Named patterns for series/season and episode have been changed to s and e.
• 23 Jul 2007 (0.0.3)
  • RSS icon
  • Ability to disable filters
• 1 Aug 2007 (0.0.4)
  • Cookies.
  • Toolbar button (optional). I also changed the icon a bit to differentiate from SatNav's plugin because of this.
  • Automatic filter creator can now handle 2-digit years for dated tv shows.
• 4 Aug 2007 (0.0.5)
  • Update feeds in a separate thread.
• 5 Aug 2007 (0.0.6)
  • Make feed retrieval in another thread optional, disabled by default. It is still buggy--takes a *long* time, but seems to work if you're patient enough. Hopefully someone more familiar with threading in Python can take a look.
  • Remove button from toolbar when the plugin is unloaded (thanks to markybob's bug report)
• 6 Aug 2007 (0.1.0)
  • "History restriction": You can now specify a range of episodes to download.
• 13 Aug 2007 (0.1.1)
  • Added an option to download the torrent to the popup menu for feed items (where "create filter" is)
  • Fixed display of from series/episode for TV shows (always worked, but the filter wasn't displayed properly)
  • Selecting an item in the feed will now make that item the test pattern on the filters tab.
• 18 Aug 2007 (0.1.2)
  • Fix output dirertory capability--thanks to gazpachoking for the bug report
• 18 Aug 2007 (0.1.3)
  • Filter test area now takes history restriction into account (thanks gazpachoking)
  • Changing the output directory will now automatically select the "Choose:" radio
• 19 Aug 2007 (0.1.4)
  • Fix bug where filter id is to 'feeds' unless feed is selected (another reported by gazpachoking)
  • Replace calls to strptime(asctime()) with localtime(), hopefully (edit: yep, it worked) fixing chaky's bug.
• 20 Aug 2007 (0.1.5)
  • Added options (in filter) to send the torrent to the top of the queue and/or set its state to paused. Requires SVN. Thanks to gazpachoking for the idea.
  • Each section in the filters tab now uses expanders to (optionally) hide content. There are quite a few options, and I was worried about people with lower screen resolutions.
• 21 Aug 2007 (0.1.6)
  • Fix Hellmark's bug in the CalendarButton widget.
  • Changes in feed type are now reflected in the history restriction section.
• 22 Aug 2007 (0.1.7)
  • Prevent an error message when adding a new filter when not using the automatic generator. Thanks gazpachoking.
• 24 Aug 2007 (0.1.8)
  • Catch the exception when we are unable to download a torrent. Thanks to gazpachoking.
• 9 Sep 2007 (0.1.9)
  • Fix a bug causing deluge to crash when there are problems adding the torrent.
  • Changing the filter test pattern now refreshes the result.
  • New option to delete filter after it is matched.
• 16 Sep 2007 (0.1.10)
  • Fix generic filters--thanks to psyberOne for the bug report.
• 17 Sep 2007 (0.1.11)
  • Fix the fix for generic filters, and add another. Thanks again to psyberOne
• 22 Sep 2007 (0.1.12)
  • Add capability to search the feeds tree view.
• 7 Oct 2007 (0.2.0)
  • Torrents are now downloaded using the feed URL as the referer.
  • Added "Link Rewriting", which allows you to use a regexp to rewrite a torrent's URL before downloading.
• 19 Oct 2007 (0.2.1)
  • Apply patch from pyjamasam to properly handle feeds that use enclosures instead of links. Thanks pyjamasam!
• 21 Oct 2007 (0.2.2)
  • Fixed a bug with generic filters.
• 27 Oct 2007 (0.2.3)
  • If an item in a feed doesn't contain a link, don't stop processing the feed--just skip the entry.

[markybob]

I just wrote another RSS plugin. SatNav's is good, but I wanted something a bit more powerful, and don't mind if it is harder to use. I hope this isn't seen as stepping on SatNav's toes--I think these two plugins compliment one another nicely, and I'm guessing most people will prefer his plugin. That said, I should probably start explaining mine.

Most stuff is pretty self-explanatory, but the filters are where things get interesting. FlexRSS allows for multiple patterns per filter, and each pattern is regular expression.

There are (currently) three types for filters: TV Show, TV Show (dated), and Generic. Generic filters will simply check to see if a torrent has already been downloaded by looking at the URL. The other two types check the filter's history based on named patterns in the regular expressions. Recognized names for TV shows are 'season', 'series', and 'episode' (where season and series are functionally equivalent). Names for dated TV shows are Y, y, m, and d, which correspond to a 4-digit and 2-digit representations of the year for Y and y (respectively), month for m, and day for d. Additionally, the patterns are pre-processed so you can use %s, %e, %Y, %y, %m, and %d for aliases for the appropriate regular expressions.

If there is any interest I'll write up some proper documentation. For now, any feedback (bug reports, feature requests, whatever) would be appreciated.

I'll keep this thread updated when/if newer versions are released.

no one says we cant have more than one plugin. users can choose to use whichever best fits their needs and desires. thanks for the work, i've included this in svn. feel free to email me updates markybob@gmail.com and i'll keep svn up-to-date

[bidossessi]

i'm assuming the install directory is ~/.config/deluge/plugins/FlexRSS
plugin appears in the list but on install, it hangs.
running 0.5.3RC1 from tar ball on an opensuse 10.2
anybody experienced the same thing, or am i missing something?

[daddy]

biodossessi,

Try the version I just uploaded (0.0.2)... it should work better under 0.5.3. If you still have problems, I can give you a version loaded with debugging stuff and we can try to figure out what is going wrong.\

My guess is it is taking a long time for your feeds to download. Making the feed retrieval threaded is on my todo list.

[markybob]

biodossessi,

Try the version I just uploaded (0.0.2)... it should work better under 0.5.3. If you still have problems, I can give you a version loaded with debugging stuff and we can try to figure out what is going wrong.\

My guess is it is taking a long time for your feeds to download. Making the feed retrieval threaded is on my todo list.

just committed it to svn

[markybob]

just committed it to svn

i spoke too soon. i just removed FlexRSS from SVN because not even 0.0.2 works with it. please fix when you get around to it

[daddy]

I just uploaded another version, which works perfectly for me with SVN on Gutsy and 0.5.2 on Feisty. If you have problems with this version, let me know.

Also, what about the patch for a method to add a torrent with an output directory?

[daddy]

New version. Adds support for cookies (which was a pain in the ass) for both torrent and feed downloads. It uses a Netscape-style cookie file (~/.config/deluge/flexrss-cookies.txt), so if you're not comfortable with the editor (which is pretty simple), you can just do `grep ~/.mozilla/firefox/$profile/cookies.txt 'site-requiring-cookies.com' >> ~/.config/deluge/flexrss-cookies.txt`

The only thing I really want that this release lacks is threading support for feed retrieval, but I can't seem to get it to work right... don't suppose someone more familiar with python wants to take a look?

[markybob]

New version. Adds support for cookies (which was a pain in the ass) for both torrent and feed downloads. It uses a Netscape-style cookie file (~/.config/deluge/flexrss-cookies.txt), so if you're not comfortable with the editor (which is pretty simple), you can just do `grep ~/.mozilla/firefox/$profile/cookies.txt 'site-requiring-cookies.com' >> ~/.config/deluge/flexrss-cookies.txt`

The only thing I really want that this release lacks is threading support for feed retrieval, but I can't seem to get it to work right... don't suppose someone more familiar with python wants to take a look?

i'm having some problems using this with svn. for example, if i load the plugin and then click on preferences, you cant actually do anything like add or delete feeds until you close the plugin manager window.

[daddy]

You're right. But, now that you mention it, you can't do anything in the main deluge window, either, so I have a guess as to what is happening...

I'm noticing that everything is a GtkDialog, which, when you call gtk_dialog_run() (or gtk.Dialog.run() in python) "Blocks in a recursive main loop until the dialog either emits the "response" signal, or is destroyed." (http://developer.gnome.org/doc/API/2.0/ ... dialog-run). Everything else seems to work because they are also GtkDialogs. This doesn't seem like proper behavior to me, and, after playing around with a few other gnome apps, it doesn't seem to be the standard behavior.

Anyways, IMHO most of these GtkDialogs (browsing through the glade directory now) should really be GtkWindows. At the very least, a *lot* of gtk_dialog_run() calls should be removed. If you disagree, I can re-work my plugin to use a dialog instead... I'm just not sure this is the best solution.