Un-official FlexRSS Plugin for 1.1.x

[tyskis]

In short, yes, it does. Nothing really happens in the daemon atm. The only thing it does is grab config files (which it doesn't use), and get interfaces to do stuff (which it doesn't do). It makes no requests to the gtk-ui, either. So, it is quite tame. In order to hit a filter / refresh a feed, a/the gtk-ui has to be up and running. If the gtk-ui was already down and the daemon freezes, I can only reason that it'd have to be something else. (It could still be related to flexrss, it'd just be for some other reason unbeknown to me.)

Ok, good to know. It seems the daemon was functioning normally, and the gtk-ui/flexrss too, except it locked up for a longer time because it had to wait for a timeout on the offline feed.

I also hadn't tried testing a feed where the requested feed is valid, but not available / times out, so it is possible. I have noticed a slight pause when it takes a bit of time to resolve and download a feed. I have tested a bad url, though; the response is near instant. What kind of response did your web browser give when accessing the offline feed?

The browser waits for ages and then times out. Maybe a (admitedly not very clean) solution for offline feeds would be to shorten the timeout on the connection, so it at least doesn't lock up the interface for so long?

How about the experimental threaded feed retrieval, how experimental is it? And what happens when something goes wrong? Does it crash something or does it just not update?

[SGSeries2]

How about the experimental threaded feed retrieval, how experimental is it? And what happens when something goes wrong? Does it crash something or does it just not update?

It is considered experimental due to its multi-threaded nature and the lack of additional experience in python coding.

For example, there are currently no mutexes/locks/access controls to restrict writing to the same data; accessing the same data is inherently dangerous in a multi-threaded environment. By dangerous, I mean that there is a possibility of the loss of feed data (and only feed data). This data of course would be overwritten with new feed data on the next refresh anyways, so it is not necessarily a crucial impediment to using flexrss unless it becomes a continuous problem.

Generally speaking, such an incident in this scenario is highly unlikely. Heck, I'd even go as far as saying extremely highly unlikely. (You can improve your chances by setting the refresh interval to, say, 1, which would be rather absurd.)

Using threads in flexrss for feed retrieval could potentially be faster than using a single thread, but not exactly a necessary feature in this scenario if that is the reason it is enabled. However, what is particularly nice about using threads here is that if the thread failed to update a feed because of, say, someone not able to keep their rss feed online, or something bogus happens where the thread hits an unhandled exception and dies, it doesn't impede on other feeds getting updated. Aside from a single feed lacking an update, you would've never even known it was there.

It seemed harmless under the circumstances it was being used, seemed to do what I suspected it to do, and during the brief period of time I was testing the feature, it didn't seem to do any harm, so I just left it.

[SGSeries2]

I had some strange behavior when I had to reboot.
Deluge didn't want to start.. got stuck in:

Code: Select all

[DEBUG   ] 23:46:52 gtkui:66 Calling flexRSS UI init

Well, I deleted ~/.config/deluge/flexrss.dat and it got passed it.
So, I added a feed and a filter, shutdown deluge and try to start it again. Crashed in the same spot.
Info: Using Deluge 1.0.0
¿Anyone has come into this behavior?

I haven't seen/heard of that behavior yet. What os are you running on? Are you running deluged on the same machine or on a different machine? Was deluged running at all, and in what state?

Reason I ask: I recall seeing something peculiar when there is more than one instance of deluged running (which I suspect shouldn't really happen anyways). In that scenario, killing all of the deluged processes fixed the problem. Starting deluge spawned a new deluged process, but that could be a different/unrelated problem.

Other than that, I can't think of anything else.

[hordak]

I am using Windows XP SP3 32 bit version of Deluge with flexRSS-0.1.egg added to C:\Program Files\Deluge\lib\site-packages\deluge-1.0.2-py2.5-win32.egg\deluge\plugins directory.

Thank you for converting this great plugin.

The only issue that I am currently having is that I can not download torrents with this plugin that include spaces. For example, I can download "The X Factor S05E11 (11th October 2008)[WS PDTV(Xvid)Tiggzz.torrent" from a certain private tracker using FlexRSS direct download or filter. Is there something that I set up wrong?

Everything else appears to work fine.

[SGSeries2]

The only issue that I am currently having is that I can not download torrents with this plugin that include spaces. For example, I can download "The X Factor S05E11 (11th October 2008)[WS PDTV(Xvid)Tiggzz.torrent" from a certain private tracker using FlexRSS direct download or filter. Is there something that I set up wrong?

Everything else appears to work fine.

I suspect it is related to a known issue. There's likely some additional characters that aren't visible, or vice for the filter. I've narrowed it down to the fact that the strings are saved and used in unicode, but I haven't decided on a good way to fix it.

My suggestion would be to try and see if there's some hidden characters in the filter/match, replace the spaces with '.*' in the filter, and use backspace or delete to remove any hidden characters in the filter, then try again to see if it matches.

Good to hear back from a winxp user. Now I can update the original post with a suggestion as to where to install it on winxp.

[tyskis]

How about the experimental threaded feed retrieval, how experimental is it? And what happens when something goes wrong? Does it crash something or does it just not update?

It is considered experimental due to its multi-threaded nature and the lack of additional experience in python coding.

For example, there are currently no mutexes/locks/access controls to restrict writing to the same data; accessing the same data is inherently dangerous in a multi-threaded environment. By dangerous, I mean that there is a possibility of the loss of feed data (and only feed data). This data of course would be overwritten with new feed data on the next refresh anyways, so it is not necessarily a crucial impediment to using flexrss unless it becomes a continuous problem.

Generally speaking, such an incident in this scenario is highly unlikely. Heck, I'd even go as far as saying extremely highly unlikely. (You can improve your chances by setting the refresh interval to, say, 1, which would be rather absurd.)

How about if a filter gets a match on two different feeds for exactly the same torrent (different trackers, but the file contents to download are exactly the same)? I experienced that with the threaded-setting turned on, and it started downloading both of them, to the same place, and as far as i can tell, it wrote to the same files, which I would guess is not a very good behaviour. The feeds were updated at the same time at the start of the gtk-ui. I assume it starts a new thread for each feed, or just one which then "works through" all the feeds that are to be updated? If it is one thread for all feeds, i can't see how it happened, as it was a TV-series-filter, so it should have blocked that episode from being downloaded again in the other feed update. If they on the other hand were updated at the same time, I assume they download the torrent first (at the same time), and only after that do they update the list of downloaded episodes.

On another note, thanks for porting the plugin.

[SGSeries2]

How about if a filter gets a match on two different feeds for exactly the same torrent (different trackers, but the file contents to download are exactly the same)? ... I assume it starts a new thread for each feed, or just one which then "works through" all the feeds that are to be updated? ...

It does start a thread for each feed if threading is enabled, yes, otherwise one thread works through all of them. The history list is updated after the torrent was successfully added, yes. In that case, a lock would have to be put around the filter that is being tested and its history until a given thread is done using it. I hadn't thought of that. So... yes, it is experimental. Thanks for bringing that to my attention. I'll have to see if I can recreate that.

To see if they were really writing to the same file, you could stop one of them, let the other one download for a bit, and then run a check on the one you stopped to see if the percentage goes up (even though you didn't actually 'download' anything). Anyways, yes, downloading the same file twice at the same time would be a bit redundant and a waste of resources, if nothing else.

What you mentioned is really interesting, actually, because if that were the case, then regardless of whatever flexrss does, (and I'm just throwing this out here,) would that also mean that if you had two users who 'just so happened' to add the exact same file/torrent from different user interfaces accessing the same deluged process, they'd both be accepted? Makes you think.

I'm not saying that what you described is not a fault of flexrss, since it should certainly catch that. I'm just surprised that deluge didn't see it as the same torrent and just attempt to either merge the trackers or just discard it altogether (since it is already being downloaded). After all, that's how it works if you try to add an already-existing torrent after you just added it.

(Man, my posts are always long. Sheesh.)

[tyskis]

What you mentioned is really interesting, actually, because if that were the case, then regardless of whatever flexrss does, (and I'm just throwing this out here,) would that also mean that if you had two users who 'just so happened' to add the exact same file/torrent from different user interfaces accessing the same deluged process, they'd both be accepted? Makes you think.

I'm not saying that what you described is not a fault of flexrss, since it should certainly catch that. I'm just surprised that deluge didn't see it as the same torrent and just attempt to either merge the trackers or just discard it altogether (since it is already being downloaded). After all, that's how it works if you try to add an already-existing torrent after you just added it.

There were a few warnings about a torrent already added in the deluged.log, but it got added anyway. Sorry i didn't save the log, it seems it cuts old parts to make room for new log messages.

[tyskis]

Ok, now it happened again, it added two torrents from different feeds with the same filter (and same episode). I use the threaded setting in flexrss. Here is the log:

Code: Select all

[DEBUG   ] 11:25:07 signalmanager:61 Registering http://192.168.1.3:40675 as a signal reciever..
[DEBUG   ] 11:25:08 torrentmanager:248 torrentmanager.add
[DEBUG   ] 11:25:08 torrentmanager:310 options: {'max_download_speed_per_torrent': -1, 'add_paused': False, 'prioritize_first_last_pieces': False, 'max_upload_speed_per_torrent': -1, 'auto_managed': True, 'max_connections_per_torrent': -1, 'download_location': '/media/superraid/download', 'compact_allocation': False, 'max_upload_slots_per_torrent': -1}
[DEBUG   ] 11:25:08 component:149 Pausing component AlertManager..
[DEBUG   ] 11:25:08 torrentmanager:342 handle id: eaf35e56a35257eab69f4701088d324b22d1d23e
[DEBUG   ] 11:25:08 torrent:51 Creating torrent object eaf35e56a35257eab69f4701088d324b22d1d23e
[DEBUG   ] 11:25:08 configmanager:88 Getting config 'core.conf'
[DEBUG   ] 11:25:08 torrent:254 set_state_based_on_ltstate: 0
[DEBUG   ] 11:25:08 torrent:138 Torrent object created.
[DEBUG   ] 11:25:08 component:159 Resuming component AlertManager..
[DEBUG   ] 11:25:08 torrentmanager:551 Saving torrent state file.
[DEBUG   ] 11:25:08 pluginmanager:132 run_post_torrent_add
[DEBUG   ] 11:25:08 alertmanager:98 torrent_resumed_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION resumed
[DEBUG   ] 11:25:08 torrentmanager:736 on_alert_torrent_resumed
[DEBUG   ] 11:25:08 torrent:254 set_state_based_on_ltstate: 0
[DEBUG   ] 11:25:08 alertmanager:98 state_changed_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION: state changed to: downloading
[DEBUG   ] 11:25:08 alertmanager:98 torrent_checked_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION checked
[DEBUG   ] 11:25:08 alertmanager:98 tracker_announce_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION (http://tracker.a.org/announce) sending announce (started)
[DEBUG   ] 11:25:08 torrentmanager:742 on_alert_state_changed
[DEBUG   ] 11:25:08 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:08 torrentmanager:654 on_alert_torrent_checked
[DEBUG   ] 11:25:08 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:08 torrentmanager:678 on_alert_tracker_announce
[DEBUG   ] 11:25:08 alertmanager:98 tracker_reply_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION (http://tracker.a.org:2710/announce) received peers: 50
[DEBUG   ] 11:25:08 torrentmanager:661 on_alert_tracker_reply
[DEBUG   ] 11:25:09 torrentmanager:248 torrentmanager.add
[DEBUG   ] 11:25:09 torrentmanager:310 options: {'max_download_speed_per_torrent': -1, 'add_paused': False, 'prioritize_first_last_pieces': False, 'max_upload_speed_per_torrent': -1, 'auto_managed': True, 'max_connections_per_torrent': -1, 'download_location': '/media/superraid/download', 'compact_allocation': False, 'max_upload_slots_per_torrent': -1}
[DEBUG   ] 11:25:09 component:149 Pausing component AlertManager..
[DEBUG   ] 11:25:09 torrentmanager:342 handle id: 5e2fcc0b36c9e9ed23eac88521564bc18df7ec7d
[DEBUG   ] 11:25:09 torrent:51 Creating torrent object 5e2fcc0b36c9e9ed23eac88521564bc18df7ec7d
[DEBUG   ] 11:25:09 configmanager:88 Getting config 'core.conf'
[DEBUG   ] 11:25:09 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:09 torrent:138 Torrent object created.
[DEBUG   ] 11:25:09 component:159 Resuming component AlertManager..
[DEBUG   ] 11:25:09 alertmanager:98 state_changed_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION: state changed to: downloading
[DEBUG   ] 11:25:09 alertmanager:98 torrent_checked_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION checked
[DEBUG   ] 11:25:09 torrentmanager:551 Saving torrent state file.
[DEBUG   ] 11:25:09 pluginmanager:132 run_post_torrent_add
[DEBUG   ] 11:25:09 torrentmanager:742 on_alert_state_changed
[DEBUG   ] 11:25:09 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:09 torrentmanager:654 on_alert_torrent_checked
[DEBUG   ] 11:25:09 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:09 alertmanager:98 torrent_resumed_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION resumed
[DEBUG   ] 11:25:09 alertmanager:98 tracker_announce_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION (http://tracker.b.net/announce) sending announce (started)
[DEBUG   ] 11:25:09 torrentmanager:736 on_alert_torrent_resumed
[DEBUG   ] 11:25:09 torrent:254 set_state_based_on_ltstate: 3
[DEBUG   ] 11:25:09 torrentmanager:678 on_alert_tracker_announce
[DEBUG   ] 11:25:09 torrentmanager:248 torrentmanager.add
[DEBUG   ] 11:25:09 torrentmanager:310 options: {'max_download_speed_per_torrent': -1, 'add_paused': False, 'prioritize_first_last_pieces': False, 'max_upload_speed_per_torrent': -1, 'auto_managed': True, 'max_connections_per_torrent': -1, 'download_location': '/media/superraid/download', 'compact_allocation': False, 'max_upload_slots_per_torrent': -1}
[DEBUG   ] 11:25:09 component:149 Pausing component AlertManager..
[WARNING ] 11:25:09 torrentmanager:334 Error adding torrent: torrent already exists in session
[DEBUG   ] 11:25:09 torrentmanager:337 torrent handle is invalid!
[DEBUG   ] 11:25:09 component:159 Resuming component AlertManager..
[DEBUG   ] 11:25:09 alertmanager:98 tracker_reply_alert: Awesome.Series.S05E03.720p.HDTV.X264-DIMENSION (http://tracker.b.net/announce) received peers: 39
[DEBUG   ] 11:25:09 pluginmanager:132 run_post_torrent_add
[DEBUG   ] 11:25:09 torrentmanager:661 on_alert_tracker_reply

Hope it helps with the debugging!

[SGSeries2]

Ok, now it happened again, it added two torrents from different feeds with the same filter (and same episode). I use the threaded setting in flexrss. Here is the log:

...

Hope it helps with the debugging!

Lol! Deluge even gives a warning, replies saying there was an error adding it, but added it anyway? Late response, maybe?

Oh well. Flexrss shouldn't be doing that anyways. Thanks for the info.