IP blocklist support patch

Suggestions and discussion of future versions
markybob
Compulsive Poster
Compulsive Poster
Posts: 1230
Joined: Thu May 24, 2007 11:27 pm
Location: Chicago, IL, USA
Contact:

Re: IP blocklist support patch

Post by markybob »

shirish wrote:I do not know what happened but this is the error which has come up with the latest svn 735. The error is persistent. I even removed the deluge config file (just in case) also but still the same error happens :-

Code: Select all

shirish@ubuntu:~$ deluge
no existing Deluge session
Starting new Deluge session...
deluge_core; using libtorrent 0.13.0.0. Compiled with NDEBUG value: 1
Applying preferences
Starting DHT...
No DHT file to resume
/usr/lib/python2.5/site-packages/deluge/core.py:757: DeprecationWarning: integer argument expected, got float
  PREF_FUNCTIONS[pref](self.get_pref(pref))
Scanning plugin dir /usr/share/deluge/plugins
Loading module ExamplePlugin
Initialising plugin ExamplePlugin
Loading module TorrentCreator
Initialising plugin TorrentCreator
Loading module TorrentSearch
Initialising plugin TorrentSearch
Loading module BlocklistImport
Traceback (most recent call last):
  File "/usr/bin/deluge", line 93, in <module>
    start_deluge()
  File "/usr/bin/deluge", line 67, in start_deluge
    interface = deluge.interface.DelugeGTK()
  File "/usr/lib/python2.5/site-packages/deluge/interface.py", line 65, in __init__
    self.plugins.scan_for_plugins()
  File "/usr/lib/python2.5/site-packages/deluge/plugins.py", line 60, in scan_for_plugins
    mod = __import__(modname, globals(), locals(), [''])
  File "/usr/share/deluge/plugins/BlocklistImport/__init__.py", line 75
    filename, headers = urllib.urlretrieve(self.config.get('url'),
           ^
IndentationError: expected an indented block
i fixed this in 736
User avatar
irfan
Leecher
Leecher
Posts: 70
Joined: Sun Jun 03, 2007 5:36 pm
Location: Poland

Re: IP blocklist support patch

Post by irfan »

markybob wrote:i fixed this in 736

Code: Select all

irfan@ubuntu:~$ svn update deluge
W wersji 736.
irfan@ubuntu:~$ deluge
no existing Deluge session
Starting new Deluge session...
deluge_core; using libtorrent 0.13.0.0. Compiled with NDEBUG value: 1
Applying preferences
Starting DHT...
No DHT file to resume
/usr/lib/python2.5/site-packages/deluge/core.py:757: DeprecationWarning: integer argument expected, got float
  PREF_FUNCTIONS[pref](self.get_pref(pref))
Scanning plugin dir /usr/share/deluge/plugins
Loading module TorrentSearch
Initialising plugin TorrentSearch
Loading module TorrentCreator
Initialising plugin TorrentCreator
Loading module NetworkHealth
Initialising plugin NetworkHealth
Loading module BlocklistImport
Traceback (most recent call last):
  File "/usr/bin/deluge", line 93, in <module>
    start_deluge()
  File "/usr/bin/deluge", line 67, in start_deluge
    interface = deluge.interface.DelugeGTK()
  File "/usr/lib/python2.5/site-packages/deluge/interface.py", line 65, in __init__
    self.plugins.scan_for_plugins()
  File "/usr/lib/python2.5/site-packages/deluge/plugins.py", line 60, in scan_for_plugins
    mod = __import__(modname, globals(), locals(), [''])
  File "/usr/share/deluge/plugins/BlocklistImport/__init__.py", line 90
    reader = readers[ltype][1](self.blockfile)
         ^
IndentationError: expected an indented block
irfan@ubuntu:~$ 
it still doesn't work for me
shirish
Seeder
Seeder
Posts: 163
Joined: Fri May 25, 2007 4:01 am
Location: South Asia, India

Re: IP blocklist support patch

Post by shirish »

The same error is persistent even after building clean rev. 736 :(

Code: Select all

shirish@ubuntu:~$ rm -rf .config/deluge
shirish@ubuntu:~$ deluge
no existing Deluge session
Starting new Deluge session...
deluge_core; using libtorrent 0.13.0.0. Compiled with NDEBUG value: 1
Applying preferences
Starting DHT...
No DHT file to resume
/usr/lib/python2.5/site-packages/deluge/core.py:757: DeprecationWarning: integer argument expected, got float
  PREF_FUNCTIONS[pref](self.get_pref(pref))
Scanning plugin dir /usr/share/deluge/plugins
Loading module ExamplePlugin
Initialising plugin ExamplePlugin
Loading module TorrentCreator
Initialising plugin TorrentCreator
Loading module TorrentSearch
Initialising plugin TorrentSearch
Loading module BlocklistImport
Traceback (most recent call last):
  File "/usr/bin/deluge", line 93, in <module>
    start_deluge()
  File "/usr/bin/deluge", line 67, in start_deluge
    interface = deluge.interface.DelugeGTK()
  File "/usr/lib/python2.5/site-packages/deluge/interface.py", line 65, in __init__
    self.plugins.scan_for_plugins()
  File "/usr/lib/python2.5/site-packages/deluge/plugins.py", line 60, in scan_for_plugins
    mod = __import__(modname, globals(), locals(), [''])
  File "/usr/share/deluge/plugins/BlocklistImport/__init__.py", line 90
    reader = readers[ltype][1](self.blockfile)
         ^
IndentationError: expected an indented block
Debian Sid, Intel Dual-Core, 512 kbps DL , 64 Kbps UL, deluge-1.3.5, pieces plugin, GNOME 3.4.x

All posts under creative commons http://creativecommons.org/licenses/by-nc/3.0/
markybob
Compulsive Poster
Compulsive Poster
Posts: 1230
Joined: Thu May 24, 2007 11:27 pm
Location: Chicago, IL, USA
Contact:

Re: IP blocklist support patch

Post by markybob »

shirish wrote:The same error is persistent even after building clean rev. 736 :(

Code: Select all

shirish@ubuntu:~$ rm -rf .config/deluge
shirish@ubuntu:~$ deluge
no existing Deluge session
Starting new Deluge session...
deluge_core; using libtorrent 0.13.0.0. Compiled with NDEBUG value: 1
Applying preferences
Starting DHT...
No DHT file to resume
/usr/lib/python2.5/site-packages/deluge/core.py:757: DeprecationWarning: integer argument expected, got float
  PREF_FUNCTIONS[pref](self.get_pref(pref))
Scanning plugin dir /usr/share/deluge/plugins
Loading module ExamplePlugin
Initialising plugin ExamplePlugin
Loading module TorrentCreator
Initialising plugin TorrentCreator
Loading module TorrentSearch
Initialising plugin TorrentSearch
Loading module BlocklistImport
Traceback (most recent call last):
  File "/usr/bin/deluge", line 93, in <module>
    start_deluge()
  File "/usr/bin/deluge", line 67, in start_deluge
    interface = deluge.interface.DelugeGTK()
  File "/usr/lib/python2.5/site-packages/deluge/interface.py", line 65, in __init__
    self.plugins.scan_for_plugins()
  File "/usr/lib/python2.5/site-packages/deluge/plugins.py", line 60, in scan_for_plugins
    mod = __import__(modname, globals(), locals(), [''])
  File "/usr/share/deluge/plugins/BlocklistImport/__init__.py", line 90
    reader = readers[ltype][1](self.blockfile)
         ^
IndentationError: expected an indented block
try 737
User avatar
irfan
Leecher
Leecher
Posts: 70
Joined: Sun Jun 03, 2007 5:36 pm
Location: Poland

Re: IP blocklist support patch

Post by irfan »

it works now :)
shirish
Seeder
Seeder
Posts: 163
Joined: Fri May 25, 2007 4:01 am
Location: South Asia, India

Re: IP blocklist support patch

Post by shirish »

its the revision 737 which did the trick, right :D
Last edited by shirish on Mon Jun 25, 2007 4:58 am, edited 1 time in total.
Debian Sid, Intel Dual-Core, 512 kbps DL , 64 Kbps UL, deluge-1.3.5, pieces plugin, GNOME 3.4.x

All posts under creative commons http://creativecommons.org/licenses/by-nc/3.0/
shirish
Seeder
Seeder
Posts: 163
Joined: Fri May 25, 2007 4:01 am
Location: South Asia, India

Re: IP blocklist support patch

Post by shirish »

tarka wrote:
shirish wrote: Looking forward to it while hopefully you can do also something about the the importing thing happening each time. It is going to be a nuisance rather than an asset for users if its going to take 20-30 mins. each time (showing importing) if it did the thing the first time around. But unless we have the patch which gives better error handling & completion statements on the CLI, nothing can be said.
I'm afraid that's probably always going to be the case. libtorrent does not store the lists in-between runs, therefore the list must be imported each time. These blocklists have in the region of 180,000 IP ranges in them; without support in libtorrent for storing them in some sort of external DB it's just going to take a long time.
Whoopie, well with the pipfilter, it had imported something between 214,000 and 215,000 (went for a snack) , the import was complete. On the CLI now I get this :-

Code: Select all

Showing window
Loading blocklist plugin ...
importing with gzmule
GZMuleReader loading /home/shirish/.config/deluge/blocklist.cache
MuleReader loading
TextBase loading
Starting import
Import complete
It would be nicer, lot better if it said something like :-

Code: Select all

Showing window
Loading blocklist plugin ...
importing with gzmule
GZMuleReader loading /home/shirish/.config/deluge/blocklist.cache
MuleReader loading
TextBase loading
Starting import
Import complete 215,0000 IPs added to blocklist 
or something like that. The idea behind that how much import was done is known.

Couple of other things/observations , Just to know I saw the blocklist.cache

Code: Select all

shirish@ubuntu:~/.config/deluge$ ls -la blocklist.cache
-rw-r--r-- 1 shirish shirish 2319250 2007-06-25 01:26 blocklist.cache
Apparently it just shows the .gz file & not the extracted file. The file by default is the same.

Code: Select all

ls -la pipfilter.dat.gz
-rw-r--r-- 1 shirish shirish 2319250 2007-06-24 11:23 pipfilter.dat.gz
The extracted version

Code: Select all

 ls -la pipfilter.dat
total 10500
drwxr-xr--  2 shirish shirish     4096 2007-06-24 11:26 .
drwxr-xr-x 70 shirish shirish    12288 2007-06-25 07:59 ..
-rw-r--r--  1 shirish shirish 10716928 2007-06-24 11:25 pipfilter.dat
Lastly, after the import is done, the blocklist config does not remember the url & is empty, should it not remember?


tarka,
I would recommend you talk with markybob & zachtib & see if we can get some kind of external db working with libtorrent (upsream). They also use trac so its good http://code.rasterbar.com/libtorrent/timeline as well as have a mailing list on which you could ask questions:-

https://lists.sourceforge.net/lists/lis ... nt-discuss

markybob does use the list whenever there is some issue with libtorrent or something. So if possible, see what those guys think about it. who knows it might be in the roadmap, somewhere down the road or there might be something which we do not know about.
Debian Sid, Intel Dual-Core, 512 kbps DL , 64 Kbps UL, deluge-1.3.5, pieces plugin, GNOME 3.4.x

All posts under creative commons http://creativecommons.org/licenses/by-nc/3.0/
tarka
Member
Member
Posts: 22
Joined: Sat Jun 09, 2007 6:49 am

Re: IP blocklist support patch

Post by tarka »

shirish wrote: It would be nicer, lot better if it said something like :-
At some point I'll add something to the status bar at the bottom.
Couple of other things/observations , Just to know I saw the blocklist.cache
...
Apparently it just shows the .gz file & not the extracted file. The file by default is the same.
Yes, it just downloads the file to save downloading it every time. blocklist.cache is an internal implementation detail, you shouldn't need to look at it. In fact you shouldn't rely on it being any particular format, or even being there at all.
Lastly, after the import is done, the blocklist config does not remember the url & is empty, should it not remember?
Good point, I'll add that at some point.
shirish
Seeder
Seeder
Posts: 163
Joined: Fri May 25, 2007 4:01 am
Location: South Asia, India

Re: IP blocklist support patch

Post by shirish »

hi tarka,
I just made ticket #376 http://dev.deluge-torrent.org/ticket/376 which tells what the issue is, dunno if its a bug or a feature, lemme know if it should behave like that?
Debian Sid, Intel Dual-Core, 512 kbps DL , 64 Kbps UL, deluge-1.3.5, pieces plugin, GNOME 3.4.x

All posts under creative commons http://creativecommons.org/licenses/by-nc/3.0/
tarka
Member
Member
Posts: 22
Joined: Sat Jun 09, 2007 6:49 am

Re: IP blocklist support patch

Post by tarka »

Update: Markybob noticed that the pulse action was taking up a fair amount of time and tried disabling it, which seems to speed things up considerably. There is still a visual update in the form of a "number imported" count so this seems like a pretty good resolution.
Post Reply