IP blocklist support patch

[tarka]

I've just dropped some basic IP blocklist filtering here: http://dev.deluge-torrent.org/ticket/185

This version includes the core patches and the plugin rework. It only works with Peerguardian *.p2b.gz files. There's the start of a download/import progress gui but it doesn't work at the moment. The config dialog looks like crap, input welcome.

[shirish]

Would love to see it at work, good showing tarka

[tarka]

This should be in SVN now. As mentioned before it only supports *.p2b.gz. At the moment it's unclear if these are maintained. If anyone has a current source of blocklists can they msg me, as my source appears to be out of date.

[shirish]

instead of doing p2b.gz it would be cool if we could do just .gz there is http://www.bluetack.co.uk/forums/index.php & I esp. like either this http://www.bluetack.co.uk/config/pipfilter.dat.gz (I'm slightly paranoid ) or this one http://www.bluetack.co.uk/config/nipfilter.dat.gz (not paranoid) . I dunno of any other site which manages to stay other, there is phoenixlabs also http://phoenixlabs.org/ but quite a bit of their stuff seems to be in beta http://test.blocklist.org/

Another link http://en.wikipedia.org/wiki/PeerGuardian which tells the state of where PeerGuardian is heading to

Hope we can do something useful. Let us know what u think?

[tarka]

Hmm, I'm seeing that problem with "endpoint not connected". Looks like it's due to resetting the IP filter before loading the blocklist. I'll have to look at this tomorrow.

[tarka]

Hmm, I'm seeing that problem with "endpoint not connected". Looks like it's due to resetting the IP filter before loading the blocklist. I'll have to look at this tomorrow.

OK, this appears to be a problem with libtorrent. At the time the blocklist is loaded there are cached connections that haven't been connected yet, and we're trying to filter them, which causes libtorrent to chuck an 'endpoint not connected' exception. Try the following:

Code: Select all

Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp     (revision 599)
+++ libtorrent/src/session_impl.cpp     (working copy)
@@ -599,7 +599,13 @@
                for (session_impl::connection_map::iterator i
                        = m_connections.begin(); i != m_connections.end();)
                {
-                       tcp::endpoint sender = i->first->remote_endpoint();
+                       tcp::endpoint sender;
+                       try {
+                               sender = i->first->remote_endpoint();
+                       } catch (asio::system_error& e) {
+                               continue;
+                       }
+ 
                        if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
                        {
 #if defined(TORRENT_VERBOSE_LOGGING)

[tarka]

Oops, infinite loop. Try this:

Code: Select all

Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp     (revision 599)
+++ libtorrent/src/session_impl.cpp     (working copy)
@@ -599,7 +599,15 @@
                for (session_impl::connection_map::iterator i
                        = m_connections.begin(); i != m_connections.end();)
                {
-                       tcp::endpoint sender = i->first->remote_endpoint();
+                       tcp::endpoint sender;
+                       try {
+                               sender = i->first->remote_endpoint();
+                       } catch (asio::system_error& e) {
+                               // May be a cached connection
+                               i++;
+                               continue;
+                       }
+ 
                        if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
                        {
 #if defined(TORRENT_VERBOSE_LOGGING)

[markybob]

Oops, infinite loop. Try this:

Code: Select all

Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp     (revision 599)
+++ libtorrent/src/session_impl.cpp     (working copy)
@@ -599,7 +599,15 @@
                for (session_impl::connection_map::iterator i
                        = m_connections.begin(); i != m_connections.end();)
                {
-                       tcp::endpoint sender = i->first->remote_endpoint();
+                       tcp::endpoint sender;
+                       try {
+                               sender = i->first->remote_endpoint();
+                       } catch (asio::system_error& e) {
+                               // May be a cached connection
+                               i++;
+                               continue;
+                       }
+ 
                        if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
                        {
 #if defined(TORRENT_VERBOSE_LOGGING)

i've just applied this to svn and it did fix not being able to unload the blocklist plugin. thanks.

[scrapmetal]

what kinda format should the end file be in if loaded from disk?
ive got a script that downlaods the blocklists im interested in, extracts them and cats them into ipfilter.dat for amule and blocklist.cache for azureus and moves the files to their appropriate dirs. this way i only need to download the files once instead of once per application
so what format should the end file be for deluge? and can it load from disk instead of url?
can it be scheduled to reload the file at a certain time?

[tarka]

what kinda format should the end file be in if loaded from disk?

It currently handles P2B format v1 and v2, gzipped: http://wiki.phoenixlabs.org/wiki/P2B_Format

I assume ipfilter.dat/blocklist.cache are v3, I plan to look at that next. The problem is that by default the v3 files are 7zipped and there's no built-in support for 7zip in Python. There's an external library available but I haven't tested it yet.

BTW, can you msg me your blocklist sources, I'm gathering data.