Page 1 of 5
IP blocklist support patch
Posted: Mon Jun 11, 2007 6:14 am
by tarka
I've just dropped some basic IP blocklist filtering here:
http://dev.deluge-torrent.org/ticket/185
This version includes the core patches and the plugin rework. It only works with Peerguardian *.p2b.gz files. There's the start of a download/import progress gui but it doesn't work at the moment. The config dialog looks like crap, input welcome.
Re: IP blocklist support patch
Posted: Mon Jun 11, 2007 6:35 am
by shirish
Would love to see it at work, good showing tarka
Re: IP blocklist support patch
Posted: Tue Jun 12, 2007 8:17 am
by tarka
This should be in SVN now. As mentioned before it only supports *.p2b.gz. At the moment it's unclear if these are maintained. If anyone has a current source of blocklists can they msg me, as my source appears to be out of date.
Re: IP blocklist support patch
Posted: Tue Jun 12, 2007 10:32 am
by shirish
instead of doing p2b.gz it would be cool if we could do just .gz there is
http://www.bluetack.co.uk/forums/index.php & I esp. like either this
http://www.bluetack.co.uk/config/pipfilter.dat.gz (I'm slightly paranoid
) or this one
http://www.bluetack.co.uk/config/nipfilter.dat.gz (not paranoid) . I dunno of any other site which manages to stay other, there is phoenixlabs also
http://phoenixlabs.org/ but quite a bit of their stuff seems to be in beta
http://test.blocklist.org/
Another link
http://en.wikipedia.org/wiki/PeerGuardian which tells the state of where PeerGuardian is heading to
Hope we can do something useful. Let us know what u think?
Re: IP blocklist support patch
Posted: Tue Jun 12, 2007 11:50 am
by tarka
Hmm, I'm seeing that problem with "endpoint not connected". Looks like it's due to resetting the IP filter before loading the blocklist. I'll have to look at this tomorrow.
Re: IP blocklist support patch
Posted: Wed Jun 13, 2007 1:43 pm
by tarka
tarka wrote:Hmm, I'm seeing that problem with "endpoint not connected". Looks like it's due to resetting the IP filter before loading the blocklist. I'll have to look at this tomorrow.
OK, this appears to be a problem with libtorrent. At the time the blocklist is loaded there are cached connections that haven't been connected yet, and we're trying to filter them, which causes libtorrent to chuck an 'endpoint not connected' exception. Try the following:
Code: Select all
Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp (revision 599)
+++ libtorrent/src/session_impl.cpp (working copy)
@@ -599,7 +599,13 @@
for (session_impl::connection_map::iterator i
= m_connections.begin(); i != m_connections.end();)
{
- tcp::endpoint sender = i->first->remote_endpoint();
+ tcp::endpoint sender;
+ try {
+ sender = i->first->remote_endpoint();
+ } catch (asio::system_error& e) {
+ continue;
+ }
+
if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
{
#if defined(TORRENT_VERBOSE_LOGGING)
Re: IP blocklist support patch
Posted: Wed Jun 13, 2007 1:50 pm
by tarka
Oops, infinite loop. Try this:
Code: Select all
Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp (revision 599)
+++ libtorrent/src/session_impl.cpp (working copy)
@@ -599,7 +599,15 @@
for (session_impl::connection_map::iterator i
= m_connections.begin(); i != m_connections.end();)
{
- tcp::endpoint sender = i->first->remote_endpoint();
+ tcp::endpoint sender;
+ try {
+ sender = i->first->remote_endpoint();
+ } catch (asio::system_error& e) {
+ // May be a cached connection
+ i++;
+ continue;
+ }
+
if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
{
#if defined(TORRENT_VERBOSE_LOGGING)
Re: IP blocklist support patch
Posted: Wed Jun 13, 2007 4:19 pm
by markybob
tarka wrote:Oops, infinite loop. Try this:
Code: Select all
Index: libtorrent/src/session_impl.cpp
===================================================================
--- libtorrent/src/session_impl.cpp (revision 599)
+++ libtorrent/src/session_impl.cpp (working copy)
@@ -599,7 +599,15 @@
for (session_impl::connection_map::iterator i
= m_connections.begin(); i != m_connections.end();)
{
- tcp::endpoint sender = i->first->remote_endpoint();
+ tcp::endpoint sender;
+ try {
+ sender = i->first->remote_endpoint();
+ } catch (asio::system_error& e) {
+ // May be a cached connection
+ i++;
+ continue;
+ }
+
if (m_ip_filter.access(sender.address()) & ip_filter::blocked)
{
#if defined(TORRENT_VERBOSE_LOGGING)
i've just applied this to svn and it did fix not being able to unload the blocklist plugin. thanks.
Re: IP blocklist support patch
Posted: Wed Jun 13, 2007 5:33 pm
by scrapmetal
what kinda format should the end file be in if loaded from disk?
ive got a script that downlaods the blocklists im interested in, extracts them and cats them into ipfilter.dat for amule and blocklist.cache for azureus and moves the files to their appropriate dirs. this way i only need to download the files once instead of once per application
so what format should the end file be for deluge? and can it load from disk instead of url?
can it be scheduled to reload the file at a certain time?
Re: IP blocklist support patch
Posted: Wed Jun 13, 2007 10:11 pm
by tarka
scrapmetal wrote:what kinda format should the end file be in if loaded from disk?
It currently handles P2B format v1 and v2, gzipped:
http://wiki.phoenixlabs.org/wiki/P2B_Format
I assume ipfilter.dat/blocklist.cache are v3, I plan to look at that next. The problem is that by default the v3 files are 7zipped and there's no built-in support for 7zip in Python. There's an external library available but I haven't tested it yet.
BTW, can you msg me your blocklist sources, I'm gathering data.