[Plugin] ltConfig v2.0.0

Suggest, post, or discuss plugins for Deluge
eldervamp
New User
New User
Posts: 4
Joined: Sat Aug 29, 2020 9:25 am

Re: [Plugin] ltConfig v2.0.0

Post by eldervamp »

ItConfig v2.0.0 mentions that this version supports Deluge 2.x and Python 3 but what about the Libtorrent version. It installs on Deluge 2 with libtorrent 1.2x but ItConfig are showing settings that shouldn't be there with libtorrent 1.2x because they been deprecated. https://github.com/arvidn/libtorrent/issues/6161 Anyone else know of more ItConfig settings to ignore while using libtorrent 1.2x ?
mhertz
Compulsive Poster
Compulsive Poster
Posts: 1145
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

The deluge2 update of ltconfig originally was made with libtorrent 1.1.x, and don't know if subsequent updates where made for libtorrent 1.2.x support, but many use it with that, including myself, without issue.

I don't know the list of currently deprecated options, probably can be googled, and should be listed somewhere in code-base presumably, but keep in mind the wording is little misleading, i.e deprecated options is marked for feature removal, so people using and depending on those, have ample time to find alternative usage scenarios instead of breaking there configs imidiately upon updating. If checking libtorrent changelog, then occasionally states this and that deprecated option removed, also there's a build-option for libtorrent to remove all deprecated options from the code base, to save space if not depending on any of those, which also is warned upon in the documentation of said build-option, which is what would be needed for e.g that hashing-thread-option in your example to not be shown, so not really wrong to show that setting in ltconfig, though would be nice if e.g it could show these options with some sort of symbol next to it or something, to make you know probably wise to rethink your approach regarding that. Maybe the deluge-log will list warnings when using such, as forwards libtorrent messages, and I don't know if all deprecated options trigger such warning message from libtorrent, but some do atleast, so I would guess all, but not sure as said.

In short, not an indication of mis-support of libtorrent 1.2.x.

Edit: Most(maybe all) settings comes comes from settings_pack.cpp(and some additional dht settings I believe), and so I just checked there, and you can see all deprecated settings there, as called with 'DEPRECATED_SET' instead of just 'SET'. I've not looked much at ltconfig sources before, but presumably not very hard to filter through from that, I'm suspecting, since I'm pretty sure it parses settings_pack.cpp to obtain settings and there default values, though personally I wouldn't myself omit them, as this system of deprecating settings, specifically are made for not abruptly removing such, and hence breaking people's configs left and right, but as said, to symbol there deprecated state indeed makes sence imho.

Libtorrent's RC_1_2 branch's latest settings_pack.cpp here(and same as in v1.2.13): https://github.com/arvidn/libtorrent/bl ... s_pack.cpp

Edit2: Very quickly checked out ltconfig code, and sorry it doesn't parse settings.pack.cpp as I guessed, but directly queries libtorrent session settings through deluge's core.session object, connecting to lt.session and get_settings/set_settings of libtorrent, roundabout, and I don't know if there's options available for restricting deprecated settings from showing in get_settings, as didn't look up, and as said, personally neither wouldn't myself. Just posting this to correct myself.
eldervamp
New User
New User
Posts: 4
Joined: Sat Aug 29, 2020 9:25 am

Re: [Plugin] ltConfig v2.0.0

Post by eldervamp »

mhertz wrote:The deluge2 update of ltconfig originally was made with libtorrent 1.1.x, and don't know if subsequent updates where made for libtorrent 1.2.x support, but many use it with that, including myself, without issue.

I don't know the list of currently deprecated options, probably can be googled, and should be listed somewhere in code-base presumably, but keep in mind the wording is little misleading, i.e deprecated options is marked for feature removal, so people using and depending on those, have ample time to find alternative usage scenarios instead of breaking there configs imidiately upon updating. If checking libtorrent changelog, then occasionally states this and that deprecated option removed, also there's a build-option for libtorrent to remove all deprecated options from the code base, to save space if not depending on any of those, which also is warned upon in the documentation of said build-option, which is what would be needed for e.g that hashing-thread-option in your example to not be shown, so not really wrong to show that setting in ltconfig, though would be nice if e.g it could show these options with some sort of symbol next to it or something, to make you know probably wise to rethink your approach regarding that. Maybe the deluge-log will list warnings when using such, as forwards libtorrent messages, and I don't know if all deprecated options trigger such warning message from libtorrent, but some do atleast, so I would guess all, but not sure as said.

In short, not an indication of mis-support of libtorrent 1.2.x.

Edit: Most(maybe all) settings comes comes from settings_pack.cpp(and some additional dht settings I believe), and so I just checked there, and you can see all deprecated settings there, as called with 'DEPRECATED_SET' instead of just 'SET'. I've not looked much at ltconfig sources before, but presumably not very hard to filter through from that, I'm suspecting, since I'm pretty sure it parses settings_pack.cpp to obtain settings and there default values, though personally I wouldn't myself omit them, as this system of deprecating settings, specifically are made for not abruptly removing such, and hence breaking people's configs left and right, but as said, to symbol there deprecated state indeed makes sence imho.

Libtorrent's RC_1_2 branch's latest settings_pack.cpp here(and same as in v1.2.13): https://github.com/arvidn/libtorrent/bl ... s_pack.cpp

Edit2: Very quickly checked out ltconfig code, and sorry it doesn't parse settings.pack.cpp as I guessed, but directly queries libtorrent session settings through deluge's core.session object, connecting to lt.session and get_settings/set_settings of libtorrent, roundabout, and I don't know if there's options available for restricting deprecated settings from showing in get_settings, as didn't look up, and as said, personally neither wouldn't myself. Just posting this to correct myself.


Thank You for looking into this and providing info :). Guess I wasn't typing in the right words while doing a google search.
mhertz
Compulsive Poster
Compulsive Poster
Posts: 1145
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

Just wanted to add that ltconfig has it's libtorrent preset settings hardcoded, and not updated since 4 years, so if using e.g. high performance seed preset from plugin, then it's not fully up-to-date.

I wen't through the hardcoded settings of that preset in plugin, one by one, compared to latest libtorrent RC_1_2 preset settings in source-code('src/session.cpp'), so matches latest libtorrent 1.2.13(actually matches exactly libtorrent 1.2.5 to 1.2.13 to be exact) and here's the resulting manually updated ltconfig.conf representing a now fully up-to-date high performance seed preset of libtorrent 1.2.13(as said, 1.2.5 to 1.2.13 - in 1.2.4 one option removed and two others added, compared to now, etc):

Code: Select all

{
    "file": 1,
    "format": 1
}{
    "apply_on_start": true,
    "settings": {
        "active_dht_limit": 600,
        "active_limit": 20000,
        "active_seeds": 2000,
        "active_tracker_limit": 2000,
        "aio_threads": 8,
        "alert_queue_size": 10000,
        "allowed_fast_set_size": 0,
        "cache_expiry": 30,
        "cache_size": 65536,
        "checking_mem_usage": 2048,
        "choking_algorithm": 0,
        "close_redundant_connections": true,
        "coalesce_reads": false,
        "coalesce_writes": false,
        "connection_speed": 500,
        "connections_limit": 8000,
        "file_pool_size": 500,
        "inactivity_timeout": 20,
        "listen_queue_size": 3000,
        "max_allowed_in_request_queue": 2000,
        "max_failcount": 1,
        "max_peer_recv_buffer_size": 5242880,
        "max_out_request_queue": 1500,
        "max_queued_disk_bytes": 7340032,
        "max_rejects": 10,
        "mixed_mode_algorithm": 0,
        "no_atime_storage": true,
        "peer_timeout": 20,
        "read_cache_line_size": 32,
        "request_timeout": 10,
        "send_buffer_low_watermark": 1048576,
        "send_buffer_watermark": 3145728,
        "send_buffer_watermark_factor": 150,
        "send_not_sent_low_watermark": 524288,
        "suggest_mode": 1,
        "unchoke_slots_limit": -1,
        "use_read_cache": true,
        "write_cache_line_size": 256
    }
}
Some added commentary to go along:

Settings removed:

Code: Select all

"cache_buffer_chunk_size": 128, (default 0 - deprecated)
"dht_upload_rate_limit": 20000, (default 8000)
"low_prio_disk": false, (default true - deprecated)
"recv_socket_buffer_size": 1048576, (default 0, meaning kernel default, replaced with send_not_sent_low_watermark in 1.2.5)
"send_socket_buffer_size": 1048576, (default 0, meaning kernel default, replaced with send_not_sent_low_watermark in 1.2.5)
"use_disk_cache_pool": true, (default false - deprecated)
"allow_multiple_connections_per_ip": true, (commented out - docs state unrecommended)
"max_http_recv_buffer_size": 6291456,  (default 835584)
Settings changed:

Code: Select all

"unchoke_slots_limit": 2000, > -1
"active_limit": 2000, > 20000
"checking_mem_usage": 320, > 2048
New settings:

Code: Select all

"max_peer_recv_buffer_size": 5242880,
"alert_queue_size":, 10000,
"no_atime_storage": true,
"use_read_cache": true,
"read_cache_line_size": 32,
"coalesce_reads": false,
"coalesce_writes": false,
"close_redundant_connections": true,
"send_not_sent_low_watermark": 524288,
"choking_algorithm": 0.
I was about to also update the hardcoded settings in the source-code and build/upload ltconfig here, and probably submit PR to ratanakvlun additionally, but decided against, since e.g some use libtorrent 1.1.x etc, and the documentation of plugin even states based upon that, so probably better leave alone anyway, and just let libtorrent 1.2.x users update there configs themselves from above.

References and more:

https://github.com/arvidn/libtorrent/bl ... ession.cpp
https://github.com/ratanakvlun/deluge-l ... presets.py
https://github.com/arvidn/libtorrent/bl ... s_pack.cpp
https://github.com/deluge-torrent/delug ... manager.py

Here's the preset settings also directly queried from libtorrent 1.2.11(my used distro's libtorrent package is little outdated currently, package maintainer seemingly busy, but still nothings changed meanwhile regarding that preset):

Code: Select all

martin@arch ~ % python -c "import libtorrent; print(libtorrent.high_performance_seed())" 
{'user_agent': '', 'announce_ip': '', 'mmap_cache': '', 'handshake_client_version': '', 'outgoing_interfaces': '', 'listen_interfaces': '', 'proxy_hostname': '', 'proxy_username': '', 'proxy_password': '', 'i2p_hostname': '', 'peer_fingerprint': '', 'dht_bootstrap_nodes': '', 'tracker_completion_timeout': 0, 'tracker_receive_timeout': 0, 'stop_tracker_timeout': 0, 'tracker_maximum_response_length': 0, 'piece_timeout': 0, 'request_timeout': 10, 'request_queue_time': 0, 'max_allowed_in_request_queue': 2000, 'max_out_request_queue': 1500, 'whole_pieces_threshold': 0, 'peer_timeout': 20, 'urlseed_timeout': 0, 'urlseed_pipeline_size': 0, 'urlseed_wait_retry': 0, 'file_pool_size': 500, 'max_failcount': 1, 'min_reconnect_time': 0, 'peer_connect_timeout': 0, 'connection_speed': 500, 'inactivity_timeout': 20, 'unchoke_interval': 0, 'optimistic_unchoke_interval': 0, 'num_want': 0, 'initial_picker_threshold': 0, 'allowed_fast_set_size': 0, 'suggest_mode': 1, 'max_queued_disk_bytes': 7340032, 'handshake_timeout': 0, 'send_buffer_low_watermark': 1048576, 'send_buffer_watermark': 3145728, 'send_buffer_watermark_factor': 150, 'choking_algorithm': 0, 'seed_choking_algorithm': 0, 'cache_size': 65536, 'cache_buffer_chunk_size': 0, 'cache_expiry': 30, 'disk_io_write_mode': 0, 'disk_io_read_mode': 0, 'outgoing_port': 0, 'num_outgoing_ports': 0, 'peer_tos': 0, 'active_downloads': 0, 'active_seeds': 2000, 'active_checking': 0, 'active_dht_limit': 600, 'active_tracker_limit': 2000, 'active_lsd_limit': 0, 'active_limit': 20000, 'active_loaded_limit': 0, 'auto_manage_interval': 0, 'seed_time_limit': 0, 'auto_scrape_interval': 0, 'auto_scrape_min_interval': 0, 'max_peerlist_size': 0, 'max_paused_peerlist_size': 0, 'min_announce_interval': 0, 'auto_manage_startup': 0, 'seeding_piece_quota': 0, 'max_rejects': 10, 'recv_socket_buffer_size': 0, 'send_socket_buffer_size': 0, 'max_peer_recv_buffer_size': 5242880, 'file_checks_delay_per_block': 0, 'read_cache_line_size': 32, 'write_cache_line_size': 256, 'optimistic_disk_retry': 0, 'max_suggest_pieces': 0, 'local_service_announce_interval': 0, 'dht_announce_interval': 0, 'udp_tracker_token_expiry': 0, 'default_cache_min_age': 0, 'num_optimistic_unchoke_slots': 0, 'default_est_reciprocation_rate': 0, 'increase_est_reciprocation_rate': 0, 'decrease_est_reciprocation_rate': 0, 'max_pex_peers': 0, 'tick_interval': 0, 'share_mode_target': 0, 'upload_rate_limit': 0, 'download_rate_limit': 0, 'local_upload_rate_limit': 0, 'local_download_rate_limit': 0, 'dht_upload_rate_limit': 0, 'unchoke_slots_limit': -1, 'half_open_limit': 0, 'connections_limit': 8000, 'connections_slack': 0, 'utp_target_delay': 0, 'utp_gain_factor': 0, 'utp_min_timeout': 0, 'utp_syn_resends': 0, 'utp_fin_resends': 0, 'utp_num_resends': 0, 'utp_connect_timeout': 0, 'utp_delayed_ack': 0, 'utp_loss_multiplier': 0, 'mixed_mode_algorithm': 0, 'listen_queue_size': 3000, 'torrent_connect_boost': 0, 'alert_queue_size': 10000, 'max_metadata_size': 0, 'hashing_threads': 0, 'checking_mem_usage': 2048, 'predictive_piece_announce': 0, 'aio_threads': 8, 'aio_max': 0, 'network_threads': 0, 'ssl_listen': 0, 'tracker_backoff': 0, 'share_ratio_limit': 0, 'seed_time_ratio_limit': 0, 'peer_turnover': 0, 'peer_turnover_cutoff': 0, 'peer_turnover_interval': 0, 'connect_seed_every_n_download': 0, 'max_http_recv_buffer_size': 0, 'max_retry_port_bind': 0, 'alert_mask': 0, 'out_enc_policy': 0, 'in_enc_policy': 0, 'allowed_enc_level': 0, 'inactive_down_rate': 0, 'inactive_up_rate': 0, 'proxy_type': 0, 'proxy_port': 0, 'i2p_port': 0, 'cache_size_volatile': 0, 'urlseed_max_request_bytes': 0, 'web_seed_name_lookup_retry': 0, 'close_file_interval': 0, 'utp_cwnd_reduce_timer': 0, 'max_web_seed_connections': 0, 'resolver_cache_timeout': 0, 'send_not_sent_low_watermark': 524288, 'rate_choker_initial_threshold': 0, 'upnp_lease_duration': 0, 'max_concurrent_http_announces': 0, 'allow_multiple_connections_per_ip': False, 'ignore_limits_on_local_network': False, 'send_redundant_have': False, 'lazy_bitfields': False, 'use_dht_as_fallback': False, 'upnp_ignore_nonrouters': False, 'use_parole_mode': False, 'use_read_cache': True, 'use_write_cache': False, 'dont_flush_write_cache': False, 'coalesce_reads': False, 'coalesce_writes': False, 'auto_manage_prefer_seeds': False, 'dont_count_slow_torrents': False, 'close_redundant_connections': True, 'prioritize_partial_pieces': False, 'rate_limit_ip_overhead': False, 'announce_to_all_trackers': False, 'announce_to_all_tiers': False, 'prefer_udp_trackers': False, 'strict_super_seeding': False, 'lock_disk_cache': False, 'disable_hash_checks': False, 'allow_i2p_mixed': False, 'low_prio_disk': False, 'volatile_read_cache': False, 'guided_read_cache': False, 'no_atime_storage': True, 'incoming_starts_queued_torrents': False, 'report_true_downloaded': False, 'strict_end_game_mode': False, 'broadcast_lsd': False, 'enable_outgoing_utp': False, 'enable_incoming_utp': False, 'enable_outgoing_tcp': False, 'enable_incoming_tcp': False, 'ignore_resume_timestamps': False, 'no_recheck_incomplete_resume': False, 'anonymous_mode': False, 'report_web_seed_downloads': False, 'rate_limit_utp': False, 'announce_double_nat': False, 'seeding_outgoing_connections': False, 'no_connect_privileged_ports': False, 'smooth_connects': False, 'always_send_user_agent': False, 'apply_ip_filter_to_trackers': False, 'use_disk_read_ahead': False, 'lock_files': False, 'contiguous_recv_buffer': False, 'ban_web_seeds': False, 'allow_partial_disk_writes': False, 'force_proxy': False, 'support_share_mode': False, 'support_merkle_torrents': False, 'report_redundant_bytes': False, 'listen_system_port_fallback': False, 'use_disk_cache_pool': False, 'announce_crypto_support': False, 'enable_upnp': False, 'enable_natpmp': False, 'enable_lsd': False, 'enable_dht': False, 'prefer_rc4': False, 'proxy_hostnames': False, 'proxy_peer_connections': False, 'auto_sequential': False, 'proxy_tracker_connections': False, 'enable_ip_notifier': False, 'dht_prefer_verified_node_ids': False, 'piece_extent_affinity': False, 'validate_https_trackers': False}

Last, if wondering(like me) why the send/recieve buffer options where removed from high-performance seed preset in libtorrent 1.2.5, then I investigated and some info here, especially last post by Arvid for direct answer but good info throughout. BTW, the option added instead of them, is a linux/Mac kernel feature, but Arvid states Linux-specific, so don't know if used for Mac or not. https://github.com/arvidn/libtorrent/issues/4328
mhertz
Compulsive Poster
Compulsive Poster
Posts: 1145
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

I apologise, my previous post with updated settings for high performance seed preset, was correct in posted updated list/config-file(ltconfig.conf), but the added commentary underneath, listing changed, new and removed, had flaws in them, so i'm sorry for posting wrong info.

What happened was that I had already a ltconfig.conf saved for the high performance seed preset and then found disreprencies between that and the list I found in libtorrent's code, and so further investigation lead me to find ltconfig using hardcoded values and 4 years outdated, but the commentary I posted underneath list, was from comparing to config-file and not code, which I thought was exact same(in just selecting to load preset and save), but now find that config-file misses a bunch of settings from hardcoded values in code, but then see also what I believe to be ltconfig in code dicthing values already same, which makes sence of what I see here also.

So there are changes, but my commentary was exageratted because included default entries ignored from written to config-file as already set. Many of the entries then also likewise could be eliminated from the updated config-file I posted, as seemingly redundant, however I just think it's somewhat of a bad idea doing this, as if libtorrent gets updated and default values change, then aren't changed back, because not defined anymore, unless reloading preset again of-course, but which I wouldn't personally do normally. Anyway, if I decide to clean up the list and remove redundant values and maybe the commentary, then i'll make an edit to previous post about it.

Anyway, from some PM comunication, I rebuilt plugin with high performance seed preset values updated matching libtorrent 1.2.5 to 1.2.13, so uploaded it here if others maybe wanted it too:

ltConfig-2.0.0.egg
Post Reply