[Plugin] ltConfig v2.0.0

Suggest, post, or discuss plugins for Deluge
eldervamp
New User
New User
Posts: 6
Joined: Sat Aug 29, 2020 9:25 am

Re: [Plugin] ltConfig v2.0.0

Post by eldervamp »

ItConfig v2.0.0 mentions that this version supports Deluge 2.x and Python 3 but what about the Libtorrent version. It installs on Deluge 2 with libtorrent 1.2x but ItConfig are showing settings that shouldn't be there with libtorrent 1.2x because they been deprecated. https://github.com/arvidn/libtorrent/issues/6161 Anyone else know of more ItConfig settings to ignore while using libtorrent 1.2x ?
mhertz
Moderator
Moderator
Posts: 2182
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

The deluge2 update of ltconfig originally was made with libtorrent 1.1.x, and don't know if subsequent updates where made for libtorrent 1.2.x support, but many use it with that, including myself, without issue.

I don't know the list of currently deprecated options, probably can be googled, and should be listed somewhere in code-base presumably, but keep in mind the wording is little misleading, i.e deprecated options is marked for feature removal, so people using and depending on those, have ample time to find alternative usage scenarios instead of breaking there configs imidiately upon updating. If checking libtorrent changelog, then occasionally states this and that deprecated option removed, also there's a build-option for libtorrent to remove all deprecated options from the code base, to save space if not depending on any of those, which also is warned upon in the documentation of said build-option, which is what would be needed for e.g that hashing-thread-option in your example to not be shown, so not really wrong to show that setting in ltconfig, though would be nice if e.g it could show these options with some sort of symbol next to it or something, to make you know probably wise to rethink your approach regarding that. Maybe the deluge-log will list warnings when using such, as forwards libtorrent messages, and I don't know if all deprecated options trigger such warning message from libtorrent, but some do atleast, so I would guess all, but not sure as said.

In short, not an indication of mis-support of libtorrent 1.2.x.

Edit: Most(maybe all) settings comes comes from settings_pack.cpp(and some additional dht settings I believe), and so I just checked there, and you can see all deprecated settings there, as called with 'DEPRECATED_SET' instead of just 'SET'. I've not looked much at ltconfig sources before, but presumably not very hard to filter through from that, I'm suspecting, since I'm pretty sure it parses settings_pack.cpp to obtain settings and there default values, though personally I wouldn't myself omit them, as this system of deprecating settings, specifically are made for not abruptly removing such, and hence breaking people's configs left and right, but as said, to symbol there deprecated state indeed makes sence imho.

Libtorrent's RC_1_2 branch's latest settings_pack.cpp here(and same as in v1.2.13): https://github.com/arvidn/libtorrent/bl ... s_pack.cpp

Edit2: Very quickly checked out ltconfig code, and sorry it doesn't parse settings.pack.cpp as I guessed, but directly queries libtorrent session settings through deluge's core.session object, connecting to lt.session and get_settings/set_settings of libtorrent, roundabout, and I don't know if there's options available for restricting deprecated settings from showing in get_settings, as didn't look up, and as said, personally neither wouldn't myself. Just posting this to correct myself.
eldervamp
New User
New User
Posts: 6
Joined: Sat Aug 29, 2020 9:25 am

Re: [Plugin] ltConfig v2.0.0

Post by eldervamp »

mhertz wrote:The deluge2 update of ltconfig originally was made with libtorrent 1.1.x, and don't know if subsequent updates where made for libtorrent 1.2.x support, but many use it with that, including myself, without issue.

I don't know the list of currently deprecated options, probably can be googled, and should be listed somewhere in code-base presumably, but keep in mind the wording is little misleading, i.e deprecated options is marked for feature removal, so people using and depending on those, have ample time to find alternative usage scenarios instead of breaking there configs imidiately upon updating. If checking libtorrent changelog, then occasionally states this and that deprecated option removed, also there's a build-option for libtorrent to remove all deprecated options from the code base, to save space if not depending on any of those, which also is warned upon in the documentation of said build-option, which is what would be needed for e.g that hashing-thread-option in your example to not be shown, so not really wrong to show that setting in ltconfig, though would be nice if e.g it could show these options with some sort of symbol next to it or something, to make you know probably wise to rethink your approach regarding that. Maybe the deluge-log will list warnings when using such, as forwards libtorrent messages, and I don't know if all deprecated options trigger such warning message from libtorrent, but some do atleast, so I would guess all, but not sure as said.

In short, not an indication of mis-support of libtorrent 1.2.x.

Edit: Most(maybe all) settings comes comes from settings_pack.cpp(and some additional dht settings I believe), and so I just checked there, and you can see all deprecated settings there, as called with 'DEPRECATED_SET' instead of just 'SET'. I've not looked much at ltconfig sources before, but presumably not very hard to filter through from that, I'm suspecting, since I'm pretty sure it parses settings_pack.cpp to obtain settings and there default values, though personally I wouldn't myself omit them, as this system of deprecating settings, specifically are made for not abruptly removing such, and hence breaking people's configs left and right, but as said, to symbol there deprecated state indeed makes sence imho.

Libtorrent's RC_1_2 branch's latest settings_pack.cpp here(and same as in v1.2.13): https://github.com/arvidn/libtorrent/bl ... s_pack.cpp

Edit2: Very quickly checked out ltconfig code, and sorry it doesn't parse settings.pack.cpp as I guessed, but directly queries libtorrent session settings through deluge's core.session object, connecting to lt.session and get_settings/set_settings of libtorrent, roundabout, and I don't know if there's options available for restricting deprecated settings from showing in get_settings, as didn't look up, and as said, personally neither wouldn't myself. Just posting this to correct myself.


Thank You for looking into this and providing info :). Guess I wasn't typing in the right words while doing a google search.
mhertz
Moderator
Moderator
Posts: 2182
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

Just wanted to add that ltconfig has it's libtorrent preset settings hardcoded, and not updated since 4 years, so if using e.g. high performance seed preset from plugin, then it's not fully up-to-date.

I wen't through the hardcoded settings of that preset in plugin, one by one, compared to latest libtorrent RC_1_2 preset settings in source-code('src/session.cpp'), so matches latest libtorrent 1.2.13(actually matches exactly libtorrent 1.2.5 to 1.2.13 to be exact) and here's the resulting manually updated ltconfig.conf representing a now fully up-to-date high performance seed preset of libtorrent 1.2.13(as said, 1.2.5 to 1.2.13 - in 1.2.4 one option removed and two others added, compared to now, etc):

Code: Select all

{
    "file": 1,
    "format": 1
}{
    "apply_on_start": true,
    "settings": {
        "active_dht_limit": 600,
        "active_limit": 20000,
        "active_seeds": 2000,
        "active_tracker_limit": 2000,
        "aio_threads": 8,
        "alert_queue_size": 10000,
        "allowed_fast_set_size": 0,
        "cache_expiry": 30,
        "cache_size": 65536,
        "checking_mem_usage": 2048,
        "choking_algorithm": 0,
        "close_redundant_connections": true,
        "coalesce_reads": false,
        "coalesce_writes": false,
        "connection_speed": 500,
        "connections_limit": 8000,
        "file_pool_size": 500,
        "inactivity_timeout": 20,
        "listen_queue_size": 3000,
        "max_allowed_in_request_queue": 2000,
        "max_failcount": 1,
        "max_peer_recv_buffer_size": 5242880,
        "max_out_request_queue": 1500,
        "max_queued_disk_bytes": 7340032,
        "max_rejects": 10,
        "mixed_mode_algorithm": 0,
        "no_atime_storage": true,
        "peer_timeout": 20,
        "read_cache_line_size": 32,
        "request_timeout": 10,
        "send_buffer_low_watermark": 1048576,
        "send_buffer_watermark": 3145728,
        "send_buffer_watermark_factor": 150,
        "send_not_sent_low_watermark": 524288,
        "suggest_mode": 1,
        "unchoke_slots_limit": -1,
        "use_read_cache": true,
        "write_cache_line_size": 256
    }
}
Some added commentary to go along:

Settings removed:

Code: Select all

"cache_buffer_chunk_size": 128, (default 0 - deprecated)
"dht_upload_rate_limit": 20000, (default 8000)
"low_prio_disk": false, (default true - deprecated)
"recv_socket_buffer_size": 1048576, (default 0, meaning kernel default, replaced with send_not_sent_low_watermark in 1.2.5)
"send_socket_buffer_size": 1048576, (default 0, meaning kernel default, replaced with send_not_sent_low_watermark in 1.2.5)
"use_disk_cache_pool": true, (default false - deprecated)
"allow_multiple_connections_per_ip": true, (commented out - docs state unrecommended)
"max_http_recv_buffer_size": 6291456,  (default 835584)
Settings changed:

Code: Select all

"unchoke_slots_limit": 2000, > -1
"active_limit": 2000, > 20000
"checking_mem_usage": 320, > 2048
New settings:

Code: Select all

"max_peer_recv_buffer_size": 5242880,
"alert_queue_size":, 10000,
"no_atime_storage": true,
"use_read_cache": true,
"read_cache_line_size": 32,
"coalesce_reads": false,
"coalesce_writes": false,
"close_redundant_connections": true,
"send_not_sent_low_watermark": 524288,
"choking_algorithm": 0.
I was about to also update the hardcoded settings in the source-code and build/upload ltconfig here, and probably submit PR to ratanakvlun additionally, but decided against, since e.g some use libtorrent 1.1.x etc, and the documentation of plugin even states based upon that, so probably better leave alone anyway, and just let libtorrent 1.2.x users update there configs themselves from above.

References and more:

https://github.com/arvidn/libtorrent/bl ... ession.cpp
https://github.com/ratanakvlun/deluge-l ... presets.py
https://github.com/arvidn/libtorrent/bl ... s_pack.cpp
https://github.com/deluge-torrent/delug ... manager.py

Here's the preset settings also directly queried from libtorrent 1.2.11(my used distro's libtorrent package is little outdated currently, package maintainer seemingly busy, but still nothings changed meanwhile regarding that preset):

Code: Select all

martin@arch ~ % python -c "import libtorrent; print(libtorrent.high_performance_seed())" 
{'user_agent': '', 'announce_ip': '', 'mmap_cache': '', 'handshake_client_version': '', 'outgoing_interfaces': '', 'listen_interfaces': '', 'proxy_hostname': '', 'proxy_username': '', 'proxy_password': '', 'i2p_hostname': '', 'peer_fingerprint': '', 'dht_bootstrap_nodes': '', 'tracker_completion_timeout': 0, 'tracker_receive_timeout': 0, 'stop_tracker_timeout': 0, 'tracker_maximum_response_length': 0, 'piece_timeout': 0, 'request_timeout': 10, 'request_queue_time': 0, 'max_allowed_in_request_queue': 2000, 'max_out_request_queue': 1500, 'whole_pieces_threshold': 0, 'peer_timeout': 20, 'urlseed_timeout': 0, 'urlseed_pipeline_size': 0, 'urlseed_wait_retry': 0, 'file_pool_size': 500, 'max_failcount': 1, 'min_reconnect_time': 0, 'peer_connect_timeout': 0, 'connection_speed': 500, 'inactivity_timeout': 20, 'unchoke_interval': 0, 'optimistic_unchoke_interval': 0, 'num_want': 0, 'initial_picker_threshold': 0, 'allowed_fast_set_size': 0, 'suggest_mode': 1, 'max_queued_disk_bytes': 7340032, 'handshake_timeout': 0, 'send_buffer_low_watermark': 1048576, 'send_buffer_watermark': 3145728, 'send_buffer_watermark_factor': 150, 'choking_algorithm': 0, 'seed_choking_algorithm': 0, 'cache_size': 65536, 'cache_buffer_chunk_size': 0, 'cache_expiry': 30, 'disk_io_write_mode': 0, 'disk_io_read_mode': 0, 'outgoing_port': 0, 'num_outgoing_ports': 0, 'peer_tos': 0, 'active_downloads': 0, 'active_seeds': 2000, 'active_checking': 0, 'active_dht_limit': 600, 'active_tracker_limit': 2000, 'active_lsd_limit': 0, 'active_limit': 20000, 'active_loaded_limit': 0, 'auto_manage_interval': 0, 'seed_time_limit': 0, 'auto_scrape_interval': 0, 'auto_scrape_min_interval': 0, 'max_peerlist_size': 0, 'max_paused_peerlist_size': 0, 'min_announce_interval': 0, 'auto_manage_startup': 0, 'seeding_piece_quota': 0, 'max_rejects': 10, 'recv_socket_buffer_size': 0, 'send_socket_buffer_size': 0, 'max_peer_recv_buffer_size': 5242880, 'file_checks_delay_per_block': 0, 'read_cache_line_size': 32, 'write_cache_line_size': 256, 'optimistic_disk_retry': 0, 'max_suggest_pieces': 0, 'local_service_announce_interval': 0, 'dht_announce_interval': 0, 'udp_tracker_token_expiry': 0, 'default_cache_min_age': 0, 'num_optimistic_unchoke_slots': 0, 'default_est_reciprocation_rate': 0, 'increase_est_reciprocation_rate': 0, 'decrease_est_reciprocation_rate': 0, 'max_pex_peers': 0, 'tick_interval': 0, 'share_mode_target': 0, 'upload_rate_limit': 0, 'download_rate_limit': 0, 'local_upload_rate_limit': 0, 'local_download_rate_limit': 0, 'dht_upload_rate_limit': 0, 'unchoke_slots_limit': -1, 'half_open_limit': 0, 'connections_limit': 8000, 'connections_slack': 0, 'utp_target_delay': 0, 'utp_gain_factor': 0, 'utp_min_timeout': 0, 'utp_syn_resends': 0, 'utp_fin_resends': 0, 'utp_num_resends': 0, 'utp_connect_timeout': 0, 'utp_delayed_ack': 0, 'utp_loss_multiplier': 0, 'mixed_mode_algorithm': 0, 'listen_queue_size': 3000, 'torrent_connect_boost': 0, 'alert_queue_size': 10000, 'max_metadata_size': 0, 'hashing_threads': 0, 'checking_mem_usage': 2048, 'predictive_piece_announce': 0, 'aio_threads': 8, 'aio_max': 0, 'network_threads': 0, 'ssl_listen': 0, 'tracker_backoff': 0, 'share_ratio_limit': 0, 'seed_time_ratio_limit': 0, 'peer_turnover': 0, 'peer_turnover_cutoff': 0, 'peer_turnover_interval': 0, 'connect_seed_every_n_download': 0, 'max_http_recv_buffer_size': 0, 'max_retry_port_bind': 0, 'alert_mask': 0, 'out_enc_policy': 0, 'in_enc_policy': 0, 'allowed_enc_level': 0, 'inactive_down_rate': 0, 'inactive_up_rate': 0, 'proxy_type': 0, 'proxy_port': 0, 'i2p_port': 0, 'cache_size_volatile': 0, 'urlseed_max_request_bytes': 0, 'web_seed_name_lookup_retry': 0, 'close_file_interval': 0, 'utp_cwnd_reduce_timer': 0, 'max_web_seed_connections': 0, 'resolver_cache_timeout': 0, 'send_not_sent_low_watermark': 524288, 'rate_choker_initial_threshold': 0, 'upnp_lease_duration': 0, 'max_concurrent_http_announces': 0, 'allow_multiple_connections_per_ip': False, 'ignore_limits_on_local_network': False, 'send_redundant_have': False, 'lazy_bitfields': False, 'use_dht_as_fallback': False, 'upnp_ignore_nonrouters': False, 'use_parole_mode': False, 'use_read_cache': True, 'use_write_cache': False, 'dont_flush_write_cache': False, 'coalesce_reads': False, 'coalesce_writes': False, 'auto_manage_prefer_seeds': False, 'dont_count_slow_torrents': False, 'close_redundant_connections': True, 'prioritize_partial_pieces': False, 'rate_limit_ip_overhead': False, 'announce_to_all_trackers': False, 'announce_to_all_tiers': False, 'prefer_udp_trackers': False, 'strict_super_seeding': False, 'lock_disk_cache': False, 'disable_hash_checks': False, 'allow_i2p_mixed': False, 'low_prio_disk': False, 'volatile_read_cache': False, 'guided_read_cache': False, 'no_atime_storage': True, 'incoming_starts_queued_torrents': False, 'report_true_downloaded': False, 'strict_end_game_mode': False, 'broadcast_lsd': False, 'enable_outgoing_utp': False, 'enable_incoming_utp': False, 'enable_outgoing_tcp': False, 'enable_incoming_tcp': False, 'ignore_resume_timestamps': False, 'no_recheck_incomplete_resume': False, 'anonymous_mode': False, 'report_web_seed_downloads': False, 'rate_limit_utp': False, 'announce_double_nat': False, 'seeding_outgoing_connections': False, 'no_connect_privileged_ports': False, 'smooth_connects': False, 'always_send_user_agent': False, 'apply_ip_filter_to_trackers': False, 'use_disk_read_ahead': False, 'lock_files': False, 'contiguous_recv_buffer': False, 'ban_web_seeds': False, 'allow_partial_disk_writes': False, 'force_proxy': False, 'support_share_mode': False, 'support_merkle_torrents': False, 'report_redundant_bytes': False, 'listen_system_port_fallback': False, 'use_disk_cache_pool': False, 'announce_crypto_support': False, 'enable_upnp': False, 'enable_natpmp': False, 'enable_lsd': False, 'enable_dht': False, 'prefer_rc4': False, 'proxy_hostnames': False, 'proxy_peer_connections': False, 'auto_sequential': False, 'proxy_tracker_connections': False, 'enable_ip_notifier': False, 'dht_prefer_verified_node_ids': False, 'piece_extent_affinity': False, 'validate_https_trackers': False}

Last, if wondering(like me) why the send/recieve buffer options where removed from high-performance seed preset in libtorrent 1.2.5, then I investigated and some info here, especially last post by Arvid for direct answer but good info throughout. BTW, the option added instead of them, is a linux/Mac kernel feature, but Arvid states Linux-specific, so don't know if used for Mac or not. https://github.com/arvidn/libtorrent/issues/4328
mhertz
Moderator
Moderator
Posts: 2182
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

I apologise, my previous post with updated settings for high performance seed preset, was correct in posted updated list/config-file(ltconfig.conf), but the added commentary underneath, listing changed, new and removed, had flaws in them, so i'm sorry for posting wrong info.

What happened was that I had already a ltconfig.conf saved for the high performance seed preset and then found disreprencies between that and the list I found in libtorrent's code, and so further investigation lead me to find ltconfig using hardcoded values and 4 years outdated, but the commentary I posted underneath list, was from comparing to config-file and not code, which I thought was exact same(in just selecting to load preset and save), but now find that config-file misses a bunch of settings from hardcoded values in code, but then see also what I believe to be ltconfig in code dicthing values already same, which makes sence of what I see here also.

So there are changes, but my commentary was exageratted because included default entries ignored from written to config-file as already set. Many of the entries then also likewise could be eliminated from the updated config-file I posted, as seemingly redundant, however I just think it's somewhat of a bad idea doing this, as if libtorrent gets updated and default values change, then aren't changed back, because not defined anymore, unless reloading preset again of-course, but which I wouldn't personally do normally. Anyway, if I decide to clean up the list and remove redundant values and maybe the commentary, then i'll make an edit to previous post about it.

Anyway, from some PM comunication, I rebuilt plugin with high performance seed preset values updated matching libtorrent 1.2.5 to 1.2.13, so uploaded it here if others maybe wanted it too:

ltConfig-2.0.0.egg
bungada
New User
New User
Posts: 2
Joined: Wed Sep 15, 2021 9:41 pm

Re: [Plugin] ltConfig v2.0.0

Post by bungada »

Hoping you guys can help me before I pull my remaining hair out.

Trying to add ltConfig. I have Deluge running on an Ubuntu headless server, Ubuntu v20.04. Deluge v2.0.3, libtorrent v1.1.13.

I access Deluge through the Deluge WebUI. I have added the ltConfig 2.0.0 egg to the plugins older. It shows in the plugins list, is checked, but I do not have preference pane for ltConfig. ltConfig has created ltconfig.conf which reads as below
{
"file": 1,
"format": 1
}{
"apply_on_start": false,
"settings": {}
}
I have set deluge logging to debug, but the only relevant line I see is below
Config /home/deluge/.config/deluge/ltconfig.conf version: 1.1 loaded: {'apply_on_start': False, 'settings': {}}
Thank you!

Edit: One more thing I have noticed is when I go to the plugin list, under ltConfig, all of the items such as author/version/etc show not available.
mhertz
Moderator
Moderator
Posts: 2182
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

I cannot reproduce this, but have seen such happen with plugins before, don't know exactly why, but when refreshing the page in browser and wen't into preferences again, then the plugin pane came up.

If not working still, then besides configuring direct in the config-file, which works on the core nonetheless(so in all UIs), then I would reset your profile and see if works again(with deluged/deluge-web closed, rename your deluge profile dir to deluge-bak and make new empty deluge dir, copy plugins folder from deluge-bak to deluge and start deluged/deluge-web and enable ltconfig and see if works now), and then if need be copy over states and core.conf etc afterwards and see if still work, so you don't need set up from scratch again.

If eveything still fail I would test other browser, though doubt would help, but just in case - I tested with qutebrowser here, which is using a chromium backend.

You probably need run debug-logging on the actual deluge-web server/interface and not deluged, if wanting check that.

Good luck.
bungada
New User
New User
Posts: 2
Joined: Wed Sep 15, 2021 9:41 pm

Re: [Plugin] ltConfig v2.0.0

Post by bungada »

Thank you mhertz! Renaming the profile directory worked and deluge recognized the plugin instantly. Copied over the state files, but kept the new core.conf just in case (probably doesn't matter, but easy enough to modify).
mhertz
Moderator
Moderator
Posts: 2182
Joined: Wed Jan 22, 2014 5:05 am
Location: Denmark

Re: [Plugin] ltConfig v2.0.0

Post by mhertz »

bungada wrote: Sat Sep 18, 2021 7:48 pm Thank you mhertz! Renaming the profile directory worked and deluge recognized the plugin instantly. Copied over the state files, but kept the new core.conf just in case (probably doesn't matter, but easy enough to modify).
You're of-course very much welcome mate, and thank you likewise for reporting back :) I'm glad to hear you succeed in fixing this. Strange that this happens sometimes - maybe it was the session.state file in the profile which sometimes can lead to issues, and forgot to suggest that also, deleting it I mean, to get it regenerated fresh, though using fresh profile like you did, is same process anyway, though would be easier for next time, if that is the culprit, of-course.

Cheers.
tugurlann
Member
Member
Posts: 27
Joined: Sun Dec 14, 2014 1:01 am

Re: [Plugin] ltConfig v2.0.0

Post by tugurlann »

Hello!

I can't get this plugin to install in anyway. I'm running deluged and deluge-web on a server and when I try to get it to install from the Web UI, nothing happens after I click ok.

What am I missing? :)

Thank you!

Code: Select all

deluged --version
deluged 2.0.3
libtorrent: 1.2.14.0
Python: 3.8.10
OS: Linux Ubuntu 20.04 focal
Post Reply