JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 10.01.2025, 06:45
djones djones is offline
Vacuum Cleaner
 
Join Date: Nov 2022
Posts: 15
Default [Feature Req] "Greylist" for Filters in Connection Manager

In Connection Manager -> Edit (Domain Filter) the options are Whitelist and Blacklist, which are restrictive in some scenarios. I'd like to suggest an additional option:

"Greylist - Prefer this proxy for the domains below"

I'd mainly like this for the No Proxy connection which is the default local connection that uses your public IP. Greylist would be a mode that doesn't prevent the connection from being used for downloads from domains that aren't specified on the Greylist. It would simply try that connection first for the specified domains before trying any other connections.

The issue is that links from many paid sites will have your public IP embedded in the download URL and are therefore keyed, and those links need to be handled by the No Proxy connection so the IP matches. The problem is that if I set the connection to "Whitelist" and specify the domains that need to use that connection, then the connection won't be used for any other downloads from random sites too numerous to manually specify, and instead only ever gets used for the specified domains.

The alternate approach would be Blacklisting the sites with IP-keyed download URL's from all the other HTTP Proxy connections I have defined, but with 25+ HTTP Proxies, it's very tedious adding a domain to 25+ individual blacklists each time I come across one that needs to use the No Proxy connection.

Last edited by djones; 10.01.2025 at 07:04.
Reply With Quote
  #2  
Old 10.01.2025, 10:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,847
Default

Quote:
Originally Posted by djones View Post
It would simply try that connection first for the specified domains before trying any other connections.
JDownloader processes the list of connection from top to bottom (you can change order). I would say it makes management and controlling much more complicated if the order is not only depending on position in list but also its mode (greylist).

Why not move "No Proxy" at top/first position, so those IP bound links will use/try that first. Hosts with limits will then move on to the next possible entry.

I think that should result in same behaviour as suggested greylist mode? Or did I miss or not fully understood something?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 10.01.2025, 12:22
djones djones is offline
Vacuum Cleaner
 
Join Date: Nov 2022
Posts: 15
Default

Quote:
Originally Posted by Jiaz View Post
JDownloader processes the list of connection from top to bottom (you can change order). I would say it makes management and controlling much more complicated if the order is not only depending on position in list but also its mode (greylist).
Hi Jiaz thanks for reply. "No Proxy" is and has always been in the top spot of the connections list. What I'm referring to is when Clipboard Observer captures a link, JD first seems to perform a check of the URL against the server in the URL to see if it is valid, retrieve filesize, etc - and for that it seems to use a random connection on the list rather than the first one (No Proxy). Because the IP embedded in the URL doesn't match that of the random HTTP Proxy connection when JD queries the server in the URL, it resolves into "Offline Files" on the LinkGrabber list due to the mismatch. If I force it to the download list anyway, it will not download due to invalid link.

If I add the domain of the site to No Proxy's whitelist, it works. The trouble is now all other domains won't use the No Proxy connection. Whitelist/Blacklist as it is now is a very limited all-or-nothing approach.

Another workaround is to temporarily de-select all the HTTP Proxy connections so that only No Proxy remains, then it also works. Unfortunately spacebar doesn't work for enabling/disabling connections or I could highlight all but No Proxy and disable the rest at once. Currently, connections have to be disabled/enabled individually by clicking one at a time.

Last edited by djones; 10.01.2025 at 13:09.
Reply With Quote
  #4  
Old 10.01.2025, 12:38
djones djones is offline
Vacuum Cleaner
 
Join Date: Nov 2022
Posts: 15
Default

If there's no obvious fix, then another question: could I add the domains I want to force to use the No Proxy connection to its Whitelist, but then add an asterisk or other wildcard at the end of the whitelist so all other domains can also use it?

I'm trying to find any way to not have to individually edit 25+ identical blacklists each time I need to force a domain to use No Proxy, since that's like the tail wagging the dog.

Last edited by djones; 10.01.2025 at 12:57.
Reply With Quote
  #5  
Old 10.01.2025, 13:09
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,991
Default

Quote:
Originally Posted by djones View Post
The issue is that links from many paid sites will have your public IP embedded in the download URL and are therefore keyed, and those links need to be handled by the No Proxy connection so the IP matches. The problem is that if I set the connection to "Whitelist" and specify the domains that need to use that connection, then the connection won't be used for any other downloads from random sites too numerous to manually specify, and instead only ever gets used for the specified domains.
That sounds like something which could possibly be automated using EventScripter scripts.*
For links that contain an IP address, you could even go so far to let it find the IP and use some kind of pre-built mapping to find the matching proxy.

* small disclaimer: I don't know if we already have the needed EventScripter hooks to make it interact with proxy behavior.

Quote:
Originally Posted by djones View Post
I'm trying to find any way to not have to individually edit 25+ identical blacklists each time I need to force a domain to use No Proxy, since that's like the tail wagging the dog.
Even for this, you could use a script with a button-trigger that just does this for you for the domain of a pre selected link.

EventScripter subforum:
https://board.jdownloader.org/forumdisplay.php?f=52
EventScripter help article:
https://support.jdownloader.org/know...event-scripter
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 10.01.2025 at 13:09. Reason: Added hyperlinks to EventScripter related info
Reply With Quote
  #6  
Old 10.01.2025, 13:17
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,847
Default

@djones:
Linkcheck has no support for balancing and the first matching proxy will be used.
If "No proxy" is at top and no white/blacklisting in use, then this should be used for Linkcheck.

Linkcheck does not auto switch to next possible proxy, so there must be a reason why this handling fails/doesn't work as expected and we need to find out why and fix that

Can you provide some more details? Like
No proxy at top, but is it in white or blacklist mode?
What did you set Settings->Advanced Settings-> GeneralSettings.freeproxybalancemode to?
The links you're talking about are generic http links?
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 10.01.2025 at 13:33.
Reply With Quote
  #7  
Old 10.01.2025, 14:23
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,665
Default

Quote:
Originally Posted by djones View Post
... temporarily de-select all the HTTP Proxy connections so that only No Proxy remains, then it also works. Unfortunately spacebar doesn't work for enabling/disabling connections or I could highlight all but No Proxy and disable the rest at once. Currently, connections have to be disabled/enabled individually by clicking one at a time.
Right click the icon (green with checkbox) on 'use' column header in connection manager.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 15:22.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.