JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 29.03.2023, 12:56
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default how to make alert boxes NOT appear? (Topic: linkgrabber filter rules)

how to make alert boxes NOT appear?

I added some FILTER to skip any domain names but allowed domain names...

and it opens those alert boxes for youtube asking to add just that single link or entire playlist, or it asks if captcha of for example some shortener domain name should be done via browser extension....

all those domain names it brings those alert boxes are skipped by FILTER...


[ the problem is that I dont want JD to be back on main screen while I leave my desktop PC alone for several hours while processing other tasks that have to be active on main screen to work the faestest way possible (for example https uploading) ... ]
Attached Images
File Type: png jd-alert-box-prompt.PNG (22.0 KB, 4 views)
Reply With Quote
  #2  
Old 29.03.2023, 13:29
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

Hi,
looks like your filters aren't setup correctly!
Filters do work before any plugins are involved so captchas and also this YT dialog are an indicator if your filters not working.

To filter youtube playlists for example, you need to use "Sourceurls -> contains -> ..." and not "Downloadurl -> contains".

If you can't make it work, please post your exported filter rules here so we can check them.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 29.03.2023, 14:04
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default

Quote:
Originally Posted by pspzockerscene View Post
Hi,
looks like your filters aren't setup correctly!
Filters do work before any plugins are involved so captchas and also this YT dialog are an indicator if your filters not working.

To filter youtube playlists for example, you need to use "Sourceurls -> contains -> ..." and not "Downloadurl -> contains".

If you can't make it work, please post your exported filter rules here so we can check them.

thank you for your quick reply...
first of all I have created a very solid workaroud for my special needs...
(thanks to you, mgpai and jiaz for your great support over years)

please see screenshot for FILTER details

it is only about source links with domain name reddit.com

plugin setup - reddit - crawler text mode - NEVER


this issue with alert box does only happen every 500-1000 crawling source links... I do add links via .txt file using eventscripter also having a sleep interval...

(my source text files have 50k lines and more...)
Attached Images
File Type: png jd-reddit-filter-contains-not.PNG (40.0 KB, 3 views)
Attached Files
File Type: txt eventscripter-rule.txt (405 Bytes, 1 views)
Reply With Quote
  #4  
Old 29.03.2023, 14:25
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

I see multiple mistakes and wrong assumptions here:

First, coming back to your last post:
As you can see in your own screenshot, that captcha prompt is coming from "ally.sh" and not from reddit or from any site that you were trying to block.
That again is not on your "blacklist" so no wonder you're getting captcha prompts of that

Now to your most recent post:
Quote:
Originally Posted by verheiratet1952 View Post
plugin setup - reddit - crawler text mode - NEVER
Have you even read the description of that setting or ever tested it?
This will not(!) disable the reddit crawler.
This will only disable crawling text-content from reddit posts.
If you do not want JD to crawl links from provider XY, there is two ways to do this:
1. Settings -> Advanced Settings -> "GeneralSettings.crawlercrawlerpluginblacklist" and "GeneralSettings.crawlerhostpluginblacklist".
2. The filter settings you already found.

Now about the screenshot of your filters:
You got "reddit.com" in "Downloadurl...contains not" and at the same time "Sourceurl...contains" that alone makes no sense.
Without further explanation, I'd do it like this:
Screenshot:

If you want to filter YT too, you need to add |youtube\.com to that filter field.
I also want to point out that there is a test button at the bottom of that window which can be used to test this rule on links. That saves time compared to testing it in your linkgrabber.

Also, if you want to block "all dialogs" such as those captcha dialogs, there is a "Enable silent mode" checkbox in the main toolbar which, if enabled, will block some(not all!) popups.

Quote:
Originally Posted by verheiratet1952 View Post
(my source text files have 50k lines and more...)
Maybe you should think about a whitelist instead of a blacklist?
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 29.03.2023, 14:49
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default

Quote:
Originally Posted by pspzockerscene View Post
I see multiple mistakes and wrong assumptions here:

First, coming back to your last post:
As you can see in your own screenshot, that captcha prompt is coming from "ally.sh" and not from reddit or from any site that you were trying to block.
That again is not on your "blacklist" so no wonder you're getting captcha prompts of that
it is crawling reddit.com links only via .txt file using eventscripter
so there is no ally.sh link from my source links... and there is no YT links links also...

it filters nearly all links (99.9%) but sometimes there are those alert boxes about for example ally.sh and YT

if my FILTER rule would not work, it would also let pass links from redgifs

as I do understand my FILTER rule it is a whitelist for reddit.com redd.it gfycat.com and imgur.com
Reply With Quote
  #6  
Old 29.03.2023, 15:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

Quote:
Originally Posted by verheiratet1952 View Post
so there is no ally.sh link from my source links... and there is no YT links links also...
Sure there can be! If a reddit post-text contains YT links and JD crawls them, JD will try to crawl those YT URLs unless there is aa filter to prevent that.
Check Settings -> Reddit.com -> "Crawl URLs inside post-text?" -> Disable this

Please provide a screenshot of your reddit.com plugin settings.

Quote:
Originally Posted by verheiratet1952 View Post
if my FILTER rule would not work, it would also let pass links from redgifs
I've provided a working rule.
You can provide further feedback to that.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 29.03.2023, 15:45
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default

Quote:
Originally Posted by pspzockerscene View Post
Sure there can be! If a reddit post-text contains YT links and JD crawls them, JD will try to crawl those YT URLs unless there is aa filter to prevent that.
Check Settings -> Reddit.com -> "Crawl URLs inside post-text?" -> Disable this

Please provide a screenshot of your reddit.com plugin settings.


I've provided a working rule.
You can provide further feedback to that.

my current filter prevents 99.9% of urls like YT if a reddit post contains such links... all links are filtered, only getting results from "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com"

"Crawl URLs inside post-text?" is diabled (better saying no checkbox activated)



just saying it once again, my settings DO work for nearly 99.9% of all links crawled... and giving me exactly the results I want...

it is only about some POP-UP that JD is asking if it should be crawled a single YT link or the entire YT playlist even if link source is not included at "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com" or JD is asking for some captcha with also different links...


regular YT links are filtered correctly but why there is some pop-up asking this?


I did re-store filtered links and added NEW screenshot to make you believe YT and other source links like tiktok insta twitter are filtered perfectly...
Attached Images
File Type: png jd-filtered-content-added.PNG (19.1 KB, 2 views)
Reply With Quote
  #8  
Old 29.03.2023, 16:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

Quote:
Originally Posted by verheiratet1952 View Post
just saying it once again, my settings DO work for nearly 99.9% of all links crawled... and giving me exactly the results I want...
Okay. I'm just saying that obviously your rules do not yet work 100% as you'd like them to work else you wouldn't get that popups...

Quote:
Originally Posted by verheiratet1952 View Post
it is only about some POP-UP that JD is asking if it should be crawled a single YT link or the entire YT playlist even if link source is not included at "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com" or JD is asking for some captcha with also different links...
I've understood that and I told you what to do.
Update your rule accordingly!

Quote:
Originally Posted by verheiratet1952 View Post
regular YT links are filtered correctly but why there is some pop-up asking this?
Because "downloadurl" is only available after processing.
Regular YT links won't show a popup before processing but playlist-links do.
If you want to filter them in the best way, you need to filter them before they get processed!

Filter only by "sourceurl" and it will work.
If you still don't want to do this, modify your YT settings accordingly:
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing
EDIT

Alternatively go to Settings -> Advanced Settings -> GeneralSettings.crawlercrawlerpluginblacklist
Enter this:
Code:
["youtube.com"]
--> Just keep in mind that this will disable playlist- and also single video processing!
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 29.03.2023 at 16:04.
Reply With Quote
  #9  
Old 29.03.2023, 16:19
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default

Quote:
Originally Posted by pspzockerscene View Post
Okay. I'm just saying that obviously your rules do not yet work 100% as you'd like them to work else you wouldn't get that popups...


I've understood that and I told you what to do.
Update your rule accordingly!


Because "downloadurl" is only available after processing.
Regular YT links won't show a popup before processing but playlist-links do.
If you want to filter them in the best way, you need to filter them before they get processed!

Filter only by "sourceurl" and it will work.
If you still don't want to do this, modify your YT settings accordingly:
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing
EDIT

Alternatively go to Settings -> Advanced Settings -> GeneralSettings.crawlercrawlerpluginblacklist
Enter this:
Code:
["youtube.com"]
--> Just keep in mind that this will disable playlist- and also single video processing!
if I do change FILTER rule it skips all those URLs I want !?

please see attached file (I had to use .zip because it cannot upload .filter file-format)


good idea, I changed to 'do nothing'
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing
Attached Files
File Type: zip reddit-contains-sourceurl.zip (589 Bytes, 1 views)
Reply With Quote
  #10  
Old 29.03.2023, 16:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

I'm sorry but this takes too much time for me.
If you need further help with this, please contact us again via support@jdownloader.org.

I'll be back online tomorrow.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #11  
Old 29.03.2023, 18:51
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

@verheiratet1952:
sourceURLs can be any URL within the processing, for example Input -> A - > B -> C -> downloadURL D
downloadURL always is the last single/final downloadURL D

so you can apply sourceURL filtering on Input, A, B , C and D
but
you can only! apply downloadURL filtering on D

If you don't want YT links to be processed at all (no processing, no popups) then you have to use sourceURL filtering for this.
same for ally.sh, as this is a site that shorten/hide links and is part of the processing chain (eg Input, A, B or C) , so you have to use sourceURL for this as well.

but it is important to know that the rules are processed from top to bottom and what sourceURLs filtering means.
for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader.
when you want to filter for example ally.sh, then you have to create a filter for this and move it above your other filter.

Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)

In case it is still unclear or there are questions about this, please just ask

Last edited by notice; 29.03.2023 at 18:59.
Reply With Quote
  #12  
Old 30.03.2023, 16:25
verheiratet1952 verheiratet1952 is offline
JD VIP
 
Join Date: Jan 2016
Posts: 340
Default

Quote:
Originally Posted by notice View Post
@verheiratet1952:
sourceURLs can be any URL within the processing, for example Input -> A - > B -> C -> downloadURL D
downloadURL always is the last single/final downloadURL D

so you can apply sourceURL filtering on Input, A, B , C and D
but
you can only! apply downloadURL filtering on D

If you don't want YT links to be processed at all (no processing, no popups) then you have to use sourceURL filtering for this.
same for ally.sh, as this is a site that shorten/hide links and is part of the processing chain (eg Input, A, B or C) , so you have to use sourceURL for this as well.

but it is important to know that the rules are processed from top to bottom and what sourceURLs filtering means.
for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader.
when you want to filter for example ally.sh, then you have to create a filter for this and move it above your other filter.

Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)

In case it is still unclear or there are questions about this, please just ask
thank you for your time and info

"... for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader ..."

that is not true on my end, better saying it works on my end to filter all links but
'redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com'

it filters for example twitter, insta, regular youtube links, flickr, redgifs

my idea months ago was to have a whitelist not a blacklist, same like pspzockerscene was saying here at this thread...

even if it would not filter those links, it would be okay, I can delete them at linkgrabber results list...

for me it is about those pop-up alert boxes...



my .txt source file has for example up to 5mio links, works fine with that large number... it should process in the background taking its time up to 2-3 weeks... max 31gb RAM assigned with shortcut...

the thing is, if I would deactivate pop-ups or captcha in general it would not fix the issue, it would be crawled, and that interrupts the entire processing...


it is also hard to be able to make it more visible to all of you, to re-produce that issue, I think noone wants to crawl a 5mio link txt file and wait until it happens to show some alert box or captcha

Last edited by verheiratet1952; 30.03.2023 at 16:28. Reason: re-produce
Reply With Quote
  #13  
Old 30.03.2023, 16:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,498
Default

Quote:
Originally Posted by verheiratet1952 View Post
for me it is about those pop-up alert boxes...
I'm sorry but that tells me that you still haven't understood our answers.

I'll quote myself:
Quote:
Originally Posted by pspzockerscene View Post
I'm sorry but this takes too much time for me.
If you need further help with this, please contact us again via support@jdownloader.org.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #14  
Old 30.03.2023, 17:02
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

Quote:
Originally Posted by verheiratet1952 View Post
that is not true on my end, better saying it works on my end to filter all links but
'redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com'

it filters for example twitter, insta, regular youtube links, flickr, redgifs
I'm sorry but it true because that's how it works. Of course it works for you as long as your links
do not lead/contain to those links. JDownloader will process ANY link that has your sourceURL
condition in processing chain. reddit->Twitter, JDownloader will still process Twitter link because
reddit is part of the processing chain and when there is no separate filter rule above for twitter

Please read again
Quote:
Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)
Quote:
Originally Posted by verheiratet1952 View Post
for me it is about those pop-up alert boxes...
You have to create filter rules AND place/move them to top of filter list if you don't want JDownloader to process those URL.
for example you don't want ally.sh links to be processed at all, then you will have to create a rule to block these with
sourceURL and move above your *whitelist* rules.

Last edited by notice; 30.03.2023 at 17:07.
Reply With Quote
  #15  
Old 05.04.2023, 17:43
notice notice is offline
JD Supporter
 
Join Date: Mar 2023
Posts: 505
Default

@verheiratet1952: any more questions about how the filters work? anything we can help with?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 03:46.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.