[verheiratet1952]

how to make alert boxes NOT appear?

I added some FILTER to skip any domain names but allowed domain names...

and it opens those alert boxes for youtube asking to add just that single link or entire playlist, or it asks if captcha of for example some shortener domain name should be done via browser extension....

all those domain names it brings those alert boxes are skipped by FILTER...


[ the problem is that I dont want JD to be back on main screen while I leave my desktop PC alone for several hours while processing other tasks that have to be active on main screen to work the faestest way possible (for example https uploading) ... ]

[pspzockerscene]

Hi,
looks like your filters aren't setup correctly!
Filters do work before any plugins are involved so captchas and also this YT dialog are an indicator if your filters not working.

To filter youtube playlists for example, you need to use "Sourceurls -> contains -> ..." and not "Downloadurl -> contains".

If you can't make it work, please post your exported filter rules here so we can check them.

[verheiratet1952]

Originally Posted by pspzockerscene

Hi,
looks like your filters aren't setup correctly!
Filters do work before any plugins are involved so captchas and also this YT dialog are an indicator if your filters not working.

To filter youtube playlists for example, you need to use "Sourceurls -> contains -> ..." and not "Downloadurl -> contains".

If you can't make it work, please post your exported filter rules here so we can check them.

thank you for your quick reply...
first of all I have created a very solid workaroud for my special needs...
(thanks to you, mgpai and jiaz for your great support over years)

please see screenshot for FILTER details

it is only about source links with domain name reddit.com

plugin setup - reddit - crawler text mode - NEVER


this issue with alert box does only happen every 500-1000 crawling source links... I do add links via .txt file using eventscripter also having a sleep interval...

(my source text files have 50k lines and more...)

[pspzockerscene]

I see multiple mistakes and wrong assumptions here:

First, coming back to your last post:
As you can see in your own screenshot, that captcha prompt is coming from "ally.sh" and not from reddit or from any site that you were trying to block.
That again is not on your "blacklist" so no wonder you're getting captcha prompts of that

Now to your most recent post:

Originally Posted by verheiratet1952

plugin setup - reddit - crawler text mode - NEVER

Have you even read the description of that setting or ever tested it?
This will not(!) disable the reddit crawler.
This will only disable crawling text-content from reddit posts.
If you do not want JD to crawl links from provider XY, there is two ways to do this:
1. Settings -> Advanced Settings -> "GeneralSettings.crawlercrawlerpluginblacklist" and "GeneralSettings.crawlerhostpluginblacklist".
2. The filter settings you already found.

Now about the screenshot of your filters:
You got "reddit.com" in "Downloadurl...contains not" and at the same time "Sourceurl...contains" that alone makes no sense.
Without further explanation, I'd do it like this:
Screenshot:

If you want to filter YT too, you need to add |youtube\.com to that filter field.
I also want to point out that there is a test button at the bottom of that window which can be used to test this rule on links. That saves time compared to testing it in your linkgrabber.

Also, if you want to block "all dialogs" such as those captcha dialogs, there is a "Enable silent mode" checkbox in the main toolbar which, if enabled, will block some(not all!) popups.

Originally Posted by verheiratet1952

(my source text files have 50k lines and more...)

Maybe you should think about a whitelist instead of a blacklist?

[verheiratet1952]

Originally Posted by pspzockerscene

I see multiple mistakes and wrong assumptions here:

First, coming back to your last post:
As you can see in your own screenshot, that captcha prompt is coming from "ally.sh" and not from reddit or from any site that you were trying to block.
That again is not on your "blacklist" so no wonder you're getting captcha prompts of that

it is crawling reddit.com links only via .txt file using eventscripter
so there is no ally.sh link from my source links... and there is no YT links links also...

it filters nearly all links (99.9%) but sometimes there are those alert boxes about for example ally.sh and YT

if my FILTER rule would not work, it would also let pass links from redgifs

as I do understand my FILTER rule it is a whitelist for reddit.com redd.it gfycat.com and imgur.com

[pspzockerscene]

Originally Posted by verheiratet1952

so there is no ally.sh link from my source links... and there is no YT links links also...

Sure there can be! If a reddit post-text contains YT links and JD crawls them, JD will try to crawl those YT URLs unless there is aa filter to prevent that.
Check Settings -> Reddit.com -> "Crawl URLs inside post-text?" -> Disable this

Please provide a screenshot of your reddit.com plugin settings.

Originally Posted by verheiratet1952

if my FILTER rule would not work, it would also let pass links from redgifs

I've provided a working rule.
You can provide further feedback to that.

[verheiratet1952]

Originally Posted by pspzockerscene

Sure there can be! If a reddit post-text contains YT links and JD crawls them, JD will try to crawl those YT URLs unless there is aa filter to prevent that.
Check Settings -> Reddit.com -> "Crawl URLs inside post-text?" -> Disable this

Please provide a screenshot of your reddit.com plugin settings.


I've provided a working rule.
You can provide further feedback to that.

my current filter prevents 99.9% of urls like YT if a reddit post contains such links... all links are filtered, only getting results from "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com"

"Crawl URLs inside post-text?" is diabled (better saying no checkbox activated)



just saying it once again, my settings DO work for nearly 99.9% of all links crawled... and giving me exactly the results I want...

it is only about some POP-UP that JD is asking if it should be crawled a single YT link or the entire YT playlist even if link source is not included at "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com" or JD is asking for some captcha with also different links...


regular YT links are filtered correctly but why there is some pop-up asking this?


I did re-store filtered links and added NEW screenshot to make you believe YT and other source links like tiktok insta twitter are filtered perfectly...

[pspzockerscene]

Originally Posted by verheiratet1952

just saying it once again, my settings DO work for nearly 99.9% of all links crawled... and giving me exactly the results I want...

Okay. I'm just saying that obviously your rules do not yet work 100% as you'd like them to work else you wouldn't get that popups...

Originally Posted by verheiratet1952

it is only about some POP-UP that JD is asking if it should be crawled a single YT link or the entire YT playlist even if link source is not included at "redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com" or JD is asking for some captcha with also different links...

I've understood that and I told you what to do.
Update your rule accordingly!

Originally Posted by verheiratet1952

regular YT links are filtered correctly but why there is some pop-up asking this?

Because "downloadurl" is only available after processing.
Regular YT links won't show a popup before processing but playlist-links do.
If you want to filter them in the best way, you need to filter them before they get processed!

Filter only by "sourceurl" and it will work.
If you still don't want to do this, modify your YT settings accordingly:
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing
EDIT

Alternatively go to Settings -> Advanced Settings -> GeneralSettings.crawlercrawlerpluginblacklist
Enter this:

["youtube.com"]

--> Just keep in mind that this will disable playlist- and also single video processing!

[verheiratet1952]

Originally Posted by pspzockerscene

Okay. I'm just saying that obviously your rules do not yet work 100% as you'd like them to work else you wouldn't get that popups...


I've understood that and I told you what to do.
Update your rule accordingly!


Because "downloadurl" is only available after processing.
Regular YT links won't show a popup before processing but playlist-links do.
If you want to filter them in the best way, you need to filter them before they get processed!

Filter only by "sourceurl" and it will work.
If you still don't want to do this, modify your YT settings accordingly:
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing
EDIT

Alternatively go to Settings -> Advanced Settings -> GeneralSettings.crawlercrawlerpluginblacklist
Enter this:

["youtube.com"]

--> Just keep in mind that this will disable playlist- and also single video processing!

if I do change FILTER rule it skips all those URLs I want !?

please see attached file (I had to use .zip because it cannot upload .filter file-format)


good idea, I changed to 'do nothing'
Settings -> Plugins -> YT -> If Link equals playlist or channel list or user-list -> Do Nothing

[pspzockerscene]

I'm sorry but this takes too much time for me.
If you need further help with this, please contact us again via support@jdownloader.org.

I'll be back online tomorrow.

[notice]

@verheiratet1952:
sourceURLs can be any URL within the processing, for example Input -> A - > B -> C -> downloadURL D
downloadURL always is the last single/final downloadURL D

so you can apply sourceURL filtering on Input, A, B , C and D
but
you can only! apply downloadURL filtering on D

If you don't want YT links to be processed at all (no processing, no popups) then you have to use sourceURL filtering for this.
same for ally.sh, as this is a site that shorten/hide links and is part of the processing chain (eg Input, A, B or C) , so you have to use sourceURL for this as well.

but it is important to know that the rules are processed from top to bottom and what sourceURLs filtering means.
for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader.
when you want to filter for example ally.sh, then you have to create a filter for this and move it above your other filter.

Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)

In case it is still unclear or there are questions about this, please just ask

[verheiratet1952]

Originally Posted by notice

@verheiratet1952:
sourceURLs can be any URL within the processing, for example Input -> A - > B -> C -> downloadURL D
downloadURL always is the last single/final downloadURL D

so you can apply sourceURL filtering on Input, A, B , C and D
but
you can only! apply downloadURL filtering on D

If you don't want YT links to be processed at all (no processing, no popups) then you have to use sourceURL filtering for this.
same for ally.sh, as this is a site that shorten/hide links and is part of the processing chain (eg Input, A, B or C) , so you have to use sourceURL for this as well.

but it is important to know that the rules are processed from top to bottom and what sourceURLs filtering means.
for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader.
when you want to filter for example ally.sh, then you have to create a filter for this and move it above your other filter.

Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)

In case it is still unclear or there are questions about this, please just ask

thank you for your time and info

"... for example:
you have a filter that allows sourceURL reddit.com. now imagine reddit.com(Input) -> ally.sh -> YT
your rule allows sourceURL reddit.com and therefore will allow ANY links that will result from this processing and thus ally.sh and YT will be allowed as well and processed by JDownloader ..."

that is not true on my end, better saying it works on my end to filter all links but
'redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com'

it filters for example twitter, insta, regular youtube links, flickr, redgifs

my idea months ago was to have a whitelist not a blacklist, same like pspzockerscene was saying here at this thread...

even if it would not filter those links, it would be okay, I can delete them at linkgrabber results list...

for me it is about those pop-up alert boxes...



my .txt source file has for example up to 5mio links, works fine with that large number... it should process in the background taking its time up to 2-3 weeks... max 31gb RAM assigned with shortcut...

the thing is, if I would deactivate pop-ups or captcha in general it would not fix the issue, it would be crawled, and that interrupts the entire processing...


it is also hard to be able to make it more visible to all of you, to re-produce that issue, I think noone wants to crawl a 5mio link txt file and wait until it happens to show some alert box or captcha

[pspzockerscene]

Originally Posted by verheiratet1952

for me it is about those pop-up alert boxes...

I'm sorry but that tells me that you still haven't understood our answers.

I'll quote myself:

Originally Posted by pspzockerscene

I'm sorry but this takes too much time for me.
If you need further help with this, please contact us again via support@jdownloader.org.

[notice]

Originally Posted by verheiratet1952

that is not true on my end, better saying it works on my end to filter all links but
'redd.it|imgur.com|gfycat.com|external-preview.redd.it|reddit.com'

it filters for example twitter, insta, regular youtube links, flickr, redgifs

I'm sorry but it true because that's how it works. Of course it works for you as long as your links
do not lead/contain to those links. JDownloader will process ANY link that has your sourceURL
condition in processing chain. reddit->Twitter, JDownloader will still process Twitter link because
reddit is part of the processing chain and when there is no separate filter rule above for twitter

Please read again

Note
* JDownloader processes the filters from top to bottom and will stop at first match
* sourceURL filters are checked against every URL within processing chain (Input,A,B,C and D)

Originally Posted by verheiratet1952

for me it is about those pop-up alert boxes...

You have to create filter rules AND place/move them to top of filter list if you don't want JDownloader to process those URL.
for example you don't want ally.sh links to be processed at all, then you will have to create a rule to block these with
sourceURL and move above your *whitelist* rules.

[notice]

@verheiratet1952: any more questions about how the filters work? anything we can help with?