[pspzockerscene]

The "Avoid downloading 'too small' videos?" setting has been added with todays' update.
It is disabled by default.

[shinji2009]

hi. can you please show how regex whitelist for crawled urls should look like with at least two example links? i'm confused reading the description of this field

[pspzockerscene]

You mean for at least two soures/websites?
Example:

https?://(website1\.tld|website2\.tld)/.+

Learn and test regular expressions with this free webtool: regex101.com

[shinji2009]

i was just thinking that use real urls in the example will be less confusing
like
https?://(mega.nz\.tld|gofile.io\.tld)/.+
or
https?://(mega\.nz|gofile\.io)/.+

[pspzockerscene]

I just noticed that the description of said setting already contains an example.
Just keep in mind that it's up to the user to learn how regular expressions work though for this sole purpose you only really need a very basic understanding.

I've updated the example inside the settings to use "real websites" for the next update.

[shinji2009]

thank you. best description is self-contained description

[shinji2009]

by the way. haven't you thinking making blacklist for crawled urls more convinient and logical than whitelist? because when you crawl a lot of videos, you can fastly update what you don't want to crawl (mostly it is only twitter and youtube), but you never know what file host uploader will choose for his video in better quality. and there are hundreds of file hosts. after you whitelist something, you need to check every video page, because there can be some other hosts. is it possible maybe to add a blacklist?

[pspzockerscene]

Originally Posted by shinji2009

haven't you thinking making blacklist for crawled urls more convinient and logical than whitelist?

In the context of iwara.tv:
I didn't think about it at all since I'm not using the iwara.tv plugin myself.
Back then I simply decided to add a whitelist instead of just a checkbox "Crawl external URLs" as it is more versatile.

Originally Posted by shinji2009

is it possible maybe to add a blacklist?

Sure I could add the functionality for this plugin but maybe you want to check the global blacklisting feature first:
Settings -> Advanced Settings:

GeneralSettings.crawlerhostpluginblacklist

and:

GeneralSettings.crawlercrawlerpluginblacklist

Originally Posted by shinji2009

you need to check every video page

The iwara.tv crawler is logging URLs which were skipped according to your whitelist, you can find them in your logs by looking for the text string "Skipped URLs due du users' whitelist pattern".
There are a lot of possible ways to make it work and I'am just trying to make users stick to our global features vs adding plugin-specific features.
So additionally to the global blacklists I definitely recommend you to check Settings -> Linkgrabber filter
You should be able to e.g. only filter youtube links when they come from a sourceurl which contains "iwara.tv".

[shinji2009]

i'm just saing from the convinience side. changing global settings is not an option for me beacuse i download from youtube too. i just like when you don't need to change settings depeding of where are you downloading from, and everythig was fine until recenly i find that some of iwara users are uploading videos in better qulity with external links. this make me look to iwara plugin settings to automate it. and today i realise that whitelist is not an option if i don't want to miss something. i guess i'll stick to linkgrabber options then, thank you

[pspzockerscene]

Originally Posted by shinji2009

changing global settings is not an option for me beacuse i download from youtube too

I understand that blocking YT globally is not an option.

Originally Posted by shinji2009

i guess i'll stick to linkgrabber options then, thank you

Not only that but also the filters.
EDIT
The advantage when using the filters is that the items do not need to be processed in order to filter them.
This makes everything faster.

Here is an example rule - if you were to allow _everything_ in the iwara.tv whitelist, you can block stuff via rules as shown here:

[shinji2009]

this is looks like just what i needed, thank you very much!

[pspzockerscene]

There you go.
Keep in mind that my example is just a very really lazy rule.

I recommend using the full domain to block sites/sources and if you want to block more than one, enable the regular expression checkbox so that you can filter multiple domains with one rule.

[shinji2009]

there is this one user
**External links are only visible to Support Staff****External links are only visible to Support Staff**
he leaves in a description a link to a previous video god knows why.
if i filter iwara.tv/video for iwara.tv source, jd don't grab a thing
if i don't filter iwara.tv/video for iwara.tv, jd start to grab a whole iwara so it seems, because one link goes to another, and another and so on.
i personally will remember him. but can you please give a hint if this can be filtered?

[pspzockerscene]

In this case, use the whitelist regex of the iwara.tv plugin to tell it "allow everything except iwara.tv links", using a 'negative lookahead' regular expression.
Regex:

https?://(?!(www\.)?iwara\.tv)[^/]+/.+

As already explained, learning regular expressions is they key to work with such filters.
It will also be very helpful for e.g. filename replaces and much more.
Use webtools like regex101.com to test regular expressions as that is easier than using JDownloader plugins to test them.

[shinji2009]

using whitelist as a blacklist sounds like a witchcraft)
for some reason i can't make grabber filter work for youtube. i make rules for both youtube and youtu.be, and youtu.be is working, but for youtube jd still ask me about downloading a playlist.
example links
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

@shinji2009: just to be sure, you want to block youtube links getting crawled/processed from iwara, right?

[shinji2009]

yes. i make some rules to block different sites crawling from iwara and they all seem to work except "youtube"

[Jiaz]

@shinji2009: currentlly not possible but I will work on a solution for this

[pspzockerscene]

Originally Posted by shinji2009

using whitelist as a blacklist sounds like a witchcraft)

That witchcraft continues here.

Jiaz is talking about the upper filter implementation but while it would be nice to have it working in this way, you can use the same magic inside the iwara.tv plugin to make this work right now.
Simply extend the regular epxression I've posted previously:

https?://(?!(www\.)?(iwara\.tv|youtube\.com|youtu\\.be))[^/]+/.+

Again:
You need to learn how regular expressions work if you want to continue working with such filters.

@Jiaz
I'll mark this as solved since the user will get it working this way.

[shinji2009]

code is not working for iwara.tv/users link
**External links are only visible to Support Staff****External links are only visible to Support Staff**
linkgrabber rule for this case also not working
sorry i don't understand why. for me if code blocks iwara.tv/video it must block iwara.tv/users, but is it not.

[pspzockerscene]

@shinji2009
I expect you to put some work into this.
So far it looks like you haven't even bothered understanding the regular expression I made for you so one last time, I'll explain it.
Now here is the version which you are most likely using at this moment:

https?://(?!(www\.)?(iwara\.tv|youtube\.com|youtu\.be))[^/]+/.+

Here is how the iwara.tv link from said videos' description looks like:
ecchi.iwara.tv/users/sssmmddaisuki

How why doesn't irt work?
Because that link is starting with "ecchi." while the regular expression only works for links that start with "www.".

So what's the possible fix?

https?://(?!([a-z0-9]+\.)?(iwara\.tv|youtube\.com|youtu\.be))[^/]+/.+

Once again: use tools like regex101.com to test regular expressions.
Learn how to use them so you can use them to your advantage in the future.
I will not provide any further help with this one!

[shinji2009]

hi. is it possible to make jd search for links through comments also? some authors are posting links to hq videos there. examples:

comment:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

comment to comment:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

comment with emoji:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

comment with emoji to comment:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

[pspzockerscene]

Just a friendly lookahead:
I'am not going to implement this so most likely, this feature will never be implemented.
Your options:

• Use other tools that can already d othis
• Implement it yourself; JDownloader is open source: https://support.jdownloader.org/Know...up-ide-eclipse

[Snookie]

can't download from iwara anymore "Blocked by!Cloudflare Site-Protection"
i also can't add my iwara account to jd2 account manager like i used to

[pspzockerscene]

Hi,

English / Englisch
the issue you've reported has been caused by Cloudflare/DDos-Guard or similar.
Please read the first post of the above linked thread and post in this thread if you have further questions!
Please post related questions in the above linked thread only!
Attention: The first post of that linked thread contains useful information for you AND website-owners and also the hint that website-owners can solve this issue on their side which would also make our plugin work again!

Deutsch / German
Das von dir gemeldete Problem wurde durch Cloudflare, DDoS-Guard oder ähnliche Anbieter verursacht.
Bitte lies den ersten Post des oben verlinkten Threads.
Weitere Fragen bitte im oben verlinkten Thread posten!
Achtung: Im ersten Post ist u.a. beschrieben, dass auch Webseiten-Admins das Problem auf ihrer Seite lösen können --> Unser Plugin würde dann sofort wieder funktionieren - das bitte beachten, den ersten Post vollständig lesen und ggf. an den Support der betroffenen Webseite weiterleiten!

-psp-