[FBD]

Originally Posted by Jiaz

So it would be great if you can provide some details like geo location/ISP name, something that you all might have in common and reason why/how you do get blocked by cloudflare and we/others don't.

Same for me, testing both with vpn and direct it's working fine and no cloudflare block, but from the people reporting in the last few hours in the support chat (and who were willing to share their location) there is a pattern:

2x from south america/argentina
1x from europe/spain
1x from south america/bogota
1x from south america/santiago
2x from south america/lima
1x from asia/india

[Emm]

Europe/Romania blocked.

I think mediafire deploys its cloudflare protection gradually that's why for now it works for some and not for others but sooner or later it won't work for anybody.

[Topken]

USA and Spectrum here and blocked by cloudflare protection

[rocco123]

Looks like the real download link can't resloved.

Coming from a filecrypt container like this: **External links are only visible to Support Staff****External links are only visible to Support Staff**
and clicking on Click'n Load, the added files could not be downloaded.

Resolving the link in the browser by clicking the "Herunterladen" button at filecrypt and then adding the links via the "Download" button from mediafire, the added links will be downloaded just fine.

Its a workaround, but it adds manual effort...

[Jiaz]

Originally Posted by rocco123

clicking the "Herunterladen" button at filecrypt and then adding the links via the "Download" button from mediafire

That will always work because download servers rarly/never are protected by cloudflare as the cloudflare traffic costs would explode then

[Jiaz]

Originally Posted by Emm

I think mediafire deploys its cloudflare protection gradually that's why for now it works for some and not for others but sooner or later it won't work for anybody.

Does not explain why it still works for us/many others and even works when using VPN to other countries. Maybe it's also related to something else, like amount of downloads or such. Have you tried to use VPN to other country?
On what OS are you?

[Emm]

Last time I downloaded from mediafire was like two weeks ago. And they were working fine, but now they're acting up from the 1st link. I'm on Win10.

After some testing, I think I see where the issue __might be__. I've used wget (for convenience) to replicate what was written in the logs.

When requesting /file why do you add as referer api/1.5/file/get_info.php...?

When I've tried the exact header structure as in the logs, I too have gotten a 403 response in wget, but removing just that referer header the page downloaded properly. Also the page seems to load fine without the need of any cookies... both in wget and in the browser (Edge, completely blocked cookies for mediafire), it loads fine.

Also, seems strange that Cloudflare protection fires up in JD2, but not in the browser. I do encounter plenty of sites behind cloudflare and their lil' captcha but not on mediafire when browsing... like never. Buzzheavier, sure, but not mediafire.

[Jiaz]

Originally Posted by Emm

When requesting /file why do you add as referer api/1.5/file/get_info.php...?

Because same browser instance was used and thus this referer was part of it. Will be changed with next update

Originally Posted by Emm

When I've tried the exact header structure as in the logs, I too have gotten a 403 response in wget, but removing just that referer header the page downloaded properly

Possible, we will see if/how it works out once you can test the updated plugin. But still strange that it works for some, fails for others, but that most likely the *magic* behind cloudflare

Originally Posted by Emm

Also, seems strange that Cloudflare protection fires up in JD2, but not in the browser. I do encounter plenty of sites behind cloudflare and their lil' captcha but not on mediafire when browsing... like never. Buzzheavier, sure, but not mediafire.

Cloudflare does Browser and Request finger printing and thus such little differences may cause the trigger.

Please test with next update
Update: that change is live

[ElCho]

I don't know what VPN are you using, but I could share with you the one I 'm using,
so you try by yourself and see if that way you are able to reproduce the blocking.

[pspzockerscene]

Sure, name the VPN you are using.

Please still keep in mind that most likely we will not be able to do anything to fix this (see Cloudflare FAQ).

Did any of you guys already contact the mediafire.com support and ask them what's going on?
After all, we are using their APIs which are specifically ment to be used by tools like JDownloader and those are being blocked by Cloudflare:
mediafire.com/developers/
This means that this problem will also affect other projects which rely on the mediafire.com API.

[ElCho]

I'm using "ExpressVPN".

This is not a big deal anyway, but there should be a way of discovering the mystery behind
how they are blocking specifically the "JD2" tool for some users, while leaving others still use
it normally.

By the way, I just tried with "MiPony" download tool, links are resolved normally and
files get downloaded correctly. So, this kind of confirms "Cloudflare" is blocking specifically
the "JD2" tool only, at least for some users.

Mediafire support team was already informed of the issue; still waiting for an answer from them.

[Jiaz]

@ElCho: have you installed latest JDownloader updates that adressed the findings by https://board.jdownloader.org/showpo...7&postcount=87 ?

@ElCho: Can you provide endpoints/geo locations that fail for you? and does it work for you with vpn or not?

[ElCho]

Yes, I already tried with the "48254" update, and still the same.

Respect your second question:

https://board.jdownloader.org/showpo...6&postcount=72

It seems they are detecting "JD2" use, besides some other unknown parameter,
and the proceeding to block the access, disregarding the IP locations and cookies.

When I have some time, I'm gonna try with different workstations,
another OS and even a different ISP, to see what happens.

[Emm]

Originally Posted by Jiaz

But still strange that it works for some, fails for others, but that most likely the *magic* behind cloudflare

IDK if it's a lot of magic. From my end, almost all crouldflare calls go through cloudflare Bucharest, then to the host server. Right now, in browser, bunkr gives me 502 bad gateway for every page link and informs me couldflare Bucharest is OK, but end host it not... and 5 seconds later it cleared. Content goes directly to host.

It could very well be finger printing, but that also means it's bypassable. If nothing shows up in the browser, then nothing should show up in JD2's internal calls. Unless it's something deeper... I know the browser uses HTTP/3, Edge shows that, while JD2 uses v1.1, that means it's not a cloudflare issue that can't be solved, but it does point to fingerprinting. It also means a host/cdn can simply "disable" JD2's use if they don't like the excess traffic it causes, in mass and erratically so you can't pinpoint the issue. At least with buzz it's a captcha and front check, but here it's a sleazy block.

Is it possible to upgrade JD2 http engine to use v3 and a newer browser agent?


Tested with update 48254, still gets blocked.

Originally Posted by pspzockerscene

Did any of you guys already contact the mediafire.com support and ask them what's going on?
After all, we are using their APIs which are specifically ment to be used by tools like.

To what end? From their perspective it works and so does it in JD2.
But where does JD2 use the API do get the content link?
From what I can see in the logs, it doesn't. The only API call is to get metadata and that one works without a problem, it outputs a pretty json in the logs every time, hence it shows up nicely until the download has to start.

To start the download, JD2 accesses the same page we all do in the browser, mediafire.com/file/MediaID/file ... with or without the file name parameter. I mean, I suppose it follows what the API call returns but that's the same link that was added to JD2 in the first place... links normal_download has as value same initial link, not a direct download / content link.

The content link obtained manually from the page works in JD2. I made myself a Tampermonkey script that gets those direct links from problematic hosts like buzz and now mediafire, it's just cumbersome to use when the download is split in 30 links, since JD2 has a clipboard scanning lag, I have to open one at a time every 3-4 seconds. But it works, so content links get accessed and downloaded with JD2 directly without a problem. The cloudflare problem is the page.

I don't use VPN, don't have too, so I got to try VPNjantit for OpenVPN. I've tired Canada Quebec, Germany Berlin and Germany Frankfurt. As a free user don't get a lot of access:
Canada is blocked, Saint Jean Quebec/OVH SAS
Germany is blocked, Berlin/1&1 Internet AG
Germany is blocked, Frankfurt/OVH SAS

Otherwise, in all vpn's the content link works in JD2, and browser page loads without any issue. Normal links don't work.

[pspzockerscene]

Originally Posted by ElCho

It seems they are detecting "JD2" use, besides some other unknown parameter,

Who is "they"?
Cloudflare or mediafire.com?

Originally Posted by Emm

Is it possible to upgrade JD2 http engine to use v3 and a newer browser agent?

We are using a custom internal browser class.
It has nothing to do with what you would call a "browser engine".
In the Cloudflare thread, some open source solutions are linked which basically consist of setting up a special proxy which does some special requests using a "real browser" in order to get through Cloudflare. Search in this thread:
https://board.jdownloader.org/showthread.php?t=83712

The JDownloader project is open source so feel free to do local modifications and test them.
Source code:
https://support.jdownloader.org/know...up-ide-eclipse

Originally Posted by Emm

To what end?

Everything except free downloads is done via API in JD2.

Originally Posted by Emm

From what I can see in the logs, it doesn't. The only API call is to get metadata and that one works without a problem, it outputs a pretty json in the logs every time, hence it shows up nicely until the download has to start.

If that is the case and the API is not blocked, this block might even be intentionl.
After all I still can't reproduce it so I could only make assumptions.

Originally Posted by Emm

I don't use VPN, don't have too

Neither do I.
I've done some tests with Mullvad VPN (only german IPs) and none were blocked.

[ElCho]

Originally Posted by pspzockerscene

Who is "they"?
Cloudflare or mediafire.com?

Mediafire.

By the way, they sent me what, at first, I thought it was a "regular" reply from
a representative, which has a series of steps I should follow, in order to help them
to figure out which could be the problem and the possible solutions. But after reading
all the instructions, they tell me I could try with another download manager, at least
temporarily, but, to my surprise, they mention a nowadays "abandonware" program,
whose not even the developer's website is alive anymore: Speedbit's "Download Accelerator Plus".
That little detail suggests me they just delivered me a generic reply, whose template isn't even
updated (last DAP's version dates from 2013).

Alright, after reading here and there about this problem, I finally was able to make "JD2" works
as expected again in my system. Basically, someone mentioned he found a "portableapps" version
which was working correctly with "Mediafire", I tested it and it was true. I then thought about what
could be the reason, and I found it: JAVA JRE. The thing is, that specific portable was using
Oracle's JRE "v1.8.0_202 (64bit/X86)"; while my current (non-working) installation has Oracle's
last version: "1.8.0_421 (64bit/X86)". So, I downloaded "202" version from Oracle's website,
replaced it from my installation, and that was the solution. I also tried with the JRE v21.0.2 (64 bit)
from "Eclipse Adoptium" company, and it was working rightly wit hit too, but I don't like that JRE's version,
since it alters the JD2 interface I'm used to.

After testing the "202" build, I also tested going back to the original version I had installed "421",
and the downloads kept working. So I will keep using it, provided it remains working fine.

So, the mystery is finally revealed, well at least partially, since it remains
to know why the Oracle's JRE "421" build ceased to work all of a sudden.
And why doing the "421>202>421" exchange makes the last Oracle's JRE build works again.

[Jiaz]

@ElCho: To make it short but you can dig into the web for more detailed articles. Cloudflare does deep analyze the hole request (UA, headers (size, number, order, content) and https settings (protocol, offered ciphers, selected cipher) and many many more) and does fingerprint those. Now they have a good *sight* at fingerprints and can easily tell that browser A has 99% fingerprint A but 1% fingerprint B, so requests from B are *strange* and might (depending on cloudflare site settings) require javascript challenge that now checks the browser environment and fingerprints that as well. and when that doesn't help either to clearify internal stats, then turnstile will be requested too. That's why a simple change of UA might help, or changing https settings. Now with different java version, the default protocol might have changed and/or the default offered/selected cipher and many more.
I tried some stuff and was able to get blocked by cloudflare as well but it was totally random and could not find a *muster*. working fine for several minutes, blocked again...changing java...blocked...working fine again....changing java...doesn't help..changing ciphers...doesn't help.....working fine again....blocked again... To me it looks like they might be under attack and it's more random (but also depending on IP range) if cloudflare is protecting or not
Short: enjoy while it lasts but doesn't mean that it has to last long

[ElCho]

Build 421 ceased to work again after a while, so I switched back to build "202", let's see how much it lasts.

[Emm]

Originally Posted by pspzockerscene

Who is "they"?
Cloudflare or mediafire.com?

At this point I'm pretty sure it's Cloudflare, after all that's what they're getting paid for, protection, so that mediafire can be lazy. And it pretty much points to fingerprinting, most likely of communication initiation in TLS.

Originally Posted by pspzockerscene

If that is the case and the API is not blocked, this block might even be intentionl.

Seems like it, but judging by why Cloudflare exists, it's to stop automated access (be it ddos or tools). So in essence Cloudflare does what it supposed to do.

I've tried build 202 of JRE as well and yes, it works.

The only difference in the logs are the SSL/TSL handshakes, everything else is the same.

Build 202 uses
SSLCipher: AutoSwitch|JVM|Protocol:TLSv1.2|CipherSuite:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA

Build 322 (the one I've had in JD2's folder since installation) uses
SSLCipher: JVM|Protocol:TLSv1.2|CipherSuite:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Wget doesn't have ssl/tls debug capability so I can only assume it negotiates something Cloudflare likes or doesn't find suspicious enough to get triggered.

However, Curl, which can show headers, fails just like JD2 with 322 does.
SSL connection using TLSv1.2 / ECDHE-RSA-CHACHA20-POLY1305
HTTP/2 403

That's as much as I can see from the logs and debug output... probably there's more to it, but yes, fingerprinting it is. Which might be caused by JD2's popularity and use on certain Cloudflare nodes, or IDK.

After searching a bit, I ended up at a post from two years ago stating exactly that, TLS fingerprinting.
where-does-cloudflare-detect-web-and-terminal-requests-on-equal-terms
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Searching further, seems curl is Persona non grata for Cloudflare. Ended up at curl-impersonate.
**External links are only visible to Support Staff**curl-impersonate
Which looks like a mixed bag, on its own it gets blocked but with with JD2's header structure (without cookies) seems to work just like wget or build 202, i.e. fine.

Originally Posted by pspzockerscene

The JDownloader project is open source so feel free to do local modifications and test them.

You gotta love this answer... No, this type of issues looks more like a youtube vs adblockers back and forth fight. Have fun.

[pspzockerscene]

Originally Posted by Emm

Seems like it, but judging by why Cloudflare exists, it's to stop automated access (be it ddos or tools). So in essence Cloudflare does what it supposed to do.

No not necessarily.
In the end it is still the website owners (mediafire) who configure Cloudflare.
I know I'm repeating myself but has anyone asked mediafire.com about the situation yet?

Originally Posted by Emm

No, this type of issues looks more like a youtube vs adblockers back and forth fight.

Not our business at all.
We've already stated that we are not participating in any "Cloudflare vs bots war", see:
https://board.jdownloader.org/showthread.php?t=83712
Read the whole FAQ. We are not planning to put any time into this.
Use the tools that are already available and linked there and or dig deeper into existing github "Cloudflare proxy" projects.
Cloudflare has been around for a long time and if ppl really wanted it to block tools like JD, they've had their chance all of the time.
You can see this in numerous threads in our forums, just search for "Cloudflare" in titles-only.

Originally Posted by Emm

Have fun.

I'm sure the people who are working on Cf solutions and see it as a challenge will have fun doing so.

[Jiaz]

Originally Posted by Emm

No, this type of issues looks more like a youtube vs adblockers back and forth fight. Have fun.

Correct, because as soon as a working/reliable solution is known to be existing, Cloudflare will for sure analyze and find new ways to detect/block it.
I think it's more likely that sites will stop using them as soon as us laws request cloudflare to hold/verify personal details about site owners

[Rickz]

don't know exactly how Jdownloader download the files, but using browser is working fine, incognito working fine, just with JD i get blocked by cloudflare...don't even download 1 file... and just message "blocked by cloudflare" just wondering if JD can simulate the download just like the browser do... also could JD simulate incognito mode like the browser do it ??

i haven't change my IP and is not needed because with browser and incognito is working, so i should say JD needs and could be improved to bypass these "checks"

[Jiaz]

@Rickz: read https://board.jdownloader.org/showpo...4&postcount=97

Originally Posted by Rickz

so i should say JD needs and could be improved to bypass these "checks"

Originally Posted by Jiaz

Correct, because as soon as a working/reliable solution is known to be existing, Cloudflare will for sure analyze and find new ways to detect/block it.
I think it's more likely that sites will stop using them as soon as us laws request cloudflare to hold/verify personal details about site owners

[Rickz]

@Jiaz if mipony can i'm sure JD can, JD BEST download manager so far!!

there are some work around that i believe can be automated.. check:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

@Rickz
Read the Cloudflare FAQ again closely!

Short version:
By copying the direct URLs from your browser, your browser already did all of the Cloudflare verify java script stuff before (see also the Cloudflare cookies that are present).
This way you effectively download manually (or in a "semi automatic" way).

About "Mipony":
I've also read on other places that this is supposed to work.
Did you test that?
I cannot test it as I'am not blocked by Cloudflare so JD is still working for MF for me.

[Rickz]

Originally Posted by pspzockerscene

@Rickz
Read the Cloudflare FAQ again closely!

Short version:
By copying the direct URLs from your browser, your browser already did all of the Cloudflare verify java script stuff before (see also the Cloudflare cookies that are present).
This way you effectively download manually (or in a "semi automatic" way).

About "Mipony":
I've also read on other places that this is supposed to work.
Did you test that?
I cannot test it as I'am not blocked by Cloudflare so JD is still working for MF for me.

Regards,
i only get blocked by cloudflare with JD, same IP and mipony is working...

[Jiaz]

Originally Posted by Rickz

i only get blocked by cloudflare with JD, same IP and mipony is working...

Easy explanation. Different number of usage. I can bet that if we start to look to how mipony works and reproduce it, that it will be blocked as well sooner or later. Cloudflare does not simple block away unusual traffic but of course also takes in accounting the amount of traffic with a specific fingerprint. That's the simple explanation why wget works fine but JDownloader does not.

We can try to work on this but our main problem is that we have hard time to get blocked at all and thus cannot work on the problem when not getting blocked.

[ElCho]

I just tested it with recent 431 JRE's build, and JD2 is still blocked with it; went back to the still working fine "202" build.

"MyPony" works correctly, but maybe because it (still) wasn't fingerprinted. Since you aren't being blocked, do you think a portable version of mine, among the same VPN and Windows' VM would be enough to get it triggered?

[Rickz]

Originally Posted by ElCho

I just tested it with recent 431 JRE's build, and JD2 is still blocked with it; went back to the still working fine "202" build.

"MyPony" works correctly, but maybe because it (still) wasn't fingerprinted. Since you aren't being blocked, do you think a portable version of mine, among the same VPN and Windows' VM would be enough to get it triggered?

what about using a ramdon fingerprints ?

[ElCho]

Originally Posted by Rickz

what about using a ramdon fingerprints ?

I already tried with different OSES and still the same; could try with a different ISP, workstations, etc., but have not the time right now. Anyway, as long as it keeps working, I will stay with build "202".

[rocco123]

Where do I get build "202"?

[ElCho]

Originally Posted by rocco123

Where do I get build "202"?

https://www.oracle.com/java/technolo...downloads.html

https(://)sites.google.com/view/java-se-download-url-converter

I use this package:

jre-8u202-windows-x64.tar.gz

[rocco123]

Thanks, that worked for me also.

[jdmaster]

I just updated Jd and with these two links it didn't work.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**

but with this one it worked fine
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Please post your log-ID here
If your report is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.
If your report is related to a login specific problem with a plugin supported website, enable debug mode before creating logs, see previously linked instructions.

Bitte poste deine Log-ID hier.
Falls dein Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.
Geht es um einen nicht funktionierenden Account-Login, aktiviere vor dem Erstellen deines Logs den debug Modus (siehe zuvor verlinkte Anleitung).

-psp-

[jdmaster]

Originally Posted by jdmaster

I just updated Jd and with these two links it didn't work.

**External links are only visible to Support Staff**...

**External links are only visible to Support Staff**...

but with this one it worked fine
**External links are only visible to Support Staff**...

log-ID:
11.11.24 14.45.17 <--> 11.11.24 14.33.00 jdlog://3666411370661/

[pspzockerscene]

Same Cloudflare problem than reported by the other users.
Read this thread and this one:
https://board.jdownloader.org/showthread.php?t=83712

[Perene]

I checked this thread. Note: I'm from Brazil. CloudFlare is not even enabled here (using my local DNS, from my ISP), and doesn't work to download anything from this server. Also, adding a free MediaFire account (login/password) makes no difference for JDW. FlagCookies isn't used for it.

I tested Mipony, and at least that app was able to download, for example, a single MediaFire link. And you know Mipony also handles the same stuff JDW does. Whatever they do, it's working.

In my opinion, you either need to find a workaround from them, or, if this isn't possible, include MediaFire in a list (if you have one) of JDW's unsupported servers. There is nothing we can do to reverse that.

Note: not saying it's your fault - this is another shenanigan from a server, which broke things that used to work 100% before.

[Jiaz]

Originally Posted by Perene

need to find a workaround from them

When I find time, I will check how they do it/why it works for them