[aremann]

Hey, F95zone have started with masked/gated urls to certain download sites like Mega.nz download links.

Note site can be NSFW at times and requires a free account to access threads.

Spoiler:

General site: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Thread Source of Download link: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Download Link Example: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Want the parser to correctly handle these download URL's and grab the gated URL.

The Deep Link Analyse isn't doing it sadly.

[raztoki]

it wont work because site uses html redirects most likely, that requires you to be logged in which jd crawler isn't. is the ending parts of the url the actual location? maybe you could reconstruct based on knowing the expected outcome, and make it a linkcrawler rule.

[aremann]

No, the end of the URL has no relation at all to the end results sadly.

Which would be:

Spoiler:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Yeah, didn't think about it requiring to login for that specific part. Makes sense that the whole site is locked down behind an account.
That't too bad, guess one have to take twice the time to grab those URL's instead of grabbing it all at once.

Lets hope they dont add more sites to the automatic replacement like they did with Mega.nz

Maybe it can be decoded somehow from a specific type of encode. That wouldn't require a login atleast.

[aremann]

Had some friends looking into it a bit for me.

They seem to think that it sense an xhr to the server and also checks for recaptcha validation should trigger or not.

This is prob looked up in some table and the key is generated based on a timestamp or something.... maybe.

Spoiler:

$(".host_link").on("click", function(c) {
c.preventDefault();
$i.css("width", $i.outerWidth());
$(".leaving-text").fadeOut(200, function() {
$l.fadeIn(200);
$.ajax({
data: {
xhr: 1,
download: 1
},
success: function(a) {
$l.fadeOut(200, function() {
switch (a.status) {
case "ok":
window.location.href = a.msg;
b("Redirecting...", 'Redirecting you to your link, <a href="' + a.msg + '" rel="noreferrer">click here</a> if it doesn\'t happen automatically');
break;
case "error":
b("Error", a.msg, 1);
break;
case "captcha":
$c.fadeIn(200), grecaptcha.render("captcha", {
theme: "dark",
sitekey: "6LcwQ5kUAAAAAAI-_CXQtlnhdMjmFDt-MruZ2gov",
callback: function(a) {
$c.fadeOut(200, function() {
$l.fadeIn(200);
$.ajax({
data: {
xhr: 1,
download: 1,
captcha: a
},
success: function(a) {
if ("ok" !== a.status) return b("Captcha Error", a.msg, 1);
window.location.href = a.msg
}
})
})
}
})
}
})
}
})
})
})

[pspzockerscene]

You'll have to manually handle this or maybe get a browser addon which might be able to see/copy the URLs out of your browsers' active DOM.

A JDownloader plugin for this website would require a lot of work just for 1-2 users. We would then also have to reconstruct the complete forum login process ...

-psp-
EDIT

Ticket:

[pspzockerscene]

Are you still interested in a plugin for this website?

[Sminem]

Originally Posted by pspzockerscene

Are you still interested in a plugin for this website?

Made an account to say it would be great to have if it's managable.

[Jiaz]

@Sminem: can you provide some example links?

[pspzockerscene]

Added ability to add accounts for this website and added simple crawler for "/masked..." URLs.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[Sminem]

Originally Posted by Jiaz

@Sminem: can you provide some example links?

Sure. "Masked" links (user has to click a link to continue);

Mega type:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Pixeldrain:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Workupload:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

@Sminem: thanks

[pspzockerscene]

Added more features to this crawler for the next update:
F95zoneToCrawler:
- added support for captchas when crawling single "/masked" urls
- added support for crawling all external urls from "/threads/..." urls
- allow max 1 concurrent crawl thread to avoid causing (many) captchas when adding single links
- added requestLimit of 1500ms to further avoid mentioned captchas

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[pspzockerscene]

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-