JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 29.01.2020, 23:49
aremann aremann is offline
Junior Loader
 
Join Date: Nov 2014
Posts: 13
Post F95zone.to masked anti-bot links

Hey, F95zone have started with masked/gated urls to certain download sites like Mega.nz download links.

Note site can be NSFW at times and requires a free account to access threads.

Spoiler:
General site: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Thread Source of Download link: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Download Link Example: **External links are only visible to Support Staff****External links are only visible to Support Staff**


Want the parser to correctly handle these download URL's and grab the gated URL.

The Deep Link Analyse isn't doing it sadly.
Reply With Quote
  #2  
Old 30.01.2020, 00:38
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

it wont work because site uses html redirects most likely, that requires you to be logged in which jd crawler isn't. is the ending parts of the url the actual location? maybe you could reconstruct based on knowing the expected outcome, and make it a linkcrawler rule.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 30.01.2020, 18:05
aremann aremann is offline
Junior Loader
 
Join Date: Nov 2014
Posts: 13
Default

No, the end of the URL has no relation at all to the end results sadly.

Which would be:
Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff**


Yeah, didn't think about it requiring to login for that specific part. Makes sense that the whole site is locked down behind an account.
That't too bad, guess one have to take twice the time to grab those URL's instead of grabbing it all at once.

Lets hope they dont add more sites to the automatic replacement like they did with Mega.nz

Maybe it can be decoded somehow from a specific type of encode. That wouldn't require a login atleast.
Reply With Quote
  #4  
Old 30.01.2020, 18:20
aremann aremann is offline
Junior Loader
 
Join Date: Nov 2014
Posts: 13
Default

Had some friends looking into it a bit for me.

They seem to think that it sense an xhr to the server and also checks for recaptcha validation should trigger or not.

This is prob looked up in some table and the key is generated based on a timestamp or something.... maybe.

Spoiler:
$(".host_link").on("click", function(c) {
c.preventDefault();
$i.css("width", $i.outerWidth());
$(".leaving-text").fadeOut(200, function() {
$l.fadeIn(200);
$.ajax({
data: {
xhr: 1,
download: 1
},
success: function(a) {
$l.fadeOut(200, function() {
switch (a.status) {
case "ok":
window.location.href = a.msg;
b("Redirecting...", 'Redirecting you to your link, <a href="' + a.msg + '" rel="noreferrer">click here</a> if it doesn\'t happen automatically');
break;
case "error":
b("Error", a.msg, 1);
break;
case "captcha":
$c.fadeIn(200), grecaptcha.render("captcha", {
theme: "dark",
sitekey: "6LcwQ5kUAAAAAAI-_CXQtlnhdMjmFDt-MruZ2gov",
callback: function(a) {
$c.fadeOut(200, function() {
$l.fadeIn(200);
$.ajax({
data: {
xhr: 1,
download: 1,
captcha: a
},
success: function(a) {
if ("ok" !== a.status) return b("Captcha Error", a.msg, 1);
window.location.href = a.msg
}
})
})
}
})
}
})
}
})
})
})
Attached Files
File Type: txt message.txt (2.0 KB, 1 views)
Reply With Quote
  #5  
Old 31.01.2020, 00:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,911
Default

You'll have to manually handle this or maybe get a browser addon which might be able to see/copy the URLs out of your browsers' active DOM.

A JDownloader plugin for this website would require a lot of work just for 1-2 users. We would then also have to reconstruct the complete forum login process ...

-psp-
EDIT

Ticket:
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 25.11.2020 at 15:13.
Reply With Quote
  #6  
Old 20.07.2022, 14:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,911
Default

Are you still interested in a plugin for this website?
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 13.09.2022, 02:37
Sminem Sminem is offline
I will play nice!
 
Join Date: Sep 2022
Posts: 2
Default

Quote:
Originally Posted by pspzockerscene View Post
Are you still interested in a plugin for this website?
Made an account to say it would be great to have if it's managable.
Reply With Quote
  #8  
Old 26.09.2022, 12:32
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

@Sminem: can you provide some example links?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 26.09.2022, 19:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,911
Default

Added ability to add accounts for this website and added simple crawler for "/masked..." URLs.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #10  
Old 30.09.2022, 21:08
Sminem Sminem is offline
I will play nice!
 
Join Date: Sep 2022
Posts: 2
Default

Quote:
Originally Posted by Jiaz View Post
@Sminem: can you provide some example links?
Sure. "Masked" links (user has to click a link to continue);

Mega type:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Pixeldrain:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Workupload:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #11  
Old 02.10.2022, 11:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

@Sminem: thanks
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 05.10.2022, 12:37
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,911
Default

Added more features to this crawler for the next update:
F95zoneToCrawler:
- added support for captchas when crawling single "/masked" urls
- added support for crawling all external urls from "/threads/..." urls
- allow max 1 concurrent crawl thread to avoid causing (many) captchas when adding single links
- added requestLimit of 1500ms to further avoid mentioned captchas

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #13  
Old 16.11.2022, 18:53
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,911
Default

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 15:12.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.