JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 17.03.2020, 10:32
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default Plugin for get-to.link

Hi JD Team,

Please add support for host get-to.link, which displays a collection of links. I think it was linx.cloud before but doesn't get redirected, since linx.cloud is dead.
Sample links below:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

As usual, my Sincere Thanks to you guys.

Regards,
Doc MaKS aKa reddevil
Reply With Quote
  #2  
Old 17.03.2020, 12:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,913
Default

Hi,

a plugin for this simple website is not needed.
You can simply add a link crawler rule to let JD automatically crawl contents of this website e.g. via clipboard detection:
Code:
[ {
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "get-to.link example rule",
  "pattern" : "https?://get-to\\.link/.+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : null,
  "rewriteReplaceWith" : null
} ]
-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 19.03.2020, 05:39
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Above link crawler rule gets the download links fine from get-to.link but also gets unwanted files.

Example url: **External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff**padsats.com" rel="noopener noreferrer nofollow" target="_blank">**External links are only visible to Support Staff**padsats.com/7f/8a/7f/7f8a7f352b9e9c8c51daf77b6941fa24.js
**External links are only visible to Support Staff**ajax.cloudflare.com" rel="noopener noreferrer nofollow" target="_blank">**External links are only visible to Support Staff**ajax.cloudflare.com/cdn-cgi/scripts/7089c43e/cloudflare-static/rocket-loader.min.js
**External links are only visible to Support Staff****External links are only visible to Support Staff**.ico
**External links are only visible to Support Staff****External links are only visible to Support Staff**.js
and a lots of other *.js files from the main host.

Any way to avoid these?
Reply With Quote
  #4  
Old 19.03.2020, 05:57
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

deep analyse will return everything that jd supports. Not sure why we return js files. putting that aside, you can specify with deepPattern field pattern to look for links.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #5  
Old 22.03.2020, 01:01
BJN01 BJN01 is offline
JD Adviser
 
Join Date: Jan 2020
Posts: 113
Default

maybe you could try setting some linkgrabber filters ( in the "X" tab).

es for *.js file:

-file type = js,ico
-url download = It contains = the main host

Last edited by BJN01; 22.03.2020 at 01:04.
Reply With Quote
  #6  
Old 22.03.2020, 08:36
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Quote:
Originally Posted by raztoki View Post
deep analyse will return everything that jd supports. Not sure why we return js files. putting that aside, you can specify with deepPattern field pattern to look for links.
Was unable to find any FAQ or online help for LinkCrawler>deepPattern.

Quote:
Originally Posted by BJN01 View Post
maybe you could try setting some linkgrabber filters ( in the "X" tab).

es for *.js file:

-file type = js,ico
-url download = It contains = the main host
Will try linkgrabber filter, thanks.
Reply With Quote
  #7  
Old 22.03.2020, 10:29
tony2long's Avatar
tony2long tony2long is online now
English Supporter
 
Join Date: Jun 2009
Posts: 6,507
Default

Not FAQ but examples:
https://board.jdownloader.org/showthread.php?t=74150
__________________
FAQ: How to upload a Log
Reply With Quote
  #8  
Old 22.03.2020, 10:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

the premise for deepPattern, is to only add a select component of the website to add links from so, open the source code of the website, create a regular expression pattern to match.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 23.03.2020, 18:46
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Yea... Got it:

Code:
"deepPattern" : "(<meta property=\"og:description\"[^>]*/>)",

I think linkCrawler should be migrated into a plugin

Last edited by reddevil; 23.03.2020 at 18:57. Reason: Thanks
Reply With Quote
  #10  
Old 24.03.2020, 16:26
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Need a bit of help:

The above deepPattern in LinkCrawler works fine for URL's listed in a get-to.link page. But sometimes there is a *.torrent file as well and JD refused to grab the link. So i added a New LinkCrawler rule:
Code:
{
"enabled" : true,
"pattern" : "**External links are only visible to Support Staff**,
"rule" : "DIRECTHTTP"
}
And afterwards JD grabs the torrent file but i am unable to automatically send them into a folder via Packagizer rule. My Packagizer rule:
Condition Name: _Torrents
File Type: is Custom Type torrent
File: is Online - Download is possible.
Package Name: _Torrents

I even tested directly (with URL **External links are only visible to Support Staff****External links are only visible to Support Staff**) at the bottom of the Rule config dialog (Play button to test filters) and matches column shows red x. The exact same packagizer rule works for apk file type.

Please advice.
Reply With Quote
  #11  
Old 24.03.2020, 16:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,913
Default

1. Please post an example URL which contains that .torrent URL inside html code.
Your rule will not work like that - please take some time to e.g. learn json structure and regular expressions (e.g. regex101.com, jsoneditoronline.org).

2. Please post a screenshot of your packagizer rule or export your packagizer rule, make a .zip archive out of it and upload it to our forum.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #12  
Old 24.03.2020, 16:53
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

*.* kinda looks like wildcard ;

try https?://[^/]+/.+\.torrent
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #13  
Old 24.03.2020, 17:21
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 1,533
Default

Quote:
Originally Posted by reddevil View Post
... i am unable to automatically send them into a folder via Packagizer rule ...
Instead of
Code:
File Type: is Custom Type torrent

Try:
Code:
File Name : contains : \.torrent$ (Enable Regex)
Reply With Quote
  #14  
Old 24.03.2020, 17:37
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Quote:
Originally Posted by pspzockerscene View Post
1. Please post an example URL which contains that .torrent URL inside html code.
Your rule will not work like that - please take some time to e.g. learn json structure and regular expressions (e.g. regex101.com, jsoneditoronline.org).

2. Please post a screenshot of your packagizer rule or export your packagizer rule, make a .zip archive out of it and upload it to our forum.

-psp-
Main url: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Screenshot:


Quote:
Originally Posted by raztoki View Post
*.* kinda looks like wildcard ;

try https?://[^/]+/.+\.torrent
After inserting the above into pattern field, JD threw an error.

Quote:
Originally Posted by mgpai View Post
Instead of
Code:
File Type: is Custom Type torrent

Try:
Code:
File Name : contains : \.torrent$ (Enable Regex)
Tried and works. Thank You.

Last edited by reddevil; 24.03.2020 at 17:41.
Reply With Quote
  #15  
Old 24.03.2020, 17:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,913
Default

So it is all working now or is there anything left to do?

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #16  
Old 24.03.2020, 18:31
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Quote:
Originally Posted by pspzockerscene View Post
So it is all working now or is there anything left to do?

-psp-
No, all working fine now. Just curiosity as to why filetype apk worked and torrent did not in the same packagizer rule.

Recap:- Rules in my Linkcrawler are below:
Code:
{
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "get-to.link example rule",
  "pattern" : "https?://get-to\\.link/.+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "(<meta property="og:description"[^>]*/>)",
  "rewriteReplaceWith" : null
}, {
  "enabled" : true,
  "name" : ".torrent file grab",
  "pattern" : "https?://[^/]+/.+.torrent",
  "rule" : "DIRECTHTTP",
}
My packagizer torrent rule:
Condition Name: _Torrents
File Name : contains : \.torrent$ (Enable Regex)
File: is : Online - Download is possible.
Package Name: _Torrents



Tested with links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Thanks to you All.

Last edited by reddevil; 25.03.2020 at 16:27.
Reply With Quote
  #17  
Old 16.07.2021, 15:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,913
Default

Added crawler plugin for get-to.link.

@reddevil
None of your recently posted URLs contain any URLs leading to files with extension ".torrent".
Your rule should work fine for files ending with ".torrent".

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #18  
Old 18.07.2021, 16:50
reddevil reddevil is offline
JD VIP
 
Join Date: Apr 2010
Posts: 369
Default

Some new links then to test:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #19  
Old 19.07.2021, 11:40
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,286
Default

Website uses Cloudflare and thus JDownloader fails to load the page
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 19.07.2021, 13:38
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,913
Default

More information regarding Cloudflare:
https://board.jdownloader.org/showthread.php?t=83712

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 17:02.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.