JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 17.03.2020, 09:32
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default [LinkCrawler rule] Host plugin for get-to.link

Hi JD Team,

Please add support for host get-to.link, which displays a collection of links. I think it was linx.cloud before but doesn't get redirected, since linx.cloud is dead.
Sample links below:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

As usual, my Sincere Thanks to you guys.

Regards,
Doc MaKS aKa reddevil
Reply With Quote
  #2  
Old 17.03.2020, 11:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,451
Default

Hi,

a plugin for this simple website is not needed.
You can simply add a link crawler rule to let JD automatically crawl contents of this website e.g. via clipboard detection:
Code:
[ {
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "get-to.link example rule",
  "pattern" : "https?://get-to\\.link/.+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : null,
  "rewriteReplaceWith" : null
} ]
-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 19.03.2020, 04:39
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Above link crawler rule gets the download links fine from get-to.link but also gets unwanted files.

Example url: **External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff**padsats.com" target="_blank">**External links are only visible to Support Staff**padsats.com/7f/8a/7f/7f8a7f352b9e9c8c51daf77b6941fa24.js
**External links are only visible to Support Staff**ajax.cloudflare.com" target="_blank">**External links are only visible to Support Staff**ajax.cloudflare.com/cdn-cgi/scripts/7089c43e/cloudflare-static/rocket-loader.min.js
**External links are only visible to Support Staff****External links are only visible to Support Staff**.ico
**External links are only visible to Support Staff****External links are only visible to Support Staff**.js
and a lots of other *.js files from the main host.

Any way to avoid these?
Reply With Quote
  #4  
Old 19.03.2020, 04:57
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,976
Default

deep analyse will return everything that jd supports. Not sure why we return js files. putting that aside, you can specify with deepPattern field pattern to look for links.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #5  
Old 22.03.2020, 00:01
BJN01 BJN01 is offline
JD Beta
 
Join Date: Jan 2020
Posts: 56
Default

maybe you could try setting some linkgrabber filters ( in the "X" tab).

es for *.js file:

-file type = js,ico
-url download = It contains = the main host

Last edited by BJN01; 22.03.2020 at 00:04.
Reply With Quote
  #6  
Old 22.03.2020, 07:36
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Quote:
Originally Posted by raztoki View Post
deep analyse will return everything that jd supports. Not sure why we return js files. putting that aside, you can specify with deepPattern field pattern to look for links.
Was unable to find any FAQ or online help for LinkCrawler>deepPattern.

Quote:
Originally Posted by BJN01 View Post
maybe you could try setting some linkgrabber filters ( in the "X" tab).

es for *.js file:

-file type = js,ico
-url download = It contains = the main host
Will try linkgrabber filter, thanks.
Reply With Quote
  #7  
Old 22.03.2020, 09:29
tony2long's Avatar
tony2long tony2long is offline
English Supporter
 
Join Date: Jun 2009
Posts: 6,349
Default

Not FAQ but examples:
https://board.jdownloader.org/showthread.php?t=74150
__________________
FAQ: How to upload a Log
Reply With Quote
  #8  
Old 22.03.2020, 09:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,976
Default

the premise for deepPattern, is to only add a select component of the website to add links from so, open the source code of the website, create a regular expression pattern to match.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 23.03.2020, 17:46
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Yea... Got it:

Code:
"deepPattern" : "(<meta property=\"og:description\"[^>]*/>)",

I think linkCrawler should be migrated into a plugin

Last edited by reddevil; 23.03.2020 at 17:57. Reason: Thanks
Reply With Quote
  #10  
Old 24.03.2020, 15:26
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Need a bit of help:

The above deepPattern in LinkCrawler works fine for URL's listed in a get-to.link page. But sometimes there is a *.torrent file as well and JD refused to grab the link. So i added a New LinkCrawler rule:
Code:
{
"enabled" : true,
"pattern" : "**External links are only visible to Support Staff**,
"rule" : "DIRECTHTTP"
}
And afterwards JD grabs the torrent file but i am unable to automatically send them into a folder via Packagizer rule. My Packagizer rule:
Condition Name: _Torrents
File Type: is Custom Type torrent
File: is Online - Download is possible.
Package Name: _Torrents

I even tested directly (with URL **External links are only visible to Support Staff****External links are only visible to Support Staff**) at the bottom of the Rule config dialog (Play button to test filters) and matches column shows red x. The exact same packagizer rule works for apk file type.

Please advice.
Reply With Quote
  #11  
Old 24.03.2020, 15:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,451
Default

1. Please post an example URL which contains that .torrent URL inside html code.
Your rule will not work like that - please take some time to e.g. learn json structure and regular expressions (e.g. regex101.com, jsoneditoronline.org).

2. Please post a screenshot of your packagizer rule or export your packagizer rule, make a .zip archive out of it and upload it to our forum.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #12  
Old 24.03.2020, 15:53
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,976
Default

*.* kinda looks like wildcard ;

try https?://[^/]+/.+\.torrent
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #13  
Old 24.03.2020, 16:21
mgpai mgpai is offline
Script Master
 
Join Date: Sep 2013
Posts: 853
Default

Quote:
Originally Posted by reddevil View Post
... i am unable to automatically send them into a folder via Packagizer rule ...
Instead of
Code:
File Type: is Custom Type torrent

Try:
Code:
File Name : contains : \.torrent$ (Enable Regex)
Reply With Quote
  #14  
Old 24.03.2020, 16:37
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Quote:
Originally Posted by pspzockerscene View Post
1. Please post an example URL which contains that .torrent URL inside html code.
Your rule will not work like that - please take some time to e.g. learn json structure and regular expressions (e.g. regex101.com, jsoneditoronline.org).

2. Please post a screenshot of your packagizer rule or export your packagizer rule, make a .zip archive out of it and upload it to our forum.

-psp-
Main url: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Screenshot:
**External links are only visible to Support Staff**

Quote:
Originally Posted by raztoki View Post
*.* kinda looks like wildcard ;

try https?://[^/]+/.+\.torrent
After inserting the above into pattern field, JD threw an error.

Quote:
Originally Posted by mgpai View Post
Instead of
Code:
File Type: is Custom Type torrent

Try:
Code:
File Name : contains : \.torrent$ (Enable Regex)
Tried and works. Thank You.

Last edited by reddevil; 24.03.2020 at 16:41.
Reply With Quote
  #15  
Old 24.03.2020, 16:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,451
Default

So it is all working now or is there anything left to do?

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #16  
Old 24.03.2020, 17:31
reddevil reddevil is offline
Black Hole
 
Join Date: Apr 2010
Posts: 259
Default

Quote:
Originally Posted by pspzockerscene View Post
So it is all working now or is there anything left to do?

-psp-
No, all working fine now. Just curiosity as to why filetype apk worked and torrent did not in the same packagizer rule.

Recap:- Rules in my Linkcrawler are below:
Code:
{
  "enabled" : true,
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "get-to.link example rule",
  "pattern" : "https?://get-to\\.link/.+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "(<meta property="og:description"[^>]*/>)",
  "rewriteReplaceWith" : null
}, {
  "enabled" : true,
  "name" : ".torrent file grab",
  "pattern" : "https?://[^/]+/.+.torrent",
  "rule" : "DIRECTHTTP",
}
My packagizer torrent rule:
Condition Name: _Torrents
File Name : contains : \.torrent$ (Enable Regex)
File: is : Online - Download is possible.
Package Name: _Torrents

**External links are only visible to Support Staff**

Tested with links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Thanks to you All.

Last edited by reddevil; 25.03.2020 at 15:27.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 08:35.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.