JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 13.06.2021, 16:58
ELM0 ELM0 is offline
Junior Loader
 
Join Date: Mar 2019
Posts: 13
Question Link crawler help needed

Hello,

I need help creating a link crawler rule for a selfhosted webapp. The webapp is called "**External links are only visible to Support Staff**Tgindex Pro", it crawls through your telegram channels, indexes and displays the files on a website, so they can be easily downloaded with a download manager.

Unfortunatelly JD2 does not recognize the links, even though they are normally embedded into the html page.

Link Schema:

Channel overview with pages containing links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Example links to crawl:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
...

Example links to _not_ crawl: (they dont work)
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
...


There is a "Page 2" button and after clicking it, it gives a "Page 3" button and so on, after each click.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
...
Reply With Quote
  #2  
Old 14.06.2021, 13:08
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 71,445
Default

Please provide real working example links, then we can check/try to help
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 16.06.2021, 02:32
ELM0 ELM0 is offline
Junior Loader
 
Join Date: Mar 2019
Posts: 13
Default

Quote:
Originally Posted by Jiaz View Post
Please provide real working example links, then we can check/try to help
Sorry, I cant make it run on the Internet. I will keep trying to figure it out with link crawler myself. Still, thanks for the quick reply.

Last edited by ELM0; 16.06.2021 at 03:34.
Reply With Quote
  #4  
Old 16.06.2021, 09:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 71,445
Default

We can try to help via Teamviewer and check together. If you want, contact us via support@jdownloader.org
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 16.06.2021, 16:16
ELM0 ELM0 is offline
Junior Loader
 
Join Date: Mar 2019
Posts: 13
Default

I will try to provide the website and re-open the thread if help still needed. thank you for your kind support.

Last edited by ELM0; 16.06.2021 at 16:38.
Reply With Quote
  #6  
Old 16.06.2021, 17:51
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 71,445
Default

@ELM0 nothing to thank us!
We can still try to help via Teamviewer or get somehow us access to an instance of that website, so we can check/help you
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 19.06.2021, 15:22
ELM0 ELM0 is offline
Junior Loader
 
Join Date: Mar 2019
Posts: 13
Default

SOLVED. I figured it out. I had to use DEEPCRYPT mode and use the correct regex. JD2 is really awesome. :o)
Reply With Quote
  #8  
Old 20.06.2021, 09:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 71,445
Default

Great to hear that you got it working by yourself
Whenever you need help/got questions, just let us know!
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 14:15.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.