JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 16.12.2019, 13:37
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default crawl "operator" - issue

Which operator allows you to export unsupported html links first and then modify them to supported links?

directhttps
.jdeatime

Or other operators that I don't know about.
Reply With Quote
  #2  
Old 16.12.2019, 14:02
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,575
Default

.jdeatme
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 16.12.2019, 14:41
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

But if you add any html html page it will only show files with extension in the log.
Eg.
Code:
p_main_1.js
tag.js
How do you save the source of any HTML page?

Example:
Code:
<li><a href="URL"
view-source:**External links are only visible to Support Staff****External links are only visible to Support Staff**

How to export all links?
But I want to add a multi urls
Reply With Quote
  #4  
Old 17.12.2019, 05:45
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,575
Default

why are you using the log, and not the html that you wanted to save to disk?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #5  
Old 18.12.2019, 16:54
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Mr raztoki - So is it possible to save without a plugin?
Reply With Quote
  #6  
Old 20.12.2019, 12:13
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Code:
sitemap urlist.txt
I tested any page, I wanted to export all links from the page, but it turned out to be impossible.

For a long time I have been looking for a tool that will only export links, without having to save pages, which can be troublesome and time consuming.
It doesn't matter if this is a forum or website or anything but without javascript.

What is the meaning of the Open All URLs extension in the browser, fill it with thousands of tabs, and then choose export from the menu with each page. It will require a full day's work.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Quote:
An easy to use extractor or grabber for hyperlinks on an HTML page
You can also add list to another manager, but it opens thousands of pages in the taskbar, runs out of memory and use 100% CPU.
Reply With Quote
  #7  
Old 28.12.2019, 18:24
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Not work:

Example:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #8  
Old 29.12.2019, 02:59
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,575
Default

far as I can tell .jdeatme has to be before any ? or & key=values
i would recommend that you just use wget or curl be less fuss for you

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 29.12.2019, 10:48
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

This is for advanced users. I certainly won't know how to use the command line.

**External links are only visible to Support Staff****External links are only visible to Support Staff** (Unix System - Not Windows)

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #10  
Old 29.12.2019, 13:37
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,575
Default

yes command line, but its very simple
wget url
curl url

I've had both running on windows without any dramas in the past.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #11  
Old 29.12.2019, 14:15
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Cannot add full URL.
It is automatically shortened! &s=50
Please fix this.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 05:11.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.