JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 13.07.2017, 09:26
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Wink checking links suggestion

Regards,

JD is verifying unnecessary (re-checking) even after it checked the links,

i mean
i have these links:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

since JD maintain the source which is short.am and adds the final link path, my guess is that is NOT necessary to re-check over and over again, i have them already in link grabber tab, if i copied those links again for any reason, JD will start like crazy verifying again and again, conclusion: definitively this needs to be improved

other suggestion, to avoid using other thread, is the need to Add links to package name already created, from Other ----- move to new package , there should be other option like: move to another package name
this will help to make links better organized

My Best Regards

Last edited by Rickz; 13.07.2017 at 09:40. Reason: added other suggestion
Reply With Quote
  #2  
Old 13.07.2017, 11:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

I've created a ticket for this idea
But it has many flaws!
-the more links you have in list, the MORE SLOW it all will be. Imagine you have 1000 links in it and you want to add 10 links. This means JDownloader has to check 10*1000 links. this becomes very slow very fast

-imagine shorter services that return more than 1 link. now you delete some of the returned links. with your check in place you no longer can add the remaining links, because dupe check would detect the link as already parsed.


You can move Links/Packages via Drag/Drop No need for another option
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 13.07.2017, 13:46
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

also another scenario
some link services allow uid to be edited by creator, then you get situation where you need to be able to decrypt the link again in order to get the decrypted link(s)
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #4  
Old 13.07.2017, 20:39
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

most of the time are created again, most scenarios i've seen are replaced with new links, some are reported so for safety it's good to replace start and end links
Reply With Quote
  #5  
Old 13.07.2017, 20:49
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Talking

-the more links you have in list, the MORE SLOW it all will be. Imagine you have 1000 links in it and you want to add 10 links. This means JDownloader has to check 10*1000 links. this becomes very slow very fast

-imagine shorter services that return more than 1 link. now you delete some of the returned links. with your check in place you no longer can add the remaining links, because dupe check would detect the link as already parsed.

You can move Links/Packages via Drag/Drop No need for another option


i have many link, with today's processors speed and memory i don't see really a problem, running with quadcore 2.40 ad 4 GB of ram from XP, it's an old machine and runs fine, i'm sure many people has more links than me, i don't start them all together and some are disabled, having organized is really fine and i will try the drap and drop feature and will reply back, option is good because i can use a package that has been already created for X type or category of links.

of course i understand there are some link that has multiple link associate, for that case the user most decide what to do, a check mark option to ignore dupe check activated or not, end user decide for best results or case scenario
Reply With Quote
  #6  
Old 13.07.2017, 22:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Do not underestimate the overhead of n times y
For example you have 10k links in list and add 100 links
First link will check against 10k links, second link checks 10k + 1 link and so on. This becomes a lot very fast.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 13.07.2017, 22:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Quote:
Originally Posted by Rickz View Post
of course i understand there are some link that has multiple link associate, for that case the user most decide what to do, a check mark option to ignore dupe check activated or not, end user decide for best results or case scenario
That's a good idea!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 14.07.2017, 02:03
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

:biggrin: ihave many links but never 5k, i got that part:biggrin:
Reply With Quote
  #9  
Old 14.07.2017, 06:09
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

about the package names, after adding some links to a single package name, all downloaded files are scattered in main download folder without the package name, when you have 20 links or little more this is a mess, could this be improved ??

Last edited by Rickz; 14.07.2017 at 06:13.
Reply With Quote
  #10  
Old 14.07.2017, 06:20
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

enable sub directory as packagename within settings > packagiser

this works on entry into linkgrabber.

you can edit existing entries via selecting all packagenames and edit download save path, and click sub directory as packagename checkbox.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #11  
Old 14.07.2017, 10:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

When you move links to a package, all will go to the download folder of the package.
Downloadfolder are assigned to packages and not links!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 14.07.2017, 10:34
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Wink

normally when i download several part of same archive, like part01 part02, part03.rar

all parts goes(download) into same package name (folder name), i wonder if adding several links(files) into same package name, could automatically make a sub dir into main download folder with all files like the case of different parts..., did a explain myself? could this be done ?
Reply With Quote
  #13  
Old 14.07.2017, 10:55
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

as Jiaz and I indicated above, subdirectory as packagename happens on entry into linkgrabber. And packages store save path, not the links. If the package doesn't contain any use of use subdirectory as packagename you will need to enable it via the changing path editor (right click context) as I mentioned above.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #14  
Old 14.07.2017, 11:13
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Wink

thanks for support!!!

in link grabber i have a package name like: Some files

in some files i have like 20 files(links)

when i check the properties of that package name says Main download folder: G:\Jdownloader

my guess it should be: G:\Jdownloader\Some files

i'm wrong?

as per your instructions i can select all those link/files of that package name and set a download directory which in this case will be: G:\Jdownloader\Some files

as per your instructions,even if package download path from link grabber says: G:\Jdownloader

when adding to download list, all files will download to: G:\Jdownloader\Some files

is that correct?

ps in packagizer setting i have always activated the 4 main rules by default, should i add other rule for this task ? idea?
Reply With Quote
  #15  
Old 14.07.2017, 13:06
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

if you 'move to new package' it may or may not contain the subdirectory as packagename option depending on what you chose.

I do not use the bottom back overview, takes up too much screen space when the linkgrabber/download table provides the exact same information. I use the change path dialog via hotkey or double clicking quickly in table it will trigger dialog.

if you enable sub directory as packagename and add a new link, yes it be default save path + packagename.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #16  
Old 16.07.2017, 18:18
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Smile

yes it does!!!!! thanks, this way i got all files organized automatically, i was doing manually after download finished

any idea how to delete offline links from selected packages/links

only saw to delete missing splitted links from clean up, i have around 100+ offline links in different packages

a handy option will be: delete offlines links from all packages, doing this manually is a pain and much time consuming

should i request this feature ?

My Best Regards

Last edited by Rickz; 16.07.2017 at 18:29.
Reply With Quote
  #17  
Old 16.07.2017, 18:41
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

download tab ,
- use the filter on the bottom bar 'all downloads' change to 'offline'
- cleanup functions from right click context/buttons
linkgrabber,
- side panel, click the offline view, it will auto expand and select the offline items you can quickly review and press delete key or use other GUI functions
- cleanup functions from right click context/buttons
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #18  
Old 16.07.2017, 19:16
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Cool

thanks again, i will check this,

one more question, does full backup saves also links contained in linkgrabber tab ???

My Best Regards
Reply With Quote
  #19  
Old 16.07.2017, 19:17
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

backup effectively zips the /cfg directory
so that would be a big fat yes. =]

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #20  
Old 16.07.2017, 20:18
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Question

other think, JD is not recognizing links with unknown extensions, like .rar0 and .r_r and maybe others

is there anyway to handle this? maybe writting a rule ? linkgrabber is ignoring some links with those wrong written extensions, i'm trying to download some manuals from a website but need to get them all, when link grabber finished i found out i'm missing some links.
any ideas??

links for testing:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

tutorial_principiante_NET+tool_RSA.rar0


**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Rickz; 16.07.2017 at 20:54. Reason: links added
Reply With Quote
  #21  
Old 17.07.2017, 05:06
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

that's because they are not valid rar extension formatting
you will need to rename them to .rar

you could use regular expression in the search bar say '.+(?!\.rar)', that will find all thats not rar
linkgrabber side panel will also have 'file types' they wont fall under anything so be placed in 'others'
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #22  
Old 17.07.2017, 18:18
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

You can enforce JDownloader to *eat* those urls either
1.) copy and PASTE into JDownloader
2.) drag and DROP into JDownloader
3.) teach JDownloader to auto add those file extensions with normal copy and AddLinks Dialog and JDownloader will ask to deep decrypt them
4.) use Linkcrawler rules to teach JDownloader those file extensions
__________________
JD-Dev & Server-Admin
Reply With Quote
  #23  
Old 18.07.2017, 03:35
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

i added a rule in linkgrabber filter: Views/File type is custom type: *.r_r,*.rar0

still dont add those links, i checked others and does not have those links

any ideas?
Reply With Quote
  #24  
Old 18.07.2017, 03:53
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

the rule is doing nothing, i also added some more parameters, like download url contain: *.r_r *.rar0 and come from clipboard, this do nothing, only work in download tab down under where says Add new links / add and deep analyze links from clipboard
Reply With Quote
  #25  
Old 18.07.2017, 10:01
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Linkgrabber Filter is a rule to FILTER links, not to add support for links

Please see my 4 options to add those links
https://board.jdownloader.org/showpo...2&postcount=22
__________________
JD-Dev & Server-Admin
Reply With Quote
  #26  
Old 18.07.2017, 11:06
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

understood, but what about adding support for links with unknown files type?

My Best Regards
Reply With Quote
  #27  
Old 18.07.2017, 11:36
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Works the same way. r_r or rar0 is an unknown file type/extension
__________________
JD-Dev & Server-Admin
Reply With Quote
  #28  
Old 18.07.2017, 14:20
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

where can i set a rule for this case or future ones.... ?

so far only deep link extraction seems to work


Regards

Last edited by Rickz; 18.07.2017 at 14:26.
Reply With Quote
  #29  
Old 18.07.2017, 14:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

The most easiest one is to use the auto learn function
Just copy the url with unsupported file extension and click AddLinks Dialog, then continue and JDownloader will ask to deep decrypt.
Now it should have auto learned the extension and you can find the rule in
Settings-Advanced Settings-LinkCrawler.linkcrawlerrules
there you can also modify the rule set and add your own rules

Use board search for more information and examples.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #30  
Old 19.07.2017, 02:09
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

i don't see option to modify it or '"add rules" only a check mark to activate/deactivate


see: http://imgur.com/a/YHI9p
Reply With Quote
  #31  
Old 19.07.2017, 04:47
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

rule below active is empty { }

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #32  
Old 19.07.2017, 05:08
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Wink

how you add a rule there?

it was supposed to learn and the rule will be there, it's obvious this needs to be feed manually

the rule should be something generic, i believe will be better a plugin, once activated will add links with unknown file extensions , user most activated for the needs

thanks in advance

Last edited by Rickz; 19.07.2017 at 05:13.
Reply With Quote
  #33  
Old 19.07.2017, 06:03
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,674
Default

Jiaz implied that it should be auto added to crawler rule. Need to wait for his response.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #34  
Old 19.07.2017, 16:01
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Thanks for the example links. Please wait for next core update, then auto learn will work for your links as well.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #35  
Old 20.07.2017, 01:23
Rickz Rickz is offline
JD Addict
 
Join Date: Nov 2016
Posts: 159
Default

thanks you all for every support and such great program JD (The Best) !!!!
Reply With Quote
  #36  
Old 20.07.2017, 10:06
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

You're welcome
__________________
JD-Dev & Server-Admin
Reply With Quote
  #37  
Old 04.09.2018, 14:35
Derp Derp is offline
Registered / Inactive
 
Join Date: Oct 2011
Posts: 336
Default

Quote:
Originally Posted by Jiaz View Post
You can enforce JDownloader to *eat* those urls either
1.) copy and PASTE into JDownloader
2.) drag and DROP into JDownloader
3.) teach JDownloader to auto add those file extensions with normal copy and AddLinks Dialog and JDownloader will ask to deep decrypt them
4.) use Linkcrawler rules to teach JDownloader those file extensions
I'm fairly sure this automatic learning is broken.

E.g. **External links are only visible to Support Staff****External links are only visible to Support Staff**

Just copying this URL doesn't make JD2 add it to the LinkGrabber-list.
Ctrl+V or using "Add Links" triggers JD2 into adding this URL into the LinkGrabber-List once, but copying another URL with that extension, like **External links are only visible to Support Staff****External links are only visible to Support Staff** , and again nothing is added.

Plus it's not possible to edit the list of learned extension anymore, which was possible quite a while ago.

I'd very much prefer an option, maybe in the Advanced Settings, to grab every extension regardless if it's known or not.
Reply With Quote
  #38  
Old 04.09.2018, 19:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Issue is cased by server responding with *text/plain* as content. will add better support for this. wait for next core update
You can still edit the list in Settings-Advanced Settings-LinkCrawler.linkcrawlerrules
__________________
JD-Dev & Server-Admin
Reply With Quote
  #39  
Old 04.09.2018, 19:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,228
Default

Quote:
Originally Posted by Derp View Post
I'd very much prefer an option, maybe in the Advanced Settings, to grab every extension regardless if it's known or not.
You can create such a rule manually but I suggest NOT to as this will cause JDownloader to eat *EVERY* url you copy....EVERY!
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 06:57.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.