#21
|
|||
|
|||
@pspzockerscene
Great job! Can I ask you what you change to retrieve text from that tag ? What you write ? |
#22
|
||||
|
||||
@nathan1: he updated the native plugin to fetch the title as you want it
__________________
JD-Dev & Server-Admin |
#23
|
|||
|
|||
@pspzockerscene
Seems that in your last update you disable my last fetching request Please see LOG Code:
18.11.20 09.16.21 <--> 18.11.20 09.15.44 jdlog://9615425302851/ |
#24
|
||||
|
||||
@nathan1: was my fault, wait for next update
__________________
JD-Dev & Server-Admin |
#25
|
|||
|
|||
@Jiaz
ok,thanks |
#26
|
|||
|
|||
@Jiaz & psp
I see that plugin have problems to decrypt and fetch <title> names if links are from file.karelia.ru If you test these urls **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** you return this strange folder names Code:
923zjt rvw36q wn93q9 qgw44f gv79df LOG Code:
19.11.20 14.03.23 <--> 19.11.20 14.31.10 jdlog://9645425302851/ |
#27
|
||||
|
||||
Updated our file.karelia.ru crawler to only set packagenames for folders with more than 1 items.
Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#28
|
|||
|
|||
host url missing from Copy Information (Action)
I think that there are some problems with CopyToClipboard Action via rightclick -> context menu
I add these additional tags in (Action) Copy Information Code:
{type};{name};{url};{url.container};{url.origin};{packagename};{url.referrer} **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** Inside these pages are links like **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** but all external hoster's url like yadi.sk or sampo.ru are missing from (Action) Copy Information. These information don't appears. When I copy information What additional tag do I need? These are insufficient Code:
{type};{name};{url};{url.container};{url.origin};{packagename};{url.referrer} Code:
21.11.20 03.26.47 <--> 21.11.20 03.46.28 jdlog://4695425302851/ Last edited by Jiaz; 21.11.2020 at 13:25. |
#29
|
|||
|
|||
The data returned by {url} will vary, depending on the url sort order in Settings > User Interface. Use {url.content} instead, to get the download url.
|
#30
|
|||
|
|||
Thank you !
|
#31
|
||||
|
||||
@mgpai: Thanks for the fast and correct help
__________________
JD-Dev & Server-Admin |
#32
|
|||
|
|||
@psp
I follow your tips from your last post of khinsider to apply for metalarea Quote:
Note: clipboard observer is enabled LOG Code:
01.09.22 23.15.02 <--> 01.09.22 23.52.12 jdlog://7369211370661/ In theory it shouldn't just show me the package name only (and this is great), but without having to check any links in it? Why check me anyway if the files are offline even after disabling the linkcollector check? I'm only interested in coming up with the name of the packages, which it does correctly and only later can I check their status online - manually or when I download them. |
#33
|
|||||
|
|||||
Quote:
Quote:
- All of those 500 URLs - Your metalarea username + password Quote:
This can't be turned off as JD would simply do nothing then Also some crawlers will do the linkcheck right away because they've accessed the URL already so yes even with disabled linkcheck, some items will be displayed as online/offline with filename/filesize information set. Quote:
Quote:
I can imagine that some of your metalarea links go to a 404 error-page which the crawler will detect and display them as offline. This is not a real "online check" but as explained the crawler needs to access the added link anyways so that is not a bug. Also, a package in JDownloader cannot be empty it needs to contain at least one item.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#34
|
|||
|
|||
@psp
ok, the problem is that it does not detect many links: if you copy and paste all 500 links in bulk, JDownloader at some point freezes and "runs idle" for 1-2 minutes and in any case does not scan all the links. Instead, if you copy and paste 10-20 links it scans them and detects all 20. What I don't understand is why the scan is conditioned by the number of links. Maybe there is some setting or timer to improve? I sent you credentials + 500 urls metalarea links |
#35
|
||||
|
||||
Quote:
If you're frequently running into rate-limits, consider adding links with a delay of X seconds. This should be easily possible using external scripts or even an EventScripter script: https://support.jdownloader.org/Know...event-scripter Quote:
I'll check it...
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#36
|
||||
|
||||
It was indeed a rate-limit.
Metalarea will return http error-code 400 when that limit is reached. For the next update I've added some measures to try to prevent running into the rate-limit. These measures include: - Wait 1000ms between requests - Limit max simultan crawler instances for metalarea to 1 - Return dummy URLs for retrying in case rate-limit is hit Bitte auf das nächste CORE-Update warten! Please wait for the next CORE-Update! Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#37
|
|||
|
|||
@psp
Thank you for this update |
#38
|
||||
|
||||
Seems like is not a classic rate limit neither is it a sophisticated one:
They're simply blocking your current session after X (about 250) requests no matter how fast these requests are performed. I'm still working on it but I think I should be able to add auto handling and remove the requestInterval so the crawler can crawl fullspeed.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#39
|
||||
|
||||
Done. Rate-limit for metalarea links shouldn't be a problem anymore after the next set of plugin updates.
Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#40
|
|||
|
|||
@psp
Thank you very much ! |
Thread Tools | |
Display Modes | |
|
|