View Single Post
  #6  
Old 31.07.2020, 12:35
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 54,924
Default

Quote:
Originally Posted by eMuse View Post
By "max simultan crawl instances", do you mean Settings -> Advanced Settings -> Link Crawler: Max Threads -> 1, and then restarting? Yes, this largely solves the problem (95% success after first crawl, 99% after adding a second crawl, 100% after 3rd or 4th crawl, which is much better than before).
No this is a general settings.
I do not mean this.

Quote:
Originally Posted by eMuse View Post
So it can't be individually set for this specific website, and you won't include it in the plugin settings, i.e. users should realize on their own that some items are missing in the LinkGrabber? Then maybe let the links with 503 error explicitly appear in the LinkGrabber, so that users know that those are missing?
No need to add settings for everything.

After all this is an edge case and we try to make our crawlers work for everyone so we do sometimes hardcode set such limitation.

The one for this crawler will be introduced after the next CORE update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
EDIT

Quote:
Originally Posted by raztoki View Post
thought it might be due to rate limiting or something, but 503 sounds like over loaded server. here there are local servers
As said, I was only able to reproduce it when just hammering their website.
Also it seems like it doesn't happen for everyone so yeah you might be right - overloaded servers and or also GEO related.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote