JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #2341  
Old 25.11.2016, 09:42
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

raztoki - Not interval issue !!
Issue @decrypter
Change the only method of execution. URL and Referer
This method works perfectly. Tested. it never blocks. Never say never.
Add 200 link. Decrypt 1.400.000 links / 2000 packages without block, so it works perfectly method.
Only minimal improvement method.
Code:
photo or album -> al_photos.php
raztoki - not to be confused with the previous issue @Hoster (Issue Speed Download)
I believe that King of The Plug Mr. pspzockerscene will know what I mean.
Reply With Quote
  #2342  
Old 25.11.2016, 10:03
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,627
Default

my previous comment was for your #2337 post, not the urls given above. Sorry I should have used quote.
that message directly implied x per second which is interval....

hence my answer.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #2343  
Old 25.11.2016, 14:21
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

raztoki - Just a POST request (but incorrect URL and address referrer)
any differences will always have an unexpected result in the form of a blockade,
Correct use of methods and solve all problems with blocks
Reply With Quote
  #2344  
Old 25.11.2016, 17:52
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

There is no correct or incorrect way - it is just how they try to detect "bots".

Aso ,I still have no idea what you mean.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA
Reply With Quote
  #2345  
Old 25.11.2016, 18:12
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Example:
Analyze empty albums / offline links = too fast to load pages = Does not work in this case, the "sleep" 1 sec!

Blocks because of invalid URL and Referer.
Please Set correctly URL and Referer and everything will work!
Advantages:
1. always superfast decrypt = never block!
2. It does not require increasing the "sleep" !
Reply With Quote
  #2346  
Old 25.11.2016, 19:14
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

Changed Referer.
I only tested it up to 2000 photos - I don't have the time to check this deeper but if it helps, you'll see it ...
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
Update cycles depend on your installed JD version!!

---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!
Die Updatezyklen variieren je nach installierter JD Version!!


GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA
Reply With Quote
  #2347  
Old 25.11.2016, 19:54
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

I updated, still wrong URL and Referer

Code:
 You tried to load the same page more than once in one second. Please go back and try again.
Also, a bug in the advanced settings!

Code:
Define sleep time for general pagination:
1000 ms

It does not work if the links "offline" or empty albums.
directhttp_jd.plugins.hoster.DirectHTTP.log-----------------------
--ID:180TS:1480095789189-16-11-25 18:43:09 - [jd.controlling.linkcrawler.LinkCrawler(processHostPlugin)] -> Processing: direct**External links are only visible to Support Staff** Here it should also be sleep 1000 ms (minimum)

or set correctly (better method API execute method) and the URL referer (up to 25 pages per 1 second! = never block)
Reply With Quote
  #2348  
Old 25.11.2016, 20:03
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

1. No idea what you mean - the Referer is definitly set.
Please post a log and show me exactly where it is wrong.

2. Nope, for offline urls there is no waittime thisd is NOT a bug!

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA
Reply With Quote
  #2349  
Old 25.11.2016, 20:10
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

2.
Too many request = Too many pages loaded within one second! = ALWAYS BLOCK !


7 pages loaded within one second. Current method allows a maximum of only one page per 1 second.
The best method (API Execute Method) up to 25 request per second = NEVER BLOCK!.

--ID:186TS:1480095789014-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789185-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789348-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789469-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789633-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789790-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
--ID:186TS:1480095789932-16-11-25 18:43:09 - [jd.http.Browser(loadConnection)] ->
Reply With Quote
  #2350  
Old 25.11.2016, 20:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

Stop trying to download so much from VK, this will for sure solve your issue.

Again JDownloader is not made for this.

I will not pput any time into optimizing JDownloader for your personal needs.

My idea for you:
Get yourself a programmer to change the plugin for your needs.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA
Reply With Quote
  #2351  
Old 25.11.2016, 20:17
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Quote:
Originally Posted by pspzockerscene View Post
no waittime
Example: Add 100 links offline, check the log, a lot of blocks!
Code:
Error: Too fast load page per second
offline (wait 1 second) [minimum]
online (wait 1 second) [minimum]

No believe? TeamViewer = Prove!
Reply With Quote
  #2352  
Old 25.11.2016, 20:18
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

Again:
For every normal user our vk.com plugin will work just fine.

Just look back the 115+ pages of this thread and you will notice that 99% of all these vk.com "bugreports" cam from you!

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA
Reply With Quote
  #2353  
Old 25.11.2016, 20:21
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Quote:
Originally Posted by pspzockerscene View Post
Stop trying to download so much from VK,
I do not download anything, all links offline!

How to solve all the problems are 2 solutions:
1. Use of better methods
OR
2. add sleep 1 second (minimum) for offline links (only source links)
Reply With Quote
  #2354  
Old 25.11.2016, 20:26
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

You need to think intelligently.
There was no need to change the method and to continue to work everything :)

Code:
Wrong:
----------------Request-------------------------
GET /album...._.... HTTP/1.1
:unsure:

Plese set
Code:
URL: POST /al_photos.php HTTP/1.1

Referer: **External links are only visible to Support Staff**
Please help!
Reply With Quote
  #2355  
Old 25.11.2016, 20:35
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Normal Users do not analyze errors in the log.
I just like to clean logs without errors.
I think this is the idea of a plug?

In theory it works fine, but if these blocks will be a lot of analysis will take a very long time!
Change the method GET to POST, the links offline
Thank you!
Reply With Quote
  #2356  
Old 25.11.2016, 20:42
ROLO ROLO is offline
Vacuum Cleaner
 
Join Date: Dec 2014
Posts: 19
Default

@pspzockerscene
issues 2(post #2314) and 3 confirmed? has been more than a week and no answer...

Last edited by ROLO; 25.11.2016 at 20:48.
Reply With Quote
  #2357  
Old 25.11.2016, 20:57
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Quote:
Originally Posted by pspzockerscene View Post
1. No idea what you mean - the Referer is definitly set.
Please post a log and show me exactly where it is wrong.

Issue Decrypter
See:
#2284
Satisfied because the previous method (settings) were perfect.
See my answer.
https://board.jdownloader.org/showpo...postcount=2287
Reply With Quote
  #2358  
Old 25.11.2016, 21:13
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Post #2287
100% work well

Again, make any changes
Post #2289
Again, do not work as expected.
sometimes blocks, and if the blocks are too many account block for
20 min - 2 hours!
if trying to analyze offline links (source) or source mixed (offline + online)
That's why I want to avoid these blocks by using other methods, which are recommended if the request uses multiple request in a short time (1 second)
Reply With Quote
  #2359  
Old 25.11.2016, 21:14
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 69,719
Default

#2353:
1. I don't see anything to improve.
2. The sleep is not needed.
We're open source - if you want it, go ahead and add that one line of code!

#2354, #2357, #2358:
No idea what you want from me here.

#2355:
I will not simply change a method from GET to POST - this is bullsh1t again.

@ROLO:
On it ...

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?


-----------------------------------
On Vacation / Im Urlaub
Start: 2023-12-09
End: TBA

Last edited by pspzockerscene; 25.11.2016 at 21:46.
Reply With Quote
  #2360  
Old 25.11.2016, 21:27
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

GET limited to only max. 1 request per second.
Need wait after each link 1 second.
Not support analyze a multiple links in one second!
I understand. If this can not be "easy change" I will have much longer to wait for the end of the analysis.

POST Request work max. 25 many request per second.

The end of my "ineffective" requests (suggestion) here.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 18:53.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.