JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1721  
Old 02.11.2015, 21:29
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

editestowy - Moesz sobie darowac wycieczki w moj strone?, nikt nie jest nieomylny...

editestowy - jeli chcesz mi pomc to napisz mi jak rozwiza ten problem.
Dodaje prawidowy login i haso, ale pokazuje, e zy.
Poanddto bd 999 pojawia si jeli powoduje zaptlenie logowania, czyli loguj, wyloguj, loguj, wyloguj i tak w kko... wida w logach, e non-stop prbuje cay czas te same czynnoci wykona... co moe nie by dozwolone....



1. Issue 1 login - Solved!

2. Issue 2

<a id="photos_load_more" onclick="photos.load()" style="display: none"><span>Show more photos</span>
<div id="photos_more_progress" class="progress"></div>

--ID:155TS:1446498155300-11/2/15 10:02:35 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498169389-11/2/15 10:02:49 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498175510-11/2/15 10:02:55 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 9 seconds
--ID:155TS:1446498188647-11/2/15 10:03:08 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498155300-11/2/15 10:02:35 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498169389-11/2/15 10:02:49 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498175510-11/2/15 10:02:55 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 9 seconds
--ID:155TS:1446498188647-11/2/15 10:03:08 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498155300-11/2/15 10:02:35 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498169389-11/2/15 10:02:49 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498175510-11/2/15 10:02:55 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 9 seconds
--ID:155TS:1446498188647-11/2/15 10:03:08 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498155300-11/2/15 10:02:35 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498169389-11/2/15 10:02:49 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds
--ID:155TS:1446498175510-11/2/15 10:02:55 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 9 seconds
--ID:155TS:1446498188647-11/2/15 10:03:08 PM - [jd.plugins.decrypter.VKontakteRu(getPage)] -> You tried to load the same page more than once in one second, sleep 6 seconds

delay = 100 % work

no delay = always error


**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Lram32; 05.11.2015 at 19:31. Reason: merged posts
Reply With Quote
  #1722  
Old 03.11.2015, 01:23
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

@djmakinera

Quote:
Originally Posted by djmakinera View Post
Very complex bug...
always reproduce if help by TeamViewer.

Please tell me what kind of strange characters
Invalid HTTP Header


}›v›H’*»¿"‹=6¥nI…I’W¾•«·¶åru[jž a‘
€º˜¥˜ù*ݧy˜·}¯3»½ÏìuŒ™‡3‘™ H‘²d»\ž²ˆ¼FFFFFDfDžê“;{¿}z’‘<}qû7wHm½‘xiži4%?˜{èyžD¾“4Hm$ãFãøøxã ˜£~cïYã[‘±š¹:nv®œŒ†A¼]‘€žvE½’*¿]cAd¿
F]ø3b %˜†:{3ñ¶k¯¿˜]¿Ž†4ñ!«'$›µom3·)
7ƒC±v-„QLCDŒ›®5³‘#Ln€[&¯¬¦ãñ*'¬W5SF~Ï’6†A¿¶3¿jì¿e€’¶u’¶f4¶–hH7øoFS³L[-ãþ›•VːmUL‰œõt¬GN’Nq4ŠY²}ìnx¯FKèˆm\;‘?N|„*kEH'C¶m8ˆ3:²ƒ›
‘YA(êLCC‰X<Pš¶ˆ‹g·6u@a‘†x0®ld@“q¼¦GTŽhDŽ6ywꆽ(Lhz>t‘&ƒÿ*a6|qJ¬ ’~ MŒñ],*¶n<!Hšl’º^'·HM™$†saŒy@©N8ªc

that is the body of the request when 999 response code is given, Please do not bring up the same stuff over and over. As stated it's a invalid response code given by VK!


Quote:
Originally Posted by djmakinera View Post
delay = 100 % work

no delay = always error


**External links are only visible to Support Staff****External links are only visible to Support Staff**
earlier you said sleeps (delays) wasn't needed o_O

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]

Last edited by raztoki; 03.11.2015 at 01:27.
Reply With Quote
  #1723  
Old 03.11.2015, 02:08
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

sleep - No


delay - Yes !


delay <===> sleep This is not the same !!!

Understound?

Good
request per second- 1 link


Bad (Plugin) request per second - 3 link

3 link = request per second = You try to load multiple pages at the same time

Last edited by Lram32; 05.11.2015 at 19:32. Reason: merged posts
Reply With Quote
  #1724  
Old 03.11.2015, 02:12
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

sleep and delay is the same thing....

once AGAIN, the plugin only allows 1 decryption task at once! Thus can not process three at a time.

plugin does allow burst rate as I stated the other day, and this is where the issue was introduced after you complained the decrypting task was too slow! Thus you introduced what ever issues are here now back in august.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]

Last edited by raztoki; 03.11.2015 at 02:14.
Reply With Quote
  #1725  
Old 03.11.2015, 02:19
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

if plugin parse one page per second is not required to add sleep because there will appear an error ... trying to load 3 pages within one second - how many times do I repeat, explain well how much?

parse 1 page per second = No error
parse 3 page per second = Error

VK plugin does not use special execute method, so reduce to only one REQUEST should avoid virtually any Block!

VK plugin uses "entire page", so requests can be only one

VK plugin does not use the full API so 3 requests per second in this case cause the lock.
Developer can add up to 100,000 seconds "sleep" - it will not change anything - it can only delay it if errors occur.

Errors will appear endlessly because loads 3 pages within seconds!
I have already said many times - but no one can understand it.

3 pages within seconds! = multipages = Multiple

You are trying to load multiple pages per second

Last edited by Lram32; 05.11.2015 at 19:32. Reason: merged posts
Reply With Quote
  #1726  
Old 03.11.2015, 07:14
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

I understand what you've said, even though you seem to keep bringing up the same stuff over and over. Once again burst rate will ALLOW multiple browser requests in short succession. This was a direct result of you saying that decrypter was too slow in August...

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #1727  
Old 05.11.2015, 10:32
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

You are trying to load multiple pages per second, most of the links Offline


05.11.15 08.59.09to05.11.15 09.21.28 jdlog://6768834739341/




**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Lram32; 05.11.2015 at 19:33. Reason: merged posts
Reply With Quote
  #1728  
Old 05.11.2015, 11:02
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

please refrain from making vk related posts until psp has committed his source changes. Thread will be re-opened when he makes his changes and not before!
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]

Last edited by raztoki; 05.11.2015 at 11:06.
Reply With Quote
  #1729  
Old 05.11.2015, 14:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,796
Default

Added settings for all waittimes we have.
Changed global waits back to before jiaz' changes in august.
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
Update cycles depend on your installed JD version!!

---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollstndig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren wrde!
Die Updatezyklen variieren je nach installierter JD Version!!


GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #1730  
Old 05.11.2015, 14:41
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Jiaz changed (improved) speed decrypt videos| videos-
Of course, improved speed crawl and without errors

but at the same time cause any impact on other settings (too fast) for albums | albums- | album | album-

That is why there were different issues:
-Video (speed issue)
-Album (speed issue)
Reply With Quote
  #1731  
Old 05.11.2015, 14:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,796
Default

No - please stop repeating what you have posted.
Thanks.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #1732  
Old 05.11.2015, 15:05
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Sleep does not help - understand it?
Many errors: You tried to load the same page more than once in one second.<br>Please go back and try again.<br>


Ask to change the number of applications (advanced settings) - Thanks

Ask to change the number of request (advanced settings) - Thanks



EXAMPLE setting

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Edit:
Please don't use "[img]" for these images, won't work.

Last edited by Lram32; 05.11.2015 at 19:33. Reason: merged posts
Reply With Quote
  #1733  
Old 05.11.2015, 15:59
tony2long's Avatar
tony2long tony2long is offline
English Supporter
 
Join Date: Jun 2009
Posts: 6,510
Default

Do you want to customize setRequestIntervalLimitGlobal?
__________________
FAQ: How to upload a Log
Reply With Quote
  #1734  
Old 05.11.2015, 16:03
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

Quote:
Originally Posted by tony2long View Post
Do you want to customize setRequestIntervalLimitGlobal?
Code:
[jd.http.Browser (open Request Connection)]
->
I do not know exactly what it means ... I do not want to
load the same page at the same time
or multipage at the same time.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Simulate the same problem

0 ... 1 second = You tried to load the same page more than once in one second.Please go back and try again.



It really works, checking in Firefox !!!

Solution:

Example 3 second
Delay between opening sucessive locations: Value: [...] second(s)

**External links are only visible to Support Staff**...

Post:

Code:
#1602
#1603
#1605
100% WORK


Chang ticket

Code:
#1607
#1608
and again ERROR


Quote:
Originally Posted by Jiaz View Post
Its not caused by multiple packages. Because of some reason JDownloader opens the same page multiple times. Pspzocker will have to check this
100%'re right. Please fix it

Last edited by Lram32; 05.11.2015 at 19:34. Reason: merged posts
Reply With Quote
  #1735  
Old 05.11.2015, 16:43
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

its required to get the info needed, I already asked psp about that. So if that's the case there is nothing to 'fix' you need to reduce requests to that base so you don't hit limit. I've given psp some ideas about how to do that.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #1736  
Old 05.11.2015, 17:11
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

raztoki - My friend suggestion:


Quote:
May be you can try add request random pages as video, audio, feed and others between main requests. Or use requests to VK API instead site requests, if it is possible.
Well, that's it.
Reply With Quote
  #1737  
Old 05.11.2015, 17:28
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,655
Default

decrypter tasks are passed to plugin as single event. Not entirely sure if psp written to return links back into the plugin or process all links within a given task. But that said either way URL could be hit within current task or next (queued) decrypter task within short succession of each other.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #1738  
Old 06.11.2015, 14:17
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

May not believe me but I tested it...

TEST:

Quote:
@DecrypterPlugin(revision="$Revision: 31706 $"
Add 20 links vk.com/idXXXX

Not Found Error
VK Plugin works quickly (optimal, good balance delays and queue) and no errors

-----------------------------------------------------------

Quote:
@DecrypterPlugin(revision="$Revision: 31972 $",
Add 20 links vk.com/idXXXX
Immediately a lot of errors

OLD:
Browser.setRequestIntervalLimitGlobal("vk.com", 50, 18, 1500L);
-------------------------------------
NEW:
Browser.setRequestIntervalLimitGlobal("vk.com", 50, 20, 2000L);
---------------------------------------------------------------

OLD:
if (this.br.containsHTML("class="photos_no_content"")) {
throw new DecrypterException("EXCEPTION_LINKOFFLINE");
}


-------------------------------------

NEW:
if (this.br.containsHTML("class="photos_no_content"")) {
throw new DecrypterException("EXCEPTION_LINKOFFLINE");
}
----------------------------------------------------------


OLD:
Thread.sleep(1000L);
-------------------

NEW:
Thread.sleep(this.cfg.getLongProperty("SLEEP_PAGINATION_GENERAL", 1000L));
--------------
OLD:
Thread.sleep(1000L);
-----------------

NEW:
Thread.sleep(this.cfg.getLongProperty("SLEEP_PAGINATION_COMMUNITY_VIDEO", 1000L));
---------------------------
NEW:
int counter = 1;
while ((this.br.containsHTML("You tried to load the same page more than once in one second|Вы попытались загрузить более одной однотипной страницы в секунду|Pr\\bujesz za\\ładowa\\ć wi\\ęcej ni\\ż jedn\\ą stron\\ę w ci\\ągu sekundy|Sie haben versucht die Seite mehrfach innerhalb einer Sekunde zu laden")) && (counter <= 10))
{
this.logger.info("Trying to avoid block " + counter + " / 10");
Thread.sleep(this.cfg.getLongProperty("SLEEP_TOO_MANY_REQUESTS", 3000L));
getPage(this.br, parameter);
}
-----------------------
NEW:
for (int counter = 1; counter <= 10; counter++)
------------------------------
NEW:
this.logger.info("Trying to avoid block " + counter + " / 10");
Thread.sleep(this.cfg.getLongProperty("SLEEP_TOO_MANY_REQUESTS", 3000L));
------------------------------
OLD:
for (int i = 1; i <= 10; i++)
---------------
OLD:
this.logger.info("Trying to avoid block " + i + " / 10");
sleep(3000L, this.CRYPTEDLINK);
-------------------
OLD:
}

private String resolveScreenNameAPI(String screenname)
throws Exception
{
getPage(this.br, "**External links are only visible to Support Staff** + screenname);
String ownerID = this.br.getRegex(""object_id":(\\d+)").getMatch(0);

return ownerID;
-----------------------------------------
OLD:
if (!br.containsHTML("You tried to load the same page more than once in one second")) {
break;
}
Thread.sleep(2000L);
---------------------------------------------------

NEW:
}

private String resolveScreenNameAPI(String screenname)
throws Exception
{
getPage(this.br, "**External links are only visible to Support Staff** + screenname);
String ownerID = this.br.getRegex(""object_id":(\\d+)").getMatch(0);
return ownerID;
---------------------------------------

Last edited by Lram32; 20.11.2015 at 19:04. Reason: merged posts
Reply With Quote
  #1739  
Old 06.11.2015, 14:58
tony2long's Avatar
tony2long tony2long is offline
English Supporter
 
Join Date: Jun 2009
Posts: 6,510
Default

By adding settings, the idea is that if you get problem, you can change the settings yourself until no problem anymore. So it's up to you now, find settings that fit your need.
__________________
FAQ: How to upload a Log
Reply With Quote
  #1740  
Old 06.11.2015, 15:04
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,451
Default

tony2long - "More timings added to config"

tony2long - I'm not a programmer, you have to know what you're doing
In this forum, there are developers who will better know what needs to be changed

Last edited by Lram32; 20.11.2015 at 19:05. Reason: merged posts
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 00:48.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.