[djmakinera]

1. You've added a lot of links? YES/NO?
2. You tried to analyze 20 minutes (for testing and stop CRAWL )? YES/NO?

Slower is good, you can not complain "too fast" anymore.

Faster is good (!) (2 - 4 October Plugin) = Fast, Without Errors!

Edit:
1. Yes
2. No

[raztoki]

@djmakinera
Full log that caused that issue please!

Either you assist us with what we need, else we can not help you moving forward. If you decline to provide us with what we need, I/We ask that you do not many any more VK related reports.

raztoki

[djmakinera]

FULL LOG:
30.10.15 12.33.29to30.10.15 12.35.52 jdlog://9413734739341/


-------------------


NOT SUPPORT FORCE HTTPS ???
----------------Response Information------------
Connection-Time: keep-Alive
----------------Response------------------------
unknown HTTP response
InvalidHTTPHeader: ‹ í}ÛvÛH’Ã*»¿"‹=.JÝ’ HJ–¼ò*ËU¾µ*ºµ¥áI IÀ ¨‹]ú��Ú}š‡}Û÷:³Û{úÃŒnßföÌÙ‡Ý3‘™ H‘²dȻÜ3.y�ŒŒŒŒˆÃŒˆÂ¼ÃµÃ™Â½§w÷¾vŸŒÃ’±Ož}}çÑû¤Â±Ãžj}kÜmµîíÝ#ß}±÷øÑ74ò"�='mµî?i�Æ(M£ ÍVëäädãÄØãÃÖÞóÖ)¶¢c5ùs=áu6ÜÔmìܸÅ;9ûA²]ӀÞï÷E½Úôip¸Ý`Aƒä¿°
F]ø3f)%ØÆ:{=ñŽÂ·ß*½»~7G4õlŸ5ˆ)Ã’íÆÃûÛÌ=„”Ôó½Ã*ˆÄÃŒßn$£0N�IJ<(Ù £˜
·-oLYÃ’Ã’cLmÀßÛ¯¬Ö¦Qä³õ4œ8£õº:¤Â±7°´�(8lì̯šxo*¤k�v*
u*%ÒÛÚ)ü›Ã‘ä,ÓV§}
------------------------------------------------


--ID:100TS:1446204952877-10/30/15 12:35:52 PM - [] -> jd.http.Browser$BrowserException
at jd.http.Browser.loadConnection(Browser.java:1370)
at jd.http.Browser.postPage(Browser.java:1572)
at jd.http.Browser.postPage(Browser.java:1579)
at jd.plugins.hoster.VKontakteRuHoster.login(VKontakteRuHoster.java:694)
at jd.plugins.hoster.VKontakteRuHoster.fetchAccountInfo(VKontakteRuHoster.java:426)
at jd.controlling.AccountController.updateAccountInfo(AccountController.java:307)
at jd.controlling.accountchecker.AccountChecker$AccountCheckJob.check(AccountChecker.java:47)
at jd.controlling.accountchecker.AccountChecker$1.run(AccountChecker.java:154)
Caused by: java.io.IOException: 999 unknown HTTP response
at org.appwork.utils.net.httpconnection.HTTPConnectionImpl.getInputStream(HTTPConnectionImpl.java:1034)
at jd.http.Request.read(Request.java:103)
at jd.http.Request.read(Request.java:662)
at jd.http.Browser.loadConnection(Browser.java:1359)
... 7 more

--ID:100TS:1446204952880-10/30/15 12:35:52 PM - [jd.http.Browser(setProxySelector)] -> Use local proxy: jd.http.StaticProxySelector@5bdf792f wished: jd.http.StaticProxySelector@5bdf792f
--ID:100TS:1446204953264-10/30/15 12:35:53 PM - [jd.controlling.AccountController(updateAccountInfo)] -> AccountCheck: Failed because of exception, temp disable it!

No option to edit, so doubles

[raztoki]

@dj
during the login process, VK uses redirects and moves away from HTTPS, after the initial login POST request (your user:pass is communicated via https). JD follows the redirects

during this VK servers respond with a invalid http response code 999 (from the exception) and JDownloader/JDBrowser aborts. This is intended, behaviour as its out of range.

Your not using any antivirus/firewall products that I can see registered with Windows, so unlikely related to that.
So either maybe vk deliberately doing this to your profile via JDownloader,
or
internet filtering from outside means (government/isp).

raztoki

[djmakinera]

Today logged into everything worked.

Nothing I changed the system and the ESET - certainly nothing blocks (never blocked!)

Changing IP - does not solve the problem.
Changing to another account VK - does not solve the problem.

[raztoki]

What OS are you running, in the logs its identified as windows 7, but your running a server right?

The logs also indicate data about security software from windows registry (which is used to identify connection related issues, for example 'firewall blocking or internet filtering'). Eset does have a webfilter feature, and can change HTML or HTTP Requests. Not sure if you're aware or not, or have that feature installed/enabled.

raztoki

[djmakinera]

VK Site uses https

https ://vk.com

http ://vk.com => automatically changes to https ://vk.com in the browser

Broswer HTTPS (100% work)

[raztoki]

JD uses https, its the hoster which changes to http via the POST request response. The series of redirects are all without HTTPS. You will need to take this up with VK and not us.

raztoki

[djmakinera]

This problem occurs because HTTP specifications state that HTTP requests that contain separator characters in the header are not valid. The HTTP specifications also state that HTTP verbs are case-sensitive.

[raztoki]

The Java exception thrown shows a response code of 999, I don't believe this is due to any 'separator characters' or 'http verb/case sensitive'.

JD sends a single POST request with HTTPS, they respond outside (responding with http redirect instead of https) of your user vk account profile on there website. JD only follows URLs directly given by these redirects, it doesn't change or reformat any of the URLs. I still say this is VK created issue in relation to not respecting your user profile setting.

[djmakinera]

999 - This error normally occurs when too many requests are sent to VKONTAKTE and the rate limit has been reached.

[raztoki]

Far as I understand 999 is not in http RFC specification. Official reserved response codes stop at 599.

[djmakinera]

Solution: Do not use a plugin or a new version JD2 - to prevent too many requests until we resolve the issue.

Wait ... ~20 minutes

Now it works well, but on the old version JD2 and pluginss of 4 October 2015 (not causes many request = 100% work)

If I continued to use the new plug-ins and new versions JD2 - again block access applications JD2

Only the previous plug and JD2 - at the moment works perfectly ...

[raztoki]

4th October doesn't reflect well in the diffs I just read, nothing since then has been changed other than to increase wait times when you hit the threshold... to me that's a past event and doesn't solve the underlying issue.

august is the month when you complained heaps about too slow, too fast, too fast, and pile of changes made and in my opinion is the cause of current problem.

We have a saying here
ain't broke, don't fix

raztoki

[djmakinera]

Reproduce, very easily :D

Try to crawl more links (eg. 20 minutes)
Automatically detected suspicious activity ... and block

[pspzockerscene]

Originally Posted by djmakinera

Automatically detected suspicious activity ... and block

Adding millions of urls might really be suspicious ...:rolleyes:

[djmakinera]

You tried to load the same page more than once second. Please go back and try again.

Jiaz -

found the cause of the plugin isuue

Please tell what caused and how to fix it. Thanks!

[raztoki]

please use the proper quote function that way we can reference the post(id) and see time stamps etc.

that post was from 03.10.2015, 00:19 (my time) which reflects to changed on svn on the 2nd. all he placed in, is a way time which was removed in august...

[djmakinera]

raztoki -
tony2Long - today changed the plug and added: "Sleep" : to cause more and more blocks acces, !
Please delete the unnecessary "sleep" - thanks!

Moderator please enable EDIT post on my profile

Originally Posted by pspzockerscene

Adding millions of urls might really be suspicious ...:rolleyes:

Add 61 links "id"
So why do you say about the millions of links?

Debug log:
30.10.15 15.35.15to30.10.15 15.51.01 jdlog://1553734739341/

Spoiler:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Originally Posted by djmakinera

So why do you say about the millions of links?

Please stop arguing about what you do - thanks.

[tony2long]

Originally Posted by djmakinera

Debug log:
30.10.15 15.35.15to30.10.15 15.51.01 jdlog://1553734739341/

This log doesn't contain "Decrypter broken", what's the problem?

[djmakinera]

tony2long - Leave me alone...
I am sorry that this is very complex bug, but it is the best information we have.

[raztoki]

Originally Posted by djmakinera

raztoki -
tony2Long - today changed the plug and added: "Sleep" : to cause more and more blocks acces, !
Please delete the unnecessary "sleep" - thanks!

sleeps do not cause more blocks, they reduce them as it reduces how fast requests can be sent!

Originally Posted by djmakinera

Moderator please enable EDIT post on my profile

No, been disabled due to you excessive editing. Posts are edited on average 4+ times removing context and you also have tenancy to remove content after answer has been given. This means next person that reads thread gets confused/mislead.

Originally Posted by djmakinera

Add 61 links "id"
So why do you say about the millions of links?

This was generalised statement by psp! History doesn't lie, you're known to mirror the internet.


raztoki

[djmakinera]

raztoki -
You can increase and increase and increase and increase and increase and increase sleep in reduced speed crawling plug-in, but it will not change anything, will continue to block - if I add eg. 60+ links.
Jiaz - previously add a "delay"
Do not "sleep".
Delay, and sleep - is not the same.

Previous plugin does not require any addition of "sleep" and there is not even any error in the log! Because it works perfectly or with a good delay (not load multiple pages at the same time)
Add 50- 60 links Many blocks and shows a lot of links shows ONLINE.

Edit: Shows OFFLINE

[raztoki]

Originally Posted by djmakinera

raztoki -
tony2Long - today changed the plug and added: "Sleep" : to cause more and more blocks acces, !
Please delete the unnecessary "sleep" - thanks!

This is where you're wrong. Since you have very little understanding of programming, I'll explain it for you! The plugin only allows 1 task at a time to be processed. You can add 10000s of vk links, the decrypter will only process one at a time. Sleeps added are just waiting for small times frames (100ms = 1/10th of a second) in jiaz change.

Originally Posted by djmakinera

raztoki -
Previous plugin does not require any addition of "sleep" and there is not even any error in the log! Because it works perfectly or with a good delay (not load multiple pages at the same time)
Add 50- 60 links Many blocks and shows a lot of links shows ONLINE.

Once again wrong. Before you asked to change the plugin was exclusively using sleeps to prevent issue and you've even said it works (but at times 'slow'). What changed was Jiaz moved to another means to slow down requests vs general sleeps. It allows burst rate of sending requests and then rate limits once reached threshold. This doesn't keep track of which URLs are requested, thus you can request the same page 'again' and get blocked. I believe this is what causing the problem as SLEEPS have been removed in favour for newer method of keeping track of how many requests have been sent over given time. Thus introducing what your experiencing now.. this leads back to August changes and not October.

[djmakinera]

14832<!>photos.js,photos.css<!>3<!>3351<!>8<!>You tried to load the same page more than once in one second. Please go back and try again.<br><!><!>3990242

--ID:116TS:1394302625846-14-03-08 19:17:05 - [jd.plugins.decrypter.VKontakteRu(postPageSafe)] -> Trying to avoid block 4 / 10

Developer, Programmer VK He said:

This is not API and this request more crazy. It can block if you send one request per second, "similar actions". This is protection for ban bots.

API has 3 request per second, but you increase speed, using pack of 10 actions in "execute" method

1,5 sec delay for 7-10 requests. After sleep to 3-5sec, and continue.
If plugin use vk API (vk.com/dev/methods) then it has other restrictions

https://support.microsoft.com/en-us/kb/828726

The plugin is probably good.
The error in the "Browser" JD2.
It occurs a few days after the update JD2

02.11.15 17.55.49to02.11.15 17.56.53 jdlog://3791834739341/

Re-Install JD2 - Not help !!!

**External links are only visible to Support Staff****External links are only visible to Support Staff**


**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Nope - the error comes from VK.
As you can see you re the only one complaining because you re downloading abnormal amounts of data from the vk servers.

GreeZ psp

[djmakinera]

I did the test. I changed password "?......" and now I can not log in. The problem is the characters.

[pspzockerscene]

Then please PM your logindata to one of us so we can check/fix that.

GreeZ psp

[djmakinera]

logindata
http://jdownloader.org/pastebin/134517

[Jd.plugin.decrypter.VKontakteRu (decrypterMultiPages)] -> Parsing Page 5 of 11

I have a question. Please answer developer pspzockerscene & Jiaz. Maybe he's too short and causes "temporary" lock and "Crawler Out Of Date"

What is the delay between the parties?

Example:

[Jjd.plugin.decrypter.VKontakteRu(decrypterMultiPages)] -> Parsing page 5 of 11

[pspzockerscene]

@djmakinera
I can add your account fine.
The "?" in the password does not seem to be an issue.

You can find the delays in our code.
We told you several times but here again: We haven't changed the delays in any way!

GreeZ psp

[djmakinera]

https://board.jdownloader.org/showpo...postcount=1625

<====After updating plugins. Again bug that causes the same actions and continuous lock and I can not analyze anything properly ... even I will add some links and constantly inform the blockade. And that plugin out of date. Link Online, Offline aple shows since the link was blocked.


"trying to avoid 1/10" - this is not an API ... it causes the same actions.

Download - 100% work
Crawling - Block / Errors

[pspzockerscene]

I cannot reproduce any of your problems.
Maybe, after nearly 100 pages in one thread you could finally be satisfied ...

[djmakinera]

Very complex bug...
always reproduce if help by TeamViewer.

Please tell me what kind of strange characters
Invalid HTTP Header


 í}ÛvÛH’Ã*»¿"‹=6Â¥nIÄ…IÉ’W¾•Â«Úᦌru[jž� a‘
€ÂºÃ˜Â¥ßØù*ݧyØ·}¯3»½§Ã�ìöuöÌÙ‡Ý3‘™  H‘²d»\žé²ˆÂ¼FFFFFDfDÞüêî“;{¿}z�’Ñ�<}qûá7wHm½ÑxiÞi4îîÝ%ß?Ø{ô�èyžD¾“4÷×Hm�$ãÍFãøøxãØÜ£~cïYã[ѱšü¹ó:nâÖv®Ýä�œŒ†A¼]рÞívE½ÚÃ’*¿]cA�d¿°
F]ø3b %ØÆ:{3ñ�¶k߯¿Ø]¿ŽÃ†4ñí!«'$Ûµoîm3·)
¨7ôƒC±áv-„QâLâCÉDÌÛ®5üí³¸áÑ#LînÀß[&¯¬Ö¦ãñ�*'áĬW5S�F~Ã�Ã’6ÆA¿¶3¿jì¿e€’¶uÒ¶f4Ô¶–hH7´øoFS�³L[-ãþ›Ã•VË�mUL‰œ�õät¬ÎGÂN’Nóq4ŠY²}ìnx¯ëFKóÄèˆm×\;‘?N|„*kEÎHâ'C¶óm8È3:²Ã›
‘YA(êíLCC‰X<PšÓ¶È‹g·6uí@aÑÆx0®íÜldÍ@“âÁ
Éq¼¦GT¤ŽhDŽÉ6yw�êƽ(LhÂz>tÑá&
ÜÃÿ *a6|�qôßÉJݬ¯’~ MÌñ],×Ô*¶né<!Hšl’º^'·HMÙ$ÆsaŒy@©N8ªc

[editestowy]

This is fun.... Like reading a comic book about deaf and blind trying to communicate with each other
Why NOONE has the same problems as djmakinera?

[pspzockerscene]

Exactly.
Many of the things he is seeing as "bugs" are not even bugs hmm ...

[djmakinera]

Forum & Chat - not reproduce
Log - not reproduce
TeamViewer - reproduce

Do you know any other better solution?

Already many times as you said, it works, it works ...
Remember that one of the developers two hours tried to solve the problem with me by TeamViewer - because this occurs in a few machines, or for the country in which logs ...

[editestowy]

Originally Posted by djmakinera

Do you know any other better solution?

Yes we know - start to program yourself
Too much developer power is being redirected to suit your "needs".

[djmakinera]

Chomikuj, Vk.com, music.163.com and all host problem

New Update = always a problem.
How to turn off all updates ? :(

[editestowy]

There are 1370 hosters and 639 decrypter plugins.
Stop writing nonsenses: "all host problem"