JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #2601  
Old 07.12.2017, 21:45
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

Bug crawl:
Comments (photo)

Link **External links are only visible to Support Staff****External links are only visible to Support Staff** is(already) FALSE
Reply With Quote
  #2602  
Old 07.12.2017, 22:12
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

Two type URLs.
Please decrypt: **External links are only visible to Support Staff****External links are only visible to Support Staff**


Incomplete URL (wrong URL) NOT WORK
wall-71015?z=photo-71015_456241225 (NOT FOUND, OR ACCESS DENIED)

FULL URL:
wall-71015?z=photo-71015_456241225%2Fwall-71015_113892

Please test and Show ALL URLs, sometimes wrong:

Reply With Quote
  #2603  
Old 26.12.2017, 19:56
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

Code:
26.12.17 18.38.26 <--> 26.12.17 18.56.24 jdlog://0656154433151/

Do not decrypt all post wall on the wall. It is associated with all the walls on the group. How do I know that? Because if you scroll down - javascrpt-processing (all pages) and try to extract all links to photos, it is much more than shows JD2 while decrypting in the package.
See my log in the previous post and see what causes this problem. Many links do not contain a full address to some of the photos, this is associated with wrong URLs that do not lead to the right photo or incorrect offset

Example wrong decrypt wrong URL
vk.com/wall153353527_642550?z=photo153353527_422821379missing full URL
HTML Code:
Error
Post not found
Also decrypts empty posts (only text), should not decrypt such posts and not show as offline and put in a package.

Please test Decrypt FULL WALL:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

It should collect/decrypt a total of 150,000 photos.
Exceed the parser operation too quickly. After a few minutes.
It shows many wrong links in packages and only shows 4865 links +/-

Shouldbe decrypt 150,000 links
Code:
All posts 156,203
It used to work correctly Not now. So this is a bug decrypt.

------------------------Thread: 202:PluginErrors.log-----------------------
--ID:202TS:1514310456292-12/26/17 6:47:36 PM - [jd.plugins.PluginForDecrypt(errLog)] -> CrawlerPlugin out of date: jd.plugins.decrypter.VKontakteRu@3fa8c3b :38375
--ID:202TS:1514310456293-12/26/17 6:47:36 PM - [jd.plugins.PluginForDecrypt(errLog)] -> URL was: **External links are only visible to Support Staff****External links are only visible to Support Staff**
--ID:202TS:1514310456293-12/26/17 6:47:36 PM - [] -> java.io.IOException: No request available
at jd.http.Browser.getURL(Browser.java:1307)
at jd.http.Browser.createPostRequest(Browser.java:702)
at jd.http.Browser.createPostRequest(Browser.java:698)
at jd.http.Browser.createPostRequest(Browser.java:725)
at jd.http.Browser.postPage(Browser.java:1722)
at jd.http.Browser.postPage(Browser.java:1718)
at jd.plugins.decrypter.VKontakteRu.postPageSafe(VKontakteRu.java:1928)
at jd.plugins.decrypter.VKontakteRu.decryptWall_Website(VKontakteRu.java:1433)
at jd.plugins.decrypter.VKontakteRu.decryptWallLink(VKontakteRu.java:1110)
at jd.plugins.decrypter.VKontakteRu.decryptIt(VKontakteRu.java:391)
at jd.plugins.PluginForDecrypt.decryptIt(PluginForDecrypt.java:274)
at jd.plugins.PluginForDecrypt.decryptLink(PluginForDecrypt.java:309)
at jd.controlling.linkcrawler.LinkCrawler.crawl(LinkCrawler.java:3132)
at jd.controlling.linkcollector.LinkCollectorCrawler.crawl(LinkCollectorCrawler.java:33)
at jd.controlling.linkcrawler.LinkCrawler$12.crawling(LinkCrawler.java:1401)
at jd.controlling.linkcrawler.LinkCrawlerRunnable.run_now(LinkCrawlerRunnable.java:104)
at jd.controlling.linkcrawler.LinkCrawlerRunnable.run_delayed(LinkCrawlerRunnable.java:74)
at jd.controlling.linkcrawler.LinkCrawlerRunnable.run(LinkCrawlerRunnable.java:36)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.MalformedURLException: no protocol: /al_wall.php
at java.net.URL.<init>(Unknown Source)
at java.net.URL.<init>(Unknown Source)
at java.net.URL.<init>(Unknown Source)
at org.appwork.utils.net.URLHelper.createURL(URLHelper.java:165)
at jd.http.Browser.getURL(Browser.java:1303)
... 20 more
Reply With Quote
  #2604  
Old 28.12.2017, 17:08
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

update post
new bug:
Code:
Photo Albums 34
jd2 decrypt only 27 albums (27 package)
**External links are only visible to Support Staff****External links are only visible to Support Staff**

missing 7 albums
Reply With Quote
  #2605  
Old 28.12.2017, 17:16
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

and new bug:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Do not decrypt photos in the album (see screenshot)
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2606  
Old 07.02.2018, 17:51
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

1. The plugin decrypts invalid (incorrect or incomplete addresses or leading to empty posts).
2. Still does not show full addresses what causes "Access Denied"
3. Shows that the plugin is out of date when trying to download.
4. Shows many invalid packages to empty addresses.
5. Do not decrypt all links from the walls. Applies to all walls. It shows too few links compared to HUGE walls, where links should show a lot.
6. Incorrect API.
7. Do not decrypt all offsets (error)
It seems to me that everything happens incorrectly, even though it used to decode any large walls without a problem (all graphic files loaded into the package)

Example link: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Screenshot: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2607  
Old 01.03.2018, 17:56
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default vk.com It does not show the size of mp3 files when decrypting

Advanced setting plugin only *.mp3.
It does not show the size of mp3 files when decrypting.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2608  
Old 01.03.2018, 18:08
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

Account is missing. MP3 DOWNLOAD. Please fix it.
**External links are only visible to Support Staff****External links are only visible to Support Staff**

01.03.18 17.06.51 <--> 01.03.18 17.07.20 jdlog://0988454433151/
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #2609  
Old 02.03.2018, 01:04
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,306
Default

Quote:
Originally Posted by djmakinera View Post
Advanced setting plugin only *.mp3.
It does not show the size of mp3 files when decrypting.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
settings > vk > fast linkcheck for audio?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #2610  
Old 02.03.2018, 13:27
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

Quote:
Originally Posted by raztoki View Post
settings > vk > fast linkcheck for audio?
yes
Reply With Quote
  #2611  
Old 04.03.2018, 18:53
patriks's Avatar
patriks patriks is offline
Super Loader
 
Join Date: Aug 2017
Location: Brasil
Posts: 26
Default

The script is not downloading all the photos, it downloads the 80 first and jumps 80.

If a photo of the album is from instagram, it does not download, even with it hosted in vk.
Reply With Quote
  #2612  
Old 06.03.2018, 10:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

@patriks: please provide example links
__________________
JD-Dev & Server-Admin
Reply With Quote
  #2613  
Old 07.03.2018, 12:08
patriks's Avatar
patriks patriks is offline
Super Loader
 
Join Date: Aug 2017
Location: Brasil
Posts: 26
Default

Quote:
Originally Posted by Jiaz View Post
@patriks: please provide example links
example links:

871 photos, only show in linkgrabber 791: **External links are only visible to Support Staff****External links are only visible to Support Staff**

This happens on all types of albums, even people.

*I noticed now with the test, it does not find all the albums.

50 albuns, only find 27
**External links are only visible to Support Staff****External links are only visible to Support Staff**

In one of the albums, linkgrabber jumped after 90 photos, instead of 80.
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Very strange

Last edited by patriks; 07.03.2018 at 12:12.
Reply With Quote
  #2614  
Old 15.03.2018, 21:04
patriks's Avatar
patriks patriks is offline
Super Loader
 
Join Date: Aug 2017
Location: Brasil
Posts: 26
Default

I would like to have a new setting if it is easy to do, to merge the post ID with the original file name, like this:

-67991642_456364887 - xb4q9pK_3sE.jpg
"post link" - "original server file name"

It would be interesting, currently it is only possible to choose one of them, but this is a problem, the ID generates an organized list in the file explorer, but if a photo is updated with a new version*, jdownloader cant download the new photo version.

*Is possible completely replace a photo with another in vkontakte, or apply clippings and edits that change the direct link (xb4q9pK_3sE), but not the photo link (-67991642_456364887).

Last edited by patriks; 16.03.2018 at 02:24.
Reply With Quote
  #2615  
Old 16.03.2018, 11:36
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

@patriks - plugins -> vkontakte.ru -> advanced setting:
Use (temporary) server filename as finalname instead of E.g.)'oid_id.jpg? Enabled
Reply With Quote
  #2616  
Old 16.03.2018, 20:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Quote:
Originally Posted by patriks View Post
50 albuns, only find 27
**External links are only visible to Support Staff****External links are only visible to Support Staff**
will be fixed with next update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #2617  
Old 16.03.2018, 20:01
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Quote:
Originally Posted by patriks View Post
In one of the albums, linkgrabber jumped after 90 photos, instead of 80.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
does find all 170 for me
__________________
JD-Dev & Server-Admin
Reply With Quote
  #2618  
Old 16.03.2018, 20:28
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Quote:
Originally Posted by patriks View Post
871 photos, only show in linkgrabber 791: **External links are only visible to Support Staff****External links are only visible to Support Staff**
always finds 860 for me, looks like there are some duplicates
__________________
JD-Dev & Server-Admin
Reply With Quote
  #2619  
Old 16.03.2018, 21:16
patriks's Avatar
patriks patriks is offline
Super Loader
 
Join Date: Aug 2017
Location: Brasil
Posts: 26
Default

Thank you so much Jiaz, I just noticed the change.
my remote jdownloader started several new downloads suddenly, I believe that everything is working well, I did a little test and now it finds all the images and albums.

Quote:
Originally Posted by djmakinera View Post
@patriks - plugins -> vkontakte.ru -> advanced setting:
Use (temporary) server filename as finalname instead of E.g.)'oid_id.jpg? Enabled
I already use this option (oid_id.jpg), for this reason, but having the post ID in the file name along with the image name is an interesting suggestion, because the files are organized in chronological order (I also keep the original date of the server in file, but not always the date of the post is the same as the image, exemple: saved photo album), it is easy to search for id of user/page or real name of the file in explorer.

Currently I use a eventscripter that generates a small db, in case I need to find the original post of the image, however with the method I suggested would be much easier and would keep everything more organized.

I also think it is not difficult to make this, the option oid_id.jpg renames the files at the time of download, with the new option it would simply add the original name of file at the end of the file:

-67991642_456364887 - xb4q9pK_3sE.jpg

Last edited by patriks; 16.03.2018 at 21:18.
Reply With Quote
  #2620  
Old 17.03.2018, 12:29
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,355
Default

@Jiaz -

Please correct the showing of the final links. If it is not final, it may display a plug-in out of date (denied access) (if it scans the entire wall).


links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
is
Decrypt
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**



Shouldbe SHOW FULL ADRESS URLS:
**External links are only visible to Support Staff****External links are only visible to Support Staff**%2Fwall-40010106_107185
**External links are only visible to Support Staff****External links are only visible to Support Staff**%2Fwall-40010106_107185

Last edited by raztoki; 17.03.2018 at 14:32. Reason: prefix added
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:46.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.