JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 29.04.2018, 07:36
ozora ozora is offline
Modem User
 
Join Date: Apr 2018
Posts: 1
Default Pixiv plugin has problems with single images.

Hi,

The pixiv plugin seems to have issues determining the difference between a single image submission and a multi image submission when crawling galleries.

When crawling a gallery such as **External links are only visible to Support Staff****External links are only visible to Support Staff** , it correctly manages to find all of the submissions in the gallery by ID, but then treats them all as multi image submissions, which use a different URL than single image submissions. In the case of this gallery, that means most of the data links are broken and display "Plugin Defect!member_illust.php".

For example, one of the download links here was
**External links are only visible to Support Staff****External links are only visible to Support Staff**
which is broken, the correct link for this type of submission should be
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Even when crawling a single image link (eg **External links are only visible to Support Staff****External links are only visible to Support Staff**) directly, it returns "Plugin Defect!member_illust.php", so perhaps site changes have just left single image links broken.
Reply With Quote
  #2  
Old 30.04.2018, 18:03
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

I've optimized crawler to better handle single/multi-image links.
Please wait for next plugin update. Please enable debug mode in Settings-Advanced Settings-Log.debugmodeenabled and restart JDownloader.
In case you encounter Plugin defect again, please create a log, see https://support.jdownloader.org/Know...d-session-logs and post the shown logID here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 28.05.2018, 01:14
Galuthien Galuthien is offline
Vacuum Cleaner
 
Join Date: May 2018
Posts: 18
Default Download Flash / animations in Pixiv with Jdownloader

Hi, I wanted to know if it is possible to download Flash / animations in Pixiv using the Jdownloader add-on ...

I can download all the images with the help of the add-on, but the animations are downloaded in a zip file, then, when I extract the file, several images (in sequence) are extracted from the animation I want to download.

Is it possible that the animations are downloaded in Jdownloader directly as swf files and not as zip files?
Reply With Quote
  #4  
Old 28.05.2018, 10:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

JDownloader can only download what is available and as far as I know the animations are just single images that are played back in browser, so no, no swf file available to download. But of course I can check, please provide example links
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 28.05.2018, 23:44
simon_7617118 simon_7617118 is offline
Vacuum Cleaner
 
Join Date: Oct 2015
Posts: 16
Default

Hi. I'm not sure if the plugin has been updated, but I'm experiencing the same issue as the first poster. Links from image sets seem to be crawled correctly, albeit with a lot of extraneous broken links. Single image posts are not getting picked up.
28.05.18 15.37.06 <--> 28.05.18 15.41.47 jdlog://1698854433151/
Thank you.
Reply With Quote
  #6  
Old 29.05.2018, 00:38
Galuthien Galuthien is offline
Vacuum Cleaner
 
Join Date: May 2018
Posts: 18
Default

Quote:
Originally Posted by Jiaz View Post
JDownloader can only download what is available and as far as I know the animations are just single images that are played back in browser, so no, no swf file available to download. But of course I can check, please provide example links
In this link there are some animations:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #7  
Old 29.05.2018, 10:05
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@simon_7617118: Please provide example links. I could not reproduce any issue. I did test gallery links and single image links without any error. Your log does not contain a single error?!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 29.05.2018, 10:08
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@Galuthien: The animation is provided as zip and JDownloader also shows the zip when you add the link/gallery.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 29.05.2018, 19:51
simon_7617118 simon_7617118 is offline
Vacuum Cleaner
 
Join Date: Oct 2015
Posts: 16
Default

This would be a good one to test because it's huge and has a mix of single and multi image entries:
**External links are only visible to Support Staff**...

There are 317 entries, but JD is grabbing only 87 multi-image entries. All the single posts are ignored when I use the gallery url, and produce a "Plugin Defect!member illust" entry when I input them individually.

This happens with every gallery. My JD version is up to date.
Reply With Quote
  #10  
Old 30.05.2018, 04:10
Galuthien Galuthien is offline
Vacuum Cleaner
 
Join Date: May 2018
Posts: 18
Default

@Jiaz: Yes, it happens to me...

Maybe in the future it would be good to consider correcting that flaw so that you can download animations from Pixiv with the addon in Jdownloader.
Reply With Quote
  #11  
Old 30.05.2018, 14:42
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@simon_7617118: can you please send the link to support@jdownloader.org
Also please provide a debug log for the plugin defect!
Enable Settings-Advanced Settings-https://support.jdownloader.org/Knowledgebase/Article/View/47/0/how-to-create-and-upload-session-logs and restart JDownloader
Now reproduce the issue and create log, see https://support.jdownloader.org/Know...d-session-logs
and post the shown logID or send to support@jdownloader.org
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 30.05.2018, 14:42
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@Galuthien: JDownloader already supports downloading the animations as zips. What flaw do you mean then?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 30.05.2018, 14:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

I cannot reproduce any issue, so please provide logs (see post above) and example links. In case you're using an account, please also send me logins to support@jdownloader.org, maybe the issues are linked to account usage as I'm not using any account at all
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 30.05.2018, 20:23
simon_7617118 simon_7617118 is offline
Vacuum Cleaner
 
Join Date: Oct 2015
Posts: 16
Default

The problem has become more complicated with some investigation.

Pixiv seems to have been rolling out a new layout to only some accounts. My old account login uses the new layout. I created a new account, and it uses the old layout. It seems this is random and one does not have control over it, nor can it be reverted.

Urls are the same, image urls are the same, but the layout of the page is different. I've copied them to Pastebin so you can take a look.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Old layout style. Account that get this old layout can still grab links correctly in JDownloader:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

New layout style. Accounts that get this cannot grab links properly in JDownloader:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I hope this helps.
Reply With Quote
  #15  
Old 31.05.2018, 02:39
Galuthien Galuthien is offline
Vacuum Cleaner
 
Join Date: May 2018
Posts: 18
Default

Quote:
Originally Posted by Jiaz View Post
@Galuthien: JDownloader already supports downloading the animations as zips. What flaw do you mean then?
The problem is that after extracting the zip file does not appear the flash animation in swf format, but several images appear .jpg (in sequence) of the animation that I want to download ...

The normal thing would be that after downloading the animation in Jdownloader it should appear directly in .swf format, and not in a sequence of images in jpg format ...

I know a page where I can download the Pixiv animations, but I find it tedious to do it by that method since the page can not download all the animations at once, because I have to download the animations one by one ...
Reply With Quote
  #16  
Old 01.06.2018, 19:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@Galuthien: The animation does not exist as swf but the zip from server simply contains the single images and played back via javascript. There is no swf at all. I will try to find an easy solution that can put images into swf file.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 01.06.2018, 19:34
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

@simon: Thanks for the information and html source

You do get new layout only with account, don't you?
Or also without any account at all?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #18  
Old 02.06.2018, 01:11
simon_7617118 simon_7617118 is offline
Vacuum Cleaner
 
Join Date: Oct 2015
Posts: 16
Default

Without account, the plugin works fine with one caveat: it's not grabbing anything past the first gallery page, because only registered users get full access.

My personal account, which is pretty old, has been rolled to the new layout. The account I just registered uses the old layout, but I assume somewhere down the line it will switch over. I bet this is why people with newer accounts aren't getting problems in JDownloader yet.

Multi-image posts aren't getting this problem yet at all, because as far as I can tell the layout for those posts are unchanged.
Reply With Quote
  #19  
Old 04.06.2018, 09:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 70,924
Default

Could you please provide some more pastebins? one/two? if possible old/new ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 05.06.2018, 09:31
simon_7617118 simon_7617118 is offline
Vacuum Cleaner
 
Join Date: Oct 2015
Posts: 16
Default

Hello,

Single Pic
**External links are only visible to Support Staff****External links are only visible to Support Staff**
New format: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Old Format: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Single Pic
**External links are only visible to Support Staff****External links are only visible to Support Staff**
New format: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Old Format: **External links are only visible to Support Staff****External links are only visible to Support Staff**

Multi-pic (12) (the images open in a second window/page 2)
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
New format page 1: **External links are only visible to Support Staff****External links are only visible to Support Staff**
New format page 2: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Old format page 1: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Old format page 2: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 09:12.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.