JDownloader Community - Appwork GmbH
 

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 12.09.2013, 02:43
th1s_n1ght
Guest
 
Posts: n/a
Exclamation imgur.com bug

There is a problem with Content-Length: header here.
When you are using Download full resolution (imgur.com/download/{id}) Content-Length of the file is sometimes smaller than reported in header and JDownloader thinks that he can't download all file.

Workaround here is to use View Full Resolution link (http://i.imgur.com/{id}.{file_type})

Example:

Response Header of: http://imgur.com/download/Kgouh
Code:
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 11 Sep 2013 23:32:29 GMT
Content-Type: image/jpeg
Content-Length: 107546
(...)
Response Header of: http://i.imgur.com/Kgouh.jpg
Code:
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Wed, 11 Sep 2013 23:34:50 GMT
Content-Type: image/jpeg
Content-Length: 107546
(...)
Size of file from http://imgur.com/download/Kgouh
104 KB (106 610 bytes)

Size of file from http://i.imgur.com/Kgouh.jpg
105 KB (107 546 bytes)

difference in size is not constant and it is different for different files. However 18bytes is quite common (this example has 936 bytes of difference)

Any help would be appreciated

EDIT
HotFix for people:
after downloading 'incomplete' files
run this recursive batch file in download folder to delete .part from filenames:
Code:
@echo off
FOR /R . %%G in (*) do if "%%~xG"==".part" REN "%%~G" "%%~nG"

Last edited by th1s_n1ght; 12.09.2013 at 17:53.
  #2  
Old 12.09.2013, 17:35
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Ticket:


GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #3  
Old 12.09.2013, 17:52
th1s_n1ght
Guest
 
Posts: n/a
Default

I thought that it will be easier to change link decrypter (Workaround from the first post) than inventing a way to check if the file was completely downloaded, but it's your call.
  #4  
Old 12.09.2013, 19:24
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Implemented workaround!
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #5  
Old 12.09.2013, 20:34
th1s_n1ght
Guest
 
Posts: n/a
Default

Nice :D
thx!

sorry to bother but... your implementation has a little bug...

I said http://i.imgur.com/{id}.{file_type}
you used http://i.imgur.com/{id}.jpg

and png, gif files are downloaded but with wrong extension (everything is jpg) :/ To determine file extension you could use imgur api (better option of course ) or parse the page

If you are interested in api:
to get album information without need of authentication you can use api v2
http://api.imgur.com/2/album/{album_id}/images

example for: http://imgur.com/a/RIBLo:
get xml: http://api.imgur.com/2/album/RIBLo/images
or get json: http://api.imgur.com/2/album/RIBLo/images.json

In example album we have 2 gifs so to get links for them you need:
json_object->album->images->item[0]->links->original
(which is http://i.imgur.com/89cFcie.gif)
and
json_object->album->images->item[1]->links->original
(which is http://i.imgur.com/3GxlyXo.gif)

there is also information about size here:
json_object->album->images->item[0]->image->size
json_object->album->images->item[1]->image->size

=======================================

to get link for a single image (not album):
http://api.imgur.com/2/image/{image_id}

example for: http://imgur.com/Xv0spmk:
get xml: http://api.imgur.com/2/image/Xv0spmk
or get json: http://api.imgur.com/2/image/Xv0spmk.json

json_object->image->links->original
(which is http://i.imgur.com/Xv0spmk.jpg)

size information:
json_object->image->image->size

=======================================

all of it you can also do with api v3 but it's more complicated. (Mostly because of authentication)
(you can find detailed description here https://api.imgur.com/endpoints/gallery#album)

Last edited by Lram32; 13.09.2013 at 04:44. Reason: merged multiple posts
  #6  
Old 13.09.2013, 06:13
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,659
Default

heh I told psp this would happen last night ;p he will need to fix or revert back to the old method+mod
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
  #7  
Old 13.09.2013, 16:35
Lram32's Avatar
Lram32 Lram32 is offline
Spanish Supporter
 
Join Date: Jul 2012
Location: Dark side of the moon
Posts: 9,930
Default

Hi,

The problem will be checked later by pspzockerscene.

Greetings,
__________________
» Setup JD2 / Instalador de JD2 «
Spoiler:

Installer for Windows XP/Vista/Seven/Eight
|| JD2 x86 - x64 (Beta) ||

Installer for Mac
|| JD2 (Beta) ||

Installers for Linux
|| JD2 (Beta) x86 || <---> || JD2 (Beta) x64 ||

How to Create a Log
-» Click Here «-


¿Cómo crear un registro?
-» Click Aquí «-


Support Chat / Chat de Soporte
-» Click Here / Click Aquí «-


  #8  
Old 13.09.2013, 17:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Fully implemented the API so basically this plugin should never fail again
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Old 13.09.2013, 17:01
pspzockerscene
Message deleted by Lram32. Reason: duplicated post!
  #9  
Old 13.09.2013, 17:10
th1s_n1ght
Guest
 
Posts: n/a
Default

fingers crossed :D

Last edited by th1s_n1ght; 13.09.2013 at 21:48.
  #10  
Old 13.09.2013, 21:18
th1s_n1ght
Guest
 
Posts: n/a
Default

lol sory but I've found one more problem xD

For example we have album of 1000 pictures. When we add link to this album jdownloader will send one request to api and we have all links, all are avaible, online and everything is OK.

... But when we want to start download process jdownloader instead of using links collected from api is asking to the api for every picture again, but this time it's like we add every link to picture from this album by hand.

this is very fast way to reach limit of the api and get short-ban.

So imho when we have already link in http://i.imgur.com/{id}.{file_type} format we shouldn't ask api again and avaibility can be checked by http header (because this link doesnt have problems with it)

Now with album of 1000 pictures it will ask api once to collect well working links with all needed information plus ask api 1000 times for every link information again and in half it will be banned... When this will be corrected this plugin will be perfect xD .

... and of course happy Programmers' Day :D


EDIT:
I managed to write local api emulator (squid) + (c++ url_rewrite) + (mysql db) + (php api emulator) which is generating responses for api /image/ from api /album/ resolved before but i think it's very exotic fix xD and i rather won't post here instructions how to for community...

But anyway I'm looking forward for jdownloader fix And i'd love to test it for you.

Last edited by th1s_n1ght; 14.09.2013 at 15:47.
  #11  
Old 14.09.2013, 19:20
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Check again after the next update
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #12  
Old 15.09.2013, 15:32
th1s_n1ght
Guest
 
Posts: n/a
Default

yep now it's perfect
  #13  
Old 15.09.2013, 15:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,044
Default

Thanks for the feedback.

GreeZ pspzockerscene *closed*
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:11.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.