JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 27.06.2019, 12:35
ecclessia ecclessia is offline
Junior Loader
 
Join Date: Jun 2019
Posts: 14
Default Deviantart plugin out of date/broken/defect

Since deviantart had introduced new site design, plugin now downloads only deviantart logo instead of pictures itself. Also there are two more problems - when i try to put link to gallery to parse, i got error message, so i have to keep adding same link 2-8 times until it would start parsing process (that has began several days before new site design introduction). Also sometimes already parsed pics give message about plugin being too old, but after 1-5 force download startings they begin downloading and was downloaded correctly (also, this was several days before new site design was introdicted.) Flash (*.swf files) just say, that plugin is too old and refuse to be downloaded at all. Please, restore plugin work.
Reply With Quote
  #2  
Old 27.06.2019, 12:39
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

New layout/site design is nearly fully incompatible to old layout/site design. Plugins needs a full revisit

__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 27.06.2019, 13:17
ecclessia ecclessia is offline
Junior Loader
 
Join Date: Jun 2019
Posts: 14
Default

Technically, old site design still exists, but by default new design is loaded, and it requires to press the switch on top panel of new design to switch to the old one. And (at least in browser) switch doesn't always works correctly. Hope that info would help.
Reply With Quote
  #4  
Old 28.06.2019, 10:47
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

Problem is that there is no *switch to old* button. That's only available when logged in. In normal mode it's random/depending on artist if you see old/new layout
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 14.07.2019, 06:31
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 77
Default

Apologies for bumping someone else's report thread, but if it helps other users I have this EventScript that will find cases where the logo has been downloaded instead of the deviation and reset them. I have it set to run on an Interval trigger every 6 hours (21600000ms) so it can pick up any downloads that are again incorrectly downloaded after being reset. My first run found around 28000 failures, dating back to mid-June.

Code:
var logFile = JD_HOME + "/da_reset.log";

function log(message) {
    writeFile(logFile, '[' + new Date().toISOString() + '] ' + message, true);
}

// If downloads are running find those that:
// 1. match the size of the logo file (most selective)
// 2. on deviantart (just in case)
// 3. have a status text of "Finished"
if (isDownloadControllerRunning() && !isDownloadControllerStopping()) {
    log('Download controller running, attempting to reset links ...');
    var candidates = getAllDownloadLinks().filter(function(dl) {
        return dl.getBytesTotal() === 10648 && dl.getUrl().indexOf('deviantart') !== -1 && dl.getStatus() != null && dl.getStatus().equals('Finished');
    });

    log('Number of possible broken downloads found: ' + candidates.length.toString());

    for (var i = 0; i < candidates.length; i++) {
        log('Resetting item ' + (i + 1).toString());
        candidates[i].reset();
    }
    log('Reset job complete.');
}
Reply With Quote
  #6  
Old 15.07.2019, 11:22
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

@mwl: thanks for sharing your script. Deviantart did not good with this new layout. Reading many artists blog and it seems many aren't very happy with the new (limited) layout.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 05.08.2019, 05:08
netgearjd netgearjd is offline
Giga Loader
 
Join Date: Aug 2014
Posts: 92
Default

Please fix this ASAP! Now JD is not able to find any full-sized images properly. I either get a "plugin defect (sic)" error, or the site logo image, or after retrying many, many times I only get the smaller JPGs instead of the full-sized images (JPGs/PNGs). :(
Reply With Quote
  #8  
Old 05.08.2019, 16:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

@netgearjd: please provide example links
seems deviantart finally changed layout to new one for everyone.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 16.08.2019, 22:03
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 77
Default

An update on this:
As of around 1800 UTC on Wednesday, all linkgrabbers fail, but links already in the queue download successfully.

I've noticed that in the page source for the old version, there's a link to the new version, but in the new version it's a <div> with a click event which doesn't appear to send a URL at all.

There is an API (deviantart.com/developers/http/v1/20160316) and it appears to be able to get most things, but it will require parsing the URL to get the correct parameters, and grabbing individual folders may be an issue. For instance, while playing around with it I wasn't able to resolve the "Scraps" folder for a user using the /folders call. OAuth authentication flows might require to do something similar to what happens for reCaptcha to get initial credentials.

//edit
I'm currently looking at github.com/KarsonKimbrel/DeviantArt-OAuth2-Java to see if this is something that could be used.

Last edited by mwl; 16.08.2019 at 23:31. Reason: add API reference
Reply With Quote
  #10  
Old 16.08.2019, 22:54
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 77
Default

OK, in the browser when logged in, this seems to work:

Prepending the URL with **External links are only visible to Support Staff****External links are only visible to Support Staff** (e.g. **External links are only visible to Support Staff****External links are only visible to Support Staff**) will bring up the version opposite that which was just served up - so if you get the new version that will give you the old, and vice versa. From a fresh session logging in, defaulting to the new version, this method takes me directly to the old version.

Not sure if this is of any use, but I hope it helps.
Reply With Quote
  #11  
Old 23.08.2019, 18:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

Doesn't help much as we have to add support for the new layout nevertheless because
1.) account support is broken and/or user doesn't use any account
2.) sooner or later the old layout will be shutdown
3.) plugin must support URLs from old AND new layout
4.) age check now requires an account and no longer a simple checkbox

unfortunately I don't find free time to work on this at the moment :(
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 29.09.2019, 11:37
aexl
Guest
 
Posts: n/a
Default

I've donated yesterday by bank transfer for this cause.

Last edited by Jiaz; 10.10.2019 at 20:14. Reason: removed email
Reply With Quote
  #13  
Old 09.10.2019, 19:19
ecclessia ecclessia is offline
Junior Loader
 
Join Date: Jun 2019
Posts: 14
Default

I'm really sorry for bothering, but is there any progress on deviantart plugin?
Reply With Quote
  #14  
Old 10.10.2019, 20:15
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

@aexl: Thank you very much for your donation but at the moment there is no developer who finds enough time to https://board.jdownloader.org/showpo...2&postcount=11
In case you wish back your donation because you're unhappy with this, please contact us via support@jdownloader.org
__________________
JD-Dev & Server-Admin
Reply With Quote
  #15  
Old 27.10.2019, 07:49
aexl
Guest
 
Posts: n/a
Default

@Jiaz: Nah, no problem. I never donated before to JD so it's all good.
Reply With Quote
  #16  
Old 28.10.2019, 18:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

@aexl: THANK you very much for your understanding and of course for your donation!
I know that many ppl are *crying* for fixed support but time is not on my side right now :(
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 29.10.2019, 02:31
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 77
Default

After putting it off for a while, I'm going to have a go at this at some point this week. I might have some questions as I go, so what would be the best way to reach out to devs for some answers while I'm working? I'd rather not clutter this thread with lots of technical rambling.
Reply With Quote
  #18  
Old 18.11.2022, 18:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

Quote:
Originally Posted by mwl View Post
I might have some questions as I go, so what would be the best way to reach out to devs for some answers while I'm working?
best by mail support@jdownloader.org but be aware at the weekend I rarly check those

Quote:
Originally Posted by mwl View Post
I'll take a look at some other API clients to see what they can and can't do.
as far as I remember, the api was limited in features/posibilites in the past but I don't know about current api. sure, feel free to test other tools and check/compare their feature-set against JDownloader.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #19  
Old 29.10.2019, 12:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,232
Default

@mwl: either contact us via support@jdownloader.org or chat (irc, freenode, #jdteam, but can't promise if anyone will be there when you join it). I also recommend to work/experiment on smaller plugins for as deviantart is complex one. and because of major website/layout change I don't think that many parts of old plugin are useful/can be reused. please feel free and don't hesitate to contact us for question/help!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 30.10.2019, 03:21
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 77
Default

Quote:
Originally Posted by Jiaz View Post
because of major website/layout change I don't think that many parts of old plugin are useful/can be reused.
Yes, I was thinking of avoiding screen-scraping and just using their API, and figuring out how to integrate that will be the biggest challenge.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 05:41.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.