#41
|
|||
|
|||
I did and only 2 post came up the most recent being this thread where the last post was March 31st 2018. https://board.jdownloader.org/showthread.php?t=74587 this thread didn't come up.
It doesn't look like there has been any movement on this ticket in almost 2 years. any idea why the deviantart plugin is getting no love? |
#42
|
||||
|
||||
Quote:
2. Deviantart is using a protection similar to Cloudflare some time ago, see ticket: We're open source so you're free to contribute code/fix it yourself. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#43
|
|||
|
|||
I have the same issue, plugin outdated + can't grab and download from e.g.
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#44
|
||||
|
||||
Hi,
please read the previous posts in this thread. To sum it up: Please do not expect a bugfix for this in the near future/ever. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#45
|
|||
|
|||
DeviantArt Login is Down?
Hi this is the error I keep getting, hope this will be resolved soon?
https://imgur.com/a/P4qgq4A |
#46
|
||||
|
||||
Merged deviantart threads.
@H-Visage Please read the thread I've merged your post with.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#47
|
|||
|
|||
Well that's a bummer. Thanks anyways.
|
#48
|
|||
|
|||
Will it ever be possible again to download images from Deviantart using JDownloader? Until recently you could download at least half of the images, the other half just had the problem with the Deviantart logo. But now, the links aren't even pasted into JDownloader, and the plugin keeps reporting that it's out of date.-
|
#49
|
||||
|
||||
Quote:
We're open source so you're free to work on it yourself. EDIT I didn't want to waste your time but a simple "No" would also be the wrong answer. From my perspective: I don't see this being fixed.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 31.10.2022 at 14:49. |
#50
|
||||
|
||||
Update:
I've looked at this again and here are my results: Normal login attempts get blocked by their AntiDDoS protection but cookie login did work for me. I've implemented some proof of concept changes and I'm waiting for your feedback. Changes: - Fixed login: Added cookie login support (no gurantees and no idea how long self added cookies will stay active) - Fixed/added proof of concept profile crawler: Currently this will only crawl the first 24 items and only images. This will only listen to such URLs for now: deviantart.com/username Before you ask: I didn't put more time into this because if in the end, their AntiDDoS protection blocks most of those requests it wouldn't make any sense to put more time into this What I'd need from you to work more on this: - Feedback once the update is released (you may also build the dev version on your own and test right away) - More example URLs especially for profiles that contain a lot of different item types like: images, videos, journals and also things like +18 content, blocked content, officially downloaded content, .zip downloadable items For all changes mentioned in this post, the following information applies: Bitte auf das nächste CORE-Update warten! Please wait for the next CORE-Update! Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#51
|
|||
|
|||
I know very little about programming, but I just have a question. In fact, there is currently no browser extension or other program that manages to download entire galleries on DeviantArt. But I found a way. This is the gallery-dl. This works with Python and you simply enter a certain command line in cmd to download it, and all the pictures and other files in a gallery are downloaded in full quality. The disadvantage is that you have to insert each gallery into the command and then start it individually in cmd, which is of course very time-consuming if you want to download many galleries. Is it possibly somehow possible to connect that with the JDownloader? Maybe the question is stupid, but as I said, I don't really know much about programming.
|
#52
|
||||
|
||||
Quote:
Simply go to github.com and look for open source tools. Having invested some seconds (didn't test it though), here are some findings: github.com/xofred/deviantart-gallery-downloader github.com/NC22/KellyC-Image-Downloader github.com/mikf/gallery-dl Jup I found that one too. Quote:
Please re-read my last post, wait for the update, provide feedback and if my changes do work for the majority of our users, I'm willing to put more time into fixing our existing deviantart plugins
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#53
|
||||
|
||||
Updates have been released.
I've done quite some more changes to this plugin but officially I've only set the progress to 50%. Biggest changes: - fixed login - fixed official downloads ("Download button downloads") - re-designed settings for this plugin, see Settings -> Plugins -> deviantart.com - fixed profile/gallery/favourites crawler Please leave your feedback here and report anything you think is still broken.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#54
|
|||
|
|||
In some cases, instead of the actual item, it's downloading a DA logo. If you're looking for some way to detect it, it's 10648 bytes.
Also, there seems to be no way to craft a packagizer rule to differentiate between user galleries and favorites, as according to the rule tester both the Download URL and Source URL are the individual deviation URL rather than the gallery URL. I acknowledge that this might already be known. Out of interest, what was the reason for sticking with scraping rather than trying to use the API? Last edited by mwl; 18.11.2022 at 17:41. |
#55
|
||||
|
||||
@mwl: can you provide logs/example for the *downloading DA logo*? Best would be debug log, see https://support.jdownloader.org/Know...d-session-logs
and post shown logID here Quote:
The situation might have changed meanwhile Quote:
__________________
JD-Dev & Server-Admin |
#56
|
|||
|
|||
Looking more closely, it looks like a lot of the ones generating a logo are "literature", and therefore won't have an image, but I'd need to check if this is the case for all of them. I'll spin up a log later if I can.
|
#57
|
||||
|
||||
@mwl: thanks for the feedback. Maybe literature support broken/not yet supported. Maybe can provide example links?
__________________
JD-Dev & Server-Admin |
#58
|
|||
|
|||
What might be (/have been) missing from the API?
If they know what they're doing, it should have proper rate-limiting support which will communicate back to the client how long to back off for, which should work a little better than cookie-sharing and having to reset it regularly. Then again, "should" is one of my favourite English words. I'll take a look at some other API clients to see what they can and can't do. |
#59
|
||||
|
||||
I'm sorry but lost in history, I don't remember, too long ago.
__________________
JD-Dev & Server-Admin |
#60
|
||||
|
||||
"Should" not be necessary to regulary reset it as you don't have to regulary re-login at website. But I'm not familiar with current plugin at all.
__________________
JD-Dev & Server-Admin |
#61
|
||||
|
||||
Quote:
as far as I remember, the api was limited in features/posibilites in the past but I don't know about current api. sure, feel free to test other tools and check/compare their feature-set against JDownloader.
__________________
JD-Dev & Server-Admin |
#62
|
|||
|
|||
Quote:
They're almost certainly locking it out at the server side, since while it's running I occasionally get 403s in the browser. Anyway, it's good to at least have something working again. |
#63
|
||||
|
||||
To be fair, this highly depends on workload and user case. Doing few requests/downloads is complete different than (some ppl prefer to) download/mass crawl/create complete clone of sites/profiles
__________________
JD-Dev & Server-Admin |
#64
|
||||
|
||||
Absolute right. Better a working (but maybe limited) plugin that no support at all/complete broken plugin. Also, depending on use case, it might be better to use specialized tools/scripts for the job.
__________________
JD-Dev & Server-Admin |
#65
|
||||
|
||||
Quote:
Thanks for reporting. Quote:
Quote:
Quote:
Which API are you referring to? Please keep in mind that it is still heavily work in progress. I'm working on it every now and then when I'm finding time and I'm also relying on our user to provide feedback.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#66
|
||||
|
||||
The mentioned CORE-updates have been released and we've optimized quite some stuff.
We're now waiting for your feedback regarding our partly rewritten Deviantart plugins.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#67
|
|||
|
|||
Copying a gallery link to the clipboard does not crawl the gallery. Adding it manually in LinkGrabber reuslts in crawling the entire page rather than just the gallery entries.
26.11.22 06.41.34 <--> 26.11.22 06.43.39 jdlog://9141311370661/ |
#68
|
||||
|
||||
@mwl: Whenever you copy a link to clipboard and nothing happens, it is a strong sign that the site/link format is not yet supported by JDownloader. When you manually add/paste it and then links show up, than it's a 100% sign that the site/link format is not supported and deep decrypt tries to find anything that can be downloaded.
Please provide example links that you try to add, then we can check/update plugin to support those as well
__________________
JD-Dev & Server-Admin |
#69
|
|||
|
|||
This is a session that should have nothing else (apart from one stray download).
Log (should be in debug mode): 29.11.22 22.43.36 <--> 29.11.22 22.48.04 jdlog://6051311370661/ Attemped links: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** Hope these are useful. |
#70
|
|||
|
|||
Good day.
It looks like plugin generally works, yet there are some pictures, which don't have it's download button, yet there is information about them actually being way higher resolution, which current version of plugin cannot process. I had found some console app, that somehow can access API and does download full resolution versions - **External links are only visible to Support Staff****External links are only visible to Support Staff** Also, when i try to use cookies to log in, i get logged out relatively quickly (about 10-15 or so minutes), so i have to relog and readd cookies to account. |
#71
|
||||
|
||||
@ecclessia: can you please provide example links? all my tested links, JDownloader was always able to fetch highest resolution/quality version.
Quote:
Enable Settings->Advanced Settings->Log.debugmodeenabled and restart JDownloader, now wait for issue to happen and then create log and post logID here
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 01.12.2022 at 20:04. |
#72
|
||||
|
||||
@mwl: just use /gallery link and remove the trailing /all. Those urls are not yet supported but will be with next update.
with next update also /gallery/scraps will be supported
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 01.12.2022 at 20:12. |
#73
|
|||
|
|||
01.12.22 22.58.13 <--> 01.12.22 23.16.19 jdlog://2451311370661/
Here you go. I had specifically tried to add long galleries, yet it was on the first one **External links are only visible to Support Staff****External links are only visible to Support Staff** within 15 minutes or so. I hadn't seen logs myself, yet it might be due to me using vpn (no such problems with site without JDownloader though) or with great amount of request, even though i had tried to limit download speed to 100 kb/s. And, yes, after JDownloader starts saying, that cookies had expires, i'm also automatically logged out of site as well. |
#74
|
||||
|
||||
Could you also please provide example links where you think a higher quality is available but not found by JDownloader?
__________________
JD-Dev & Server-Admin |
#75
|
||||
|
||||
Quote:
@ecclessia: Quote:
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 02.12.2022 at 16:36. |
#76
|
|||
|
|||
Well, as per provided within application instruction:
1. I had installed "Flag Cookies" Firefox addon, l 2. logged into deviantart site, 3. specifically reloaded page just in case, 4. opened addon preferences and copied cookies through use of button @Export cookie data to clipboard 5. inside JDownloader application i had opened account manager, opened account login/password edit window (login was already applied earlier) 6. inserted clipboard data into password field and applied it. After check Jdownloader had indicated, that everything is ok (everything green, "free account" text), then i had added gallery to link parser, after gallery had been parsed i had started download. After a little more than 15 minutes file downloads had stopped starting with text "Account is missing", inside account manager was text "Cookies had expired" (or similar phrase). When i had created this thread (when deviantart had just changed it's layout) there was no such problem. |
#77
|
||||
|
||||
@ecclessia: I don't think that the cookies are expired but rather temp/IP blocked
JDownloader tries to connect website but fails due to shown message above. JDownloader has no special *blocked* handling and thus just tells that login via cookies has failed. Have you tried with fresh IP eg Reconnect/VPN? or wait some time. to me it looks more like IP blocked as the block comes from cloudfront
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 02.12.2022 at 16:38. |
#78
|
|||
|
|||
Quote:
Is there any way to put some timeout between downloads? I had tried to limit downloads to 1 at a time and to limit download speed, but that didn't help. |
#79
|
||||
|
||||
Quote:
Quote:
as far as I know there is a script that waits x seconds after each download. For example see https://board.jdownloader.org/showpo...postcount=2319 Once we know what *limits* there are or what might help to avoid this, then we can update plugin accordingly
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 02.12.2022 at 17:21. |
#80
|
|||
|
|||
Sometimes during downloads still being active site does throw 403 error (and stil from time to time opens with account still being logged in), but right after downloads had stopped (or, maybe, several seconds later) site definitely opens normally, but logged out of account.
|
Thread Tools | |
Display Modes | |
|
|