JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 05.10.2019, 11:54
Loberon Loberon is offline
Baby Loader
 
Join Date: Oct 2019
Posts: 7
Default Plugin for furaffinity.net

Edit: The thread title is wrong it's .net not .com sry for that I can't edit it.

Hello! I've been only able to find a 3 year old ticket regarding this and no proper forum threads, so I figured missing information/demand might be a cause for this not being a thing.^^ So I hope the following information/links help at least with that. If you find the time to work on it I'd really appreciate this feature. (I selected mostly links/images that will hopefully spare your sanity)

For being able to see mature/adult submissions it has to be activated in the account options or via SFW button in the upper menue, so account login support is necessary. I'll send account information to pspzockerscene. Also a captcha has to be solved for logging in, on there are 2 types of captcha which can be changed with a button below the captcha **External links are only visible to Support Staff****External links are only visible to Support Staff** (link from button)

links: sfw picture submission view: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw picture submission full view: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw picture source: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw picture download button link: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw audio submission view: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw flash submission view: **External links are only visible to Support Staff****External links are only visible to Support Staff**
sfw gif submission view: **External links are only visible to Support Staff****External links are only visible to Support Staff**
gallery: **External links are only visible to Support Staff****External links are only visible to Support Staff**
page 2 of a gallery: **External links are only visible to Support Staff****External links are only visible to Support Staff**
page 2 of scraps (nsfw-nudity):**External links are only visible to Support Staff****External links are only visible to Support Staff**
page 2 of favs (a bit nsfw): **External links are only visible to Support Staff****External links are only visible to Support Staff**
page 3 of favs(a bit nsfw): **External links are only visible to Support Staff****External links are only visible to Support Staff**
scraps (is separate from gallery): **External links are only visible to Support Staff****External links are only visible to Support Staff**
favorites: **External links are only visible to Support Staff****External links are only visible to Support Staff**
mature rated picture submission view (is sfw): **External links are only visible to Support Staff****External links are only visible to Support Staff**
mature rated picture submission full view (is sfw, just a zombie): **External links are only visible to Support Staff****External links are only visible to Support Staff**
mature rated picture submission source (two comic dudes fighting, kinda sfw): **External links are only visible to Support Staff****External links are only visible to Support Staff**
mature rated picture download button link (is sfw, just a zombie):**External links are only visible to Support Staff****External links are only visible to Support Staff**
adult rated story submission (sfw if you don't read the story): **External links are only visible to Support Staff****External links are only visible to Support Staff**
adult rated picture submission view (sfw just a cropped patreon preview): **External links are only visible to Support Staff****External links are only visible to Support Staff**
adult rated picture submission full (sfw just a cropped patreon preview): **External links are only visible to Support Staff****External links are only visible to Support Staff**
adult rated picture submission source(just a render of a girl in bikini): **External links are only visible to Support Staff****External links are only visible to Support Staff**
adult rated picture download button link(just a render of a girl in bikini): **External links are only visible to Support Staff****External links are only visible to Support Staff**

The file formats I've seen: jpg,png,mp3,txt,swf,gif
Also some gifs only work in full view. And saving the picture from view/full and download button yield the same picture quality, it has only thumbnail quality when taken directly from favorites for example though.
Some seemingly random number seems to be assigned to the pages of favorites I hope there's a way to still scrape the favs.
I've also read of bans regarding mass downloading there, maybe putting an optional 24h pause after ~1000 downloads would make sense (or at least putting in a warning message somewhere)

Thanks for reading and hopefully considering this request.^^

Last edited by Loberon; 05.10.2019 at 13:28. Reason: I made an error on the website link in the thread title
Reply With Quote
  #2  
Old 05.10.2019, 18:08
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Ticket:


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 19.08.2020, 13:12
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

I've added support to download single items from this website and a gallery crawler.

I haven't aded account support for now - just let me know if you still need that.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #4  
Old 19.08.2020, 21:27
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

Quote:
Originally Posted by pspzockerscene View Post
I've added support to download single items from this website and a gallery crawler.

I haven't aded account support for now - just let me know if you still need that.

-psp-
I think account access is needed, some artists lock their galleries so only logged in users can view their uploads, and most of the adult uploads require you to be logged in.

Last edited by DeusExBestia; 19.08.2020 at 22:23. Reason: Clarification
Reply With Quote
  #5  
Old 20.08.2020, 13:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Quote:
Originally Posted by DeusExBestia View Post
[...]some artists lock their galleries so only logged in users can view their uploads, and most of the adult uploads require you to be logged in.
Please post example URLs to such content/profiles.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 20.08.2020, 15:02
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Added account support.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 05.10.2020, 05:28
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

The crawler seems to be a little overzealous.

For example, I added " **External links are only visible to Support Staff****External links are only visible to Support Staff** " as a link to test out the plugin, and it grabbed only 79 out of the 100 submissions, doesn't appear to have looked at the scraps...

And then it went on a tangent and grabbed a whole mess of other things, looks like the images for some of the commenters on his page, banners, some sort of script.js from somewhere, then went on a crawling spree of a link to a twitter account posted on that page.
Reply With Quote
  #8  
Old 05.10.2020, 14:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Hi,

our crawler only supports galleries so far.
User-profiles are not (yet) supported.

-psp-
EDIT

Which "100 submissions" do you mean?

furaffinity.net/user/rionquosue
-->
furaffinity.net/gallery/rionquosue/
--> Contains 6 items
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 05.10.2020 at 14:52.
Reply With Quote
  #9  
Old 05.10.2020, 17:52
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

Quote:
Originally Posted by pspzockerscene View Post
Hi,
EDIT

Which "100 submissions" do you mean?

furaffinity.net/user/rionquosue
-->
furaffinity.net/gallery/rionquosue/
--> Contains 6 items
I created a user account. It's set to show all art, general/mature/adult etc. How do I send it to you?

Last edited by DeusExBestia; 05.10.2020 at 17:55. Reason: Removed logindata - NEVER post logindata in public!
Reply With Quote
  #10  
Old 05.10.2020, 17:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

@DeusExBestia
1. What do I need that account for?
2. Please post example URLs which lead to your proposed "100+ items gallery".

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #11  
Old 05.10.2020, 17:57
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

Quote:
Originally Posted by pspzockerscene View Post
@DeusExBestia
1. What do I need that account for?
2. Please post example URLs which lead to your proposed "100+ items gallery".

-psp-
Some of the items in user galleries are locked behind a login, so you have to be logged in with an account that can view adult artwork.

**External links are only visible to Support Staff****External links are only visible to Support Staff** for example is one of the items you can't view if you don't have an account.
Reply With Quote
  #12  
Old 05.10.2020, 17:59
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Okay now I understand.
Ticket:


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #13  
Old 05.10.2020, 20:37
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

I think there is already account support, the strange thing is that even when logged in not all the items are crawled in the gallery.
Reply With Quote
  #14  
Old 06.10.2020, 11:51
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Sorry - you're right.
I was in a rush yesterday so I didn't check everything and also, I actually got my own test-account already so - sorry again!

Apart from that I wasn't able to reproduce your issue.
Silgne URLs that are locked behind a login won't get displayed with their final filename in the linkgrabber but, if an account is present in JD, they will be downloaded just fine and filename will change accordingly once the download starts.

Please add a new problem description and more example URLs and a log if you are able to reproduce this issue.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #15  
Old 20.11.2020, 20:02
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

So, after a little bit more investigation I discovered that, as long as I load the /gallery/<username> and the /scraps/<username> as separate links, it grabs everything in those particular sections, at least on the few names I've tested. However if I add /user/<username> it grabs a bunch of random stuff, in addition to some/most of the gallery, and random things from whatever user commented on the page.
Reply With Quote
  #16  
Old 20.11.2020, 20:07
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

Only /gallery/ and /view/ are supported. The others are processed via deep crawling (auto find all kind of links) and that's causing the random stuff.
Can you provide a /scraps/ example url?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 21.11.2020, 06:59
DeusExBestia DeusExBestia is offline
Vacuum Cleaner
 
Join Date: Feb 2018
Posts: 19
Default

Quote:
Originally Posted by Jiaz View Post
Only /gallery/ and /view/ are supported. The others are processed via deep crawling (auto find all kind of links) and that's causing the random stuff.
Can you provide a /scraps/ example url?
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Looks like I have to ask someone to help create a "convert /user/" in an FurAffinity link to /gallery/ and /scraps/ event script xD Struggling to figure out how to adapt the twitter conversion script someone made for me.
Reply With Quote
  #18  
Old 21.11.2020, 12:13
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

Thanks! I think we can update plugin to support those links as well. I've updated the existing ticket
__________________
JD-Dev & Server-Admin
Reply With Quote
  #19  
Old 24.11.2020, 00:10
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Done.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #20  
Old 24.11.2020, 15:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,192
Default

Quote:
Originally Posted by DeusExBestia View Post
Looks like I have to ask someone to help create a "convert /user/" in an FurAffinity link to /gallery/ and /scraps/ event script xD Struggling to figure out how to adapt the twitter conversion script someone made for me.
I've just hardcoded this into our plugin.
It will add both of these URLs whenever you add a "/user/" URL.
Also it will create different packages depending on the added type (gallery/scraps).

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:08.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.