JDownloader Community - Appwork GmbH
 

Reply
Page 1 of 5 1 23 Last »
 
Thread Tools Display Modes
  #1  
Old 05.10.2021, 05:00
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Talking request for iwara.tv plugin improvement
when copying an iwara channel link is it possible to make iwara plugin capture all videos in that channel with correct videos filenames ?
Reply With Quote
  #2  
Old 05.10.2021, 10:50
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
What exactly do you mean by *channel*? can you provide example links?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 05.10.2021, 15:14
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
this is a channel **External links are only visible to Support Staff****External links are only visible to Support Staff**

on uploaded videos if you click on "see all" it will take you to that channel all videos
**External links are only visible to Support Staff****External links are only visible to Support Staff**

is it possible when copying one of those two urls jd2 grab all videos that channel has ?
Reply With Quote
  #4  
Old 05.10.2021, 15:20
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
Added crawler for user profiles.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 05.10.2021, 16:27
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
also when copying a video link the filename starts with the name of channel then the url of the video and at last the actual video title

is it possible to make it grab the actual video title only as it's provided below the video ?
Reply With Quote
  #6  
Old 05.10.2021, 16:31
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
You can simply auto-alter filenames using a simple Packagizer rule.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 05.10.2021, 18:14
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
it's hard to do.. how can i remove video uploader name and last part of url and only keep the actual filename, please help me do this

i tested with this link
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Last edited by Jiaz; 05.10.2021 at 18:25.
Reply With Quote
  #8  
Old 05.10.2021, 18:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
Filename - contains - .+_(.+\..+)$
enable regex
Downloadurl - contains - iwara.tv
then set
Filename - <jd:orgfilename:1>
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 05.10.2021, 18:41
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
that worked, thank you so much.
Reply With Quote
  #10  
Old 05.10.2021, 19:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
You're welcome
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 08.10.2021, 01:47
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
hey thank you so much for implementing the crawler in today's update

there are some minor things that needs fixing :

i tested with this link :
**External links are only visible to Support Staff****External links are only visible to Support Staff**

the folder that was captured by jd2 had the name in the url instead of the name inside the page, as you can see in the picture below the name in the url is different and not correct

also inside the folder grabbed by the crawler there is 2 files that aren't videos uploaded by uploader and are not supposed to be grabbed ( Plugin Defect!TSOF 402 & thumbnails.mp4 ) as you can see in the picture





there is another thing that i noticed with this particular profile

**External links are only visible to Support Staff****External links are only visible to Support Staff**

the videos titles captured aren't correct
fyi my jd2 font name is Dialog
Reply With Quote
  #12  
Old 08.10.2021, 10:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
Wait for next plugin update
Can you provide screenshot/link for *wrong titles*? I can't reproduce here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 08.10.2021, 17:00
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
after today plugin update profile name was corrected but the captured videos names are short and incorrect

this above applies only for this particular user..

as other users and the one i reported yesterday are now showing correct username and correct videos titles

here is the example i reported yesterday and today it was corrected with the plugin update
Last edited by Jiaz; 08.10.2021 at 17:48.
Reply With Quote
  #14  
Old 08.10.2021, 17:48
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
check again with next update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #15  
Old 09.10.2021, 17:26
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
last issue wasn't fixed with yesterday 2 consequential updates, in fact the updates changed nothing.

and unfortunately i discovered a new issue :
the crawler doesn't always capture the uploaded videos if the link copied isn't the /videos of the user profile URL, some profiles just doesn't have a /videos URL section like the rest of the profiles, here is an example:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

so the crawler need to grab the videos without being in /videos section of the user profile URL

also there are some user profiles that can be viewed only if you are in Japan, but if you add /videos to their URL you can actually access the videos like this example :
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

the solution here is to make the crawler grab the videos in both ways with and without /videos in the URL thus grabbing the videos if the user have no /videos URL like first example and/or in 2nd example when the user profile is restricted only to japan we add /videos to the URL and copy that link to bypass the restriction.
Last edited by Snookie; 10.10.2021 at 00:08.
Reply With Quote
  #16  
Old 10.10.2021, 16:29
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
please don't give up on this.
Reply With Quote
  #17  
Old 11.10.2021, 15:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
Hi again,
I've updated the crawler to take the URLs as they come so if you add e.g.:
Code:
bla.iwara.tv/users/xyz
...exactly this will be used.
If you want it to use the one ending with "/videos", just add that.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #18  
Old 11.10.2021, 18:34
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
thank you so much, i really appreciate you making our life easier.
Reply With Quote
  #19  
Old 11.10.2021, 18:38
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
Quote:
Originally Posted by Snookie View Post
please don't give up on this.
Weekend

Quote:
Originally Posted by Snookie View Post
last issue wasn't fixed with yesterday 2 consequential updates, in fact the updates changed nothing.
The crawler can only fetch/crawl the names that are available on website. If the name is too long, the full name is not available at crawling but only after actual download.
To fetch full name while crawling, plugin would need to crawl/check each link individually and thus would be much much slower and can easier cause IP block because of too many requests.

EDIT: found a better way to grab name, I guess this solves the issue, wait for plugin update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 11.10.2021, 19:17
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Quote:
Originally Posted by Jiaz View Post
Weekend
sometimes i feel ashamed of how stupid i can be


Quote:
Originally Posted by Jiaz View Post
EDIT: found a better way to grab name, I guess this solves the issue, wait for plugin update
after plugin update :

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Reply With Quote
  #21  
Old 11.10.2021, 19:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
@Snookie: Check our packagizer rules, they remove everything until last _ ! and the name also contains _
I will work on this by tomorrow
__________________
JD-Dev & Server-Admin
Last edited by Jiaz; 11.10.2021 at 19:23.
Reply With Quote
  #22  
Old 11.10.2021, 19:48
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Quote:
Originally Posted by Jiaz View Post
@Snookie: Check our packagizer rules, they remove everything until last _ ! and the name also contains _
I will work on this by tomorrow
wow that really worked

Reply With Quote
  #23  
Old 11.10.2021, 20:43
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
i tried modifying the Regex by learning a little about it using google but i'm out of my depth here.
Reply With Quote
  #24  
Old 12.10.2021, 13:40
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
__________________
JD-Dev & Server-Admin
Reply With Quote
  #25  
Old 12.10.2021, 15:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
Added plugin specific Packagizer properties for the next update.
You can then simply build your desired filenames based on them and completely without RegEx.

List of available tags for iwara.tv:
  • user
  • title
  • videoid

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #26  
Old 12.10.2021, 17:36
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Quote:
Originally Posted by pspzockerscene View Post
Added plugin specific Packagizer properties for the next update.
You can then simply build your desired filenames based on them and completely without RegEx.

List of available tags for iwara.tv:
  • user
  • title
  • videoid

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!

-psp-
i honestly don't know what to do here, can you guys please help me create this Packagizer rule to remove user and videoid from title
Reply With Quote
  #27  
Old 12.10.2021, 17:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
@Snookie
I don't want to be rude but your answer sounds as if you didn't even bother reading that article (see information about "Plugin specific properties" in that article).
Nevermind...here is a Packagizer rule that will alter iwara.tv filenames to only contain title + file-extension:
(Screenshot)


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #28  
Old 12.10.2021, 18:03
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
@pspzockerscene
the only link i got from you is this and i promise you i read it all and couldn't find in it how to do this..
i spend most of my internet time on iwara and having this easy way of downloading videos will make my download time significantly shorter, so thank you for your help, i'm eternally grateful to you guys.
Reply With Quote
  #29  
Old 12.10.2021, 18:34
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
now that i have read it again i think i understand how to work with it more, so many things i now want to change using this, with practice i'll get there, so thank you again for this.
Reply With Quote
  #30  
Old 12.10.2021, 18:51
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
@Snookie: In case you need help/got questions, please don't hesitate and just ask
__________________
JD-Dev & Server-Admin
Reply With Quote
  #31  
Old 12.10.2021, 20:16
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Quote:
Originally Posted by Jiaz View Post
@Snookie: In case you need help/got questions, please don't hesitate and just ask
thanks for saying that

btw i just encountered a new issue :

the crawler on the non /videos URL grab only the 16 visible videos when the user actually have 53 videos that can only be grabbed when copying the /videos URL

here is an example
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #32  
Old 13.10.2021, 14:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 72,746
Default
Fixed.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #33  
Old 13.10.2021, 21:09
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Quote:
Originally Posted by pspzockerscene View Post
Fixed.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser **External links are only visible to Support Staff**...! | Please read our **External links are only visible to Support Staff**...!

-psp-
after today fix the crawler doesn't capture the video titles on some user profiles

example :

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #34  
Old 14.10.2021, 11:39
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
@Snookie: wait for next plugin update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #35  
Old 14.10.2021, 20:02
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
i just tested it after installing latest update and i can safely say it's 100% working flawlessly, guys i'm beyond grateful for all your hard work on this, thank you honestly.
Reply With Quote
  #36  
Old 15.10.2021, 09:37
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
You're welcome
__________________
JD-Dev & Server-Admin
Reply With Quote
  #37  
Old 16.10.2021, 11:44
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
hi again.. i hate to do this but there a issue that i just discovered

most of the videos names showing as "View user profile.mp4"

**External links are only visible to Support Staff****External links are only visible to Support Staff**



also single video link when copied jd2 didt grab the video

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Last edited by Snookie; 18.10.2021 at 13:24.
Reply With Quote
  #38  
Old 18.10.2021, 11:13
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
Filename will be fixed with next update
Single Video Link did work fine here. Maybe you have enabled filter in Linkgrabber, right side Quickfilters? or video requires you to be logged in at youtube?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #39  
Old 18.10.2021, 12:13
Snookie Snookie is offline
JD Adviser
  
Join Date: Jun 2015
Posts: 104
Default
Filename is fixed after update, single link works fine i just didnt notice it because it has a youtube icon, lastly thank you for the quick fix.
Reply With Quote
  #40  
Old 18.10.2021, 12:24
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
  
Join Date: Mar 2009
Location: Germany
Posts: 80,906
Default
Thanks for the feedback and you're welcome
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply
Page 1 of 5 1 23 Last »

« Previous Thread | Next Thread »
Thread Tools
Display Modes
Linear Mode Linear Mode

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

All times are GMT +2. The time now is 09:15.
JDownloader Community Board - Archive - Top
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.