JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 06.03.2019, 09:32
michael88 michael88 is offline
Ultra Loader
 
Join Date: May 2017
Posts: 45
Default Plugin Request for tiktok.com profile crawler

A plugin request for tiktok.com. It would be great if all the videos that a user posted could be downloaded (not the one's that the user has liked). Some accounts have upwards of 500 videos, hopefully that isn't a problem.

Example Links:-
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I hope this is the right part of the forum, there was no suggestions thread in Jdownloader 2 subforum.
Reply With Quote
  #2  
Old 06.03.2019, 12:19
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 06.03.2019, 12:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 05.06.2019, 06:23
michael88 michael88 is offline
Ultra Loader
 
Join Date: May 2017
Posts: 45
Default

Any update on this one?
Reply With Quote
  #5  
Old 03.07.2019, 01:17
dageeza dageeza is offline
Linkgrabbing Monster
 
Join Date: Aug 2012
Posts: 84
Default

Hi Jiaz, any chance to work on this yet ?
Reply With Quote
  #6  
Old 10.07.2019, 16:46
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Added support for single videos - crawler for profiles will follow once I find the time ...

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 16.07.2019, 18:58
dageeza dageeza is offline
Linkgrabbing Monster
 
Join Date: Aug 2012
Posts: 84
Default

Jiaz you are simply the best !!
Reply With Quote
  #8  
Old 17.07.2019, 11:24
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@dageeza: thanks to pspzocker as he did the plugin, not me
__________________
JD-Dev & Server-Admin
Reply With Quote
  #9  
Old 18.07.2019, 09:00
dageeza dageeza is offline
Linkgrabbing Monster
 
Join Date: Aug 2012
Posts: 84
Thumbs up

Then thanks to pspzocker, and look forward to the profile crawler :-):thumbup:
Reply With Quote
  #10  
Old 28.07.2019, 19:15
nas3200 nas3200 is offline
Bandwidth Beast
 
Join Date: Jul 2015
Posts: 132
Default tiktok.com

Hello

How can i download files from tiktok.com

I can browse the site through the mobile but there is tiktok viewer Maybe it can help us

**External links are only visible to Support Staff****External links are only visible to Support Staff**

thanks
Reply With Quote
  #11  
Old 30.07.2019, 12:43
nas3200 nas3200 is offline
Bandwidth Beast
 
Join Date: Jul 2015
Posts: 132
Default

any help please
Reply With Quote
  #12  
Old 31.07.2019, 09:59
nas3200 nas3200 is offline
Bandwidth Beast
 
Join Date: Jul 2015
Posts: 132
Default

i can get video one by one i want get it all from user

**External links are only visible to Support Staff****External links are only visible to Support Staff**

thanks
Reply With Quote
  #13  
Old 31.07.2019, 16:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Merged tiktok threads.

@nas3200
There is a ticket as you can see.
Please either wait until someone works on that or add the links manually!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #14  
Old 01.08.2019, 09:08
nas3200 nas3200 is offline
Bandwidth Beast
 
Join Date: Jul 2015
Posts: 132
Default

ok thanks
Reply With Quote
  #15  
Old 18.04.2020, 07:21
nas3200 nas3200 is offline
Bandwidth Beast
 
Join Date: Jul 2015
Posts: 132
Default

Hi can you help me to grab videos into package by user name

**External links are only visible to Support Staff****External links are only visible to Support Staff**

thanks
Reply With Quote
  #16  
Old 20.04.2020, 14:52
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hi,
this is not (yet) possible - you will have to go on the users' page and add them manually.
See the ticket linked in the above post.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #17  
Old 03.09.2020, 13:15
accelheim accelheim is offline
JD Addict
 
Join Date: Jan 2020
Posts: 158
Default Download TikTok Profiles

JD2 can download single video links but downloading profiles isn't yet possible. Can this be implemented?

Example link:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Best regards
Reply With Quote
  #18  
Old 03.09.2020, 13:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Merged duplicated tiktok "Download complete user profiles" threads.
@accelheim
Next time please use the boardsearch and use for existing/similar topics before opening a new one - thanks.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #19  
Old 03.09.2020, 14:22
accelheim accelheim is offline
JD Addict
 
Join Date: Jan 2020
Posts: 158
Default

Quote:
Originally Posted by pspzockerscene View Post
Merged duplicated tiktok "Download complete user profiles" threads.
@accelheim
Next time please use the boardsearch and use for existing/similar topics before opening a new one - thanks.

-psp-
I guess I used the wrong search...

Any update on how long it will take until profiles can be downloaded?
Reply With Quote
  #20  
Old 03.09.2020, 14:30
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by accelheim View Post
Any update on how long it will take until profiles can be downloaded?
No. Probably never.

Again the hint: We're open source - you're welcome to contribute code.

Apart from that, JD can download single video items --> You can use helper browser-addons to e.g. auto-scroll down profiles and copy/add all items.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #21  
Old 03.09.2020, 14:51
accelheim accelheim is offline
JD Addict
 
Join Date: Jan 2020
Posts: 158
Default

Quote:
Originally Posted by pspzockerscene View Post
No. Probably never.

Again the hint: **External links are only visible to Support Staff**... - you're welcome to contribute code.

Apart from that, JD can download single video items --> You can use helper browser-addons to e.g. auto-scroll down profiles and copy/add all items.

-psp-
I see. You're saying there are addons that scroll down pages and collects all links from videos? Do you know of any?
Reply With Quote
  #22  
Old 03.09.2020, 15:14
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

You still ignored my hint regarding contributing code to our project ...
Quote:
Originally Posted by accelheim View Post
I see. You're saying there are addons that scroll down pages and collects all links from videos?
Yes for sure.

Quote:
Originally Posted by accelheim View Post
Do you know of any?
No not for this purpose only but there are general scripting addons such as e.g. Tampermonkey:
tampermonkey.net

There are probably also extensions available that are going more in the direction of what you want but you'll have to find them on your own.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #23  
Old 12.11.2020, 05:49
Emm Emm is offline
JD Beta
 
Join Date: Apr 2009
Posts: 50
Default

I would like to ask if it's possible to implement account login or cookie login to tiktok? Pref. the 2nd one. I've seen that in GDrive login.

The reason I ask is that downloading hits a snag, probably I'm on a shared IP, but also happens on VPN's, in that after I download 10-20 videos, I get a host problem? status. I mainly identified the issue, tiktok asks users to complete a captcha slider. If I enter the link into a chrome profile where I'm not logged in, that's what I'm getting right off, and on every new link I enter it's the same, so I suppose it's a current session verification, not a browser session one. If I do that into a profile where I'm logged in, I'm also getting that, but rarely. Or it might need a captcha implementation (something like filecrypt has) and a session that propagates on every new link attempted to download.

It used to be that I would be able to download 200-300 vids, but my IP is not static and I rarely get a clean one. Happens on other sites with recaptcha as well, rarely do I get a click and go one, mostly it's those with a few steps and some false negatives torce a redo.
Reply With Quote
  #24  
Old 12.11.2020, 11:34
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@Emm: Maybe we should just add support for that captcha slider Can you provide a debug log about this, see https://support.jdownloader.org/Know...d-session-logs
Enable Debug first, see Settings->Advanced Settings->Log.debugmodeenabled
and restart JDownloader. Now wait for issue to happen and then create log and post shown logID here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #25  
Old 12.11.2020, 16:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

@Emm
Plain cookie login will not fix your issue.
Tiktok additionally uses crypto for their API requests.

Also you're the first user who has reported this issue.
Please follow the instructions linked by Jiaz and post a log.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #26  
Old 12.01.2021, 16:07
mothos mothos is offline
Modem User
 
Join Date: Mar 2020
Posts: 1
Default TikTok crawler stops when trying to load profile.

I tried loading a couple of tiktok profiles but it stops adding videos after the first 2-6 clips. Is there a way to parse all of them or do I need to download clips individually?
Reply With Quote
  #27  
Old 12.01.2021, 16:21
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Merged tiktok profile crawler threads.

Please read the previous posts on this topic.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #28  
Old 26.02.2021, 22:46
Neurotoxin001's Avatar
Neurotoxin001 Neurotoxin001 is offline
Linkgrabbing Monster
 
Join Date: Feb 2021
Posts: 80
Default

Hello, can you please make my likes in tiktok support? I want to download all videos liked by me but now there is even no tiktok support in account manager.
Reply With Quote
  #29  
Old 01.03.2021, 15:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hello Neurotoxin001,
at this moment this feature is not planned to be implemented.
Here is how you could add those URLs:
1. Visit that "likes" page.
2. Scroll all the way down so all alements get loaded.
3. Press CTRL + A and CRTL + C afterwards to copy all URLs -> Add them to JD

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #30  
Old 02.03.2021, 09:36
Neurotoxin001's Avatar
Neurotoxin001 Neurotoxin001 is offline
Linkgrabbing Monster
 
Join Date: Feb 2021
Posts: 80
Default

Quote:
Originally Posted by pspzockerscene View Post
Hello Neurotoxin001,
at this moment this feature is not planned to be implemented.
Here is how you could add those URLs:
1. Visit that "likes" page.
2. Scroll all the way down so all alements get loaded.
3. Press CTRL + A and CRTL + C afterwards to copy all URLs -> Add them to JD

-psp-
Thanks, it's works, but I wonder how? There is even no links in my clipboard.
Reply With Quote
  #31  
Old 02.03.2021, 13:38
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by Neurotoxin001 View Post
There is even no links in my clipboard.
Yes there is.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #32  
Old 28.04.2022, 19:15
Baldachin Baldachin is offline
Baby Loader
 
Join Date: Jun 2018
Posts: 8
Default

Is there a possibility to download vids from a private profile? JDownloader shows all videos as offline when the profile is private.
Reply With Quote
  #33  
Old 29.04.2022, 10:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@Baldachin: currently there is no account support for TikTok and thus private profiles won't work
__________________
JD-Dev & Server-Admin
Reply With Quote
  #34  
Old 04.05.2022, 17:53
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Use this method as a workaround to download such content.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #35  
Old 24.05.2022, 17:44
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

The next CORE-update will add full profile crawling functionality.
You will be able to crawl all public profiles.
Provate profiles cannot be crawled (yet) because we haven't implemented account support for tiktok.com yet.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #36  
Old 25.05.2022, 14:58
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by Baldachin View Post
Is there a possibility to download vids from a private profile? JDownloader shows all videos as offline when the profile is private.
Do you have an example-account for that which we can use for testing?

I'm not yet in the mood to create a 2nd tiktok test-account just to befriend it with the first one in order to implement login support...
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #37  
Old 25.05.2022, 15:24
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #38  
Old 23.03.2023, 18:10
Baldachin Baldachin is offline
Baby Loader
 
Join Date: Jun 2018
Posts: 8
Default TikTok Full Profile Rip?

Hi,
is it at the moment possible to download all videos from a profile somehow? The API is not working as I understood and without using the API JD only finds the latest about 30 videos.

Thanks for the answer!
Reply With Quote
  #39  
Old 24.03.2023, 11:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Merged same threads.

@Baldachin
You've already asked 2022-04-28 here:
https://board.jdownloader.org/showpo...8&postcount=32
...so why again now?

Also I've provided detailed information on how to work around this none working profile crawler in our forums e.g. here:
https://board.jdownloader.org/showth...314#post511314
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #40  
Old 24.03.2023, 12:49
Baldachin Baldachin is offline
Baby Loader
 
Join Date: Jun 2018
Posts: 8
Default

I asked for private profiles that time. Now I'm asking for any profiles. Not private profiles worked fine some time ago.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:42.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.