JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #41  
Old 21.06.2023, 20:42
DukeM's Avatar
DukeM DukeM is offline
JD Adviser
 
Join Date: Sep 2019
Posts: 103
Default

Quote:
Originally Posted by pspzockerscene View Post
While I do believe you I'm unable to reproduce this.
You might want to reach out to the developers of said LinkGopher browser addon and/or open an issue on their github.
Link to their github project:
github.com/az0/linkgopher/
Thanks but I'm not too sure if it's a LinkGopher issue. Although, this happens on both Brave and Firefox for me, it only does affect TikTok it seems. And I feel like it's something with my browsers for some reason (even using Private/Incognito).

Like when you view a profile page and you hover your cursor over a video, you'd get like a link preview on the bottom left of the screen, right? But for me, it just shows **External links are only visible to Support Staff****External links are only visible to Support Staff**. That's why I initially thought perhaps TikTok changed how their website works but you disproved that on your reply earlier.

So I've been trying to figure it out intermittently the whole day because I can't get links again no matter how many refreshes I do. But very weirdly, whenever I try to get links from the example profile you used earlier, I don't encounter any problems.

But this isn't your issue to fix. Haha sorry, just wanted to let it out a bit.

Quote:
Originally Posted by pspzockerscene View Post
Depends on your settings.
In most cases you should at least get a dummy item as feedback if a crawler fails.
For tiktok, at this moment, it depends on your plugin settings:
- Profile crawl mode API: No feedback
- Profile crawl mode website: Crawler should find up to 30 items (first page of videos on tiktok website)
I see, thanks. I totally blanked out that there's a Bugtracker status on the thread but thanks for clarifying the crawl modes though.

Anyway, thank you so much again and again!
Reply With Quote
  #42  
Old 22.06.2023, 10:25
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 68,419
Default

Quote:
Originally Posted by DukeM View Post
Thanks but I'm not too sure if it's a LinkGopher issue. Although, this happens on both Brave and Firefox for me, it only does affect TikTok it seems. And I feel like it's something with my browsers for some reason (even using Private/Incognito).
As said I'm on Chrome and there it is working.
It might also be a "browser related issue" but either way I think it would be a good idea to report it to the developers of that addon as it is clearly not working as intended.
I highly recommend creating a gituhb.com account for that. There is so many more nice open source with public repositories for which you can then also report bugs / request new features or even contribute your own code

Quote:
Originally Posted by DukeM View Post
Like when you view a profile page and you hover your cursor over a video, you'd get like a link preview on the bottom left of the screen, right?
I can confirm that but looks like this is only applied for the first few items. Scroll down multiple times so the pagination will load more items and those will show the links, at least here (this time tested under Chrome and Firefox).

Also by looking deeper into this, I just found out that indeed LinkGopher is unable to find any items of the first page of a tiktok profile but it does find all items after that.
Indeed this would then not be a LinkGopher bug as it really can't see those links as it looks like they were intentionally "hidden" by tiktok.

So if I was to crawl a tiktok user profile now, this is how I would do it:
1. Add profile link to JD so JDcan grab all items of the first page.
2. Scroll all the way down in browser, grab all the remaining links via LinkGopher and add them to JD.

Good luck.
__________________

On vacation / im Urlaub
START: 2023-09-26
END: TBA


JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Reply With Quote
  #43  
Old 23.06.2023, 23:30
An0nym0usAn0n An0nym0usAn0n is offline
Modem User
 
Join Date: Nov 2022
Posts: 2
Smile

Quote:
Originally Posted by goldensun87 View Post
I made my own guide detailing how I download TikTok profiles. It doesn't have visual steps like JD guides do, but I hope it will still be helpful.

Script: pastebin.com/ZbMUf5kS (copy code into notepad, save extension as .PS1 to convert to powershell)

If the link is not visible, it is a pastebin link, and the link ID is ZbMUf5kS.
This was very helpful, thank you!!! I've made my own home brew solution to automating steps 7+ in your guide.

TikTok Video Downloader
Credit: Made with the help of ChatGPT lol, it's not perfect, and has gone through a few iterations, let me know if you have any issues with it and I'll see if I can tweak it.

Script Location: pastebin.com/QG78NJ1C

This PowerShell script enables you to download TikTok videos from a specific user profile. It uses the yt-dlp command-line tool to download videos, and PowerShell to automate the process, and downloads all the videos into the username's folder.

The following are prerequisites for using the script:

Prerequisites

1: Operating System: Windows. The script is a PowerShell script, which is a Windows-based scripting language.
2: yt-dlp: This is a command-line tool that is used to download videos from YouTube.com and other video platforms. You can get it from the yt-dlp GitHub page. Download the Windows executable and place it in the C:\URLscrape directory on your machine.
3: Browser Extension: LinkClump ONLY (not open multiple URL's, logic built into script). This add-on is used to collect the URLs of the TikTok videos to download. Follow the steps in the provided JD instructions to configure this extension and collect the URLs. (Note: I use firefox).
4: Text Editor: Notepad++ or similar (I use Sublime with Powershell syntax). This is used to view and edit the PowerShell script if you need to.

How to Use

1: Follow the provided instructions in the original paste bin (steps 1-6, pastebin.com/ZbMUf5kS, credit: goldensun87) to use LinkClump to collect the URLs of the TikTok videos you want to download. Save the copied information in a text file named URLinput.txt and place it in the C:\URLscrape directory (copied information does not need filtered via Open Multiple URL's browser extension, script will handle).
2: Run the PowerShell script (copy code from green pastebin link above into a notepad file, and save file extension as ".PS1". If it's not already running with administrative privileges, it will relaunch itself with admin rights.
3: When prompted, enter the username of the TikTok profile you want to download videos from. The username should include the '@' symbol (e.g., @tiktokusername).
4: The script will filter the URLs in URLinput.txt to only include those that contain the specified username. It will save the filtered URLs in a new text file named output_{username}_{date}.txt in the C:\URLscrape directory, where {username} is the username you entered and {date} is the current date.
6: Each video file will be named after its title on TikTok in a folder named after the username provided. Default folder creation location is C:\URLscrape.

Notes

1: The script assumes that all required files (URLinput.txt, yt-dlp.exe) and directories are in the C:\URLscrape directory. If you want to use a different directory, you will need to edit the script to change the paths / file names. You can name the C:\ directory whatever you want, same with the input file, you just need to update the script to match.
2: The script does not provide any progress indication (other than window beep when done) while yt-dlp is downloading videos. It may appear to be doing nothing, but as long as it hasn't exited, it's likely that yt-dlp is still downloading videos.
3: The script will pause after downloading the videos to keep the PowerShell window open. Press any key to exit.
4: I've not tested trying to download multiple profiles at once via the script.

Last edited by An0nym0usAn0n; 24.06.2023 at 21:40. Reason: Included logic for ALL URL filtering (browser extension no longer necessary)
Reply With Quote
  #44  
Old 26.06.2023, 16:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 68,419
Default

@An0nym0usAn0n
Just a quick sidenote:
You're using yt-dlp which does already support downloading tiktok profiles out of the box but that feature is broken at this moment.
Related ticket:
github.com/yt-dlp/yt-dlp/issues/3776

If it was working, all you'd need to do is to run a single yt-dlp command
__________________

On vacation / im Urlaub
START: 2023-09-26
END: TBA


JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 04:20.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.