JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 24.04.2024, 00:23
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default [NOT a JD bug] TikTok Missing Creation Date

Hi,

First off thanks for the amazing software.

So the issue I'm having is that most of the TikToks are missing the video creation date as part of the file name.

ex1
Code:
**External links are only visible to Support Staff**
shows the 2024-01-08 prefix date in the filename

ex2
Code:
**External links are only visible to Support Staff**
doesn't show the prefix date

Desire:
show the video creation date.

Last edited by snuffleupagus; 24.04.2024 at 01:15.
Reply With Quote
  #2  
Old 24.04.2024, 11:57
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

@snuffleupagus: TikTok API is currently not available and JDownloader can only access date on actual download/download attempt. Either wait for download to start or rightclick and do linkcheck.
Due to heavy bot/script protection, JDownloader can only access video in a way where no date is available
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 24.04.2024, 12:06
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Quote:
Originally Posted by Jiaz View Post
Either wait for download to start or rightclick and do linkcheck.
* or rightclick and do linkcheck after adding the links.
The date will appear then.

In the future (next update), you can turn off "Fast linkcheck" in the tiktok plugin settings to automate this step.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 24.04.2024 at 12:21. Reason: Highlighted part of reply
Reply With Quote
  #4  
Old 24.04.2024, 12:26
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

@pspzockerscene thanks but I'm not seeing the "link check" option. Is it the "check online status" with that I only see the site update not the date added. Please advise




Code:
**External links are only visible to Support Staff**
Reply With Quote
  #5  
Old 24.04.2024, 12:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Settings -> Plugins -> tiktok.com -> "Enable fast linkcheck?"

The setting there will only have an effect after the release of the next update.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #6  
Old 24.04.2024, 12:37
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Quote:
Originally Posted by pspzockerscene View Post
Settings -> Plugins -> tiktok.com -> "Enable fast linkcheck?"

The setting there will only have an effect after the release of the next update.
Great. I thought I was going blind since I didn't see the link check, I have enabled it and will wait until the next release. Any chance you could share when that would be?
Reply With Quote
  #7  
Old 24.04.2024, 12:40
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

Later today
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 24.04.2024, 12:51
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Quote:
Originally Posted by snuffleupagus View Post
Any chance you could share when that would be?
We got an update-FAQ which explains that:

https://support.jdownloader.org/know...pdate-behavior
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #9  
Old 24.04.2024, 15:49
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

The mentioned update has been released.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #10  
Old 24.04.2024, 18:57
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Thank you for the update. works beautifully with the TikTok link alone. However, when I use the following

Code:
https://board.jdownloader.org/showthread.php?t=87726
to append to the file name it removes the prefix date but does add the appended text. Unsure how to fix that.

Any help would be greatly appreciated.
Reply With Quote
  #11  
Old 24.04.2024, 21:19
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

@snuffleupagus: you want to add something to link, right? problem is that the date information is no longer available at crawling time, only at linkcheck/add time.
you could try to modify the rule to only modify filename after linkcheck -> linkstatus is online. Please give it a try
__________________
JD-Dev & Server-Admin
Reply With Quote
  #12  
Old 24.04.2024, 22:30
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

@Jiaz - I apologize but I can't seem to find the "modify filename after linkcheck -> linkstatus is online" in the condition. I"m not seeing any link check or link status options..please see attached image

Code:
**External links are only visible to Support Staff**
Reply With Quote
  #13  
Old 25.04.2024, 09:46
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

@snuffleupagus: File - is - Online, above the separator in the middle. with this condition, the rule only applies after the linkcheck/file status is online
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 25.04.2024, 11:27
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Thanks for the clarification. I have implemented that but didn't work.

Code:
**External links are only visible to Support Staff**
The result doesn't include the prefix date but does include the appended text.

However, the same url without the append text does include the prefix date.

Code:
**External links are only visible to Support Staff**
Code:
**External links are only visible to Support Staff**
Code:
**External links are only visible to Support Staff**
It almost seems like the both aren't executed either it's the link check or the cat not sequentially being executed not both.
Reply With Quote
  #15  
Old 26.04.2024, 00:09
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

@snuffleupagus: working fine for me. have you made sure to disable Settings->TikTok->"Enable fast linkcheck" before adding the links
__________________
JD-Dev & Server-Admin
Reply With Quote
  #16  
Old 26.04.2024, 10:32
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Quote:
Originally Posted by Jiaz View Post
@snuffleupagus: working fine for me. have you made sure to disable Settings->TikTok->"Enable fast linkcheck" before adding the links
Quote:
Originally Posted by pspzockerscene View Post
Settings -> Plugins -> tiktok.com -> "Enable fast linkcheck?"

The setting there will only have an effect after the release of the next update.

Ahhh. Ok thought it was to be left on per your post. But disabling it does return the desired functionality.

thanks for the help greatly appreciated and keep up the awesome work.
Reply With Quote
  #17  
Old 26.04.2024, 10:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

@snuffleupagus: Thanks for the feedback and sorry for the misunderstanding Glad to hear that it works now as intended
__________________
JD-Dev & Server-Admin
Reply With Quote
  #18  
Old 26.04.2024, 12:11
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Just a couple of notes.

append text.

inconsistency for append text.
- there is some inconsistency in bulk downloads. ex. 42 links. most work but there are some that don't which aren't consistent. Meaning if I redo the bulk list they aren't the same ones that don't append the text on the file. If I identify them and redownload those only, it works.
- I did noticed that if I leave them in the link grabber for longer that there is more consistency.

mp3 files
- so as know there is an .mp4 and .mp3 file now that downloads using this plugin. the .mp3 do not have the prefix date but do have the append text. this is also inconsistent.

soak period?
- noticed that the longer I wait to download the less unexpected file naming convention issue there are. Not sure if it's a soak period or something else.

I can work around this issue by redownloading or allowing a soak period.
Reply With Quote
  #19  
Old 26.04.2024, 12:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Quote:
Originally Posted by snuffleupagus View Post
mp3 files
- so as know there is an .mp4 and .mp3 file now that downloads using this plugin. the .mp3 do not have the prefix date but do have the append text. this is also inconsistent.
Log and example URLs please.

Quote:
Originally Posted by snuffleupagus View Post
soak period?
- noticed that the longer I wait to download the less unexpected file naming convention issue there are. Not sure if it's a soak period or something else.
Log and example URLs please.

Please post your log-ID here
If your report is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.
If your report is related to a login specific problem with a plugin supported website, enable debug mode before creating logs, see previously linked instructions.

Bitte poste deine Log-ID hier.
Falls dein Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.
Geht es um einen nicht funktionierenden Account-Login, aktiviere vor dem Erstellen deines Logs den debug Modus (siehe zuvor verlinkte Anleitung).

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #20  
Old 26.04.2024, 14:31
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

Quote:
Originally Posted by snuffleupagus View Post
mp3 files
- so as know there is an .mp4 and .mp3 file now that downloads using this plugin. the .mp3 do not have the prefix date but do have the append text. this is also inconsistent.
Disable Settings->Plugins->tiktok.com "Crawl soundtrack separately". Sorry for misunderstanding. It's about the date will be fixed with update
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 26.04.2024 at 14:38.
Reply With Quote
  #21  
Old 26.04.2024, 15:08
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Update is released.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #22  
Old 19.05.2024, 17:55
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

Ok still a few issues.

core issue: dates not appending to the file name or appending text.

observations:
- what is displayed in the JD window doesn't show up as the downloaded file. ex. there is no appended date but when the file is downloaded the appended date is added. It's difficult to troubleshoot which links have issues this way.
- items with availability that are ? (question marked) don't show the desired file name (no date or appended text)
- some are inconsistent
example:
first run: out of 38 links - 25 no date or append text, 12 with date and append text, 1 offline
second run: out of 38 links - 2 no date or append, 35 with date and append text, 2 had ?, 1 offline
third run: out of 38 links - 1 no date or append, 36 with date and append text, 1 had ?, 1 offline
- each re-run yields better results if the rerun is continuous.
- seems like if there is an extended soak period the results are better, if you run it after right after the crawling the results are poorer but wait longer there results are better. not sure why.. not always the case but sometimes very inconsistent.
- also, the availability for the same set of links changes. it's not always online sometimes it's ? on the same links while other times they are all online for the same set of links.

previously no matter run all were behaving the same.


links

Code:
**External links are only visible to Support Staff**
logID

Code:
19.05.24 08.28.03 <--> 19.05.24 08.40.54 jdlog://7592411370661/

Last edited by snuffleupagus; 19.05.2024 at 20:30.
Reply With Quote
  #23  
Old 21.05.2024, 13:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,031
Default

Quote:
Originally Posted by snuffleupagus View Post
ex. there is no appended date but when the file is downloaded the appended date is added. It's difficult to troubleshoot which links have issues this way.
- items with availability that are ? (question marked) don't show the desired file name (no date or appended text)
Without access to the API, date can only be known/fetched on actual download and not before as it's not available without API.
Items that could not be checked (due to no API access), then JDownloader cannot know about title/date/any meta information at all.




Quote:
Originally Posted by snuffleupagus View Post
logID

Code:
19.05.24 08.28.03 <--> 19.05.24 08.40.54 jdlog://7592411370661/
Log shows many server issues. On first check (HEAD Request) the file is available but returns 404 (File not found) or 503(Service Unavailable) on GET Request.

TikTok is cracking down more and more on tools and only matter of time till they close last *non api* doors.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #24  
Old 03.06.2024, 22:13
snuffleupagus snuffleupagus is offline
JD Adviser
 
Join Date: Feb 2020
Posts: 106
Default

@jiaz, so the interesting observation is that if you repeatedly load the links the more times you do it the better results for a period of time the the results reset. The inconsistency is still there.

any chance your magic powers can fix the issue? anything I can provide further to help?
Reply With Quote
  #25  
Old 04.06.2024, 12:20
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,970
Default

Quote:
Originally Posted by snuffleupagus View Post
any chance your magic powers can fix the issue?
No, I'am not going to add more workarounds to our existing tiktok.com plugin.

If the dates can be found, they will be added once the download is started.
If you need to see the dates inside the linkgrabber, feel free to build an EventScripter script which e.g. triggers the linkcheck up to X times whenever the date is missing.

EventScripter forum thread:
https://board.jdownloader.org/showthread.php?t=70525
EventScripter help article:
https://support.jdownloader.org/Know...event-scripter
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:10.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.