[shinji2009]

what about these two? jd says file not found, but if you get source download link from page, there is files, but different size
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

@shinji2009
Both of these links are also leading to the same 404 page just as the last two you has posted.

[shinji2009]

strange, i still can download them from source link and in browser.

[pspzockerscene]

Hm now the question is why.
Here is what I can imagine:

• Browser has cached the website and video direct-URLs are still available -> Clear your cache and/or try to open the links with another browser and see if they're still online there
• GEO-block + VPN: Maybe you are using a VPN in browser but not in JD (or vice versa) and those videos can only be watched via this VPN

Fact is, for me they're offline 100% and I can't see a JD plugin bug here.

[shinji2009]

this is not a browser cache, because i did not download this video with the browser yesterday. and i've never played a video in the browser. i see thumbnail of videos on the site, and not 404 as it was with previous videos.

i am using sort of vpn, but both browser and jd are behind it.

[pspzockerscene]

So I got no explanation for this so far but JD will only display the "File not found" error if it gets a response code 404 so I'am quite sure that your JD is getting exactly the same page as your browser.
If you want you can send me a log so I can double-check that.

One last question for now:
Can you view those videos maybe only when you are logged in into an iwara.tv account?

[pspzockerscene]

Okay I know what it was and it's not a JD bug!

Those videos look like offline when you are not logged in. The website will just display said "error 404".
Add your iwara.tv account to JDownloader and JD will recognize those.
This is either intentionally done by iwara.tv or just a bad website design.

[shinji2009]

i was authorised in both jd and site all the time.
also today jd are grab and download these videos ok. but they was "File not found" yesterday.

09.09.23 18.44.18 <--> 12.09.23 23.52.59 jdlog://8077311370661/

[pspzockerscene]

Your log is super big and contains a lot of unrelated events.
Here is other possibilities how the problem might have happened:

• Your iwara.tv login-session in JD did expire but JD failed to detect this -> This is not easy to check
• Your iwara.tv account got temporary deactivated (yellow account status) -> You didn't notice it -> Crawler ran without account -> Error

If this error happens again, please provide a fresh log and also try the following:
Rightclick on your iwara.tv account -> Refresh -> Then try to add the link again and see if it works now.

[shinji2009]

links was added sep 12 about 9:51 pm.
i don't think it was 404 or something with account, because jd crawl link and file size. but size was wrong.
ok i'll try.

[pspzockerscene]

Originally Posted by shinji2009

jd crawl link and file size. but size was wrong.

Which items' size (= which filename) was wrong?
Can you still reproduce this issue?

[shinji2009]

i has provide the links or i don't understand what do you mean by Which items.
i cannot reproduce it with this two links now, but this is a long time issue, since new iwara desighn and new jd plugin. basically at every 40 videos from iwara i get 2 with this error, so i can probably face it in the future with another links

[pspzockerscene]

Please provide fresh logs once the issue happens again.

Sorry, I can't fix an issue which I'am unable to understand/reproduce.

[lafhej]

Hi I noticed that sometimes I'm not downloading the source video. I am using this script
https://board.jdownloader.org/showpo...64&postcount=9
And pointing it at a text file that contains a bunch of profile urls.

The idea is that it'll go through the text file on a set interval (I have mine set for 6 hours I think) and it'll auto download any new videos.

I noticed that this video
**External links are only visible to Support Staff****External links are only visible to Support Staff**
the one jdownloader downloaded is around 96mb
But source video is 256mb

This is just one example that I found I'm sure there are many more but I don't know how to identify. I have logs if it helps.

25.10.23 15.56.30 <--> 24.10.23 02.19.05 jdlog://2568311370661/

[pspzockerscene]

Your log is very big and I can't find any problems in it.
For me, JDownloader is correctly downloading the ~256MB version of that video.

Please post a screenshot of your iwara.tv plugin settings.

[lafhej]

https://imgur.com/a/LLAGtt9
Yes now they seem to be source video when you try to download, I suspect it's when video is recently uploaded source video isn't downloaded. I wonder if some form of check can be implemented.

I have found three more videos that suffered the same
https://imgur.com/a/3WtgDyR
just to show added and finished timestamp

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

A common sign is that most of them are below 100mb and have 960x540 resolution, but those are not sure signs that they are not source videos.

[pspzockerscene]

Very strange!

Also those 3 other URLs are all over 100 MiB here.
Maybe this is some kind of IP based block or something?
Are you logged in into an iwara.tv account in JD?
If yes, try without - if you are not logged in, add an iwara account and try again.
I'll be going into details for one example here:

iwara.tv/video/CsKiBgbCCEt7ni

API call:

api.iwara.tv/video/CsKiBgbCCEt7ni

API response:

{"id":"CsKiBgbCCEt7ni","slug":"napoleon-eta","title":"Napoleon ETA","body":"**External links are only visible to Support Staff**,"status":"active","rating":"ecchi","private":false,"unlisted":false,"thumbnail":5,"embedUrl":null,"liked":false,"numLikes":302,"numViews":3878,"numComments":4,"file":{"id":"78833a09-a2be-4306-81ac-b7a771eecb30","type":"video","path":"2023/10/26","name":"78833a09-a2be-4306-81ac-b7a771eecb30.mp4","mime":"video/mp4","size":191063557,"width":null,"height":null,"duration":153,"numThumbnails":12,"createdAt":"2023-10-26T10:52:27.000Z","updatedAt":"2023-10-26T11:18:38.000Z"},"customThumbnail":null,"user":{"id":"1994d304-cf6e-4787-b921-6ab5de27fe7a","name":"avharu","username":"avharu","status":"active","role":"user","followedBy":false,"following":false,"friend":false,"premium":true,"locale":null,"seenAt":"2023-10-27T12:33:34.000Z","avatar":{"id":"53d6d6d3-4730-4f73-97d3-f75e47a04160","type":"image","path":"pictures","name":"picture-165525-1581035162.jpg","mime":"image/jpeg","size":121620,"width":null,"height":null,"duration":null,"numThumbnails":null,"createdAt":"2020-02-07T00:26:02.000Z","updatedAt":"2020-02-07T00:26:02.000Z"},"createdAt":"2017-06-28T15:32:35.000Z","updatedAt":"2023-10-27T12:36:20.000Z"},"tags":[{"id":"mikumikudance","type":"category"},{"id":"moeoh_ex","type":"general"},{"id":"napoleon","type":"general"}],"createdAt":"2023-10-26T11:18:38.000Z","updatedAt":"2023-10-27T12:33:10.000Z","fileUrl":"**External links are only visible to Support Staff**}

Download/Stream API call:

files.iwara.tv/file/78833a09-a2be-4306-81ac-b7a771eecb30?expires=1698414276542&hash=675ceeb5242caca36fb98cce4a231a9f70fb9a42654661a72374ce919537b8ce

API response:

[{"id":"2aba5d92-1eeb-4308-bfe7-62dd4059b460","name":"360","src":{"view":"//sukone.iwara.tv/view?filename=78833a09-a2be-4306-81ac-b7a771eecb30_360.mp4&path=2023%2F10%2F26&expires=1698414278&hash=59b82c91dcd12301a0e3aceb03b889052cfa8f34be519fff0688cc6275b7a76b","download":"//sukone.iwara.tv/download?filename=78833a09-a2be-4306-81ac-b7a771eecb30_360.mp4&path=2023%2F10%2F26&expires=1698414278&hash=59b82c91dcd12301a0e3aceb03b889052cfa8f34be519fff0688cc6275b7a76b"},"createdAt":"2023-10-26T11:23:41.000Z","updatedAt":"2023-10-26T11:23:41.000Z","type":"video/mp4"},{"id":"45e036a1-ea09-4e6a-9b5b-1eca94da68d5","name":"540","src":{"view":"//aku.iwara.tv/view?filename=78833a09-a2be-4306-81ac-b7a771eecb30_540.mp4&path=2023%2F10%2F26&expires=1698414278&hash=8b929275a2055863c61bbe303cb53a485b3d297629d2a6b058fa336249ea2e51","download":"//aku.iwara.tv/download?filename=78833a09-a2be-4306-81ac-b7a771eecb30_540.mp4&path=2023%2F10%2F26&expires=1698414278&hash=8b929275a2055863c61bbe303cb53a485b3d297629d2a6b058fa336249ea2e51"},"createdAt":"2023-10-26T11:28:10.000Z","updatedAt":"2023-10-26T11:28:10.000Z","type":"video/mp4"},{"id":"9eb60ef2-a6d8-4c33-9cd9-34e6d16a8128","name":"Source","src":{"view":"//piko.iwara.tv/view?filename=78833a09-a2be-4306-81ac-b7a771eecb30_Source.mp4&path=2023%2F10%2F26&expires=1698414278&hash=44d75f9d7afcb7f577b20b0bb5686db97146d9d78428e4eb6f740bfa60610f26","download":"//piko.iwara.tv/download?filename=78833a09-a2be-4306-81ac-b7a771eecb30_Source.mp4&path=2023%2F10%2F26&expires=1698414278&hash=44d75f9d7afcb7f577b20b0bb5686db97146d9d78428e4eb6f740bfa60610f26"},"createdAt":"2023-10-26T12:39:26.000Z","updatedAt":"2023-10-26T12:39:26.000Z","type":"video/mp4"},{"id":"473b1d0d-def2-4b2e-84da-dc9711a2cc2f","name":"preview","src":{"view":"//uta.iwara.tv/view?filename=78833a09-a2be-4306-81ac-b7a771eecb30_preview.mp4&path=2023%2F10%2F26&expires=1698414278&hash=892298a137df5580bd229ab8520282b44619bf51476148c94b527562e345e803","download":"//uta.iwara.tv/download?filename=78833a09-a2be-4306-81ac-b7a771eecb30_preview.mp4&path=2023%2F10%2F26&expires=1698414278&hash=892298a137df5580bd229ab8520282b44619bf51476148c94b527562e345e803"},"createdAt":"2023-10-26T17:06:08.000Z","updatedAt":"2023-10-26T17:06:08.000Z","type":"video/mp4"}]

As you can see, I'am getting the video quality/type "source" here.
Filesize: ~180MiB

My test results are the same with- and without active iwara.tv account.

[lafhej]

It is very hard to replicate as when I reset those links now they'll download the correct source videos. I do not know how to reliably replicate but it is very random. I am logged into my iwara account as I need to be able to download private videos that are only available to friends.

[pspzockerscene]

The question is if at this moment, the "source" type of video is just missing.

If that was the case, I could at least add handling to display an error message whenever "source" is missing.
It would be good to get a log that shows the problem / response of iwara.tv when the problem happens -> Without this I won't be able to help as long as I can't reproduce the problem.

[lafhej]

Originally Posted by pspzockerscene

It would be good to get a log that shows the problem / response of iwara.tv when the problem happens -> Without this I won't be able to help as long as I can't reproduce the problem.

Any idea how to get the logs?

[pspzockerscene]

Sure.
Log instructions:
https://support.jdownloader.org/Know...d-session-logs

[lafhej]

I have enabled debug mode, should I wait until I encounter the problem again or resubmit the previous logs?

*edit*

I think I might have a example right now.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

As of time of writing there is only 540 download and no source

https://imgur.com/a/fUcMTRc

https://imgur.com/a/nZqFGA7
27.10.23 23.39.10 <--> 27.10.23 23.36.45 jdlog://0668311370661/

Added log hopefully it captured it.

[pspzockerscene]

Wait until you encounter it again and the best would be if you would be able to reproduce it:
Then restart JD so you got a fresh session -> Reproduce the problem and upload a fresh log.

What would be interesting to me:
Usually JD will get the same response as your browser so technically if all of the sudden the "source" version is missing in JD, it should be missing in browser too when you're playing the video there.

Sure it could also be a measure of iwara.tv to block/shadow-ban bots/tools like JDownloader but that would be a very vague assumption at this moment.

[lafhej]

How to reproduce, I can only identify, cause I only find out if when I reset the link and check if it will download source version. I can only show the initial download is not source but when I reset and it downloads the source video it will only download source video from then on.
I cannot say every video less than 100mb or have 540p res is not source video cause sometimes actual video is really less than 100mb and user only upload 540p.

For example the above video I linked, linkgrabber show 20.51MiB size, but when download only 8.51 MiB. Here's even a recording.
**External links are only visible to Support Staff****External links are only visible to Support Staff**

That was earlier, at time of this post when I try to download it I cannot get the 8.51 MiB file anymore. Infact the current file is 20.54 MiB which is also different from the earlier.

[pspzockerscene]

EDIT The video you uploaded to streamable.com for me is down.


Also in your newest example, I get the ~20.5MiB video so for some reason you must get another API response from iwara.tv when downloading a lot of items.

Please try to use a new IP or a VPN and check the results then.
Maybe this is some kind of "shadow ban" by iwara.tv.
I remember hat the famous porn website "xvideos.com" added a shadow ban. Whenever this was active, a user would only always get the lowest video quality.

[lafhej]

I didn't know you won't be around in weekend so video expire.

It's kinda time sensitive I show you link and then three days later you test high chance of being source file. Now when I grab old link it'll also be the 20mb file.

How about this one

**External links are only visible to Support Staff****External links are only visible to Support Staff**

No VPN
01.11.23 12.03.33 <--> 01.11.23 12.05.37 jdlog://6378311370661/

Use VPN
01.11.23 12.08.01 <--> 01.11.23 12.09.32 jdlog://7378311370661/

I use no vpn and with vpn to download the same file, same result. When linkgrab it show 65MiB, download switch to 27MiB.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Originally Posted by lafhej

I use no vpn and with vpn to download the same file, same result. When linkgrab it show 65MiB, download switch to 27MiB.

Maybe we were talking past each other.
In this case / for this video, this is not a bug:
When you add the link, JDownloader will first take the filesize which the iwara.tv Web-API shows:
iwara.tv/video/Aoa36TfzaxxaNl
-->
api.iwara.tv/video/Aoa36TfzaxxaNl
--> "size":201881 -> That equals 64,76 MiB

Then later when the download is started, JDownloader will show the real filesize.
In this case, the source file is 27,41 MiB in size.
--> You can verify this in your browser.

Now you could argue that the iwara.tv Web-API is displaying wrong values but apart from that I'm still unable to find any kind of JDownloader bug here.

My theory:

• The iwara.tv Web-API is returning wrong filesize information
• or: The iwara.tv Web-API is returning filesize-information about the original file while the stream API "source" file is not really the original but also a transcoded file

Either way I'm unable to find any version of the above video that is 60+ MiB in size.

[lafhej]

I really don't know how to show you the problem. I submit log and recording multiple times already, but you say no problem. Maybe not jdownloader bug but maybe jdownloader not checking if video is really source.

The best example was this video **External links are only visible to Support Staff****External links are only visible to Support Staff**, where in my screenshot you can see it was download button was "Download (540p)" maybe cause recently uploaded. But if you check the page now it will show "Download (Source)". I even show you screenshot of jdownload when it download 8mb file at that time. But now I'll always get 20mb file and I can't reproduce the problem even though I got proof in screenshot that jdownloader didn't download source video, I don't know what is included in logs so I can't say but I submit logs also looks like useless cause you can't find issue.

The problem only occur during the time I post and lasts maybe a few hours. I understand if weekend no support but this problem is pretty time sensitive if I show you now and again it's almost weekend by the time you check the problem will almost 100% be unable to reproduce.

You can say just wait and download again, the problem I automate the downloads and many files so I don't know which videos suffered from that issue, it's only when I suspect and check manually then I'll find out that jdownloader didn't download source at that time.

[pspzockerscene]

Originally Posted by lafhej

but you say no problem

I didn't just look at it and say "no problem".
I've provided a detailed explanation.
Can you download a 64,76 MiB version of the example video which I was using in my last reply using other tools?

Originally Posted by lafhej

where in my screenshot you can see it was download button was "Download (540p)" maybe cause recently uploaded. But if you check the page now it will show "Download (Source)". I even show you screenshot of jdownload when it download 8mb file at that time. But now I'll always get 20mb file and I can't reproduce the problem even though I got proof in screenshot that jdownloader didn't download source video, I don't know what is included in logs so I can't say but I submit logs also looks like useless cause you can't find issue.

When I'm checking this video in my browser now, the max file size is ~20MiB so maybe that has changed serverside.
Please re-check.

Please re-read my last post.
It explains in details, why JDownloader is first showing a higher filesize which then changes.
It also explains why/how this can happen.


Until you got a proof that you can get a higher quality version of the video "Aoa36TfzaxxaNl", this is not a JDownloader bug!
Max in browser is: ~27MB
Max in JD is: ~27MB

Originally Posted by lafhej

I understand if weekend no support

Maybe you should also understand that our support is free of charge.

[lafhej]

For this video **External links are only visible to Support Staff****External links are only visible to Support Staff**
I cannot get 64MB, but I show as example cause it had the same effect
as this video **External links are only visible to Support Staff****External links are only visible to Support Staff** as first time I showed so I thought maybe can help you find problem.

The effect being when I link grab **External links are only visible to Support Staff****External links are only visible to Support Staff** it'll show 20MB but overwrite and download become 8mb. I show in video but expired. It is similar to the non expire video I show, link grab show 64MB but overwrite and download become 20MB. But I say many time it was then, now if I download same video **External links are only visible to Support Staff****External links are only visible to Support Staff** it'll be 20MB every time. So now when you check you'll say it's 20MB not 8mb.

I cannot get proof for this video **External links are only visible to Support Staff****External links are only visible to Support Staff** 64MB file.
But I have proof in screenshot for **External links are only visible to Support Staff****External links are only visible to Support Staff** that when I first download is 8MB, but only after many hours later when I try to download again now it'll be 20MB, so what am I supposed to do.

[pspzockerscene]

I still can't reproduce your problem.

Please either use other tools to download iwara.tv videos or deep-dive into this topic, re-read my post and provide the information needed to reproduce- and/or detect the "quality downgrade".
Repeating your previous posts will not help me to gather more findings.

I'm done here for now.

[lafhej]

Hi please check this video

**External links are only visible to Support Staff****External links are only visible to Support Staff**

~285MiB linkgrab
~79MiB download

I have use vpn and no vpn to download same result
Here are the logs
03.11.23 12.15.34 <--> 03.11.23 12.22.26 jdlog://1978311370661/

I have asked on iwara forum regarding "ip ban" "shadowban" and am sure it is not the cause. Instead it is slow encoding server affecting new uploads, that why so hard to reproduce. The video at time of post is up for 51 min but still serving 540p version. I will keep try until get source then upload logs again.

*edit*

Adding extra information

Hi you keep thinking it is "ip ban" "shadowban" even though I disagree, so I ask on iwara and mod respond that is cause of slow encoding server.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
That why so hard to reproduce, when I identify a video and by the time you check already hours past and encoding server process so you will say cannot reproduce. Now I can only say check new video but also difficult cause you see time range is so big and random, from minutes to hours cause depend on server load.

If you still don't accept, I have another reasoning why it is not a "shadowban".

If I download 100 videos and iwara have shadowban limit at for example, after 50 videos ban you, then I should get source video from video #1 to #50 then #51 to #100 will be 540p, that will mean a lot of my videos will be 540p. But this is not happen, I get random videos so maybe video #1 to #30 source, #31 will be 540p, then #32 to #80 will be source and than maybe #81 might be 540p than #82 to #100 is source. The occur is scattered and very random, so I ask why shadowban only 1 or 2 video when I donwloading 100 videos? What is point? What you save? If shadowban really happening then I should have a lot of these sort of videos not a few scattered in between.


So iwara have lousy server, can't be help, I already set profile check to be 12 hours from 6 hours, but I cannot control when people will upload and what server load will be, maybe someone upload only half hour before when I start checking profile and server still encoding then I will still get the random non source videos.

I suggest need some way to identify when you are downloading non source videos for example tag video resolution so when you see at least you can rectify instead of have archive polluted with non source and not know. I tried to change name convention from plugin customized to server original cause name convention is include 123_xxxYYY_Resolution.mp4 so at least can see if have source tag, but no work as it will still take date_uploader_videoid_title.ext

[pspzockerscene]

You didn't even read or understand my post #147(?)

One last time:
- The iwara.tv API claims that this video is 285MiB in size. This is that you can see in JDownloader when adding that link.
- Then when the download is started, the best quality / highest size is 79MiB
--> You can verify this information via browser for example by using VideoDownloadHelper.

Only the iwara.tv support can know what the 285MiB value that is in their Web-API stands for.

[lafhej]

Hi I understand, but I think you mistaken, I added extra information first but I need to post video first make you see right now download is 79MiB, please wait maybe 1 hour later and try download again if is 285MiB.

[pspzockerscene]

Let's put it this way:
If you want I can add a setting so that if the size of the final file is less than 75% of the file-size value provided via API, you will get an error-message.
...though I'm not sure if this will really help.
In the worst case, this will just happen for all downloads.
EDIT

I will soon be offline and back next week on monday.

[lafhej]

Hi, if you check video again Monday, you'll see now is source video, not ~285MiB but ~206MiB.
Logs
03.11.23 13.56.20 <--> 03.11.23 14.28.49 jdlog://5978311370661/

I mention in post earlier big problem is cannot identify these files are hiding in your downloads.
Is there no way to see from my logs the difference and either tag to filename or delay download?

I notice if you check website, if encoding still you will find download button only show 540p and 360p

1 hour from upload with 540p

But once encode finish, download button will show Source, 540p and 360p.

3 hour from upload with source

Can check website for this?

Also in your post about
api.iwara.tv/video/Aoa36TfzaxxaNl

I don't know how to read api but I see two size, one 201881, another 68950834 one bigger than other is that help?

[pspzockerscene]

201881 is the size of an image (thumbnail).
You can view the json of that API request via this webtool: jsoneditoronline.org

Again:
My offer on what I can do is in my last post #155.

I'm off into the weekend now.

[pspzockerscene]

Added simple detection of "too small files".
If the final file has less than 75% of the size previously detected via Web-API, an error can be displayed.
Screenshot:

This feature hasn't been released yet!
If you want you can already test it in the developer version:
https://support.jdownloader.org/Know...up-ide-eclipse

[ntljc16]

After making packagizers for iwara, for example packagizer about "**External links are only visible to Support Staff**. if 729 updates new video is there anyway to auto scrap those new videos and download? Should I just put that link to jdownloader again?

[pspzockerscene]

There is no official feature for this but with EventScripter scripts you could do it by creating a script which will only move all items which are not in downloadlist already to your downloadlist.

EventScripter forum thread:
https://board.jdownloader.org/showthread.php?t=70525
EventScripter help article:
https://support.jdownloader.org/Know...event-scripter