JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #21  
Old 12.01.2022, 20:21
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Did you check the logs from post #16? I don't get it why you ask for logs then don't post your findings here. I assume the logs show the errors I am getting here, don't they?

BTW, I checked again and stopped working even with the GD account there. It took less than a day to break things.

Then I added another acc and it started working. Again. It is NOW..

This was the last folder I tested:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

What may be happening here is JDW being required to:

a) Having a GD account there to download specific old/resourcekey URLs

b) After you add then you start downloading. If this is how it would fix for me, no problem. But eventually the credentials put there sooner or later become invalid.

You either need to fix that area or explain why this happened with me.

That would account for why in the interval one 1 minute I was able to bypass the skipped - acc missing.

The idea of using my IP as a proxy would not be feasible, at least I have no idea how to make this work.

And I don't know why the insistence in this or using any GD account I can give you because they are not special. Anyone you have in your computer for testing is no different than mine.

I explained this before and you didn't listen:

- The affected files/folders are NOT PRIVATE;
- Any account I own has no special clearance to access them. It can only download with the full GD URL which has 'resourcekey" written to it.

For example: if I create a new acc tomorrow, then I add this ACC to JDW, and later I tell it to download any folder...

This in no way is different than you doing the same in the United States or Sweden.

Because it's not an IP blocking we are discussing here.

And if I didn't mention this before, my IP is DYNAMIC. It changes every X days automatically.

Last edited by Perene; 12.01.2022 at 20:29.
Reply With Quote
  #22  
Old 12.01.2022, 20:34
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Perene View Post
Did you check the logs from post #16?
Yes.

Quote:
Originally Posted by Perene View Post
I don't get it why you ask for logs then don't post your findings here.
No new findings. I don't need new logs.

Quote:
Originally Posted by Perene View Post
b) After you add then you start downloading. If this is how it would fix for me, no problem. But eventually the credentials put there sooner or later become invalid.
Cannot confirm.
My Google cookies were imported over 6 months ago and are still valid.

Quote:
Originally Posted by Perene View Post
You either need to fix that area or explain why this happened with me.
I can't do either of those at this moment.

Quote:
Originally Posted by Perene View Post
I explained this before and you didn't listen:
That's just wrong. You didn't read my posts.

Quote:
Originally Posted by Perene View Post
- The affected files/folders are NOT PRIVATE;
I never stated that.

Quote:
Originally Posted by Perene View Post
- Any account I own has no special clearance to access them. It can only download with the full GD URL which has 'resourcekey" written to it.
I never stated anything else.

Quote:
Originally Posted by Perene View Post
This in no way is different than you doing the same in the United States or Sweden.
Yes it is. The same way it is different for other users...all of which don't seem to haver the issue you're facing.

Quote:
Originally Posted by Perene View Post
Because it's not an IP blocking we are discussing here.
You can't be sure of that.
There are many ways/combinations of blocking.
Google may as well recognize JDownloader all of the time and simply block it whenever they want to.

I'm not meaning to be rude but I don't know how else I should answer this time:
You didn't read my answer!
I'll quote myself one more (3rd!) time and if you do not provide the information I need in order to look into this I will not answer this thread anymore!

Quote:
Originally Posted by pspzockerscene View Post
I already told you what I need to be able to check this (apart from logs).
Here again:
If possible, please provide one or both of the following information:
- A google account (send login credentials to support@jdownloader.org)
- A way to use your IP as proxy for testing
-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #23  
Old 12.01.2022, 23:30
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

The credentials from a new free acc were sent to this email. Just now.

Ticket ID: FVY-189-90587

Unfortunately I can't download anymore regardless of putting a new Google account there. But before the skipped - acc missing error appeared here and prevented any new downloads from the targeted (only those) old/resourcekey URLs I was able to see in ALL links this other error message:

"insufficient permissions/private file or quota reached"

Then ALL links had this replaced for skipped...

Funny to realize that if this is some sort of bandwidth restriction, it's only affecting the old URLs.

It's sad only these specific files can't be downloaded. Among a million others.

And there shouldn't be ANY restriction, this cannot be possible because there were days I downloaded over 300 GB from new stuff. Using free accounts.

JDW itself says unlimited quota in the account area. But with these companies you never know the latest shit they are pulling.
Reply With Quote
  #24  
Old 13.01.2022, 12:41
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,342
Default

@Perene: Thanks for the mail. I will forward to pspzocker.

To be honest, I did not read the full conversation, but some more ideas:
-you don't logout of the session that you have exported to JDownloader, do you?
-sounds like you are using/having several accounts? maybe when you switch account in browser,
the existing session will be logged out and thus no longer working. You explained that using firefox and fresh login finally worked.
- have you tried to download those files without being logged in (eg incongito tab) in browser?
- are you using sort of VPN/Proxy in browser? maybe some account settings to harden security and when IP changes the cookies won't work anymore?

just some thoughts.

@Perene and @pspzocker: Just want to remind you to stay calm and be respectful to each other. I know it's a long thread and we just want to help and you just want to get help I'm confident that in small steps we will find out what is going on
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 13.01.2022 at 12:47.
Reply With Quote
  #25  
Old 13.01.2022, 13:51
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Quote:
Originally Posted by Jiaz View Post
- are you using sort of VPN/Proxy in browser? maybe some account settings to harden security and when IP changes the cookies won't work anymore?
No VPN here.


Quote:
Originally Posted by Jiaz View Post
To be honest, I did not read the full conversation, but some more ideas:
-you don't logout of the session that you have exported to JDownloader, do you?
-sounds like you are using/having several accounts? maybe when you switch account in browser,
the existing session will be logged out and thus no longer working. You explained that using firefox and fresh login finally worked.
- have you tried to download those files without being logged in (eg incongito tab) in browser?
In my case once I put the credentials in JDW I may or may not log out of the GD account.

All my browsers except one (Opera, the one I use most) are configured this way:

a) Always open in private/incognito mode
or
b) Never save anything after being closed, so all cookies/session data/etc. are erased. (this is valid for Firefox)

This is the equivalent of CCleaner erasing anything except passwords of your browser.

Once you insert the credentials in JDW then anything should work, if in this case the bug requires a logged account, right? And you might as well erase the session saved in all browsers.

The problem is that I tried the opposite of what I usually do, too.

I tried inserting an account and letting the browser with a session active, so not closed. While running JDW at the same time.

I tried putting the same GD account that is actually logged permanently in Opera and without erasing anything there; no luck.

I tried informing to JDW the Google account that is sharing the same URLs I am trying to download.

Same skipped - acc missing error.

Whatever GD account I put there seems to be prevented from downloading only certain URLs.

This is how I know this is Google's doing or your plugin not working 100%.

That's also why I reject the idea that JDW is somehow being prevented due to bandwidth quota.

You aren't going to convince me of that, because if you think about it, this quota is only applied to these specific files? No.

This isn't how it works when Google blocks you from further downloading from their servers.

The URL needs to meet these requirements to not be downloaded by JDW:

a) Last modif. date from 2018 or older, regardless of the URL being or not a folder (yes, even individual files, not folders, are not being downloaded! *) and also having this below applied:

* This one wasn't:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I tried yesterday and it was rejected. Tried today, the same! This is a file. Not a folder. But it's an old one I uploaded there.

b) It needs to have this thing applied to it:

********
What's a resource key?

A resource key is an additional security measure in a file’s URL. It helps protect your file from unintended access. To see if a file URL has a resource key, look for resourcekey=.

Who's impacted, what to do, and when

If you don’t take any action, the security update impacts files, folders, and shared drives that don’t have a resource key. However, Google Docs, Sheets, and Slides files are not impacted by this change. By default, all users not in a Google Workspace for Education organization can remove the update from files that they own or manage.

Even if you have no impacted content, this update might affect your organization in the future. Impacted files can come to your organization through shared drives or a change in file ownership. Look for an email alert with the subject Security update for Drive. For details about impacted files, shared drives, and users in your organization, go to Use the alert center.

What's changing

On September 13, 2021, Drive applies a security update to make file sharing more secure. As an administrator, you can choose how to apply this update.

This update changes the links used for some files. Access to impacted files won’t change for people who have already viewed them or who have direct access, but others might need to request access.

Important: For files and folders that you use in internal sites or documentation, make sure to update the links to the new links that include resource keys. You can get the new link from the item’s owner if it’s not shared directly with you.

*********

Since this started to happen September 13, 2021, then you all need to take into consideration JDW was working properly before this date.

Now it isn't.

The criteria for which Google applies this change I don't know, all I can say is that they did for ALL my older files. ALL OF THEM. Not the new, the old!

If the URL is not a resourcekey JDW can download anything. If I tell it to download 1 TB of data it will right now, if the last modification date is 2019 or newer it will...

I even put examples here proving that among many other files/URLs JDW deliberately rejected downloading from only those OLDER/resourcekey files.

Among 100 files JDW downloaded 99. Rejected 1 that was old and using resourcekey.

If this isn't indisputable evidence I don't know what else is.

Why do I sound like I am speaking with such a confidence? Because I have more than 21 TB of data spread over free 15 GB accounts created in the span of 6-7 years.

That's why.

So whenever anyone here says I don't know what I am talking about and that my examples are irrelevant, you can see why I would be annoyed because I have a lot of proof to affirm the plugin broke.

At least on my side.

pspzockerscene is claiming he can download anything he wants even without informing a free Google account in his computer, I can say the same, if the links are not old and using resourcekey URLs.

Not only I have all this data, I also have records from ALL the accounts that store them and all the URLs from all the contents.

And the irony is that I need the mass-download of all my files because I am moving away from Google, due to their antics such as locking me out of random accounts and requiring a phone number (which can't be reused) to go back to them again, or having to log into all to prevent a 2 year inactivity period (which erases the data), or wasting time having to log into them... so it would be better to save elsewhere or decide which data is essential and what's not.

Quote:
Originally Posted by Jiaz View Post
- have you tried to download those files without being logged in (eg incongito tab) in browser?
I can download all affected files using a browser or program like Internet Download Manager.

And they aren't private. These are public URLs.

The problem is that I can't manually download so many links, this would be a waste of time. For example: say it's 100 files. Instead of informing JDW 10 URLs and letting it download all 100, I would need instead to visit one by one and hit download.

For example, I need to download this content below:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Take a look what JDW downloaded so far:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I tried a couple of minutes ago the rest. First JDW said:

>>>>>>>>>>>>>>
Insufficient permissions/private file or quota reached.
>>>>>>>>>>>>>>

For all of them.

Then the next second it changed the error message to

>>>>>>>>>
SKIPPED - ACC MISSING.
>>>>>>>>>

Whatever this bug is it's telling JDW that all these links are private. When they are not.

The 1st IMGBOX picture shows I downloaded 13 files using JDW.

94 were not downloaded and are giving me the "SKIPPED - ACC missing" crap.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

So what do I need to do, to download the remaining 94?

Visit one by one and hit download. Using my browser or IDM.

Only because JDW doesn't work with them. Why?

I don't know.

Last edited by Perene; 13.01.2022 at 17:08.
Reply With Quote
  #26  
Old 13.01.2022, 18:35
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Perene View Post
Once you insert the credentials in JDW then anything should work, if in this case the bug requires a logged account, right?
Wrong and not a bug:
Some files can sometimes by some users/IPs only be downloaded via account.
Google may change this at any time but then again:
Your files do not seem to be quota blocked in any way and I've never claimed that they are!

Please keep in mind that it is sometimes really hard to interpret Googles errors which is why JD shows "Insufficient permissions (private file) or quota limit reached".
Basically Google is returning a basic 403 error in your case which could mean anything see the definition of a 403 error:
en.wikipedia.org/wiki/HTTP_403

Quote:
Originally Posted by Perene View Post
I tried inserting an account and letting the browser with a session active, so not closed. While running JDW at the same time.
As long as you do not press any button like "logout" in your browser before closing it, your cookies should stay active in JD even after they've been deleted in your browser...
Maybe google only allows X valid sessions so if you keep using your browser in incognito mode, over the time your older sessions will get deleted -> That's only a speculation though.

Other question:
When logging in into your Google account, did you tick the box like "keep this session" or "Trust this computer"?
Google will show this option on the 2nd login step when you enter your password.
It is important to tick this box.

Quote:
Originally Posted by Perene View Post
You aren't going to convince me of that, because if you think about it, this quota is only applied to these specific files?
I'm not trying to convince you of anything.
Theissue you're having doesn't look like a typical GDrive quota issue anyways...

Quote:
Originally Posted by Perene View Post
What's a resource key?...
I've already gone into that so no need for me to do this again.
In any way:
If you can reproduce this issue consistenly with the new "resource key" files, this might at least help to enclose the issue but we're still far away from finding a solution to it.

Quote:
Originally Posted by Perene View Post
I can download all affected files using a browser or program like Internet Download Manager.
IDM works totally different - it doesn't ajutomate the process of downloading GDrive items 100% like JD does...at least as far as I know.

Quote:
Originally Posted by Perene View Post
Only because JDW doesn't work with them. Why?

I don't know.
I don't know yet either...

So I've tested it using the GDrive account you've sent me and some of the URLs you claim are not working.
Even with your account I wasn't able to reproduce this issue so my current/last guess is that it is somehow related to your IP.

Please try the following:
1. Delete all google accounts in JD.
2. Get a VPN - for testing, a free VPN should be fine e.g. "windscribe.com" (this is not any kind of recommendation/affiliate!).
3. Activate that VPN, then open a new browser window, then logout of google if logged in, then login again.
Add the cookies of this browser window to JD.
4. Now while the VPN is still active, try to download files which you were not able to download before.


To further investigate this, I'd either need to be able to use your IP/connection as a proxy and/or do a Teamviewer session with you.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #27  
Old 13.01.2022, 18:38
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Additional idea:
Please test if you can do automated GDrive downloading of the same files using other tools/scripts like e.g.:
github.com/wkentaro/gdown
and:
github.com/Akianonymus/gdrive-downloader

(Again: IDM doesn't count as "Automated tool" for GDrive because you still have to init the download manually via browser and then all it does is to download the detected direct-URL.)

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #28  
Old 13.01.2022, 20:25
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Using Windscribe didn't help. It didn't change the issue a bit. It modified my IP to Florida, but using Firefox at least still says in the account settings which is my local town, so Google is capable of bypassing VPNs by knowing our * TRUE * IP, perhaps because javascript needs to be enabled for you to visit these accounts.

Or flash cookies, or anything else they are using to identify from where I am.

Oddly below the "Windows" name of the computer used to log in, there is nothing only in Opera (browser). But Opera is not configured to be using private mode. So once I logged out the current Google account and used a new one, then it should have worked.

As for Firefox, I reported incorrectly, it was using incognito mode all the time. So I went to Opera and used the cookie plugin, and exported from there. No luck.

Quote:
Originally Posted by pspzockerscene View Post
Wrong and not a bug:
Some files can sometimes by some users/IPs only be downloaded via account.
JDW was reported in the past not working with private folders. This was something I discovered yesterday:

https://support.jdownloader.org/Know...roubleshooting

This will not work even if we add the account owner.

But in my case none of the affected files/URLs are private. I also checked in a private tab if they are (if YES, then Google should have asked you to log in).

The "bug" (let's assume this to be 100% confirmed) affects:

- Folders
- Files
- Resourcekey URLs from 2018 or older (forget about 2019 or newer)
- Random files from these resourcekey FOLDERS.

It looks like (my guess) that JDownloader is considering them private in a way that it will not download no matter what we do, just like that limitation from * REAL * private URLs.

I also remember that in the past Google allowed us to choose between 3 modes: public, unlisted (but public for those that know the URL) and private.

Now there's only public and private. But all old contents became private after September 2021. It took me 1 week to visit several accounts and get the new URLs with resourcekey in it. Probably more than 70% of what I have is not in this condition.

Quote:
Originally Posted by pspzockerscene View Post
Google may change this at any time but then again:
Your files do not seem to be quota blocked in any way and I've never claimed that they are!
Then using VPN or trying in another location doesn't matter.

If I can download as much as I want from the * other files * then forget about this being some sort of IP block only for me.

Quote:
Originally Posted by pspzockerscene View Post
As long as you do not press any button like "logout" in your browser before closing it, your cookies should stay active in JD even after they've been deleted in your browser...
I don't remember if I asked this before, so I'll do now:

- Are we required to add any GD account to download old/resourcekey URLs? I mean the ones that became private and I had to go into all the trouble to visit the account that owns the files, and generate a new URL (which now have the "resourcekey").

Because if we are not, then all this discussion about adding or not a Google account to JDW is also pointless.

And if an account is not needed, then you only add them to bypass bandwidth quota.

Quote:
Originally Posted by pspzockerscene View Post
When logging in into your Google account, did you tick the box like "keep this session" or "Trust this computer"?
I am not seeing this option. Google will ask what is the recovery email if doesn't recognize the device and sometimes trigger this:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

This is the worst thing that could happen to an account because if you don't log often (and I don't for 99% of mine, for months), they will lock you out of yours and randomly (it doesn't happen to all of them) will ask you to provide a phone number to send a code and then allow you back.

All of this has no relation with cookies, though.

Quote:
Originally Posted by pspzockerscene View Post
Google will show this option on the 2nd login step when you enter your password. It is important to tick this box.
In here it's only displaying "show password".

Quote:
Originally Posted by pspzockerscene View Post
So I've tested it using the GDrive account you've sent me and some of the URLs you claim are not working.

Even with your account I wasn't able to reproduce this issue so my current/last guess is that it is somehow related to your IP.
Were you able to download ALL problematic URLs I listed?

Try this one:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

If you can download the 2 files from this URL then this is only happening with me and I don't know how to fix it.

Quote:
Originally Posted by pspzockerscene View Post
Please try the following:
1. Delete all google accounts in JD.
Deleting and reinserting doesn't help and I did even more than this. I removed JDW completely and reinstalled from scratch.

Oddly yesterday I think a few of these problematic URLs were downloaded, but moments later the "bug" came back and this time for good.

The other solutions from your 2nd post I don't know how to implement... IDM is useless for me because it can only download one by one.

I'll ask someone else to see if they can download what I can't. If a totally unrelated person is not affected, then it's more evidence it only happens with me.

Now, why would Google target specific files among so many others?

Don't you agree this makes zero sense?

Last edited by Perene; 14.01.2022 at 00:47.
Reply With Quote
  #29  
Old 14.01.2022, 12:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Perene View Post
Or flash cookies, or anything else they are using to identify from where I am.
No.

Quote:
Originally Posted by Perene View Post
JDW was reported in the past not working with private folders. This was something I discovered yesterday:
...
But in my case none of the affected files/URLs are private. I also checked in a private tab if they are (if YES, then Google should have asked you to log in).
Why are you coming up with this?
I've never assumed that your links are private and you've never claimed it.
Yes, private folders are not supported but that has nothing todo with the issue you're facing as all URLs you've provided for testing were public links...

Quote:
Originally Posted by Perene View Post
It looks like (my guess) that JDownloader is considering them private in a way that it will not download no matter what we do, just like that limitation from * REAL * private URLs.
No.

Quote:
Originally Posted by Perene View Post
Then using VPN or trying in another location doesn't matter.
As you tested it, this assumption should be correct, yes.

Quote:
Originally Posted by Perene View Post
I don't remember if I asked this before, so I'll do now:

- Are we required to add any GD account to download old/resourcekey URLs?
No.

Quote:
Originally Posted by Perene View Post
Because if we are not, then all this discussion about adding or not a Google account to JDW is also pointless.
Exactly.

Quote:
Originally Posted by Perene View Post
And if an account is not needed, then you only add them to bypass bandwidth quota.
Yes...or to download private files (single private files are supported, private folders are not/cannot be crawled.).

Quote:
Originally Posted by Perene View Post
I am not seeing this option.
...
In here it's only displaying "show password".
Indeed. You're right and I was wrong.


Quote:
Originally Posted by Perene View Post
Were you able to download ALL problematic URLs I listed?
Yes.

Quote:
Originally Posted by Perene View Post
Try this one:
I can download this one too.

Quote:
Originally Posted by Perene View Post
I removed JDW completely and reinstalled from scratch.
As I've told you two times before: This will not fix your issue and as you've seen yourself, it didn't.

Quote:
Originally Posted by Perene View Post
The other solutions from your 2nd post I don't know how to implement... IDM is useless for me because it can only download one by one.
As described in my lasst post, yes, IDM doesn't have the automatic download capabilities JD has.

Quote:
Originally Posted by Perene View Post
I'll ask someone else to see if they can download what I can't. If a totally unrelated person is not affected, then it's more evidence it only happens with me.
Well your forum post is public.
If any other users are affected by this they can see this thread and simply post in it too...

Quote:
Originally Posted by Perene View Post
Now, why would Google target specific files among so many others?
It doesn't. Something on your side seems to be different than for other users.

Please read my last (second) reply again and provide feedback on it:
https://board.jdownloader.org/showpo...0&postcount=27

Lastly, I will be here for the next ~3 hours so here is what we can try next:
Contact us again via support@jdownloader.org and ask for a Teamviewer session.
Please keep in mind that:
- We're located in germany (timezone-wise)
- We do not provide support during the weekends --> Do not expect new replies here before monday!

Have a nice weekend.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #30  
Old 14.01.2022, 22:26
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

I think I just solved this mystery! Finally I was able to get somewhere.

14.01.22 17.03.26 <--> 14.01.22 17.17.42 jdlog://0273925302851/

New debug log if this one is capable of showing what I will below.

**********

I discovered what it was. You are not going to believe it...

First, please note that with or without any Google Drive account configured (except the acc that owns these files!) I can't download the contents of this folder.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

It will say SKIPPED - ACC MISSING in any attempt.

To make things easier I am just going to quote this one file above. Acc owner is dublamos...

Now I tried something I hadn't done before. What if I export the Google cookies from the account owner? I wouldn't need that because we aren't talking about private files, right?

https://i.postimg.cc/4X5BTVCg/TENTA2.png

You can see for yourself in this #2 image that episode 1x13 said skipped - acc missing, same thing for 93 others.

Now look what I tried:

Logged using Opera and exported the Google cookies from the acc that owns this episode 1x13.

https://i.postimg.cc/6tCMYFYT/TENTA1.png

What did happen?

I was able to download the file!!!!!!!

https://i.postimg.cc/4J2PYGN2/TENTA3.png

All other files were still blocked and only this one, ONLY NOW I WAS ABLE TO DOWNLOAD!!!!!!!!!!!

So, the observation you made here:

http://privatelink.de/?http://privat...roubleshooting

- That private files were downloadable once we logged into the acc that owns them, but never folders;

Is the same issue that is affecting these old, resourcekey files!!!!!!!!!!

It's the same problem!!!!!!!!!!!! (I mean, not a problem, a limitation. Google turned my old links into private ones).

The difficulty here for you to realize this is a problem is that perhaps you don't have access to specific files from 2018 or older, with the resourcekey URL. Or perhaps because this only started to happen 4 months ago.

If this is a confirmed bug and if it can't be fixed then for every one of those links I would need to export the cookie from the acc that owns them.

Last edited by Perene; 14.01.2022 at 23:16.
Reply With Quote
  #31  
Old 15.01.2022, 17:41
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Please disregard my previous post, since now it's not working even if we put the credentials from the acc that owns these files! I tried with the acc from 1x01.

So it's a wrong assumption (another I made...) to say it's the same limitation.

What I don't understand is why, why in the world this worked for a short while and then broke all over AGAIN!

All of this suggests this is an intermittent problem that is exclusive to these specific URLs.

Google clearly has done all it could to mess with this program, at least for me. I added the Google cookies and not even a full 1 minute later it's not downloading anything. None of the links from the other accounts, and the links it owns.

I'll try the rest of the options you suggested and if this isn't fixed we might as well give up.

*****
EDIT: Ops, spoke too soon! Again it worked, only after I entered one of the accounts that owns the files!

Proof:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

As you can see, the contents of this folder are being downloaded:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

And who owns these files? valeriduarte.

Guess which account I just added to JDOWNLOADER?

https://i.postimg.cc/Ns0MhQ7D/EVID.png

valeriduarte, which owns these "Ascent of Man" files!!!!!!!!

Once again this proves my point that owners can download and crush this "bug".

At the same time I posted above this isn't working!

Why? Because I did the same cookie export for the owner of these files:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Which is "lexadug".

But JDW failed to download in spite of that.

This doesn't make any SENSE! Because the files are different, yet they are all public!!!!!!!!

Again, another mystery and I have no idea what causes this erratic behavior.

**************
**************
**************
**************

Another log for you:

15.01.22 14.05.04 <--> 15.01.22 14.13.18 jdlog://0773925302851/

Last edited by Perene; 15.01.2022 at 19:15.
Reply With Quote
  #32  
Old 15.01.2022, 19:50
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

For the 1st time ever I was able to use a different account besides the one that owns these files and use JDW to download!!!!!!!

Currently I have a specific account with cookies exported to JDW, one that is logged 24/7 using the Opera browser.

This shouldn't work if there's a bug, right? So what did I do to this time?

Here's what I think is going on with these old/resourcekey URLs (pay attention to this workaround I just found!):

- Google has explicitly told this:

********
What's changing

On September 13, 2021, Drive applies a security update to make file sharing more secure. As an administrator, you can choose how to apply this update.

This update changes the links used for some files. Access to impacted files won’t change for people who have already viewed them or who have direct access, but others might need to request access.
********

So this means if the account B (A = the owner) has accessed (viewed) the affected files/links, it can download whenever it wants. It has the same clearance as account A.

And it doesn't need account A to inform the new URL with resourcekey in it. It will know because of past association!!!!

Note I am not talking about account A sharing contents with B. It's not about sharing!!!!

Sharing is not needed!

If account B (the one I have configured in JDW right now) has simply viewed the file in the past, it can access something only the owners can.

And why all of this is important?

>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>

Because I just discovered that having viewed the file in the browser is what JDW needs to start downloading! Using whatever account we have configured there.

MANUALLY visiting the file. For example: this folder has 7 episodes:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

From 1x06 to 1x12.

If I just add this folder URL to JDW, nothing will be downloaded. No matter what. No matter what we do, it doesn't matter if there's a GOOGLE DRIVE account there.

How can we fix this?

Here's an idea (which worked!): what if I visit episode 1x06 (manually):

**External links are only visible to Support Staff****External links are only visible to Support Staff**

And then hit download? It will go here:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Both things need to be done using the Opera browser, logged in this account B (or C, D, whatever).

So with JDW running, I only need to try downloading this same episode 1x06. With account B (C, D, whatever) configured there.

Only then it will work.

In other words, JDW cannot download old resourcekey URLs:

a) Without a GD account configured

and

B) If such account had not visited the contents BEFORE.

Meaning just informing any old resourcekey URL to JDW is not enough.

It needs, just as a blind man, to "touch" beforehand (to figure out what it is) whatever it will try to download LATER.

It needs this:

Access to impacted files won’t change for people who have already viewed them or who have direct access, but others might need to request access.

Otherwise it will not start any download.

To buy some time we only need to preview the video!

Look how it's done:

Step #1: visit the folder/file. Do it in the GD account configured in JDW. In my case, I use the Opera browser.

https://i.postimg.cc/kXJpDTJv/STEP-1.png

Step #2: PREVIEW the content.

https://i.postimg.cc/zXxPhT3Z/STEP-2.png

Step #3: ask JDW to download.

https://i.postimg.cc/g2rtHr6w/STEP-3.png

If this is all it takes, then despite not being a solution, it will save me considerable time having to download so many of those files manually.

Last edited by Perene; 15.01.2022 at 20:03.
Reply With Quote
  #33  
Old 17.01.2022, 18:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Perene View Post
Now I tried something I hadn't done before. What if I export the Google cookies from the account owner? I wouldn't need that because we aren't talking about private files, right?
Correct.

Quote:
Originally Posted by Perene View Post
If this is a confirmed bug and if it can't be fixed then for every one of those links I would need to export the cookie from the acc that owns them.
Right now it is still a bug that only you can confirm.

To this point there is still no way for me to reproduce this so I won't do any changes to our code.
Without the ability to either reproduce this here or live on your machine in order to work on this issue, I won't be able to help you.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #34  
Old 17.01.2022, 19:58
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

I am getting in touch with someone that claims it can't download one of the old links with resourcekey in it. Of course I can't confirm anything unless that person provides me evidence. I asked for a screenshot.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

You can read his message saying it couldn't download, too.

Basically, what I am getting of this issue is this:

- If you simply visit the link in your browser (we are talking about PC ones, such as Opera, Chrome, etc.), with your Google account logged, then you are granted perpetual clearance/access to such content, and this is precisely why JDOWNLOADER can download the file.

That also explains why you @pspzockerscene could download ALL the files I mentioned here.

You didn't say if you added the links directly.

If you previewed any of them, that explains why you were able to download.

The last days I was able to download all episodes (more than 90 from that TV show, and 200 from another) doing this trick: simply preview the content logged in a specific GOOGLE account, then add the cookies from the Google acc that previewed them to JDW, and after you preview each episode, you try downloading.

(But doing this also wastes a lot of time, it's simply a workaround and not a fix)

The bug (if you agree there is one) consists of JDW being unable to download old/resourcekey URLs directly.

Note the emphasis on the word: DIRECTLY. Without the "middleman" called BROWSER.

For example:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I asked someone else (that same guy that confirmed it couldn't) to try downloading this short video. Without of course looking what it is in their browser.

Don't open this link in your browser first. Do a control + C and then control + V (paste) in JDW.

Directly.

Add this link to JDW and then try downloading.

How would I fix this problem? If this can be considered a "fix"....

>>>>>>>>>>>>>>>

Instruct JDW to analyse these old resourcekey URLs as if it is "previewing" them, similar to the command issued by the browser. At the same time (perhaps seconds later) only then it tries to download.

Also change JDW to only download resourcekey URLs if you have an account logged. You need to treat them as if they are all 100% private and the only way to access them is after your past association.

What I can't say is how all these matter to determine if I can download them or not:

**********
a- Exporting cookies from the Google acc that owns them
b- Previewing from the browser while not logged to a Google acc
c- Simply sharing from account A to B, and then exporting cookies from B to JDW, and then trying
**********

If a- and c- somehow make a difference, it's not relevant to me because I don't want these options.

If b- also makes this work, then this also explains why you said you could download my links without * ANY * exported cookies from a Google account.

But that also means you needed to preview the affected URLs prior to use JDW to download them.

Once I have any further evidence someone else can't download these old links, I'll present here.

For the time being the only method of downloading them it's to visit one by one in my browser and previewing the contents.

--------------
Make no mistake, for me if the link is new/doesn't use resourcekey, JDW downloads imediatelly, and the second I preview any of the **** skipped - acc missing links ****, they start to download after I restart/try one more time!

Previewing in my browser = not some sort of magic to fix this issue, it's doing what is explained below:

***********
Once the security update is applied to a file, a resource key included in the URL will be required to access the file. Users who previously accessed or viewed the file won’t need the resource key in the URL. Only people who haven’t previously accessed the file will need the resource key to gain access. If they don’t have the URL with the resource key, they’ll need to request access to the file.
***********

Last edited by Perene; 17.01.2022 at 20:06.
Reply With Quote
  #35  
Old 17.01.2022, 20:59
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

I did a 3 minute capture from my entire screen. Saved in a 20 MB video. I suggest you ALL download and see for yourself how everything I said in the previous post is proven:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Watch in fullscreen and note these specific moments:

- Nothing is being downloaded from this TV show. All links are old/resourcekey.

- 20 seconds into this video: proof I have exported cookies from the acc "perenep..."

- 30 seconds: more evidence: Opera browser logged into acc "perenep..."

So what the browser does is reflected into JDW.

- 35s: I look into my records and decide to visit one of the affected URLs.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

This one. With episodes 2x16 and 2x22.

- 48s: I do a preview from episode 2x16. Dated from 2017.

Note that same folder has episode 2x22, from the year 2020.

2x22 is not even listed among the ones with "skipped - acc missing"!!!!! Why? It was already downloaded. Like I said, the problem affects old stuff. Not 2019 or newer!!!!!!!

- Now skip to 1 minute, or 1m15s. Still not downloading anything. Doing a preview didn't work, uh?

Wrong! 1m30s = episode 2x16 is NOW (only now, not before!) ​being downloaded! Using JDW.

All the rest of the video shows further examples of I using the browser to preview stuff and only then JDW starts to download.

If you wanted me to provide a very strong evidence, then this is the best one posted here so far. Screenshots are nowhere near as convincing as this video capture.

Last edited by Perene; 17.01.2022 at 21:01.
Reply With Quote
  #36  
Old 17.01.2022, 22:23
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Someone from a tech forum (not the user from my country which I mentioned before) said this:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

*********
I have never used JD2 for google drive before your test, but use it a lot for other sites. That doesn't mean I'm SUPER familiar with all its bells and whistles but it usually works fine for me to just drag or copy links, one at a time for sites it supports. That's all I needed of it. Every now and then it fails to auto-start downloads despite being set to do so, but that's not related, I can't even force start your first link.

Your first GD link would not download, stated Skipped - Account is missing. I have not added a google account to JD2. I have no problem opening that link in my browser instead. I am logged into google with that browser I tried, if that matters.

Your second GD link downloads fine with JD2. I am using the latest version - Jan 10 build.

*********

This just proves what I explained to you so far, that old resourcekey URLs are not being downloaded.

I asked in this thread if someone could download one of the "targeted" URLs which are problematic precisely because they are resourcekey and old.

The guy said it couldn't download the old, only the new URL.

As you can see a random guy saying it couldn't confirms that without a GD account (exported cookies there) this doesn't work, too.

It's further proof Google turned these old URLs into private ones (despite them being public).

Or someone else saying it's OK, it's your computer (which it's your case). Very unlikely - I suspected from the start this to be your plugin's fault (or Google messing with it).

As suggested before, if you are not using a premium Google Drive acc or your computer used to analyze these plugins isn't special compared to ours, then try adding the problematic URLs without visiting them first.

Don't visit them in your browser because if you do this then you aren't going to see anything wrong.

When you visit one of these URLs you inform that GD account you have permanent access to that content. That might explain why you are not seeing what we do.

I'll give you two more to add directly to JDW:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by Perene; 17.01.2022 at 22:38.
Reply With Quote
  #37  
Old 17.01.2022, 22:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,140
Default

Quote:
Originally Posted by Perene View Post
You can read his message saying it couldn't download, too.
No I can't because an account is required to open your link.
Also I'm not interested in that - I'm telling you for the last time:
I do believe you - no need for any kind of proofs!
The only real proof is the content of your log and I've seen that...

Quote:
Originally Posted by Perene View Post
That also explains why you @pspzockerscene could download ALL the files I mentioned here.

You didn't say if you added the links directly.
...
If you previewed any of them, that explains why you were able to download.
No I didn't but I'll add this information now:
I've added the links directoy to JD without opening the in my browser...

Quote:
Originally Posted by Perene View Post
Once I have any further evidence someone else can't download these old links, I'll present here.
Please don't do that as it won't help.
If another person is having this issue, tell them to contact us directly or in this thread.

Quote:
Originally Posted by Perene View Post
sers who previously accessed or viewed the file won’t need the resource key in the URL. Only people who haven’t previously accessed the file will need the resource key to gain access
No news.

Quote:
Originally Posted by Perene View Post
I did a 3 minute capture from my entire screen. Saved in a 20 MB video. I suggest you ALL download and see for yourself how everything I said in the previous post is proven:
I'm repeating myself:
I do believe you - no need for any kind of proofs!
The only real proof is the content of your log and I've seen that...

Quote:
Originally Posted by Perene View Post
If you wanted me to provide a very strong evidence, then this is the best one posted here so far. Screenshots are nowhere near as convincing as this video capture.
I'm repeating myself:
I do believe you - no need for any kind of proofs!
The only real proof is the content of your log and I've seen that...

Quote:
Originally Posted by Perene View Post
This just proves what I explained to you so far, that old resourcekey URLs are not being downloaded.
No it doesn't.
They may as well just be quota limited for that other user.
Without a log you can't tell the exact reason he got the "Skipped" errormessage...
I'm repeating myself:
I do believe you - no need for any kind of proofs and/or feedback of other users outside of our support forum.
The only real proof is the content of your log and I've seen that...


Quote:
Originally Posted by Perene View Post
Don't visit them in your browser because if you do this then you'll spoil the idea of testing them.
I've answered this point in the upper part of this reply.

Quote:
Originally Posted by Perene View Post
I'll give you two more to add directly without visiting them in your browser:
Guess what:
I can download those just fine (did not open them in browser, did not use a GD acc for downloading).

I'll repeat myself again:
If you want to get this resolved, do one of the following things:
1. Contact us again via support@jdownloader.org and ask for a Teamviewer session.
or:
2. Setup a proxy on your machine and give us access to that.
If you don't know how, either find that out or use option 1.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #38  
Old 18.01.2022, 01:13
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Quote:
Originally Posted by pspzockerscene View Post
If you want to get this resolved, do one of the following things:
1. Contact us again via support@jdownloader.org and ask for a Teamviewer session.
OK, I just sent an email...

Ticket: #BTB-781-66062

Like I said there I am always online between 5 AM to 5 PM (Pacific Time)... Just installed this program.

I hope this can be resolved because the irony is that I am trying to store my content from Google elsewhere, and having to deal with hundreds of still pending old links is not worth the trouble. That explains my eagerness.

Everything was going fine until Google decided to change these URLs...

Last edited by Perene; 18.01.2022 at 01:17.
Reply With Quote
  #39  
Old 18.01.2022, 09:30
dave9 dave9 is offline
Baby Loader
 
Join Date: Jun 2021
Posts: 5
Default

This is way above my pay grade, but I saw this post on Anandtech forum, and tried to do the google download myself, and had the same fault.

I am no expert using JD2 by any stretch, but do use it very often, on other sites, and as a member of the forum, I noticed the post and tried to help, and have the exact same problem.

I am too lazy to read this entire thread, lol, just came here to confirm that I have the same problem, but also......... better things to do than get involved in whatever this issue is, lol!!! JD2 works fine for me, besides the recent !@#$ issue in updating it, but I can confirm that the problem cited, happens for me too.
Reply With Quote
  #40  
Old 18.01.2022, 14:52
Perene Perene is offline
JD Adviser
 
Join Date: Jun 2019
Posts: 114
Default

Hold on, I need to ask why TeamViewer would be of any help.

I didn't check the way this program works until last night and after my last message. This software gives full access to someone's computer. It's as if you were here sitting in my chair.

For obvious reasons I can't allow that without removing all sensitive data, which would probably need DAYS and no one would be at ease even if did that. I didn't allow the computer technician to snoop into my SSD when I built my current config. I formatted the entire hard drive first.

Would also have to set up a specific GD account in the browser that I may create now and then add to JDW.

The thing is, even if I do all I am saying or do nothing and let you run this program, you wouldn't advance one bit into discovering or fixing anything.

How do I know that? Because this can't be some wrong config from JDW, since I uninstalled and reinstalled your app. So if with JDW's default settings this problem is still there...

And don't bother trying an old version because it can't download anything from Google.

The ammount of evidence posted here and now this other user saying the same suggests the plugin broke.

Unless you are implying I needed to format my SSD and reinstall Windows 11 too, just to check if the plugin would work after that. I don't think this is needed to remove all possible traces from a previous install.

Also can't be due to my IP because in here the ISP changes it every X days and I did that myself. I can download as much as I want from all other (new) URLs. Except old and resourcekey.

The very fact I was able to download these old/resourcekey URLs after previewing them in the Google account added to JDW (you do this in the browser first) dismiss this idea of Google confusing JDW and you sending me the wrong error message.

Quote:
Originally Posted by dave9 View Post
This is way above my pay grade, but I saw this post on Anandtech forum, and tried to do the google download myself, and had the same fault.
Probably everyone had the same issue, however I may be among the few with hundreds of links in that situation.

It wasn't until these last days I remembered that if you visit/preview the content, then the account is granted access to it, and that explains why JDW can download after you do that.

What happened here is that Google turned all these public URLs into private. Or unlisted, if you want to call that.

The same thing happened with Youtube.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

The problem described here is if the Google Drive link is in this condition (resourcekey/dated 2018 or older), you can't download it directly after adding to JDOWNLOADER.

That creates a huge problem for people like me, since I have to visit HUNDREDS of now "broken" links before hitting "DOWNLOAD".
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 12:11.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.