#1
|
|||
|
|||
Request for plugin: Google Drive Directory Index (GoIndex)
Hello,
Can you please add support for websites built using **External links are only visible to Support Staff****External links are only visible to Support Staff** An example website is **External links are only visible to Support Staff****External links are only visible to Support Staff** There are plenty of websites using it, and I would like to crawl all the links at once instead of manually downloading one file at a time. One problem is that there is no common host, i.e. people are free to create their own www. So I am not sure how the linkcrawler will know to use this plugin. But, if it is possible I would love to have a plugin for this. |
#2
|
||||
|
||||
Hi,
I don't see how/what this website does. It is a wrapper for Google Drive content? JDownloader already supports normal Google Drive URLs / files / folders / subfolders. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
|||
|
|||
Some websites do not give out direct gdrive links, instead they provide using some wrapper or index service.
Parsing links from here is not too difficult, just get the href, and remove ?a=view if present, and you get the download link. But manually creating the folder structure and remove the ?a=view from the url gets annoying after a while, therefore this request for link crawler plugin. |
#4
|
|||
|
|||
>JDownloader already supports normal Google Drive URLs / files / folders / subfolders.
The last time I tried, login failed? And I searched the forums, it said that login has bug. Is that fixed now. |
#5
|
||||
|
||||
No
Please post example URLs and a guide on how to use this website. Is there any kind of API? Is there any way to get the real Google Drive URLs? -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#6
|
|||
|
|||
> Please post example URLs and a guide on how to use this website.
**External links are only visible to Support Staff****External links are only visible to Support Staff** Folders have class=folder Files have class=file (This can be seen by html inspector in browsers.) Some files (class=file) have a href ending with "?a=view" For eg. **External links are only visible to Support Staff****External links are only visible to Support Staff**3.txt?a=view Removing ?a=view gives - "**External links are only visible to Support Staff**3.txt". This is a downloadable url now. If a file does not have a href ending with "?a=view", then it can be directly downloaded. (e.g. last file in this url) Some folders can also have password/encryption: **External links are only visible to Support Staff****External links are only visible to Support Staff** > Is there any kind of API? AFAIK, no. > Is there any way to get the real Google Drive URLs? AFAIK, no So, the algorithm to download from the page is - 1a. For all links of class File, remove ?a=view from the link 1b. Add to download 2. For all links of class Folder, fetch that link, and use these algorithms recursively. The only issue as I said is that there is no single host, and everybody is free to use their own host. So some heuristic would be required to detect when this plugin is used (because the website is hosted on cloudflare workers, so they have url ending with worker.dev typically. not a great heuristic though). or maybe a manual option to invoke the plugin for some url. |
#7
|
|||
|
|||
I don't know how jdownloader handles subdirectories, because I never used it to download such links till now, but if you choose to implement this plugin, please try to make it output files in proper subdirectories.
|
#8
|
||||
|
||||
Ticket:
I'll check it out once I find the time ... -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#9
|
|||
|
|||
Okay, thank you.
|
#10
|
||||
|
||||
I've added a simple crawler for this website for now.
Please provide the following types of test-URLs so that I can add more of your requested functionality: - Folder containing subfolders AND files - Password protected URLs with the corresponding password Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
Sorry for late reply. I found a link containing subfolders and files - **External links are only visible to Support Staff****External links are only visible to Support Staff**
I could not find any link with password, so leave it for now. If i find it, i will reply here. |
#12
|
|||
|
|||
I dunno why my previous post is not showing up, so I am posting again -
> Folder containing subfolders AND files **External links are only visible to Support Staff****External links are only visible to Support Staff**[Book]Freelancing%20and%20Marketing%20for%20Web%20Developers%20+%20Web%20Designers%20(The%20Complete%20Bun dle)/ This url contains a folder, and inside that folder are subfolders and files both. > Password protected URLs with the corresponding password I could not find any such links. Maybe leave password-protected links for now. |
#13
|
||||
|
||||
Quote:
Your URLs are offline and none of them seems to lead to the website I previously implemented e.g. "index.gd.workers.dev". -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#14
|
|||
|
|||
> Your URLs are offline
Use this link for now - **External links are only visible to Support Staff****External links are only visible to Support Staff** > none of them seems to lead to the website I previously implemented e.g. "index.gd.workers.dev" umm.. yeah. I said in the beginning that GoIndex is a self hosted solution, so the url host might be different but the page structure is the same, so the crawler should work. Last edited by raztoki; 18.04.2020 at 06:41. |
#15
|
|||
|
|||
Also, here is a password protected link -
link - **External links are only visible to Support Staff****External links are only visible to Support Staff** password - see edit history Last edited by raztoki; 18.04.2020 at 11:20. |
#16
|
||||
|
||||
I've added simple subfolder support and support for all "workers.dev" subdomains for the next update.
Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#17
|
|||
|
|||
Thank you. But it does not work properly. The parsed urls are wrong because they contain an ending slash in the url so file download fails with File Not Found. This happens with every url I tried.
Also I am getting files with "unknownFilename" if i crawl this link for eg. - **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#18
|
||||
|
||||
Yeah I see, sorry.
Only folders/subfdolders have to end with "/". Fixed with the next update. I've also added subfolder-support meaning you should get the same folder-structure you got on that website also after downloading via JD. Please keep in mind that this will lead to a LOT of packages as our package system does not allow sub-packages with e.g. other download paths. If you do not wish to have this feature, you can turn it off here: Settings --> Packagizer --> "Adopt folder structure". Wartest du auf einen angekündigten Bugfix oder ein neues Feature? Updates werden nicht immer sofort bereitgestellt! Bitte lies unser Update FAQ! | Please read our Update FAQ! --- Are you waiting for recently announced changes to get released? Updates to not necessarily get released immediately! Bitte lies unser Update FAQ! | Please read our Update FAQ! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
|
|