JDownloader Community - Appwork GmbH
 

Go Back   JDownloader Community - Appwork GmbH > English Support > Suggestions & Requests
Reply
 
Thread Tools Display Modes
  #1  
Old 14.04.2022, 14:43
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default [LinkCrawler Rule] aliexpress.com image and videos

so when i'm enabled this rule

Code:
 "name"               : null,
  "id"                 : 1649932308721,
  "enabled"            : true,
  "pattern"            : "https?://www\\.aliexpress\\.com/item/\\d+\\.html",
  "formPattern"        : null,
  "passwordPattern"    : null,
  "updateCookies"      : true,
  "logging"            : true,
  "rewriteReplaceWith" : null,
  "deepPattern": "<img src=\"(https?://[^\"]+)",
  "packageNamePattern" : "<title>(.+?)</title>",
  "maxDecryptDepth"    : 1
and copied **External links are only visible to Support Staff****External links are only visible to Support Staff** , nothing found

but when i'm deleted this line
"deepPattern": "<img src="(https?://[^"]+)",
it will have so many resized images like this

**External links are only visible to Support Staff****External links are only visible to Support Staff**

which can remove _50x50.jpg_.webp for non-resized images but i dont know how to rewrite it to removed \d+x\d+.jpg

and that link have one mp4 file but jd is not captured by JD

**External links are only visible to Support Staff****External links are only visible to Support Staff**

what i'm tried so far is :
1 - make deeppattern working
2 - rewrite image url

Last edited by wanko; 14.04.2022 at 15:24.
Reply With Quote
  #2  
Old 14.04.2022, 16:11
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,561
Default

Quote:
Originally Posted by wanko View Post
but when i'm deleted this line
"deepPattern": "<img src="(https?://[^"]+)",
I've checked html and your deepPattern has no single match! that's why JDownloader doesn't find anything. there is no single "img src" within the html. You have to check the raw html source, not the dom within your browser after javascript has been executed and website is rendered.
The images are available plain in json within the html. In case you need help with it, please let us know
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 14.04.2022 at 18:48.
Reply With Quote
  #3  
Old 14.04.2022, 19:13
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

so not highlighted element and view selection source ?
if you mean this
data: {"actionModule":{"


but if so, i only see product img in source view, not description images
Reply With Quote
  #4  
Old 14.04.2022, 19:18
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,561
Default

@wanko: yes, you cannot use the rendered dom. JDownloader only *see* the pure/raw downloaded html file, no dom/no css/no js.
The descriptions are also available in source, eg
Quote:
propertyValueDisplayName":"L27A"
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 14.04.2022, 20:14
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

oh i see, i just found description but it's inside desc.htm

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by wanko; 14.04.2022 at 20:32.
Reply With Quote
  #6  
Old 15.04.2022, 17:27
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

update2: ok with this i feel like it work ok
"deepPattern" : "actionModule|(https?://[^_"]+)|^(?=.*zip).*",

1- capture all link have _(resized images) and zip (description)
2- packagizer add prefix into name to prevent mirrored link problems

but look like JD can't render that zip htm file

Last edited by wanko; 15.04.2022 at 19:15.
Reply With Quote
  #7  
Old 16.04.2022, 11:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,561
Default

Quote:
Originally Posted by wanko View Post
but look like JD can't render that zip htm file
what do you mean? JDownloader only loads raw html files, no rendering/no js/no css.
your deepPattern doesn't make much sense, for example actionModule just matches on the "actionModule"!
the second will only match for https links until it hits " or _
the third part of the pattern also doesn't make any sense as it will start at line start ^ I think you wanted to match on zip urls but that's not how it will work. what kind of links are you looking for ?
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 16.04.2022 at 11:34.
Reply With Quote
  #8  
Old 17.04.2022, 00:05
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

yeah that | look like OR , not condition and that actionmodule is useless in this case, zip too,
about [^_"]+ i'm not noticed ", thank you



Quote:
what kind of links are you looking for ?
i think
ae01.alicdn.com looking for all jpg (but without suffix \d+x\d+ (resized)

**External links are only visible to Support Staff****External links are only visible to Support Staff**

->
**External links are only visible to Support Staff****External links are only visible to Support Staff**


that zip link htm contains images in description, i want images inside this sector too

"descriptionUrl":"**External links are only visible to Support Staff**
Reply With Quote
  #9  
Old 18.04.2022, 16:12
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,561
Default

can you provide one, two more example links of aliexpress ?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #10  
Old 19.04.2022, 21:08
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #11  
Old 04.05.2022, 15:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,117
Default

Two chained rules could do what you want e.g.:
Code:
[
  {
    "enabled": true,
    "logging": false,
    "maxDecryptDepth": 1,
    "name": "example rule for aeproductsourcesite.alicdn.com/product/description",
    "pattern": "https?://aeproductsourcesite\\.alicdn\\.com/product/description/pc/.+",
    "rule": "DEEPDECRYPT",
    "packageNamePattern": null,
    "passwordPattern": null,
    "deepPattern": "class=\"detail-desc-decorate-image\" src=\"(https?://[^\"]+.jpg)\""
  },
  {
    "enabled": true,
    "logging": false,
    "maxDecryptDepth": 1,
    "name": "example rule for aliexpress.com single items",
    "pattern": "**External links are only visible to Support Staff**,
    "rule": "DEEPDECRYPT",
    "packageNamePattern": null,
    "passwordPattern": null,
    "deepPattern": "(https?://aeproductsourcesite\\.alicdn\\.com/[^\"]+)"
  }
]
Rule as plaintext for easier copy & paste:
pastebin.com/raw/h2GmChSp

Please invest some time to learn this yourself as we've already created a lot of linkcrawler rules for you...

-psp-
EDIT
To get videos too, you will need to slightly modify the first rule...
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #12  
Old 08.05.2022, 03:13
wanko wanko is offline
JD VIP
 
Join Date: Aug 2015
Posts: 300
Default

oh thank you for that rule

i must modified it with "|"

and look like trying to get video will not work because it's required script render, so view source wont help, so it's impossible to get video
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by wanko; 08.05.2022 at 04:27.
Reply With Quote
  #13  
Old 09.05.2022, 13:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,117
Default

I'm unable to find any video on that website.
If it's not possible using LinkCrawler Rules you might want to try this method.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #14  
Old 09.05.2022, 15:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,561
Default

@wanko: video is done via javascript player so JDownloader doesn't see it
but method hinted by pspzockerscene should work fine
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:45.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.