#1
|
|||
|
|||
Redfin.com and Zillow.com and other realty sites
So one of the favorite pastimes of my husband and I is to browse super fancy over priced well out of our price range homes on websites like Redfin or Zillow. Problem is that some of our favorites eventually get removed because they are either sold or no longer on the market.
It’s just something we poor peasants like to do so we can see what the rich buttheads in society live like. I would like to request that we add support for those sites. Maps and address I’m not not that concerned with saving, because it’s not like I can go visit them, and it could lead to privacy concerns. But the pictures of the homes; how they’re presented for sale, and even the blue prints they may upload alongside the pictures can help a man dream and fantasize about a fancier, and maybe better future. **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#2
|
||||
|
||||
@DeusExBestia: so basically you are interested in downloading the images and, if available, the blueprints, right?
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
Yeah! The blueprints are uploaded as pictures as well, so just whatever they have in the house picture gallery. Maybe a .txt of the house description and whatever? “Beds: 7482, baths: 1.”
I don’t want to be all picky, but I presume if it’s at all possible they’d be saved as one house/address per package? |
#4
|
||||
|
||||
@DeusExBestia: i will check, we have public holiday here in germany on monday, so expect answer by next week
__________________
JD-Dev & Server-Admin |
#5
|
|||
|
|||
Awesome! Thank you kindly, and enjoy your holiday/weekend!
|
#6
|
||||
|
||||
Hi again and sorry it seems like this thread has been forgotten which rarely happens here.
Both websites look pretty simple so you should be able to do this by either only using our default deep-parser (= just add the links and let JD scan for results) or using LinkCrawler Rules. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#7
|
|||
|
|||
No worries! I’ve been dealing with a new puppy so lots of things have fallen by the wayside. Thank you for replying.
I will give them a shot once I get home after work. |
#8
|
||||
|
||||
Okay.
Just come back to us if you can't get it working. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#9
|
|||
|
|||
So I tried it out, Zillow comes up as a mess. If I just tell JD to crawl it without putting in a manual packagename it comes out as a giant mess, tons of folders with jpg/png and webp files, for every resolution available from the site, and it grabs all the analytics and some random stuff from their tiktok and their twitter. If I manually put in a package name everything for zillow is neatly tucked into the folder, with dozens of copies of each picture at every resolution, leading to thousands of files downloaded.
Redfin, for the same address gave a ton of twitter links? Still waiting for it to finish downloading. |
#10
|
||||
|
||||
Looks to me like you didn't understand any of the concepts of our LinkCrawler Rules(?)
Please post the LinkCrawler Rule you've created which lead to those results. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
||||
|
||||
Here is a really really basic rule for redfin.com that simply grabs all .jpg images from the source code and puts them into a single package.
You should adjust the deepPattern to further limit the results. Rule: Code:
[ { "enabled": true, "logging": false, "maxDecryptDepth": 1, "name": "example rule for redfin.com", "pattern": "https?://(www\\.)?redfin\\.com/.*/home/[0-9]+", "rule": "DEEPDECRYPT", "packageNamePattern": "<title>(.*?)</title>", "deepPattern": "(https?[^\"]+\\.jpg)" } ] pastebin.com/raw/tz7B45Pq -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by Jiaz; 11.05.2022 at 15:28. |
#12
|
|||
|
|||
Quote:
I added the linkcrawler rules you wrote up, it only downloaded 17 pictures, and none of the ones of the house. Looked like it grabbed a bunch of the sample houses on there. I'll try giving another crack at it once I can dedicate time to it. |
#13
|
||||
|
||||
@DeusExBestia: I will try to help when I find some free time for this
__________________
JD-Dev & Server-Admin |
#14
|
||||
|
||||
@DeusExBestia
Indeed it seems like the way those image-URLs are encoded cannot yet be handled by our generic parser. Here are possible solutions/workarounds you can do even without us changing anything (LinkCrawler rule is not required in those cases):
-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|