- You can now add a listing by pasting its URL
- Fixing bot detection in docker
Adding a listing by pasting its URL
Sometimes a job just misses a flat. Filter too tight, ad only online for a few hours, a friend sends a link. Untill now the only way to track one of those was a throwaway job built around it. Now there's a small + on the listings page.
Paste the link, pick one of your jobs, done. Fredy works out the portal from the host, reads the exposé and stores it under that job, so it gets the deal type, distances, commute times and price alerts like everything else. No notification, you know about it allready. The detail page says it was added by hand, and it goes on the watch list by default.
Providers get an optional fetchListingByUrl. It builds the same card a search would and runs it through normalize, so link and hash match what the next job run produces and the advert isnt stored twice. Works for ImmoScout (DE), immowelt, SeLoger, Kleinanzeigen, WG-Gesucht, Sparkasse, Deutsche Wohnen, Engel & Völkers, Tecnocasa, Tecnorete, immobilien.de, betterhomes, imaxx and schwarzesbrett. Everything else says "not supported yet". ImmoScout Neubau project pages get their own hint, since they aren't one listing.
Docker bot detection
Add a listing by pasting its URL, and fix bot detection in Docker
Closes #492
Sometimes a job just misses a flat. Filter too tight, ad only online for a few hours, a friend sends a link. Untill now the only way to track one of those was a throwaway job built around it. Now there's a small + on the listings page.
Paste the link, pick one of your jobs, done. Fredy works out the portal from the host, reads the exposé and stores it under that job, so it gets the deal type, distances, commute times and price alerts like everything else. No notification, you know about it allready. The detail page says it was added by hand, and it goes on the watch list by default.
Providers get an optional fetchListingByUrl. It builds the same card a search would and runs it through normalize, so link and hash match what the next job run produces and the advert isnt stored twice. Works for ImmoScout (DE), immowelt, SeLoger, Kleinanzeigen, WG-Gesucht, Sparkasse, Deutsche Wohnen, Engel & Völkers, Tecnocasa, Tecnorete, immobilien.de, betterhomes, imaxx and schwarzesbrett. Everything else says "not supported yet". ImmoScout Neubau project pages get their own hint, since they aren't one listing.
Left out: idealista (no fixtures to prove the hash), ImmoScout AT and inberlinwohnen. immowelt's hash isn't proven by the fixtures either, so refreshing them at some point would be good.
Migration 52 adds listings.added_manually.
Docker on Linux CloakBrowser pretended to be Windows, but the image has none of the Windows fonts and we can't ship them. DataDome noticed the mismatch, immowelt challenged the warm-up and then answerd every search with a 403. The browser now poses as Linux on Linux hosts, so UA, client hints and fonts tell the same story, and that gets through. CLOAKBROWSER_SUPPRESS_FONT_WARNING is gone from the Dockerfile too.
Full Changelog: 30.0.0...30.1.0
I build and maintain Fredy in my free time. If it saves you some, consider supporting me ❤️
Full Changelog: 29.0.0...29.1.0