Scrape KPSDK protected Property website

[tho-exe]

Been trying to automate scraping of a property site using GitHub Actions due to fingerprint issue with Docker. But Kasada has been blocking all my attempt.
Scraping has only been successful on local but not on GitHub Actions.

from seleniumbase import Driver

driver = Driver("chrome", headless=True, uc=True)
driver.uc_open_with_reconnect("https://www.property.com.au/nsw/", reconnect_time=2)
print(driver.page_source)
driver.quit()

[mdmintz]

uc_open_with_reconnect() is the old way. activate_cdp_mode() is the newer, stealthier way.
You'll want to use CDP Mode with CDP methods: https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/ReadMe.md
You definitely shouldn't use headless mode or Docker if you want to remain stealthy.

You can use CDP Methods with the driver format: https://github.com/seleniumbase/SeleniumBase/blob/master/examples/cdp_mode/raw_driver.py
(but note that you can't use WebDriver methods from the driver while in CDP Mode)

[tho-exe]

Thank you so much for the timely reply and glad to learn about the upgrade with tips for more stealthy way. However, my script is still be detected by Kasada with the new CPD mode.

import atexit
from seleniumbase import Driver

driver = Driver(uc=True, guest=False, headless=False)
atexit.register(driver.quit)
url = "https://www.property.com.au/nsw/"
driver.activate_cdp_mode(url)
driver.sleep(2)
print(driver.get_page_source())

[mdmintz]

I ran it locally and it worked for me. Maybe something with your environment?

[tho-exe]

One of my enquiry is about GitHub Actions vs local. I have seen previously that Docket leaves a fingerprint and as a result, is unable to be stealth and you suggested GitHub Actions. So, I tried using GitHub Action since I need to run this on a schedule. But it is detected with GitHub Actions.

[mdmintz]

Could be your GitHub Actions configuration. Here's an example of mine: https://github.com/mdmintz/undetected-testing/actions/runs/21121897305/job/60736100208
(It bypasses bot-detection to scrape Walmart, SeatGeek, and Nike)

[tho-exe]

Thank you so much for providing the example and apologies for the late reply. I have follow the example and set up my GitHub file to mirror that as well. I have had success scrapping all the provided example website. But is still unable to access "https://www.property.com.au/nsw/" with the same configuration, even incorporating playwright library.