fix: rewrite east ayrshire council scraper for recollect api#2081
fix: rewrite east ayrshire council scraper for recollect api#2081InertiaUK wants to merge 1 commit into
Conversation
The council moved from a UPRN-based lookup on their own website to the ReCollect platform. The old scraper returned empty results because the council portal now requires a session cookie. This rewrites the scraper to use the ReCollect address-suggest and events APIs directly. Accepts postcode + house number (or full address string). Falls back through multiple resolution strategies when the first attempt returns no match. Test address updated to KA1 3RB which resolves correctly on ReCollect.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe EastAyrshireCouncil module is migrated from BeautifulSoup-based HTML parsing to Recollect API integration. The implementation resolves addresses via the address-suggest endpoint, retrieves bin collection events, and transforms them into a standardized output. Test configuration is updated to reflect the new API-driven approach. ChangesRecollect API Migration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2081 +/- ##
=======================================
Coverage 86.67% 86.67%
=======================================
Files 9 9
Lines 1141 1141
=======================================
Hits 989 989
Misses 152 152 ☔ View full report in Codecov by Sentry. |
Summary
East Ayrshire Council migrated their bin collection lookup from a direct UPRN-based page to the ReCollect platform. The old scraper returned empty results because the council portal now requires session cookies to set an address.
This rewrites the scraper to hit the ReCollect address-suggest and events APIs directly:
Testing
Summary by CodeRabbit
Release Notes