Skip to content

Fix LibraryThing false negatives with Cloudflare bypass#2810

Open
jake112-AO wants to merge 3 commits intosherlock-project:masterfrom
jake112-AO:fix-librarything-cloudflare
Open

Fix LibraryThing false negatives with Cloudflare bypass#2810
jake112-AO wants to merge 3 commits intosherlock-project:masterfrom
jake112-AO:fix-librarything-cloudflare

Conversation

@jake112-AO
Copy link

Problem

LibraryThing was returning false negatives (failing to find existing users) because the site uses Cloudflare bot protection that blocks standard HTTP requests.

Testing

  • Real user test: FlorenceArt - Now correctly detected
  • Fake user test: asdkjfhakjsdhf123456789 - Correctly returns "not found"
  • Regression test: Other sites still work normally

Solution

  • Added cloudscraper library to bypass Cloudflare challenges
  • Implemented useCloudScraper flag in site configuration
  • Modified session creation to conditionally use cloudscraper
  • Updated LibraryThing configuration in data.json

Changes

  • sherlock_project/sherlock.py: Added cloudscraper import and conditional session creation
  • sherlock_project/resources/data.json: Added "useCloudScraper": true to LibraryThing
  • pyproject.toml: Added cloudscraper dependency

Notes

  • This change is backward compatible - existing sites continue to use requests
  • Only sites with "useCloudScraper": true will use cloudscraper
  • Other sites with Cloudflare protection can now use this same approach

@github-actions
Copy link
Contributor

Automatic validation of changes

Target F+ Check F- Check
LibraryThing ❌   Fail ❌   Fail

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant