Skip to content

Commit a5accc5

Browse files
authored
Merge pull request #20 from ScrapingAnt/feature/add-exception-handling-example-to-readme
feature/add-exception-handling-example-to-readme: done
2 parents 9ab3847 + 6732478 commit a5accc5

File tree

1 file changed

+38
-0
lines changed

1 file changed

+38
-0
lines changed

README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,44 @@ result = client.general_request(
134134
print(result.content)
135135
```
136136

137+
### Exception handling and retries
138+
139+
```python
140+
from scrapingant_client import ScrapingAntClient, ScrapingantClientException
141+
142+
client = ScrapingAntClient(token='<YOUR-SCRAPINGANT-API-TOKEN>')
143+
144+
RETRIES_COUNT = 3
145+
146+
def parse_html(html: str):
147+
... # Implement your data extraction here
148+
149+
parsed_data = None
150+
for retry_number in range(RETRIES_COUNT):
151+
try:
152+
scrapingant_response = client.general_request(
153+
'https://example.com',
154+
)
155+
except ScrapingantClientException as e:
156+
print(f'Got ScrapingAnt exception {repr(e)}')
157+
except Exception as e:
158+
print(f'Got unexpected exception {repr(e)}') # please report this kind of exceptions by creating a new issue
159+
else:
160+
try:
161+
parsed_data = parse_html(scrapingant_response.content)
162+
break # Data is parsed successfully, so we dont need to retry
163+
except Exception as e:
164+
print(f'Got exception while parsing data {repr(e)}')
165+
166+
167+
if parsed_data is None:
168+
print(f'Failed to retrieve and parse data after {RETRIES_COUNT} tries')
169+
# Can sleep and retry later, or stop the script execution, and research the reason
170+
else:
171+
print(f'Successfully parsed data: {parsed_data}')
172+
```
173+
174+
137175
## Useful links
138176
- [Scrapingant API doumentation](https://docs.scrapingant.com)
139177
- [Scrapingant JS Client](https://github.com/scrapingant/scrapingant-client-js)

0 commit comments

Comments
 (0)