Failed to parse scripts

[Cageeeeee]

Im currently experiencing what seems to be the same as others a few days regarding the parsing and now i keep getting this error:

WARNING | twscrape.queue_client:req:291 - Unknown error. Account timeouted for 15 minutes. Create issue please: https://github.com/vladkens/twscrape/issues If it mistake, you can unlock accounts with twscrape reset_locks. Err: <class 'Exception'>: Failed to parse scripts

I checked #280 and saw some people had a fix but after implementing i still have the issue.

Anyone has solutions for this?

[stip1101]

Fixed it with a monkey patch

import json
import re

def script_url(k: str, v: str):
    return f"https://abs.twimg.com/responsive-web/client-web/{k}.{v}.js"

def patched_get_scripts_list(text: str):
    scripts = text.split('e=>e+"."+')[1].split('[e]+"a.js"')[0]
    
    try:
        for k, v in json.loads(scripts).items():
            yield script_url(k, f"{v}a")
    except json.decoder.JSONDecodeError:
        # Fix unquoted keys like: node_modules_pnpm_ws_8_18_0_node_modules_ws_browser_js
        fixed_scripts = re.sub(
            r'([,\{])(\s*)([\w]+_[\w_]+)(\s*):',
            r'\1\2"\3"\4:',
            scripts
        )
        for k, v in json.loads(fixed_scripts).items():
            yield script_url(k, f"{v}a")

# Apply before importing twscrape
from twscrape import xclid
xclid.get_scripts_list = patched_get_scripts_list

The issue is that Twitter started returning malformed JSON with unquoted keys (e.g., node_modules_pnpm_ws_8_18_0_node_modules_ws_browser_js:). The regex wraps these keys in quotes before parsing.

Works perfectly now

[JAAVAAD]

Excellent! Excellent! Excellent!

[Cageeeeee]

i just wanted to make a new ticket but saw your comment! how can i add the patch you made because there isnt a folder for scripts and im not sure where it should be placed? @JAAVAAD

Appreciate you brotha 🤝🏽

[Cageeeeee]

@stip1101 Did you experience the same issue here, the previous monkey patch for the xclid file stopped working?

[stip1101]

@stip1101 Did you experience the same issue here, the previous monkey patch for the xclid file stopped working?

My previous monkey patch is still working fine on my side.

[Cageeeeee]

hmmm alright weird, let me check a few things on my end.

[Cageeeeee]

so this was the original part in the xclid.py file that originally didnt work for me. After i replaced the code that wasnt working with the code you sent which worked and now im having failed parsing error again.

I tried to go back to the old one, save, replace your code back into the xclid.py file and for some reason its still giving me the same error. You have any ideas what i could check or maybe twitter changed the format again?

Could you maybe send me the xclid file you have so i can compare it with mine? @stip1101

[Cageeeeee]

Alright guys im back in action. for people that might run into this issue also let me explain what i did and how i fixed it.

I made sure to completly remove all the twscrape files from my VS Code and reinstall it. After that i applied the original monkey patch into the xclid.py file. For some reason i kept getting the failed parsing error and i manually logged in to the twitter account and got a suspended pop up but one that i didnt see before because everything worked the same on the account and there were no limitations of what i could do. I bought some new accounts and after got everything back up and running.

I want to say i really appreciate everyones help here and will keep a eye here if there might be any issues for somebody else so if possible i can help just like i have received now 🤝🏽