The issue was that when I originally uploaded the challenge yaml, I had a text answer for the question. I subsequently switched the answer to use a variable. When I updated the yaml on the platform, the db entry for the question never removed the static text answer. If the check_answer function sees there is a static text answer, it compares against that and never actually uses the variable
Steps to reproduce:
- Upload a YAML that uses a static value as the answer
- Update the YAML to use a challenge variable as the answer
- Attempt to make an answer submission - the old static value should be what is compared