Unicode Support?

[amccloud]

When passing a unicode object to

parsley.parse(string=u'\u2019blah blah blah\u2019')

I get an UnicodeEncodeError
[Failure instance: Traceback: <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\u2019' in position 0: ordinal not in range(128)

Does PyParsley or even Parsley support unicode?

[fizx]

In theory, unicode should work. All internal data structures are utf-8. In practice, there's no test coverage of this, so I don't know. I'm going to add some now, and see where I end up.

[fizx]

So the problem is that I'm relying on libxml2 to correctly guess the encoding of the parsed string. This works great on the web, where content-type headers are sent. I added an option (install the latest master branch of both parsley and pyparsley) to manually specify that the input is utf-8. This should resolve your problem, but the api is still a little clunky. I'm open to suggestions for improvement.

parsley.parse(string=u'\u2019blah blah blah\u2019'.encode('UTF-8'), utf8 = 1)