support encoding

[wumpus]

I am converting a webpage which is not utf8. There's no way to specify an input encoding to your tool, so only utf8 is permissible. Most of my favorite webpage downloading tools preserve the webpage charset.

I ran it through iconv, no big deal, but it would be nice if your tool supported --encoding FOO and perhaps control over encoding errors.

[hanwentao]

What kind of encoding support do you need, for input or for output? In my understanding for input is more realistic and useful.

[wumpus]

Just input encoding. I figure anyone who wants a different output for encoding should do their own work! But it's no surprise that input html pages don't always have utf-8 encoding.