-
Notifications
You must be signed in to change notification settings - Fork 771
Expand file tree
/
Copy pathhistory.txt
More file actions
1154 lines (1050 loc) · 65 KB
/
Copy pathhistory.txt
File metadata and controls
1154 lines (1050 loc) · 65 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
HTTrack Website Copier release history:
--------------------------------------
This file lists all changes and fixes that have been made for HTTrack
3.49-21
+ New: --host-alias (-%C) folds the several hostnames of one site onto a single canonical host, so a site that answers under more than one hostname is fetched once instead of once per name (#16)
+ New: the WebHTTrack wizard can set the host-alias rules and the guide documents them for every front end; a malformed rule is refused where it can still be corrected, through a newly exported hts_host_alias_rule_ok() (#1161, #1172, #1192)
+ Fixed: any signal caught during a mirror killed every transfer in flight, so a burst of them silently truncated the mirror; a stop now ends an HTTP transfer as the user's stop rather than a fabricated receive error (#1110)
+ Fixed: stopping an FTP transfer against a silent server waited out --timeout twice, once on the data socket and again on a QUIT reply nobody was going to send (#1096)
+ Fixed: a crawl that ended early ran --update's purge against the files it had never reached, deleting good ones (#1109)
+ Fixed: a long URL overflowed the duplicate-detection key buffer (#1160)
+ Fixed: a long command line, a large doit.log or a large .httrackrc aborted the engine while it rebuilt its arguments (#1189, #1198)
+ Fixed: a scan rule beginning with a dash ended the whole run (#1179)
+ Fixed: --index=0 and --noindex enabled the index rather than disabling it, and --purge-old=1 killed the run; an option that takes no value now refuses one instead of silently dropping it (#1195, #1200)
+ Fixed: reloading a WebHTTrack project lost every zero and every cleared field, so a rate limit of 0 came back as 100 KB/s and a cleared user-agent refilled itself (#1177, #1171, #1186)
+ Fixed: the wizard kept only the last of several --strip-query rules, and its domain answers matched beyond the domain they named (#1168, #1119)
+ Fixed: the offer to resume an interrupted mirror printed the program path instead of the command line it would rerun (#1204)
+ Fixed: the build failed where getrandom() has no declaration, and the installed headers did not compile under some include orders (#1139, #1150, #1151)
+ Changed: the documentation and the WebHTTrack pages compute their copyright year instead of carrying a fixed one (#1163, #1165)
+ Changed: refreshed the guide screenshots for all three interfaces, and repaired the documentation's dark mode and its search-target tabs (#1164, #1158)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-20
+ Fixed: a link to another host whose name looked like a file was scanned as a page, so one link could mirror the whole foreign site on the referring page's depth budget (#121)
+ Fixed: a size-bounded filter built both of its filter URLs wrong when the path had no leading slash (#1095)
+ Fixed: a mirror told to quit while an FTP file was being received announced the exit and then waited forever on that slot (#1106)
+ Fixed: stopping an FTP transfer closed its data socket twice, so a threaded crawl could lose another worker's connection (#1090)
+ Fixed: WebHTTrack's welcome page opened its language menu blank instead of marking the language in use (#1084)
+ Fixed: two suite tests failed on the hppa, powerpc and ppc64 Debian buildds, holding the package back on those ports (#1108)
+ Fixed: a growable string could wrap its capacity to zero instead of saturating (#1062)
+ Changed: the macOS disk image carries its architecture in its name, so an Intel user can tell before downloading that the build is for Apple Silicon (#1083)
+ Changed: refreshed the GUI guide, its screenshots, the documentation typography and the footer contrast
+ Changed: settled on one spelling of the product name across the docs and the site
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-19
+ Fixed: an over-long user name in an ftp:// URL was truncated at the credential boundary, so the login went to a different account (#1032)
+ Fixed: an over-long FTP path or host name aborted the process instead of failing that one link (#1019)
+ Fixed: an FTP crawl ignored --timeout and --max-time, so a server that went quiet held a slot for five minutes (#1039)
+ Fixed: an FTP crawl of a dead host took two minutes to give up, and no signal could shorten it (#1059)
+ Fixed: the engine released the transfer slots while an FTP thread was still writing to one, crashing a mirror that hit its size or time limit (#1051)
+ Fixed: Ctrl-C did not shorten a connection or a name lookup already in flight, so ending a crawl against an unreachable host took minutes (#1073)
+ Fixed: a crawl that gave up mid-loop, on the link limit or an abort, lost its WARC archive and its change report, and a WARC written past teardown was stranded in a .tmp file (#1061, #1060)
+ Fixed: a crawl that rolled back for want of data still replaced the previous WARC archive, stranding its temporary and never writing its index; a stale or unwritable index is now reported rather than left silently in place (#1053)
+ Fixed: a resume the server answered with an unusable Content-Range deleted the partial file and never fetched it again; the restart no longer spends the retry budget, and is latched to one per link (#581, #1052)
+ Fixed: a long argument to -n or -O smashed the stack through an unbounded panic message, and the growable arrays could wrap their capacity to zero (#1040)
+ Fixed: --single-file could write a reference mark longer than it reads back, leaving it in the delivered page as visible text (#1054, #1069)
+ Fixed: on a terminal with no input left, a crawl toggled between the spinner and the full panel on every pass (#1072)
+ Fixed: -%v1 ended each line with a carriage return, so the lines overwrote each other and a redirected log was unreadable; -%v2 now paints its panel only on a terminal
+ Fixed: the installed socket headers did not compile under -std=c99 without _POSIX_C_SOURCE (#1001)
+ Fixed: the 16 px application icon rendered every stem as grey (#938)
+ Fixed: the desktop files installed outside the data directory, and the AppStream metainfo declared an icon name that is not a stock one
+ Fixed: the build failed on distributions whose coreutils are not GNU's (#1042)
+ Changed: --single-file drops its own HTML and CSS scanner and reuses the crawler's parser, so it follows the same references the mirror does (#749)
+ Changed: updated Turkish translation, and repaired two mangled Polish strings (#306)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-18
+ Fixed: the 3.49.17 package build failed on four architectures: the test suite's mmap interposer did not compile on armhf, powerpc or hppa, and the alternate-stack test rejected loong64's shorter backtrace (#1023)
+ Fixed: the test suite failed on a host with no ps command, such as a Fedora build root; its hang diagnostics now read /proc directly (#1021)
+ Changed: multiple internal test and CI improvements, including a cross-compile matrix covering the Debian ports architectures
3.49-17
+ New: a pkg-config file, libhttrack.pc, ships with the development headers (#1018)
+ Fixed: a crawled page could read the WebHTTrack session id and drive the control panel; the id is now unguessable and cross-origin commands are refused (#877)
+ Fixed: an ftp:// link carrying encoded line breaks could append commands to the FTP control channel, and an unquoted LIST argument let a path holding a space list something else (#1010)
+ Fixed: a crawled link could inject header lines into the requests HTTrack sends (#1008)
+ Fixed: a control byte in a link made the crawler fetch a URL the page never wrote, and stripping one left the original tail glued to the result (#982, #974)
+ Fixed: the WebHTTrack progress panel rendered crawled URLs into HTML and JavaScript unescaped (#973)
+ Fixed: the WebHTTrack command box rendered its settings unescaped, so a posted value could break out of the box it was shown in (#998)
+ Fixed: the WebHTTrack settings pages interpolated values into attributes unescaped, lost a value containing a quote, and mangled a Windows path such as C:\ab (#989)
+ Fixed: a worker thread exhausting its stack died with no diagnostic (#969)
+ Fixed: a crash report named no frame of the executable on a non-PIE build, and named the dynamic loader rather than the program when launched through it (#995, #996)
+ Fixed: the crash report could hang, its symbolizer being forked from the signal handler (#968)
+ Fixed: the installed headers declared symbols the library does not export, and scoped struct addrinfo inside a parameter list (#977, #987)
+ Fixed: an install moved away from its configured prefix could not find libhttrack on macOS (#978)
+ Fixed: 3.49.16 did not build on armhf, loong64, hppa or hurd-i386 (#1015)
+ Changed: mirrored pages are served sandboxed, so a saved page can no longer script the WebHTTrack panel, and loses cookies, storage and cross-frame access (#877)
+ Changed: htsserver exits with a diagnostic rather than fall back to a guessable session id when no entropy source answers (#877)
+ Changed: an FTP file whose name contains a tab is no longer fetched (#1010)
+ Changed: on macOS, libhttrack records @rpath/libhttrack.3.dylib, so anything linked against it afterwards supplies its own rpath (#978)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-16
+ New: macOS ships a signed and notarized HTTrack.app with its own icon, bundling the OpenSSL it needs so a downloaded copy launches (#890, #900, #901, #950)
+ New: the desktop icons install into the hicolor theme, scalable SVG included, so Icon=httrack resolves in a launcher (#932, #933)
+ Fixed: an unauthenticated PROPFIND overflowed the ProxyTrack DAV item buffer (#836)
+ Fixed: response headers from the server overflowed the stack buffer holding the cache header block, in both httrack and ProxyTrack (#841)
+ Fixed: a chunked response carrying trailers was rejected as "Invalid chunk" (#855)
+ Fixed: an over-long URL aborted the whole mirror inside the cache instead of reading as a miss (#935, #936)
+ Fixed: ProxyTrack could not re-read the .arc it writes, and crashed on a record whose body it could not read (#834, #929, #931)
+ Fixed: ProxyTrack logged a hashtable stats line and every PROPFIND body to stderr (#911, #918)
+ Fixed: the frozen-slot spool was written inside the mirror namespace, so it landed in the mirrored tree (#859)
+ Fixed: a stack overflow killed httrack with no diagnostic, crash reports named no frame of the executable, and on armhf they had no frames at all (#866, #889, #892)
+ Fixed: an install moved away from its configured prefix could not find its data directory, its shared library, or the html symlink (#885, #887, #894, #906)
+ Fixed: tooltips in the web GUI broke on a translation containing an apostrophe; the escaping now also covers quotes, backslashes, markup and DBCS lead bytes (#864)
+ Fixed: nine strings of the Windows GUI's option dialogs were untranslated in 26 of the 30 language files, and six language files disagreed with their own declared charset (#863, #963)
+ Fixed: the AppStream metainfo still advertised WebHTTrack 3.49.8 (#884)
+ Fixed: the installed development headers did not compile standalone (#943)
+ Fixed: configure discarded a user-supplied BASH_SHELL, resolved bash to /bin/sh on macOS, and hung on one pointing at a FIFO (#891, #895, #908, #922)
+ Changed: the masthead wordmark and the rings background are SVG, so they stay sharp on a hi-DPI screen (#910, #916)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-15
+ New: --single-file rewrites each saved page with its assets inlined as data: URIs (#713)
+ New: --changes reports what a crawl added, updated or removed against the previous mirror (#714)
+ New: --sitemap and --sitemap-url ingest sitemaps, so pages nothing links to are still found (#712)
+ New: webhttrack exposes --warc-cdx, --wacz and --warc-max-size (#862)
+ Fixed: --update and --purge-old destroyed a good local copy when the re-fetch got no response, was aborted mid-read, or when the backup meant to protect it failed (#746, #748, #758, #775)
+ Fixed: an FTP re-fetch truncated the mirrored file, resumed a complete mirror with REST and spliced the old body into the new one, and a successful transfer was blanked when its backlog slot was swapped out (#771, #797, #798, #823)
+ Fixed: a chunked response cut at a chunk boundary was stored and cached as complete (#840)
+ Fixed: cache repair deleted the old cache before a rename it never checked, and the unlink-then-rename fallback could lose the destination (#779, #786, #790, #824)
+ Fixed: several backward scans from strlen(s) - 1 read before their buffer on an empty string (#730, #768, #770, #814, #821)
+ Fixed: a URL could be saved onto the engine's own temporary files, and the final path segment was never clamped (#774, #842, #852)
+ Fixed: a query-string character reference the page charset cannot represent was left unescaped, changing how the query parses (#854)
+ Fixed: a second --update pass overwrote the previous WARC and regenerated a page-less WACZ (#759)
+ Fixed: WARC output dropped URLs of 1005 bytes or more, archived nothing for a 304 revisit, and marked an engine-forced not-modified as one the server sent (#778, #785, #826, #838, #839)
+ Fixed: ProxyTrack overflowed its .arc header block, walked past the end of an .ndx buffer, trusted an unparsed offset, and crashed on a PROPFIND or on an entry with no usable Last-Modified (#793, #820, #825, #828)
+ Fixed: webhttrack leaked the session id into crawled pages, overflowed its command line so a quoted value could inject flags, let a posted project path repoint the served root, and built its redirect Location in a 256-byte stack buffer (#700, #706, #707, #710)
+ Fixed: in the web GUI, options ticked on by default could not be un-ticked, "max site size" set a per-file cap instead of the HTML one, and the mirror link on the finished page could not be followed (#708, #709, #725)
+ Fixed: htsserver labelled every PNG as image/gif and offered JPEG as a download, and an unauthenticated GET of a directory spun the server forever (#724, #875)
+ Fixed: webhttrack hung at exit when no mirror had been launched (#753)
+ Fixed: oversized cache and header fields aborted the engine or overflowed a neighbouring field instead of being clipped (#701, #715, #717, #722, #732)
+ Fixed: a -%S rules file of 4 GB or more overran the heap (#702)
+ Fixed: the CLI display did not repaint when the terminal was resized (#97)
+ Fixed: a document with no declared charset double-encoded the title lifted for the local index (#848)
+ Fixed: a fragment on an inlined reference was dropped, losing an SVG sprite selector (#766)
+ Fixed: several time helpers handed out libc's shared gmtime/localtime static rather than a reentrant breakdown (#794, #806)
+ Changed: fatal-signal backtraces name engine frames instead of a bare module and offset (#705)
+ Changed: --without-zlib is rejected at configure time rather than failing at link (#735)
+ Changed: the offline documentation gains one GUI guide with screenshots, an Android option reference, and a restructured index
+ Changed: multiple internal hardening, test and CI improvements
3.49-14
+ New: WARC/1.1 archive output (--warc), with a sorted CDXJ index (--warc-cdx) and WACZ packaging (--wacz), also available from webhttrack (#668)
+ New: -%F takes named footer fields such as {url}, {lastmodified}, {mime}, {charset} and {status} instead of a fixed layout (#667)
+ New: a command-line guide organized by task ships with the offline documentation (#649)
+ Fixed: on Windows, paths beyond MAX_PATH truncated files, and mirroring into a long or non-ASCII directory silently failed (#133)
+ Fixed: the top index showed mojibake for non-ASCII project names and categories on Windows (#216)
+ Fixed: webhttrack handed the engine the web form's charset rather than UTF-8, so a non-ASCII path mirrored into a mojibake directory (#629)
+ Fixed: a non-ASCII single -O left the logs and the cache in a mangled twin directory on Windows (#630)
+ Fixed: a path-ceiling truncation dropped the .delayed marker, losing the file (#623)
+ Fixed: an oversized -%F footer aborted the crawl instead of being skipped (#669)
+ Fixed: a rejected 206 resume could loop and lose the file rather than refetch it whole (#581)
+ Fixed: default-port stripping was scheme-blind and dropped explicit ports from https and ftp URLs, and a :80 written with leading zeros mangled the host (#627, #638)
+ Fixed: -K silently reset the -c socket count (#650)
+ Fixed: signed-shift undefined behaviour in the zip-repair local-header read (#639)
+ Changed: the offline documentation drops stale facts, gains an Android help page, and documents the filter wildcards and the real long option forms
+ Changed: multiple internal hardening, test and CI improvements
3.49-13
+ New: SOCKS5 proxy support, with scheme-aware -P URLs (socks5://, socks5h://, connect://) and plain HTTP tunneled through a CONNECT-only proxy (#563, #564)
+ New: decode brotli and zstd content codings, advertised over TLS only as browsers do (#556)
+ New: webhttrack exposes the engine options added since 3.49-2, among them --cookies-file, --pause and --strip-query (#587)
+ Fixed: files of 2 GB or more were mishandled on Windows and on every 32-bit build (#569)
+ Fixed: --update destroyed a good local copy when the re-fetch returned an HTTP error, was aborted by -M/-E, failed to decode, or came in short (#176, #521, #557, #562)
+ Fixed: a self-redirect cookie wall was dropped instead of being re-fetched with the cookie (#15)
+ Fixed: a stalled TLS handshake ignored --timeout, and synchronous DNS resolution could wedge a crawl past --max-time (#607, #613)
+ Fixed: -M metered saved bytes rather than received volume, and overshot its cap under a slow server (#77, #520)
+ Fixed: several network-facing overflows and denial-of-service paths in the Content-Range, chunked-transfer, cookie, filter and ProxyTrack cache parsers
+ Fixed: a failed connect did not fall back to the next address on Windows (#579)
+ Fixed: -P took an out-of-range port as a garbage port, and scanned past an IPv6 literal's closing bracket (#598, #602)
+ Fixed: reject a port outside 1..65535 wherever one is parsed (a crawled URL, the htsserver and proxytrack listen arguments, an ftp:// URL), instead of letting a bare sscanf wrap a huge value into a plausible port and silently use it (#614, #626)
+ Fixed: a configured proxy still resolved and dialed the origin itself (#592)
+ Fixed: ~/ in the -O base path was never expanded (#270)
+ Fixed: a non-ASCII -O output path was double-encoded on Windows once argv became UTF-8 (#621)
+ Fixed: files under a non-ASCII project path were saved to a mangled directory on Windows (#217)
+ Fixed: --build-top-index (-%i) and --protocol (-@i) were taken for the -i continue flag, wiping the URL list and exiting on the usage screen (#615)
+ Fixed: webhttrack ignored LC_ALL/LC_MESSAGES and picked the wrong Chinese and Portuguese (#95)
+ Fixed: webhttrack wrote its base path and httrack.ini to the filesystem root when $HOME was empty (#625)
+ Fixed: crawls on a non-default port were slowed by a per-request pre-resolve (#181)
+ Changed: Windows builds moved to Visual Studio 2022 and OpenSSL 3.x, the VS2008 project files are retired, and the binaries carry a version resource
+ Changed: removed the obsolete Java-applet (.class) parser and the dead SWF module remnants
+ Changed: multiple internal hardening, test and CI improvements (Windows and macOS crawl suites, HTML-parser fuzzing, parallel make check)
3.49-12
+ New: --why explains which filter rule accepts or rejects a given URL, then exits (#505)
+ Fixed: links carrying raw UTF-8 bytes were fetched double-encoded and 404'd (#516)
+ Fixed: an uncompressed body mislabeled as gzip no longer loses the page (#515)
+ Fixed: remote stack overflow and uninitialized read in Content-Type/-Encoding parsing (#506)
+ Fixed: several over-reads and a leak in the filter, URL and IDNA parsers (#499)
+ Fixed: bound the cache-index (.ndx) parser to its buffer (#507)
+ Fixed: catastrophic backtracking on '*'-heavy filter patterns (#513)
+ Fixed: cookies.txt was created world-readable (#511)
+ Fixed: a single corrupt cache entry no longer aborts the whole mirror (#494)
+ Fixed: cache-reconcile policy was broken for zip caches (#491, #493, #495)
+ Fixed: cancelling a crawl mid type-check no longer orphans .delayed placeholders (#496)
+ Fixed: detect URLs after the first inline script and in mid-tag attributes (#497)
+ Changed: build with _FORTIFY_SOURCE and -fstack-protector-strong (#504)
+ Changed: removed the pre-3.31 (.dat/.ndx) cache import (#512)
+ Changed: multiple internal hardening and build improvements (libFuzzer harnesses, CodeQL, dead-code removal)
3.49-11
+ New: parse robots.txt Allow rules and path wildcards per RFC 9309 (#452)
+ New: advertise deflate in Accept-Encoding and decode deflate responses (#450)
+ New: follow <source> and <track> media elements as embedded links (#451)
+ New: added modern web MIME types to the type/extension table (#448)
+ Fixed: enforce the -E time limit during a slow transfer instead of only between files (#481)
+ Fixed: sniff the leading bytes of a download so a misdeclared Content-Type no longer renames a correct URL extension
+ Fixed: fast transfers could be saved under their temporary .delayed placeholder name (#5, #107)
+ Fixed: follow a redirect that maps to the same saved file instead of writing a self-pointing stub (#159)
+ Fixed: several network-facing buffer overflows in the FTP, Java and HTML parsers
+ Fixed: the htsjava plugin could not be loaded (hidden entry points, stale library name)
+ Fixed: HTML-escape truncation and a cache-buffer leak in the parser
+ Changed: modernized the default User-Agent to an honest HTTrack identifier (#449)
+ Changed: decode the full WHATWG set of HTML named character references (#443)
+ Changed: refreshed stale HTTP status, proxy-port and TLS-floor constants (#453)
+ Changed: multiple internal hardening, build, test and CI improvements
3.49-10
+ New: --cookies-file to preload a Netscape cookies.txt before crawling (#215)
+ New: --pause to space out file downloads by a random delay (#185)
+ New: --strip-query to drop selected query keys from the dedup naming (#112)
+ Changed: split the -%u URL hacks into independent --keep-www-prefix, --keep-double-slashes and --keep-query-order toggles (#271)
+ Fixed: follow a redirect Location after dropping its #fragment, instead of requesting the fragment and polluting the saved name (#204)
+ Fixed: escaped brackets inside a *[...] filter character class (#148)
+ Fixed: honor the server's Content-Range when resuming a partial download, instead of appending overlapping bytes (#198)
+ Fixed: abort the download as soon as the response type is excluded by -mime:, instead of fetching then discarding the body (#58)
+ Fixed: keep size-based filter rules neutral until the file size is known (#143)
+ Fixed: stop the mirror with a clean fatal error on a cache write failure, instead of crashing (#174, #219)
+ Fixed: stop the 412/416 partial re-get loop on --continue and --update (#206)
+ Fixed: keep an unrecognized URL tail instead of mangling it to .html (#115)
+ Fixed: honor --tolerant (-%B) on a broken Content-Length, and fix an out-of-bounds read it exposed (#32, #41)
+ Fixed: fall back to the next resolved address when a connection fails or stalls, instead of hanging on a dead IPv6 address
+ Fixed: report why a -%L URL list could not be loaded (#49)
+ Changed: multiple internal hardening, build and CI improvements
3.49-9
+ Fixed: file-type detection from the Content-Type header: trust a declared type over a binary URL extension, honor --assume under the delayed type check, and keep a known extension against a bogus or empty Content-Type (#267, #29, #56)
+ Fixed: an uninitialized-buffer read when the Content-Type is empty (#411)
+ Fixed: restored C++ source-compatibility of the installed headers so reverse dependencies (httraqt) build again (#413)
+ Changed: multiple internal build, packaging and test-harness improvements
3.49-8
+ New: tunnel HTTPS downloads through the configured HTTP proxy via CONNECT (#85)
+ New: parse every candidate URL in <img> and <source> srcset lists (#326)
+ Changed: dropped the obsolete OpenSSL linking exception (OpenSSL 3.0+ is Apache-2.0 and GPL-compatible); httrack is now plain GPLv3-or-later
+ Fixed: several out-of-bounds reads in the HTML/CSS parser on hostile input (#94, #396)
+ Fixed: stored XSS via an unescaped URL in the generated page footer (#165)
+ Fixed: hardened buffer copies throughout the engine against overflow
+ Fixed: capture conditional CSS @import URLs (#94)
+ Fixed: don't crawl xmlns namespace declarations as links (#191)
+ Fixed: don't mistake the method argument of XMLHttpRequest.open for a URL (#218)
+ Fixed: percent-encode parentheses when rewriting CSS url() targets (#163)
+ Fixed: collapse ../ in file:// URLs and widen relative-link handling (#137, #162)
+ Fixed: drop the obsolete $Version/$Path attributes from the request Cookie header, per RFC 6265 (#151)
+ Fixed: keep empty quoted arguments when reloading doit.log for --update/--continue (#106)
+ Fixed: raise the User-Agent and custom-header length limits (#152)
+ Fixed: abort on a long log path (lock-file buffer too small) (#183)
+ Fixed: race in lazy mutex initialization (#297)
+ Fixed: sub-second mtime precision when comparing local files on POSIX (#383)
+ Fixed: modernize OpenSSL TLS initialization for the 3.x to 4.x transition (#308)
+ Fixed: in-place changes made by the postprocess callback were not applied (Roman Sęk)
+ Fixed: "preffered" typo in the help text and man page (yosinn1-blip)
+ Fixed: corrections and updates of the Russian translation (German Aizek)
+ Fixed: corrections and updates of the Danish translation (scootergrisen)
+ Fixed: link libhtsjava and the libtest examples directly against libc
+ New: documented the public library API headers and typed the option fields as named enums
+ Fixed: numerous build, packaging, CI and test-coverage improvements (out-of-tree builds, sanitizer/distcheck CI, shell and Python linting, AppStream metainfo)
3.49-7
+ Fixed: keep generated config.h architecture-independent (Debian #1133728)
+ Fixed: man page rendered the -%! warning as bogus options (Debian #1061053)
3.49-6
+ Fixed: puny_decode CVE-2017-14062
3.49-5
+ Fixed: MiniZip CVE-2023-45853
+ Lintian fixes, multiple build fixes
3.49-4
+ New: Push default bandwidth to 100kiB/s, max to 10MiB/s
+ Fixed: Multiple packaging and build fixes, added security flags
3.49-3
+ Fixed: Corrections and updates of the Brazilian Portuguese translations (Paulo Neto)
+ Fixed: Fixed for Android
+ Fixed: Add mimetypes for documents (Pablo Castellano)
+ Fixed: Add firefox-developer-edition (sickcodes)
+ New: Support data-src and data-srcset for img (Brian 'Redbeard' Harrington)
+ Fixed: When using the '%N' structure naming key, separate the filename and extension with a dot (Steve Mokris)
+ Fixed: Multiple build updates
3.49-2
+ Fixed: Buffer overflow in output option commandline argument (VL-ID 2068) (Hosein Askari)
+ Fixed: Minor fixes
3.48-23
+ Fixed: on Linux, FTBFS with openssl 1.1.0
3.48-22
+ Fixed: on Windows, fixed possible DLL local injection (Tunisian Cyber)
+ Fixed: various typos
3.48-21
+ Fixed: Google RPMs use /usr/bin/google-chrome as program location (Cickumqt)
+ Fixed: Fixed htsserver not dying (immediately) on quit
+ New: Updated WIN32 OpenSSL to 1.0.1j (Evgeniy)
+ Fixed: webhttrack incompatibility with Chrome
+ Fixed: assertion failure at htslib.c:3458 (strlen(copyBuff) == qLen) seen on Linux
+ Fixed: infamous crashes inside the DNS cache due to a corruption within the option structure (E.Kalinowski/karbofos)
+ New: added minimalistic crash reporting on Windows and Linux
+ Fixed: URL list not working anymore (tom swift)
+ Fixed: FTBFS on ARM
+ Fixed: buggy FFFD (replacement character) in place of leading non-ascii character such as Chinese ones (aballboy)
+ Fixed: FTBFS when compiling with zlib versions < 1.2.70 (sammyx)
+ Fixed: buggy SVG (Smiling Spectre)
+ Fixed: do not uncompress .tgz advertised as "streamed" (Smiling Spectre)
+ Fixed: NULL pointer dereferencing in back_unserialize (htsback.c:976)
+ Fixed: library development files
+ Fixed: --advanced-maxlinks broken (Localhost)
+ Fixed: -devel package should now be standalone
+ Fixed: assertion failure at htscore.c:244 (len + liensbuf->string_buffer_size < liensbuf->string_buffer_capa)
+ Fixed: injection-proof templates
+ Fixed: htshash.c:330 assertion failure ("error invalidating hash entry") (Sergey)
+ Fixed: Windows 2000 regression (fantozzi.usenet)
+ Fixed: code cleanup (aliasing issues, const correctness, safe strings)
+ New: handle --advanced-maxlinks=0 to disable maximum link limits
+ New: updated ZIP routines (zlib 1.2.8)
+ Fixed: broken 32-bit version
+ Fixed: assertion "segOutputSize < segSize assertion fails at htscharset.c:993"
+ Fixed: new zlib version fixing CVE-2004-0797 and CVE-2005-2096
+ Fixed: more reliable crash reporting
+ Fixed: fixed infamous "hashtable internal error: cuckoo/stash collision" errors
+ Fixed: safety cleanup in many strings operations
+ Fixed: buggy option pannels
+ New: Enforce check against CVE-2014-0160
+ New: improved hashtables to speedup large mirrors
+ New: added unit tests
+ New: Added %a option, allowing to define the "Accept:" header line.
+ New: Added %X option, to define additional request header lines.
+ New: Added option '-%t', preserving the original file type (which may produce non-browseable file locally)
+ Fixed: remove scope id (% character) in dotted address resolution (especially for catchurl proxy)
+ Fixed: build fixes, including for Android, non-SSL releases
+ Fixed: buggy keep-alive handling, leading to waste connections
+ Fixed: removed chroot and setuid features (this is definitely not our business)
+ Fixed: removed MMS (Microsoft Media Server) ripping code (mmsrip) (dead protocol, unmaintained code, licensing issues)
+ Fixed: type mishandling when processing a redirect (such as a .PDF redirecting to another .PDF, with a text/html type tagged in the redirect message)
+ Fixed: infinite loop when attempting to download a file:/// directory on Unix (gp)<br/>
+ Fixed: removed background DNS resolution, prone to bugs
+ Fixed: do not choke on Windows 2000 because of missing SetDllDirectory() (Andy Hewitt)
+ Fixed: %h custom build structure parameter not taken in account (William Clark)
3.47-27
+ New: support for IDNA / RFC 3492 (punycode) handling
+ New: openssl is no longer dynamically probed at stratup, but dynamically linked
+ Fixed: random closing of files/sockets, leading to "zip_zipWriteInFileInZip_failed" assertion, "bogus state" messages, or random garbage in downloaded files
+ Fixed: libssl.dylib is now in the search list for libssl on OSX (Nils Breunese)
+ Fixed: bogus charset because the meta http-equiv tag is placed too far in the html page
+ Fixed: incorrect \\machine\dir structure build on Windows (TomZ)
+ Fixed: do not force a file to have an extension unless it has a known type (such as html), or a possibly known type (if delayed checks are disabled)
+ Fixed: HTML 5 addition regarding "poster" attribute for the "video" tag (Jason Ronallo)
+ Fixed: memory leaks in proxytrack.c (Eric Searcy)
+ Fixed: correctly set the Z flag in hts-cache/new.txt file (Peter)
+ Fixed: parallel patch, typo regarding ICONV_LIBS (Sebastian Pipping)
+ Fixed: memory leak in hashtable, that may lead to excessive memory consumption
+ Fixed: on Windows, fixed possible DLL local injection (CVE-2010-5252)
+ Fixed: UTF-8 conversion bug on Linux that may lead to buggy filenames
+ Fixed: zero-length files not being properly handled (not saved on disk, not updated) (lugusto)
+ Fixed: serious bug that may lead to download several times the same file, and "Unexpected 412/416 error" errors
+ Fixed: images in CSS were sometimes not correctly detected (Martin)
+ Fixed: links within javascript events were sometimes not correctly detected (wquatan)
+ Fixed: webhttrack caused bus error on certain systems, such as Mac OSX, due to the stack size (Patrick Gundlach)
+ Fixed: bogus charset for requests when filenames have non-ascii characters (Steven Hsiao)
+ Fixed: bogus charset on disk when filenames have non-ascii characters (Steven Hsiao)
+ Fixed: fixed 260-characters path limit for Windows (lugusto)
+ Fixed: non-ascii characters encoding issue inside query string (lugusto)
+ Fixed: HTML entities not properly decoded inside URI and query string
+ Fixed: URL-encoding issue within URI
+ Fixed: --timeout alias did not work
+ Fixed: more windows-specific fixes regarding 260-character path limit
+ Fixed: escaping issue in top index
+ Fixed: Linux build cleanup (gentoo patches merge, lintian fixes et al.)
+ Fixed: Fixed div-by-zero when specifying more than 1000 connections per seconds (probably not very common)
+ Fixed: Mishandling of '+' in URLs introduced in 3.47-15 (sarclaudio)
+ Fixed: "Wildcard domains in cookies do not match" (alexei dot co at gmail dot com )
+ Fixed: buggy referer while parsing: the referer of all links in the page is the current page being parsed, NOT the parent page. (alexei dot com at gmail dot com)
+ Fixed: Russian translation fixes by Oleg Komarov (komoleg at mail dot ru)
+ New: Added .torrent => application/x-bittorrent built-in MIME type (alexei dot co at gmail dot com)
+ Fixed: unable to download an URL whose filename embeds special characters such as # (lugusto)
+ New: Croatian translation by Dominko Aždajić (domazd at mail dot ru)
+ Fixed: url-escaping regression introduced in the previous subrelease
+ Fixed: content-disposition NOT taken in account (Stephan Matthiesen)
+ Fixed: buggy DNS cache leading to random crashes
+ Fixed: fixed logging not displaying robots.txt rules limits by default
+ Fixed: license year and GPL link, libtool fixes (cicku)
+ Fixed: Keywords field in desktop files (Sebastian Pipping)
3.46-1
* New: Unicode filenames handling
* Fixed: fixed bug in handling of update/continue with erased files or renamed files, leading to "Unexpected 412/416 error (Requested Range Not Satisfiable)" and/or "Previous cache file not found" (-1)" errors
* Fixed: escape characters >= 128 when sending GET/HEAD requests to avoid server errors
* Fixed: do not use "delayed" extensions when the mirror is aborting
* Fixed: generate error pages when needed (Brent Palmer)
* Fixed: parsing issue with js files due to "script" tags (Vasiliy)
* Fixed: anonymous FTP without password (Luiz)
* Fixed: Makefile issues regarding parrallel build and examples (Sebastian Pipping)
* Fixed: removed deprecated and annoying "Accept-Charset" header in requests (Piotr Engelking) (closes:#674053)
3.45-4
* New: source license is now GPLv3
* New: added a "K5" feature to handle transparent proxies (Brent Palmer)
* New: option -y to control ^Z behavior (Julian H. Stacey)
* Fixed: replace // by / when saving rather than _/ (Brent Palmer)
* Fixed: do not interpret ^C before mirror is finished, or after
* Fixed: webhttrack: do not use md5sum to produce a temporary filename, but mktemp (Ryan Schmidt)
* Fixed: document %k for custom structure (full query string)
3.45-3
* Fixed: spurious "Previous file not found (erased by user ?)" messages leading to retransfer existing files in cache (Alain Desilets)
* Fixed: --max-time now stops the mirror smoothly (Alain Desilets)
3.45-2
* Fixed: number of simultaneous connections was often only one
* Fixed: "Unexpected 412/416 error" leading to have broken files on disk
3.45-1
* Fixed: interrupting an update/continue mirror session should not delete anymore previously downloaded content (William Roeder, Alain Desilets and many others)
* Fixed: --continue/--update bug leading to download again already cached data in some cases (especially redirect/error pages)
3.44-5
* Fixed: crash when using -d with non-fully-qualified hostname (Alain Desilets)
* Fixed: typo in logs (Pascal Boulerie)
3.44-4
* Fixed: random crash when interrupting the mirror (spotted by -fstack-protector) in htscoremain.c (closes:#657878)
3.44-3
+ Fixed: Linux build (closes:#657334)
3.44-2
+ Fixed: malformed format htslib.c (Moritz Muehlenhoff)
+ Fixed: default footer print format
+ New: clever "^C" handling
+ New: added --do-not-generate-errors option
+ New: increased maximum cookie name
3.44-1
+ Fixed: Randomly corrupted files during updates due to "engine: warning: entry cleaned up, but no trace on heap"/"Unexpected 412/416 error" errors (Petr Gajdusek ; closes:#614966)
3.43-12
+ Fixed: buffer overflow while repairing httrack cache if a damaged cache is found from a previous mirror (closes:#607704)
3.43-11
+ Fixed: webhttrack fixes for icecat (closes:#605140)
3.43-10
+ Fixed: capture URL not working properly when IPv6 is installed (John Bostelman)
3.43-9
+ Fixed: application/xhtml+xml not seen as "html" (Peter Fritzsche)
+ Fixed: various linux fixes for desktop files (closes:#563691)
3.43-8
+ Fixed: URL encoding bugs with filenames containing '%' characters (sandboxie32)
+ Fixed: MacPorts Darwin/Mac fixes to webhttrack (Ross Williams)
+ Fixed: Flash link extraction has been improved (Vincent/suei8423)
3.43-7
+ Fixed: "Open error when decompressing" errors due to temporary file generation problems (Scott Mueller)
3.43-6
+ Shell: WIN32 setup cosmetic fixes: do not probe the proxy on non-local network, do not force *.whtt registration
3.43-5
+ Fixed: code tag handling bug in certain cases leading to produce invalid links (Tom-tmh13 and William Roeder)
3.43-4
+ Fixed: horrible SSL slowdowns due to bogus select() calls (Patrick Pfeifer)
+ Fixed: Konqueror fixes
3.43-3
+ Updated: Portugues-Brasil language file
3.43-2
+ Fixed: wizard question buggy, and commandline version did not print it (Maz)
+ Fixed: do not rename xml subtypes (such as xsd schemas) (Eric Avrillon)
3.43
+ Fixed: Fixed too aggressive javascript url= parsing (Chris)
+ Fixed: fixed --urllist option "sticking" the list content to the list of URL (Andreas Maier)
+ Fixed: "Previous cache file not found" not redownloading file when deleted before an update (William Roeder)
+ Fixed: *.rpm.src files renamed to *.src.src with bogus servers (Hippy Dave)
+ Fixed: "pause" is pausing much faster (William Roeder)
+ Fixed: binary real media files and related files are no longer being parsed as html (William Roeder)
+ Fixed: "File not parsed, looks like binary" check no longer corrupt the checked binary file
+ Fixed: multiple download of error pages (several identical '"Not Found" (404) at link [identical link]') leading to a slowdown in certain cases (William Roeder)
+ Fixed: sometimes, a double request was issued to update a broken file
+ Fixed: display bug "link is probably looping, type unknown, aborting .."
+ Fixed: missing library references at build time and other build related issues (Debarshi Ray)
+ Fixed: on windows, switched from wsock32.dll to ws2_32.dll
+ Fixed: minor argument size validation error for "-O" option (Joan CALVET)
3.42-3
+ Fixed: Bad URL length validation in the commandline (CVE-2008-3429) (Joan CALVET)
3.42-2
+ Fixed: Random crashes at the end of the mirror due to a dangling file pointer (Carlos, angus at quovadis.com.ar)
3.42
+ Fixed: size limits are stopping the mirror gently, finishing pending transfers (David Stevenson)
3.41-3
+ Fixed: text/plain advertised files renamed into .txt
+ Fixed: broken configure.in
3.41-2
+ Fixed: major WIN32 inlined function bug caused the cache not to be used at all, causing update not to work
3.41
+ New: changed API/ABI to thread-safe ones (libhttrack1 2), big cleanup in all .h definitions
+ Fixed: Major memory usage bug when downloading large sites
+ Fixed: do not rename files if the original MIME type was compatible
+ Fixed: several source fixes for freeBSD (especially time problems)
+ New: option %w to disable specific modules (java, flash..)
+ Fixed: 'no space left in stack for back_add' error
+ Fixed: fixed redirected images with "html" type
+ Fixed: 'Crash adding error, unexpected error found.. [4268]' error
3.40-2
+ Fixed: bogus '.del' filenames with ISO-9660 option
+ Fixed: now merges the header charset even with an empty footer string
+ New: --port option for webhttrack
3.40
+ New: mms:// streaming capture (thanks to Nicolas Benoit!)
+ New: proxyTrack project released
+ New: new experimental parser that no longer needs link testing ('testing link type..')
+ New: Redirect handled transparently with delayed type check and broken links made external when the "no error page" option is enabled
+ New: improved background download to handle large sites
+ New: '--assume foo/bar.cgi=text/html' is now possible
+ New: MIME type scan rules (such as -mime:video/* +mime:video/mpeg)
+ New: size scan rules now allows to rewrite uncaught links as external links
+ Fixed: crash fixed when ftime()/localtime()==NULL
+ Fixed: iso-9660 option now using '_' for collision character
+ Fixed: collision problems with CaSe SeNsItIvItY
+ Fixed: a href='..' fixed!
+ Fixed: redirects are now handled by the new experimental parser
+ Fixed: "./" links generated with fixed outbound links (../../)
+ Fixed: 'base href' bogus in many cases
+ Fixed: enforce security limits to avoid bandwidth abuses
+ Fixed: bogus external (swf) parser, fixed remaining .delayed files
+ New: new check-mime and save-file2 callbacks
+ New: "always delayed type check" enabled
+ Fixed: totally bogus finalizer causing compressed files not to be uncompressed, and many files to be truncated
+ Shell: new Finnish interface added!
+ Fixed: "..html" bogus type
+ Fixed: remaining bogus .delayed entries
+ Fixed: flush before user-defined command
+ Fixed: fixed user-defined command call and background cleaner
+ Fixed: fixed 'Crash adding error, unexpected error found.. [4250]' error
+ Fixed: fixed cache absolute file reference (the reference is now relative) preventing the cache form being moved to another place
+ Fixed: webhttrack 'Browse Sites' path bug
+ Fixed: old httrack cache format fixes (import of older versions did not work anymore)
+ Fixed: port fixes in htsnet.h
+ Fixed: -N option with advanced extraction (bogus "not found" member)
+ Fixed: javascript: location=URL was not recognized
+ Fixed: no more character escaping when not needed (such as UTF-8 codes)
+ Fixed: possibly temporary files left on disk with bogus servers giving compressed content on HEAD reuests
+ Fixed: URL hack caused unexpected filename collisions (index.html vs INDEX.HTML)
+ Fixed: "do not erase already downloaded file" option now correctly works (it leaves files linked in the mirror)
+ Fixed: UCS2 encoded pages are now converted properly into UTF-8
+ New: "near" option now also catch embedded (images, css, ..) files
+ Fixed: bogus chunked multimedia link text files (such as x-ms-asf files)
+ Fixed: compilation problems on Un*x version
3.33
+ Fixed: Bogus redirects with same location in https
+ Fixed: Bogus file naming with URL hack
+ Fixed: Extremly slow redirections and empty files
+ Fixed: Bogus names with directories ending with a "."
+ New: Number of connection per second can now be.. decimal, to delay even more
+ New: Enforce stronger ISO9660 compliance
+ Shell: "URL Hack" in interface
+ Shell: "Save settings" now rebuild categories
+ Shell: "Shutdown PC after mirror" option
+ Shell: Sound at the beginning/end or the mirror (configurable through system sound properties)
+ Shell: Fixed drag & drop, .url import
+ Shell: Fixed "wizard" mode (crash)
+ Fixed: Crash at the end due to unterminated pending threads
+ Fixed: \ is not anymore transformed into / after the query (?) delimiter
+ New: Two new callbacks for pre/post-processing html data
+ New: link-detected2 callback (additional tag name parameter)
+ Fixed: Broken ISO9660
+ Fixed: Crash on file:// links
+ Fixed: Unescaped ampersands (&) in URLs
+ Fixed: Transfer hangs introduced in 3.33-beta-2
+ Fixed: Display bug "Waiting for scheduled time.."
+ Fixed: Bug "Waiting for scheduled time.." (NOT a display bug, actually)
+ Fixed: CaSe SenSiTiViTy bugs with mutliple links reffering to the same URL but using different case styles
+ Fixed: Failed to build from sources (FTBFS) on amd64 archs because of cast problems (Andreas Jochens)
+ Fixed: & were converted into (Leto Kauler)
+ Shell: Fixed crash with long URL lists (Libor Striz)
+ Fixed: connection/seconds limiter replugged
+ Fixed: "no files updated" display bug
+ Fixed: bogus links encoded with UTF (Lukasz Wozniak)
+ New: --assume can be used to force a specific script type (Brian Schröder)
3.32
+ Fixed: css and js files were not parsed!
+ Fixed: again broken file:// (infinite loops with local crawls)
+ Fixed: Bandwidth limiter more gentle with low transfer rate
+ Fixed: external wrappers were not called during updates/continue
+ New: additional callback examples
+ Fixed: overflow in unzip.c fixed
+ New: tests are now cached for better performances!
+ New: %r (protocol) option for user-defined structure
+ Fixed: Broken engine on 64-bit archs
3.31
+ New: Experimental categories implemented
+ New: New cache format (ZIP file)
+ New: .m3u files now crawled
+ New: .aam files now crawled
+ Fixed: Broken ftp
+ Fixed: Broken file://
+ Fixed: Broken cookies management and loading
+ Fixed: HTTrackInterface.c:251 crash
+ Fixed: "N connections" means "N connections" even in scan phase
+ Fixed: javascript:location bug
+ Fixed: libtool versioning problem fixed
+ Fixed: More javascript bugs with \' and \"
+ Fixed: .HTM files not renamed into .html anymore
+ Fixed: OSX fixes in the Makefile script
+ New: Default "referer" and "from" fields
+ New: Full HTTP headers are now stored in cache
+ Fixed: ftp transfer not logged/properly finalized
+ Fixed: Missing symbolic link in webhttrack install
+ Fixed: path and language not saved in webhttrack
+ Shell: Avoid invalid project names
+ Fixed: Javascript bug with src=
+ Fixed: Keep-alive consistency problems on Linux/Unix with bogus servers (SIGPIPE)
+ Fixed: Parsing bug inside javascript (bogus parsing with empty quotes in function: foo(''))
+ Fixed: static compiling on Linux/Unix
+ Fixed: bloated .h headers (internal function definitions)
+ Fixed: Bogus query strings with embedded ../ and/or ./
+ New: Added "change-options" call in the crawl beginning
+ New: Query arguments now sorted for normalized URL checks (when "url hack" option is activated)
+ Fixed: Previous dependency to zlib.dll to zlib1.dll
+ Fixed: Broken static files were not correctly updated with the new cache format
+ Shell: Launch button in Internet Explorer
+ Fixed: Crash when dealing with multiple '?' in query string with 3.31-alpha
3.30
+ New: Webhttrack, a linux/unix/bsd Web GUI for httrack
+ New: "URL hack" feature
+ New: HTTP-headers charset is now propagated in the html file
+ New: loadable external engine callbacks
+ New: Experimental ".mht" archives format
+ Fixed: Query ?? bug
+ Fixed: Bogus base href without http://
+ Fixed: Several javascript bugs
+ Fixed: UCS2 pages badly detected
+ Fixed: Build structure change does not redownload files
+ Fixed: "?foo" URL bug (link with only a query string) fixed
+ Fixed: ' or " inside non-quoted URLs
+ Fixed: keep-alive problems with bogus servers
+ Fixed: Broken .ra files
+ Fixed: More javascript bugs
+ Fixed: ftp transfers not properly monitored in the shell
+ Fixed: various fixes in webhttrack
+ Fixed: Blank final page in webhttrack
+ Fixed: Javascript comments (//) are skipped
+ Fixed: Temporary fix for "archive" bug with multiple java archives
+ Fixed: Inlined js or css files have their path relative to the parent
+ Fixed: Unescaped quotes ("") when continuing/updating in commandline mode
+ Fixed: Null-character in html page bug
+ Fixed: External depth slightly less bogus
+ Fixed: Filters based on size bogus ("less than 1KiB" is now functionning)
+ Fixed: Strange behaviour with filters (last filter "crushed")
+ Fixed: Bogus downloads when using ftp (unable to save file)
+ Fixed: Freeze with keep-alive on certain sites due to bad chunk encoding handling
+ Fixed: Problems with javascript included paths
+ Fixed: The mirror now aborts when the filesystem is full
+ Fixed: "No external pages" option fixed
+ Fixed: Javascript and \" in document.write bug fixed
+ Fixed: Two memory leaks in temporary file generation, and in link build fixed
+ Fixed: Bogus compression with non-gzip format
+ Fixed: Larger range of charsets accepted
+ Fixed: Bogus robots.txt when using comments (#)
+ Fixed: Missing MIME types for files such as .ico
+ Shell: Fixed continuous proxy search
+ Shell: Fixed missing HelpHtml/ link
+ Fixed: Overflow in htsback.c:2779
+ Fixed: Bogus style and script expressions due to too aggressive parsing
+ Fixed: Javascript parsing bugs with \" and \'
+ Fixed: Javascript link detection bugs when comments were inserted between arguments
+ Fixed: Bug when valid empty gzip content was received
+ New: More aggressive "maximum mirroring time" and "maximum amount of bytes transfered" options
+ New: Windows file://server/path syntax handled
+ Fixed: mht archive fixes
+ Fixed: Serious bugs with filters given in commandline erased by the engine
+ Fixed: Bogus parsing of javascript: generated inside document.write() inside javascript code removed
3.23
+ New: Keep-alive
+ New: URLs size limit is now 1024 bytes
+ New: Bogus UCS2 html files hack
+ Fixed: base href bugs
+ Fixed: windows "dos devices" bug fixed
+ Fixed: dirty parsing now avoids ","
+ Fixed: "get non-html files near a link" option sometimes caused huge mirrors
+ Fixed: Bugs if zlib library is not found
+ Fixed: Bug with "near" and "no external pages"
+ Fixed: "Link empty" crash
+ Fixed: Several javascript bugs
+ Fixed: Keep-alive problems ("unknown response structure")
+ Fixed: Major keep-alive bug (connection not closed)
+ Fixed: 8-3 options not working, ISO9660 option improved
+ Fixed: Bogus links with embedded CR, TAB..
+ Fixed: small ../ external link bug fixed
3.22-3
+ Fixed: Slow engine due to (too strict) memory checks
+ Fixed: Overflow in htscore.c:2353
+ Fixed: Bogus chunked files with content-length fixed
+ Fixed: Folders renamed into ".txt" on Un*x platforms bug fixed!
+ New: Scan rule list (-%S) added
+ New: Cache debugging tool (-#C) added
3.21-8
+ New: Basic Macromedia Flash support (links extraction)
+ New: Modular design for https, flash parser and zlib
+ New: Standard autoconf/configure design on Un*x platforms
+ New: Modular design also on Windows platforms (dll/lib)
+ Fixed: Text files without extension not renamed "html" anymore
+ Fixed: Bug with "?foo" urls
+ Fixed: No chmod 755 on home anymore
+ Fixed: Stability problems due to bad file structure checks
+ Fixed: Overflow in GUI/commandline when displaying statistics
+ Fixed: Directory creation error
3.20-2
+ New: HTTPS support (SSL)
+ New: ipv6 support
+ New: 'longdesc' added
+ New: new file 'new.txt' generated for transfer status reports
+ New: ISO9660 compatibility option
+ New: empty mirror/update detection improved
+ New: Update hack now recognizes "imported" files
+ New: Option to disable ipv4/ipv6
+ New: Filters now recognize patterns like -https://*
+ Fixed: The engine should be now fully reentrant
+ Fixed: Fixes for alpha and other 64-bit systems
+ Fixed: Files downloaded twice if not found in cache
+ Fixed: ftp problems with 2xx responses
+ Fixed: ftp problems with multiple lines responses
+ Fixed: ftp %20 not escaped anymore
+ Fixed: ftp RETR with quotes problems
+ Fixed: now tolerent to empty header responses
+ Fixed: hts-log closed
+ Fixed: Compressed pages during updates
+ Fixed: Crash when receiving empty compressed pages
+ Fixed: Random crashes in 'spider' mode
+ Fixed: bcopy/bzero not used anymore..
+ Fixed: various code cleanups
+ Fixed: Better UTF8 detection
+ Fixed: External links now work with https and ftp
+ Fixed: Top index.html corrupted or missing
+ Fixed: URL list crashes
+ Fixed: Random crashes with large sites due to bogus naming handler
+ Fixed: Freezes on some robots.txt files
+ Fixed: Compressed files not stored
+ Fixed: SVG fixes
+ Fixed: Raw HTML responses
+ Fixed: 406 error workaround
+ Fixed: Crashes due to binary files with bogus HTML type (not parsed anymore)
+ Fixed: External https and ftp links broken, relative https links broken
+ Fixed: Automatic resizing of filter stack
+ Fixed: Various ampersand (&) elements added
+ Fixed: https with proxy temporary workaround (direct connection)
+ Fixed: "base href" with absolute uris
+ Fixed: stack frame too large on some systems
+ Fixed: random bad requests due to bogus authentication
+ Shell: Several fixes, including registration type problems
+ Shell: "template files not found" fixed
3.16-2
+ Fixed: Zlib v1.4
+ Fixed: Gzipped files now downloaded without problems (HTTP compression bug)
+ Fixed: Ending spaces in URLs now handled correctly
+ Fixed: META-HTTP bug
+ Shell: Type registration done only once
3.15
+ Fixed: Bogus HTTP-referer with protected sites
+ Fixed: Fatal IO/socket error with large sites (handles not closed)
+ Fixed: K4 option now works
+ Fixed: --continue+URL(s) now clears previous URLs
+ Fixed: Parsing bug with 'www.foo.com?query'
+ Shell: 'Soft cancel' documented
+ Shell: 'Kx' options added
3.10
+ Fixed: Broken pipes on Linux version
+ Fixed: Commandline version bug with gzipped files
+ Fixed: Crash when reaching compressed error pages
+ Fixed: Bogus html-escaped characters in query strings
+ Fixed: Files skipped (bogus anticipating system)
+ Fixed: Crash when showing stats (div by zero)
+ Fixed: Problems with URLs/redirects containing spaces or quotes
+ Fixed: Slash added when ~ detected
+ Fixed: Ugly VT terminal
+ New: Faster and cleaner mirror interrupt
3.09
+ Fixed: Several problems with javascript parsing
+ Fixed: Elements after onXXX not parsed
+ New: Source update wrapper
+ New: Style url() and @import parsed
+ Shell: Word database and maximum number of links
+ Shell: Option changes taken in account immediately
+ Shell: Cleaner installer (registry keys)
3.08
+ New: HTTP compression is now supported
+ New: Faster response analysis
+ Fixed: External page in html if cgi
+ Fixed: Mix between CR and CR/LF for comments
+ Fixed: Top index corrupted
+ Shell: Better refresh during parsing
+ Shell: DLL error
3.07
+ Fixed: Random crashes with HTTP redirects
+ New: New rate limiter (should be sharper)
+ New: Code cleaned up, new htscore.c/httrack.c files
3.06
+ Fixed: Redirect to https/mailto now supported
+ New: Top index/top dir for Un*x version
+ New: Sources more modular (.so)
+ New: Quicktime targetX= tags
+ New: HTTP 100 partially supported
3.05
+ Fixed: Non-scannable tag parameters ("id","name",..)
+ Fixed: Java classes not found when using "." as separator
+ Fixed: Java classes not found when missing .class
3.04
+ Fixed: URLs with starting spaces
+ Fixed: bogus URLs when using "base href"
+ Shell: --assume and -%e options included
+ New: Documentation updated a little
3.03
+ New: Parser optimizations, 10 times faster now!
+ New: New --assume option to speed up cgi tests
+ New: Option to avoid Username/password storage for external pages
+ New: Query string kept for local URIs
+ Fixed: RFC2396 compliant URLs accepted (//foo and http:foo)
+ Fixed: foo@foo.com not considered as URL anymore
+ Fixed: Space encoded into %20 in URIs
+ Fixed: "Unable to save file" bug
+ Fixed: Corrupted top index.html
+ Fixed: Cookies disabled with --get
+ Fixed: Cache bug for error pages
3.02
+ Fixed: Pages without title recorded in top index
+ Fixed: Error with Content-type+Content-disposition
+ Fixed: backblue.gif/external.html files not purged anymore
+ Fixed: Encoding problems with files containing %2F or other characters
+ Fixed: Write error reported for HTML files
+ New: hts-stop.lock file to pause the engine
+ New: New install system using InnoSetup
3.01
+ New: HTTP real media files captured
+ Fixed: Bogus statitics
+ Fixed: Minor fixes
3.00
+ New: New interface, with MANY improvements!
+ New: Better parsing (enhanced javascript parsing, sharper HTML parsing)
+ New: Faster and more efficient background download system
+ New: ETag properly handled
+ New: Optional URL list
+ New: Optionnal config file
+ New: New structure options
+ New: New filters options (size filters)
+ New: Better password site handling
+ New: Traffic control to avoid server overload
+ New: Setuid and Chroot for Unix release
+ New: limited 64-bit handling
+ New: .js files are now parsed
+ New: Single hts-log.txt file, error level
+ New: New top index.html design
+ New: "Update hack" option to prevent unnecessary updates
+ New: Default language sent for mirrors
+ New: Searchable index
+ Fixed: Bogus ftp routines (Linux version)
+ Fixed: Bug that caused to mirror a complete site from a subdir
+ Fixed: Bug that caused restart to be very slow
+ Fixed: Bug that caused loops on several query-string pages (?foo=/)
+ Fixed: Corrupted cache bug
+ Fixed: Random broken links (pages not downloaded)
+ Fixed: Shared links problems
+ Fixed: Bogus URLs with commas (,)
+ Fixed: Bogus / and \ mixed
+ Fixed: Bogus addresses with multiple @
+ Fixed: Bogus links with %2E and %2F
+ Fixed: Bogus empty links
+ Fixed: "Unexpected backing error" bug fixed
+ Fixed: Files with incorrect size no more accepted
+ Fixed: Top index.html created even for untitled pages
+ Fixed: Bogus N100 option (unable to save file)
+ Fixed: Deadlock when using many hosts in URLs
+ Fixed: Password stored internally to avoid access errors
+ Fixed: Fixed /nul DOS limit
+ Fixed: Bogus -* filter (nothing mirrored)
+ Fixed: .shtml now renamed into .html
+ Fixed: Content-disposition without ""
+ Fixed: External html page for /foo links
+ Fixed: Username/password % compliant
+ Fixed: Javascript parser sometimes failed with " and ' mixed
+ Fixed: Some Range: bugs when regeting complete files
+ Fixed: Range: problems with html files
+ Fixed: HTTP/1.1 407 and 416 messages now handled
+ Fixed: Bogus timestamp
+ Fixed: Null chars in HTML bug
+ Fixed: Error pages cache bug
+ Fixed: Connect error/site moved do not delete everything anymore!
+ Fixed: Bogus garbage ../ in relative URL
+ Shell: New transfer rate estimation
+ Shell: Fixed crash when using verbose wizard
+ Shell: dynamic lang.h for easier translation updates
+ Shell: Fixed some options not passed to the engine
+ Fixed: A lots of minor fixes!
2.2
Note: 3.00 alpha major bug fixes are included in the 2.2
2.02
+ New: Cache system improved, compatible with all platforms
+ New: Update process improved (accurate date)
+ New: Remote timestamp for files
+ New: ETag (HTTP/1.1) supported
+ Shell: Portugese interface available
+ Fixed: Bug with links containing commas
+ Fixed: 'file://' bug with proxy
+ New: Engine a little bit faster
+ Shell: Some bugs fixed in the interface
2.01
+ New: ftp through proxy finally supported!
+ New: Sources cleaned up
+ New: Again some new marvelous options
+ New: Speed improved (links caught during parsing, faster "fast update")
+ New: Tool to catch "submit" URL (forms or complex javascript links)
+ Shell: German interface available
+ Shell: Dutch interface available
+ Shell: Polish interface available
+ Fixed: Level 1 bug fixed
+ Fixed: Still some parsing/structure problems
+ Fixed: Referer now sent to server
+ Fixed: Cookies did not work properly
+ Fixed: Problems with redirect pages
+ New: Better javascript parsing
+ Fixed: Problems with URL-parameters (foo.cgi?param=2&choice=1)
+ Fixed: Problems with ftp
+ New: ftp transfers are now in passive mode (firewall compliant)
2.00 -- The First Free Software Release of HTTrack!
+ New: HTTrack sources (command line), now free software, are given
+ Shell: Interface rewritten!
+ New: Documentation rewritten
+ Shell: Drag&Drop abilities
+ Shell: More URL informations
+ Shell: Fixed: Remote access problems
+ Fixed: Loop problems on some sites causing crashes
+ Fixed: URL encoding problems
+ Fixed: Some file access problems for ../
+ Fixed: Some fixes for updating a mirror
+ Shell: Crazy progress bar fixed
+ Fixed: Form action are rewritten so that cgi on form can work from an offline mirror
+ Fixed: Crashes after continuing an "hand-interrupted" mirror
+ Fixed: Bogus files with some servers (chunk bug)
1.30
+ Shell: Interface improved
+ New: robots.txt are followed by default
+ New: Parsing speed improved on big (>10,000 links) sites with an hash table
+ New: Mirror Link mode (mirror all links in a page)
+ New: Cookies are now understood
+ New: No external pages option (replace external html/gif by default files)
+ New: Command line version improved, background on Unix releases
+ Fixed: Problems with javascript parsing
+ Fixed: Username/password not set to lowercase anymore
+ Fixed: Problems with base href
+ New: Links in level 1 html files now patched
+ New: Expurge now deletes unused folders
+ New: Option -V executes shell command for every new file
+ Shell: Primary filter now works
1.24
+ Fixed: Ftp protocol bogus (with login/pass)