After three and a half years of development the release of the first stable version of the project "GNU Wget2 2.0", which is being developed as a completely redesigned version of the program to automate the recursive loading of "GNU Wget" content.
GNU Wget2 is designed and rewritten from scratch, and is notable for the removal of basic web client functionality in the libwget library, which can be used in standalone applications.
Instead of gradually reworking the existing code base, decided to redo everything from scratch and found a separate branch of Wget2 to implement ideas to restructure, increase functionality, and make changes that break compatibility. With the exception of the end of support for FTP and the WARC format, wget2 can act as a transparent replacement for the classic wget utility in most situations.
With the release of this version functionality has been moved to libwget library along with which a transition has been made to a multi-threaded architecture with which the possibility of configuring multiple connections in parallel and downloading in multiple flows has also been provided. It is even possible to parallelize the download of a file with a division in blocks using the option "–chunk-size".
Another novelty that stands out is the support for the HTTP / 2 protocol next to the If-Modified-Since HTTP header to download only the modified data.
While the OpenSSL-specific changes part fixes the CRL check, ALPN has been implemented and fixes have been made to fix problems with memory leaks.
On the other hand, it is also mentioned that license information has been updated, various fixes have been made in compilations to support lzip acceptance encoding, as well as to allow a list of tokens for connection and to fix the directory conflict with –no-clobber.
While for the part of the options that have been added we can find a improving method for backward compatibility, data improvements, the –body-file option has been added improving compatibility with previous versions, as well as the –ignore-length option, –convert-file-only option and –download-attr option to make use of the 'download attribute 'from HTML5
Of the other notable changes that stand out from the last version:
- –Robots = off option added for robots.txt downloads
- Added pkg-config support for GPGME
- Conversion corrections (-k) were made in combination with -E
- Fixed the cookie file header to be recognized by the 'file' command
- Fixed loading CA certificates when 'system' is not supported
- Renamed from –retry-on-http-status to –retry-on-http-error
- Page requirements limit for leaf pages only
- Correct NULL dereference with –convert-links
- Supports terminal hyperlinks on output
- Set the –disable-manylibs switch to disable building small libraries
- Support - background on Windows
- Add –bind-interface option
- Add HTTP2 payload
- Supports HTML download attribute (for to and area tags)
- Add the –download-attr = [strippath | usepath] to control download attribute support
- OpenSSL: add OCSP support
- OpenSSL: implement OCSP stapling
- Supporting data: URL in srcset attribute
- Fixed various issues
- Enhanced code, documentation, build, testing, CI, and more
Finally if you are interested in knowing more about it About this new released version of Wget2, you can check the details In the following link.
How to install Wget2 on Linux?
For those who are interested in being able to install this utility, they should know that in some of the Linux distributions they can find the package within their repositories.
Although they can also compile the package by following these instructions. The first thing we must do is obtain the source code with:
git clone https://gitlab.com/gnuwget/wget2.git cd wget2 ./bootstrap ./configure
We proceed to compile with:
make setarch x86 ./configure --prefix=/boot/home/config/non-packaged rm /boot/home/config/non-packaged/wget2 mv /boot/home/config/non-packaged/wget2_noinstall /boot/home/config/non-packaged/wget2 make check
And finally we install with:
sudo make install