Wget commands: examples. Console program for downloading files over Wget network

It would seem that in a world where half the world's population actively uses computers and is accustomed to stylish graphic interfaces and rich features of such, there is no place for text applications. Some 20 years ago, the computer interface was a terrible black panel that did not display anything except a set of white characters, and all control was carried out through text commands. Surprisingly, how much these programs are ugly, so they are functional and reliable.

What is Wget?

Wget (also called GNU Wget Linux) is a console, non-interactive application for working with files and web pages. In fact, it is a program for downloading files from the Internet, characterized by a lack of a graphical interface and a number of management features that are not typical for more modern solutions.

The program was developed by Croatian engineer Hrvoji Niksic, and now the development is supported by Giuseppe Skrivano and the Linux community. The first build utility (then still called GetURL) was introduced in January 1996. The program was written in the C programming language and was released under the GNU license. Initially, the tool was released on the Linux platform. Over time, the utility has been translated into many languages and ported to all popular platforms, including Wget for Windows 7 (and newer), Wget for MacOS and so on.

This program is not interactive, which means that any process launched within its capabilities can not be controlled, it can be managed by referring to the controls of the operating system itself.

Despite the fact that modern browsers usually have a built-in download manager, they are still designed for interactive interaction, therefore, work in manual mode becomes difficult. Wget, on the contrary, allows a number of processes to automate. For example, it is possible to prepare a text document with a list of web pages in advance, using which you can download several files and pages at once.

Main functions of the program

Still, in the first place it is a program for downloading files, which implies the obviousness of its main function.

Wget can download any material from the Internet, be it documents, executive files or whole web pages in HTML format (by various protocols). Files can be downloaded from directories via FTP.

A client can perform a recursive download by automatically navigating through links within a specific website (or multiple websites), while restoring the original structure of the web page. It is possible to view downloaded websites without access to the network (by default it is downloaded as addresses for online viewing).

Initially, Wget adapted to slow and unstable connection types, so its important feature is the ability to restore an interrupted file download (this is possible if the server supports RANGE headers). Also Wget can monitor the status of the servers (their availability) for downloading and updating (Wget checks the file headers and supports the latest version) of the files and the web page.

The program also has a number of features typical for so-called spider tools that explore different websites and servers for the presence of certain files on them.

Supported Platforms

Wget Linux became the base for this product. As mentioned above, the utility initially worked only on Linux, but quickly gained popularity and was adapted to other systems. For example, you can find the version of Wget for Windows 7 and more modern operating systems, and there are assemblies with a graphical interface. As for Linux, there the program is often stitched into the distribution. The terminals in some Linux builds initially support Wget commands. Ubuntu, Fedora, openSUSE and other consumer distributions work with the program, as they say, out of the box. Next, consider the program in more detail.

Wget commands: list of basic parameters

The basic list of commands includes:

• -V - allows you to find out which version of the program is currently in use;
• -h - shows the most complete list of commands for working with the application;
• -b - allows you to put the program into the background;

List of message options

Wget commands allow you to fully control the information displayed. Here is their list:

• -o logfile - creates a log file, which records the entire process of the application;
• -a logfile - allows you to add an existing log file instead of creating a new one;
• -d - displays the information necessary for developers and debugger (helps to solve problems and correct errors in the client's work);
• -q - disables the display of messages;
• -v - this option is enabled by default, provides a detailed report on all processes occurring in the program (in real time);
• -nv - greatly reduces information received by the user in messages. Only the most necessary data and error notifications remain;
• -i file (with the path to the file) - the command allows to scan a file, extracting a link from it and performing a download; You can use files of different formats, including HTML, but if you use this command, do not specify the path to the file on the command line;
• -f - causes the program to read the file as HTML; When using this command to work with a local HTML document, an error may occur in order to avoid it, just specify in the document

Boot Options List

The basic commands for working with the download of files with the help of Wget. Here is their basic set:

• --bind-address = ADDRESS - this command allows you to transfer the bind data to the ADDRESS of the local machine. When using a TCP / IP connection, the command is required to specify the IP address accurately (if there are more than one).
• -t number (specify number) - allows the user to select the number of times to download. To avoid duplication, you must specify the value of number equal to zero.
• -nc - this command allows to restore the previous state of the downloaded file in case the network fails and the download is interrupted. If you do not use one of the boot commands (-N, -nc, -r), Wget will create a copy of it and start downloading again. The -r option allows you to replace existing files.
• -c - also contributes to the loading of the file, in case of loss of connection (and when Wget stopped self-attempting to restore the connection). It is enough to specify the given command before the link to the file. If there is a similar file in the directory in use, Wget will identify it and automatically decide whether to reboot it. The command can be used only on those servers where there is support for RANGE headers.

• --progress - allows you to select the type of download progress display. You can specify -progress = bar or progress = dot.
• --spider - replaces the file download command to check its presence on the server;
• -w (in seconds) - indicates the time interval between downloads.
• -q (specified in the number of kilobytes or megabytes) - allows you to accurately regulate the size of downloaded data and limit excessive volumes. Nevertheless, it's worth noting that one file will be loaded regardless of the quota set (that is, with one file this option will not work, no matter how hard it is).

HTTP Parameters List

• --http-user = and --http-passwd = - allow to specify login and password for authorization on the web resource from which files will be downloaded.
• -C = on / off - allows you to enable or disable the caching of data on the server side.
• --cookies = on / off - the same as for cache, only for cookies. Allows either to stop or resume the exchange of cookie data necessary for the server to maintain visitor statistics. By default, the option is enabled, but cookies are not saved to the hard drive.
• --save-cookies - allows you to save cookies (only fresh data).
• --proxy-user and -proxy-passwd - allows you to specify the login and password for authorization on the proxy server (if one is involved).

FTP Settings List

Advanced options for working with Wget. FTP Commands:

• -nr - disables deleting temporary files from the .listing directory that are spontaneously generated when working with FTP;
• -g on / off - allows the utility to use special characters in FTP directories;
• --passive-ftp - this command can be used with a Firewall to activate passive FTP mode (when the user himself is responsible for establishing a connection to the server).

Wget commands: examples of use

The simplest and most common example for working with the program is, of course, downloading files. To test Wget commands, you can start with a simple one, find a link to a file on the network, and enter a terminal with the exact and full path to the file *.

To continue downloading the file, use: Wget -c * a link specifying the exact and full path to the file *.

If there is a scheduled list with links to files and documents, then you can use the Wget -i filelist.txt command.

As for the loading of web pages, everything is somewhat more complicated here, but at the same time the number of possibilities increases significantly. In order not to re-enter the parameters every time you need to download a website, you can create scripts or set all the settings to a .wgetrc file.

To save a web page as links (it will be difficult to open them on the local machine), you can use the Wget -m * command to specify the exact and full path to the file *.

To make the same page available for viewing without connecting to the Internet, you can use the Wget -r -10 -k * link with the exact and full path to the file *.

Wget format error a command not found can mean any typo and an incorrectly specified parameter.