linux

How To Download Entire Websites Using wget.

An invaluable tool that I use a lot is wget. wget is a free terminal utility for downloading files from the Internet.

I do not have a reliable Internet connection and when I do get online, I am usually on a metered network. Not being able to connect to the Internet when I want to makes accessing online resources difficult. To solve this problem I use wget to download websites to my computer. I will show you how to do this yourself in this post.

The command to download a site is wget url
Here is an example using the Python Documentation Website wget http://docs.python.org/
This will download everything on the python docs website’s first page. Adding an “r” option to the wget command will cause it to download the entire site.
wget --recursive http://docs.python.org/

Other options you can make use of are:

  • –domains : don’t follow links outside the domain in url
  • –convert-links: convert links so that the work offline
  • –no-clobber: don’t overwrite any existing file. Use this option to resume an interrupted download

wget can download files over FTP, HTTPS and even retrieve files through proxies.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s