Mirror or download a website with a tool much better than wget

If you want to mirror a website, the simplest tool I know if is wget. Once you have used wget, you know the troubles associated with it. You also get added couple of files as search html files, which are useless and wastage of bandwidth for you as well as the webserver.

lftp -e \”mirror -c\”  <URL>

This will mirror the website for you and will not leave with all those un-necessary files.

\"Reblog

About Amit Agarwal

12 Comments on “Mirror or download a website with a tool much better than wget”

  1. EzraSF

    The rsync command is another way of accomplishing this. It also has sophisticated tools for ensure the two locations result in the same data. What I most like about rync is the efficiency in comparing the bits to determine whether the data ought to be transferred. For something like this where the changes are taking place in existing files, rsync will do well.

    Be careful with it though. Tossing in random flags could cause a headache.

    1. Amit Agarwal

      Though I agree that rsync is a very good tool but you can use it only if rsh or rsync daemon in running on the remote side. For local copy, YES it is a very good tool.
      Thanks for reminding me of rsync 🙂

  2. Allen Demar

    I want to send kindly hi and want to say thanks for enlightened I digg and trying to find through the web for some kind of very useful like this, or at least a website. That coveredwhat i looked in to
    Thank you.

  3. Betty Votaw

    Simply want to say your article is as tonishing. The clearness in your post is simply spectacular and i can assume you are an expert on this field. Well with your permission allow me to grab your rss feed to keep up to date with forthcoming post. Thanks a million and please keep up the delightful work.

3 Trackbacks on “Mirror or download a website with a tool much better than wget”

  1. Pingback: Free Movie Online
  2. Pingback: car radio wire