Create a mirror of an FTP site with wget

Thoughts

People have asked how I downloaded FTP mirrors I mentioned on the archived hat episode, especially in light of recent news.

There are so many tools you can use, but wget makes it simple:

$ wget -m ftp://site.tld/folder/

And If you need authentication:

$ wget -m ftp://host.tld/folder/ --user=username --password=password

There are tools that will let you do batch downloads, but wget works sequentially on one file at a time. I think that’s a reasonable balance between getting the material and respecting the target site’s bandwidth.

As I said on that show, I’m realising that if you treasure something online, you need to preserve it yourself. Or if you’re allowed to, send it to the Internet Archive.

Author bio and support

Me!

Ruben Schade is a technical writer and IaaS engineer in Sydney, Australia who refers to himself in the third person in bios. Wait, not BIOS… my brain should be EFI by now.

The site is powered by Hugo, FreeBSD, and OpenZFS on OrionVM, everyone’s favourite cloud infrastructure provider.

If you found this post helpful or entertaining, you can shout me a coffee or buy some silly merch. Thanks!