I want to suck my files!

I would put my list of URLs to download in a file named URLs, and do something likecat URLs | while read url; do curl -o `basename "$url"` "$url"doneThat uses shell to loop over the URLs. Using basename for the file name isn't awesome. Often I prefer wget to curl, simply because it has a decent default filename.

Can't you just rename the files after they're downloaded? Have a bash script that looks something like this:#!/bin/bash#curl command(s) here, assuming we write to a directory called 'example.com'for file in ./example.com/*; do if grep -qi html "$file" then mv "$file" "$file.html" fidoneIf you need recursion you can adapt find to the task at hand, but that's the basic idea.

I'm not opposed to something other than curl, but I am going to do this from a dynamic site. So I would be generating that file on the fly. It won't be static. Is that a way to cat the content of a URL? I could pull that info way easy. Basically, I want to download every entry on my blog as an individual static page (from a template specifically designed for this).

axion, I can rename after downloading. If that's the easier option I'll take it. The filenames would just need .html appended.

If you're trying to archive your blog, that's basically what http://www.gnu.org/software/wget/manual/html_node/Recursive-Retrieval-Options.html is for. There are a variety of wget options for MacOS; I installed mine via homebrew.

You can use bash's word expansion to make this simple:while read -r url; do curl -so "${url##*/}.html" "$url"; done <file_with_urls.txtChange the final ; to a & and you automatically turn it into a parallel job, where all the files are downloaded simultaneously. (Don't do this if you have a ton of URLs in the file.)