我有一个文件,有我需要下载的所有url。 不过,我需要一次限制一个下载。 即下一次下载应该只在上一次完成时才开始。 这可能使用curl吗? 或者我应该使用其他的东西。
wget(1)默认顺序运行,内置此选项:
-i file --input-file=file Read URLs from a local or external file. If - is specified as file, URLs are read from the standard input. (Use ./- to read from a file literally named -.) If this function is used, no URLs need be present on the command line. If there are URLs both on the command line and in an input file, those on the command lines will be the first ones to be retrieved. If --force-html is not specified, then file should consist of a series of URLs, one per line. However, if you specify --force-html, the document will be regarded as html. In that case you may have problems with relative links, which you can solve either by adding "<base href="url">" to the documents or by specifying --base=url on the command line. If the file is an external one, the document will be automatically treated as html if the Content-Type matches text/html. Furthermore, the file's location will be implicitly used as base href if none was specified.
xargs -n 1 curl -O < your_files.txt
这可以在shell脚本中使用curl,就像这样,但是你需要为自己研究适当的curl选项
while read URL curl some options $URL if required check exit status take appropriate action done <fileontainingurls