我正在尝试多次并行执行长时间运行的进程。 每个执行过程的参数都存储在一个空格分隔的环境variables中。 这是我正在尝试执行的一个人为的例子:
$ echo -e 1 2 3 4 | xargs --max-procs=3 --max-args=1 --replace=% echo % is the number being processed
这是该命令的输出:
1 2 3 4 is the number being processed
为什么max-args似乎被忽略? 然后,我试图明确设置分隔符,以获得更好的结果:
$ echo -e 1 2 3 4 | xargs -d " " --max-procs=3 --max-args=1 --replace=% echo % is the number being processed 1 is the number being processed 2 is the number being processed 3 is the number being processed 4 is the number being processed
xargs在处理第四个参数时做了什么?
经过一番search,我确实设法得到我想要的东西。 参数处理正确,但并行性不起作用(用另一个命令validation这里没有显示):
$ echo -e 1 2 3 4 | xargs -n 1 | xargs --max-procs=3 --max-args=1 --replace=% echo % is the number being processed 1 is the number being processed 2 is the number being processed 3 is the number being processed 4 is the number being processed
我错过了什么?
是否
echo -e 1 2 3 4 | sed -e 's/\s\+/\n/g' | xargs --max-procs=3 --max-args=1 --replace=% echo % is the number being processed
完成任务? 输出结果似乎是正确的:
1 is the number being processed 2 is the number being processed 3 is the number being processed 4 is the number being processed
我也尝试用sleep来replaceecho来确认它并行执行,它会:
echo -e 1 2 3 4 5 6 7 8 9 9 9 9 9 9 9 9 9 9 9 9 | sed -e 's/\s\+/\n/g' | xargs --max-procs=20 --max-args=1 --replace=% sleep 1