xlat/gen.sh: speedup xlat generation
This commit implements the following changes: - increases the number of concurrent jobs to twice the CPU count; - creates a circular buffer, so instead of running multiple jobs at once, the generator tries to keep about the same number of jobs being run concurrently; - runs gen_git and gen_make concurrently in order to squeeze in one more bit of concurrency. With my deeply scientific measurements, this approach achieves up to 15% speedup with dash and about 30-40% with bash as /bin/sh on a 4-core machine. * xlat/gen.sh (main): Declare pids local variable, append pid of every run job to it; increase the limit of jobs to ncpus * 2; when the limit is reached, wait for the first pid in pids instead of resetting jobs to zero and waiting for all the run jobs; put gen_git and gen_make into background.
This commit is contained in:
parent
4d8eedbd4f
commit
04a65a183b
18
xlat/gen.sh
18
xlat/gen.sh
@ -290,6 +290,7 @@ main()
|
||||
local name
|
||||
local jobs=0
|
||||
local ncpus="$(getconf _NPROCESSORS_ONLN)"
|
||||
local pids=
|
||||
[ "${ncpus}" -ge 1 ] ||
|
||||
ncpus=1
|
||||
|
||||
@ -300,15 +301,20 @@ main()
|
||||
name=${f##*/}
|
||||
name=${name%.in}
|
||||
gen_header "${f}" "${output}/${name}.h" "${name}" &
|
||||
pids="$pids $!"
|
||||
names="${names} ${name}"
|
||||
: $(( jobs += 1 ))
|
||||
if [ ${jobs} -ge ${ncpus} ]; then
|
||||
jobs=0
|
||||
wait
|
||||
fi
|
||||
if [ "${jobs}" -gt "$(( ncpus * 2 ))" ]; then
|
||||
read wait_pid rest
|
||||
pids="$rest"
|
||||
wait -n 2>/dev/null || wait "$wait_pid"
|
||||
: $(( jobs -= 1 ))
|
||||
fi <<- EOF
|
||||
$pids
|
||||
EOF
|
||||
done
|
||||
gen_git "${output}/.gitignore" ${names}
|
||||
gen_make "${output}/Makemodule.am" ${names}
|
||||
gen_git "${output}/.gitignore" ${names} &
|
||||
gen_make "${output}/Makemodule.am" ${names} &
|
||||
wait
|
||||
else
|
||||
name=${input##*/}
|
||||
|
Loading…
Reference in New Issue
Block a user