GNU parallel (https://www.gnu.org/software/parallel/) is a powerful tool for evoking multiple processes. For example:
ls *.log | parallel -j8 xz -v9z
Will execute 8 xz processes in parallel to compress all the files with .log extension. By default, it will block until all the child processes return. You can also use it as the semaphore.
for i in *.log ; do echo $i parallel --semaphore -j8 gzip $i ";" echo donedoneparallel --semaphore --wait
@meisam All good, but if you can, try to use zstd :D. I used it once and never looked back.
Fediscience is the social network for scientists.