Progress of long processes in BASH#
You can view progress of your process execution in UNIX using pv or bar.
With pv
, you can even report progress of multiple modules of your pipeline.
This is very useful for tracing large database dump/restore progress:
pv -cN gzip backup.sql.gz | gzip -d | pv -cN mysql | mysql
mysql: 799MiB 0:06:30 [1.68MiB/s] [ <=> ]
gzip: 173MiB 0:06:30 [ 250kiB/s] [=> ] 4% ETA 2:25:09
TAR random access
Multiprocessing in Python and garbage collection