pv (Pipe Viewer) – Shell pipeline element to meter data passing through

pv (Pipe Viewer) can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

To use it, insert it in a pipeline between two processes, with the appropriate options. Its standard input will be passed through to

Install pv using the following command

#apt-get install pv

pv Syntax

pv [-h|-V]

pv Examples

pv will copy each supplied FILE in turn to standard output (- means standard input), or if no FILEs are specified just standard input is copied. This is the same behaviour as cat.

A simple example to watch how quickly a file is transferred using nc

pv file | nc -w 1 somewhere.com 3000

A similar example, transferring a file from another process and passing the expected size to pv:

cat file | pv -s 12345 | nc -w 1 somewhere.com 3000

A more complicated example using numeric output to feed into the dialog program for a full-screen progress display:

(tar cf – . \
| pv -n -s `du -sb . | awk ‘{print $1}’` \
| gzip -9 > out.tgz) 2>&1 \
| dialog –gauge ‘Progress’ 7 70

Frequent use of this third form is not recommended as it may cause the programmer to overheat.

Available Options

If no display switches are specified, pv behaves as if -p, -t, -e, -r, and -b had been given (i.e. everything is switched on). Otherwise, only those display types that are explicitly switched on will be shown.

-p, –progress
Turn the progress bar on. If standard input is not a file and no size was given (with the -s modifier), the progress bar cannot indicate how close to completion the transfer is, so it will just move left and right to indicate that data is moving.
-t, –timer
Turn the timer on. This will display the total elapsed time that pv has been running for.
-e, –eta
Turn the ETA timer on. This will attempt to guess, based on previous transfer rates and the total data size, how long it will be before completion. This option will have no effect if the total data size cannot be determined.
-r, –rate
Turn the rate counter on. This will display the current rate of data transfer.
-b, –bytes
Turn the total byte counter on. This will display the total amount of data transferred so far.
-n, –numeric
Numeric output. Instead of giving a visual indication of progress, pv will give an integer percentage, one per line, on standard error, suitable for piping (via convoluted redirection) into dialog. Note that -f is not required if -n is being used.
-q, –quiet
No output. Useful if the -L option is being used on its own to just limit the transfer rate of a pipe.

Output Modifiers

-W, –wait
Wait until the first byte has been transferred before showing any progress information or calculating any ETAs. Useful if the program you are piping to or from requires extra information before it starts, eg piping data into gpg or mcrypt which require a passphrase before data can be processed.
-s SIZE, –size SIZE
Assume the total amount of data to be transferred is SIZE bytes when calculating percentages and ETAs. The same suffixes of “k”, “m” etc can be used as with -L.
-l, –line-mode
Instead of counting bytes, count lines (newline characters). The progress bar will only move when a new line is found, and the value passed to the -s option will be interpreted as a line count.
-i SEC, –interval SEC
Wait SEC seconds between updates. The default is to update every second. Note that this can be a decimal such as 0.1.
-w WIDTH, –width WIDTH
Assume the terminal is WIDTH characters wide, instead of trying to work it out (or assuming 80 if it cannot be guessed).
Assume the terminal is HEIGHT rows high, instead of trying to work it out (or assuming 25 if it cannot be guessed).
-N NAME, –name NAME
Prefix the output information with NAME. Useful in conjunction with -c if you have a complicated pipeline and you want to be able to tell different parts of it apart.
-f, –force
Force output. Normally, pv will not output any visual display if standard error is not a terminal. This option forces it to do so.
-c, –cursor
Use cursor positioning escape sequences instead of just using carriage returns. This is useful in conjunction with -N (name) if you are using multiple pv invocations in a single, long, pipeline.

Data Transfer Modifiers

-L RATE, –rate-limit RATE
Limit the transfer to a maximum of RATE bytes per second. A suffix of “k”, “m”, “g”, or “t” can be added to denote kilobytes (*1024), megabytes, and so on.
-B BYTES, –buffer-size BYTES
Use a transfer buffer size of BYTES bytes. A suffix of “k”, “m”, “g”, or “t” can be added to denote kilobytes (*1024), megabytes, and so on. The default buffer size is the block size of the input file’s filesystem multiplied by 32 (512kb max), or 400kb if the block size cannot be determined.
-R PID, –remote PID
If PID is an instance of pv that is already running, -R PID will cause that instance to act as though it had been given this instance’s command line instead. For example, if pv -L 123k is running with process ID 9876, then running pv -R 9876 -L 321k will cause it to start using a rate limit of 321k instead of 123k. Note that some options cannot be changed while running, such as -c, -l, and -f.

General Options

-h, –help
Print a usage message on standard output and exit successfully.
-V, –version
Print version information on standard output and exit successfully.

Sponsored Link

6 thoughts on “pv (Pipe Viewer) – Shell pipeline element to meter data passing through

  1. Hi…

    I’m a half-n00b, half-bash user . . . .

    Can I use pv to display progress in a cp command?
    I cant figure out how and what to pipe exactly….

  2. There’s probably a better way to use pv to copy files, I’ve used the following when I wanted to monitor copy progress:

    pv big_file.tgz > /some_dir/sub_dir/big_file.tgz

    The only real annoyance doing it that way is having to specify the desitnation file name (whereas you would normally just specify the destination dir when using cp)

  3. Isaac:
    Simple, beautiful, elegant.
    It works and that’s very very good 🙂

    Thanks so much 🙂

  4. Hm – of course it is not that elegant, but to simulate cp syntax one may use:

    if [ -d $2 ]; then
    pv $1 > $2/$1
    pv $1 > $2

  5. I recommend to use the ‘rsync’ program together with the ‘–progress’ (or ‘-P’) option if you want a progress bar while copying files.
    This works with both local and network copy processes (like ‘scp’ or ‘rcp’).

    I just recently configured the following aliases for my bash shell (located in my $HOME/.bashrc):

    alias rscp=’rsync -aP –no-whole-file –inplace’
    alias rsmv=’rscp –remove-source-files’

    Where ‘rscp’ means ‘cp’ with ‘rsync’. The ‘-a’ (archive) option ensures that all file properties (date, permissions, owner, etc.) are copied, too. ‘-P’ enables the progress bar. The next two options are a matter of taste but are usefull if you copy files to a USB stick: ‘–inplace’ disables the usage of a hidden file for the partial data during the copy process. ‘–no-whole-file’ re-enables the synchronisation modus, i.e. in this case: a previous partially copied file is continued instead of overwritten.

    THe ‘rsmv’ is the move version of ‘rscp’ and simply adds ‘–remove-source-files’ to delete the original files after the copy process.

Leave a comment

Your email address will not be published. Required fields are marked *