cpipe - copy stdin to stdout while counting bytes and reporting progress
[-b bsize] [-vt] [-vr] [-vw] [-ngr] [-s speed]
- buffer size in kB,
1 Int value between 1 and oo.
- show throughput.
- show read-times.
- show write-times.
- non-greedy read. Don't enforce a full buffer on read before
starting to write.
- throughput speed limit in kB/s,
1 Double value between 1 and oo.
copies its standard input to its standard output while measuring
the time it takes to read an input buffer and write an output buffer. If one
or more of the -vx
options is given, statistics of average
throughput and the total amount of bytes copied are printed to the standard
Normally, cpipe does its best to totally fill its buffer (option -b
before it starts writing. In some situations however, e.g. if you talk to an
interactive program via cpipe, this deadlocks the communication: said program
waits for input which it will never see, because the input is stuck in cpipe's
buffer. But cpipe itself will not see more input before the program does not
To get around this, try using -ngr
. When issuing a read call, cpipe is
then satisfied as soon as it gets at least one byte. Instead of filling the
buffer, it stops reading and writes whatever it got to the output. Note,
however, that the throughput measurements will be less exact if the number of
bytes transferred in one read/write pair becomes small, because cpipe will
spent relatively more time working on every byte.
If a throughput limit is specified with option -s
(3) in between copying buffers, thereby artificially extending
the duration of a read/write-cycle. Since on most systems there is a certain
minimum time usleep() sleeps, e.g. 0.01s, it is impossible to reach high
limits with a small buffer size. In this case increasing the buffer size
) might help. However, keep in mind that this limits the
throughput only on the average. Every single buffer is copied as fast as
tar cCf / - usr | cpipe -vr -vw -vt > /dev/null
results in an output like
in: 19.541ms at 6.4MB/s ( 4.7MB/s avg) 2.0MB
out: 0.004ms at 30.5GB/s ( 27.1GB/s avg) 2.0MB
thru: 19.865ms at 6.3MB/s ( 4.6MB/s avg) 2.0MB
The first column
shows the times it takes to handle one buffer of data
(128kB by default). The read-call took 19.541ms, the write-call to /dev/null
took just 0.004ms and from the start of the read to the end of write, it took
The second column
shows the result of dividing the buffer size (128kB by
default) by the times in the first column.
The third column
contains the average over all measured values from the
start of the program.
Finally, the last column
shows the total number of bytes transferred,
which is of course the same for reading and writing.
This program uses precious processor cycles. Consequently the measured times
will be different from the transfer rates possible without it.
Instead of just non-greedy reading, full non-blocking I/O and use of select(2)
should be used to make sure that no deadlocks occur when communicating with
Peter Astrand <email@example.com> recommended the speed limit.
Ivo De Decker <firstname.lastname@example.org> asked for deadlock prevention, which is
(hopefully) sufficiently covered by the non-greedy read.
Bug reports, beer and postcards go to email@example.com
. New versions will
show up on