manpagez: man pages & more
man lrzip(1)
Home | html | info | man
lrzip(1)                                                              lrzip(1)




NAME

       lrzip - a large-file compression program


SYNOPSIS

       lrzip [OPTIONS] <file>
       lrzip -d [OPTIONS] <file>
       lrunzip [OPTIONS] <file>
       lrzcat [OPTIONS] <file>
       lrztar [lrzip options] <directory>
       lrztar -d [lrzip options] <directory>
       lrzuntar [lrzip options] <directory>
       lrz [lrz options] <directory>
       LRZIP=NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>



DESCRIPTION

       LRZIP is a file compression program designed to do particularly well on
       very large files containing long  distance  redundancy.   lrztar  is  a
       wrapper for LRZIP to simplify compression and decompression of directo-
       ries.



OPTIONS SUMMARY

       Here is a summary of the options to lrzip.



       General options:
        -c, --check             check integrity of file written on decompression
        -d, --decompress        decompress
        -e, --encrypt           password protected sha512/aes128 encryption on compression
        -h, -?, --help          show help
        -H, --hash              display md5 hash integrity information
        -i, --info              show compressed file information
        -q, --quiet             don't show compression progress
        -r, --recursive         operate recursively on directories
        -t, --test              test compressed file integrity
        -v[v], --verbose        Increase verbosity
        -V, --version           show version
       Options affecting output:
        -D, --delete            delete existing files
        -f, --force             force overwrite of any existing files
        -k, --keep-broken       keep broken or damaged output files
        -o, --outfile filename  specify the output file name and/or path
        -O, --outdir directory  specify the output directory when -o is not used
        -S, --suffix suffix     specify compressed suffix (default '.lrz')
       Options affecting compression:
        -b, --bzip2             bzip2 compression
        -g, --gzip              gzip compression using zlib
        -l, --lzo               lzo compression (ultra fast)
        -n, --no-compress       no backend compression - prepare for other compressor
        -z, --zpaq              zpaq compression (best, extreme compression, extremely slow)
       Low level options:
        -L, --level level       set lzma/bzip2/gzip compression level (1-9, default 7)
        -N, --nice-level value  Set nice value to value (default 19)
        -p, --threads value     Set processor count to override number of threads
        -m, --maxram size       Set maximim available ram in hundreds of MB
                                overrides detected ammount of available ram
        -T, --threshold         Disable LZO compressibility testing
        -U, --unlimited         Use unlimited window size beyond ramsize (potentially much slower)
        -w, --window size       maximum compression window in hundreds of MB
                                default chosen by heuristic dependent on ram and chosen compression

       LRZIP=NOCONFIG environment variable setting can be used to bypass lrzip.conf.
       TMP environment variable will be used for storage of temporary files when needed.
       TMPDIR may also be stored in lrzip.conf file.

       If no filenames or "-" is specified, stdin/out will be used.





OPTIONS


General options

       -c     This option enables integrity checking of the  file  written  to
              disk on decompression. All decompression is tested internally in
              lrzip with either crc32 or md5 hash checking  depending  on  the
              version  of  the  archive  already.  However the file written to
              disk may be  corrupted  for  other  reasons  to  do  with  other
              userspace  problems  such  as  faulty library versions, drivers,
              hardware failure and so on. Enabling this option will make lrzip
              perform  an  md5  hash check on the file that's written to disk.
              When the archive has the md5 value stored in it, it is  compared
              to this. Otherwise it is compared to the value calculated during
              decompression. This offers an  extra  guarantee  that  the  file
              written is the same as the original archived.


       -d     Decompress.  If  this option is not used then lrzip looks at the
              name used to launch the  program.  If  it  contains  the  string
              "lrunzip"  then  the  -d option is automatically set. If it con-
              tains the string "lrzcat" then the -d -o - options are automati-
              cally set.


       -e     Encrypt.  This  option  enables  high  grade password encryption
              using a combination of multiply sha512 hashed  password,  random
              salt  and aes128 CBC encryption.  Passwords up to 500 characters
              long are supported, and the encryption mechanism used  virtually
              guarantees  that  the  same  file created with the same password
              will never be the same. Furthermore,  the  password  hashing  is
              increased  according to the date the file is encrypted, increas-
              ing the number of CPU cycles required for each password  attempt
              in  accordance  with  Moore's law, thus making the difficulty of
              attempting brute force attacks proportional to the power of mod-
              ern computers.


       -h|-?  Print an options summary page


       -H     This  shows  the  md5  hash  value  calculated on compressing or
              decompressing an lrzip archive. By default all  compression  has
              the  md5  value calculated and stored in all archives since ver-
              sion 0.560. On decompression, when an md5 value has been  found,
              it  will  be calculated and used for integrity checking.  If the
              md5 value is not stored in the archive, it will  not  be  calcu-
              lated  unless  explicitly  specified  with this option, or check
              integrity (see below) has been requested.


       -i     This shows information about a compressed  file.  It  shows  the
              compressed  size,  the decompressed size, the compression ratio,
              what compression was used and what hash checking  will  be  used
              for internal integrity checking.  Note that the compression mode
              is detected from the first block only and it will show  no  com-
              pression  used  if  the  first block was incompressible, even if
              later blocks were compressible. If verbose options -v or -vv are
              added,  a breakdown of all the internal blocks and progressively
              more information pertaining to them will also be shown.


       -q     If this option is specified then lrzip will not  show  the  per-
              centage  progress  while compressing. Note that compression hap-
              pens in bursts with lzma compression which is the  default  com-
              pression.  This  means  that  it  will progress very rapidly for
              short periods and then stop for long periods.


       -r     If this option is specified, lrzip will  recursively  enter  the
              directories  specified,  compressing or decompressing every file
              individually in the same directory. Note for better  compression
              it  is recommended to instead combine files in a tar file rather
              than compress them separately, either manually or with the  lrz-
              tar helper.


       -t     This tests the compressed file integrity. It does this by decom-
              pressing it to a temporary file and then deleting it.


       -v[v]  Increases verbosity. -vv will print more messages than -v.


       -V     Print the lrzip version number




Options affecting output

       -D     If this option is specified then lrzip will  delete  the  source
              file  after  successful  compression or decompression. When this
              option is not specified then the source files are not deleted.


       -f     If this option is not specified (Default) then  lrzip  will  not
              overwrite  any  existing files. If you set this option then rzip
              will silently overwrite any files as needed.


       -k     This option will keep broken or damaged files instead of  delet-
              ing  them.   When  compression  or  decompression is interrupted
              either by user  or  error,  or  a  file  decompressed  fails  an
              integrity check, it is normally deleted by LRZIP.


       -o     Set  the  output  file  name. If this option is not set then the
              output file name is chosen based on the input name and the  suf-
              fix.  The -o option cannot be used if more than one file name is
              specified on the command line.


       -O     Set the output directory for the default filename.  This  option
              cannot be combined with -o.


       -S     Set the compression suffix. The default is '.lrz'.




Options affecting compression

       -b     Bzip2  compression.  Uses  bzip2  compression for the 2nd stage,
              much like the original rzip does.

       -g     Gzip compression. Uses gzip compression for the 2nd stage.  Uses
              libz compress and uncompress functions.


       -l     LZO  Compression.  If this option is set then lrzip will use the
              ultra fast lzo compression algorithm for  the  2nd  stage.  This
              mode of compression gives bzip2 like compression at the speed it
              would normally take to simply copy the  file,  giving  excellent
              compression/time value.


       -n     No  2nd stage compression. If this option is set then lrzip will
              only perform the long distance redundancy 1st stage compression.
              While this does not compress any faster than LZO compression, it
              produces a smaller file that then  responds  better  to  further
              compression  (by eg another application), also reducing the com-
              pression time substantially.


       -z     ZPAQ compression. Uses ZPAQ compression which is  from  the  PAQ
              family  of compressors known for having some of the highest com-
              pression ratios possible but at the cost of being extremely slow
              on  both  compress  and decompress (4x slower than lzma which is
              the default).




Low level options

       -L 1..9
              Set the compression level from 1 to 9. The  default  is  to  use
              level 7, which gives good all round compression. The compression
              level is also strongly related to how much  memory  lrzip  uses.
              See the -w option for details.


       -N value
              The default nice value is 19. This option can be used to set the
              priority scheduling for the lrzip backup or decompression. Valid
              nice  values  are from -20 to 19. Note this does NOT speed up or
              slow down compression.


       -p value
              Set the number of processor count to  determine  the  number  of
              threads to run.  Normally lrzip will scale according to the num-
              ber of CPUs it detects. Using this will override  the  value  in
              case  you  wish  to use less CPUs to either decrease the load on
              your machine, or to improve compression. Setting it  to  1  will
              maximise  compression  but will not attempt to use more than one
              CPU.


       -T     Disables the LZO compressibility threshold testing when a slower
              compression  back-end is used. LZO testing is normally performed
              for the slower back-end compression of LZMA and ZPAQ.  The  rea-
              soning is that if it is completely incompressible by LZO then it
              will also be incompressible by them. Thus if a block fails to be
              compressed  by the very fast LZO, lrzip will not attempt to com-
              press that block with  the  slower  compressor,  thereby  saving
              time.  If this option is enabled, it will bypass the LZO testing
              and attempt to compress each block regardless.


       -U     Unlimited window size. If this option is set, and the file being
              compressed does not fit into the available ram, lrzip will use a
              moving second buffer as a "sliding mmap" which  emulates  having
              infinite ram. This will provide the most possible compression in
              the first rzip stage which can improve the compression of  ultra
              large  files when they're bigger than the available ram. However
              it runs progressively slower the larger the  difference  between
              ram and the file size, so is best reserved for when the smallest
              possible size is desired on a very  large  file,  and  the  time
              taken is not important.


       -w n   Set  the  maximum allowable compression window size to n in hun-
              dreds of megabytes.  This is the amount  of  memory  lrzip  will
              search during its first stage of pre-compression and is the main
              thing that will determine how much benefit  lrzip  will  provide
              over  ordinary  compression with the 2nd stage algorithm. If not
              set (recommended), the value chosen will  be  determined  by  an
              internal  heuristic  in lrzip which uses the most memory that is
              reasonable, without any hard upper limit. It is limited  to  2GB
              on  32bit  machines. lrzip will always reduce the window size to
              the biggest it can be without running out of memory.




INSTALLATION

       "make install" or just install lrzip somewhere in your search path.



COMPRESSION ALGORITHM

       LRZIP operates in two stages. The first stage finds and  encodes  large
       chunks  of  duplicated data over potentially very long distances in the
       input file. The second stage is to use a compression algorithm to  com-
       press  the  output of the first stage. The compression algorithm can be
       chosen to be optimised for extreme size (zpaq), size (lzma -  default),
       speed  (lzo),  legacy  (bzip2 or gzip) or can be omitted entirely doing
       only the first stage. A one  stage  only  compressed  file  can  almost
       always improve both the compression size and speed done by a subsequent
       compression program.


       The key difference between lrzip and other well known compression algo-
       rithms  is  its  ability to take advantage of very long distance redun-
       dancy. The well known deflate algorithm used in  gzip  uses  a  maximum
       history  buffer  of  32k.  The block sorting algorithm used in bzip2 is
       limited to 900k of history. The history buffer in lrzip can be any size
       long, not even limited by available ram.

       It  is  quite  common these days to need to compress files that contain
       long distance redundancies. For example, when compressing a set of home
       directories  several  users  might  have copies of the same file, or of
       quite similar files. It is also common to have a single file that  con-
       tains  large  duplicated  chunks over long distances, such as pdf files
       containing repeated copies of the same image. Most compression programs
       won't  be  able  to  take  advantage of this redundancy, and thus might
       achieve a much lower compression ratio than lrzip can achieve.




FILES

       LRZIP recognises a configuration file that contains  default  settings.
       This   configuration   is   searched  for  in  the  current  directory,
       /etc/lrzip,  and  $HOME/.lrzip.  The  configuration  filename  must  be
       lrzip.conf.



ENVIRONMENT

       By  default,  lrzip  will  search  for  and  use  a configuration file,
       lrzip.conf.  If the user wishes to bypass the file, a startup ENV vari-
       able may be set.
       LRZIP = NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>
       which will force lrzip to ignore the configuration file.



HISTORY - Notes on rzip by Andrew Tridgell

       The  ideas behind rzip were first implemented in 1998 while I was work-
       ing on rsync. That version was  too  slow  to  be  practical,  and  was
       replaced  by  this version in 2003.  LRZIP was created by the desire to
       have better compression and/or speed by Con  Kolivas  on  blending  the
       lzma  and  lzo  compression  algorithms  with the rzip first stage, and
       extending the compression windows to scale with increasing ram sizes.



BUGS

       Nil known.




SEE ALSO

       lrzip.conf(5), lrunzip(1), lrzcat(1), lrztar(1),  lrzuntar(1),  lrz(1),
       bzip2(1), gzip(1), lzop(1), rzip(1), zip(1)




AUTHOR and CREDITS

       lrzip is being extensively bastardised from rzip by Con Kolivas.
       rzip was written by Andrew Tridgell.
       lzma was written by Igor Pavlov.
       lzo was written by Markus Oberhumer.
       zpaq was written by Matt Mahoney.
       Peter  Hyman  added  informational  output, updated LZMA SDK, and added
       lzma multi-threading capabilities.

       If you wish to report a problem, or  make  a  suggestion,  then  please
       email Con at kernel@kolivas.org

       lrzip  is  released  under  the  GNU  General Public License version 2.
       Please see the file COPYING for license details.



                                   June 2016                          lrzip(1)

lrzip 0.631 - Generated Tue Feb 19 14:05:54 CST 2019
© manpagez.com 2000-2025
Individual documents may contain additional copyright information.