Fsdb::Filter::dbsort(3)

NAME
dbsort - sort rows based on the the specified columns

SYNOPSIS
    dbsort [-M MemLimit] [-T TemporaryDirectory] [-nNrR] column [column...]

DESCRIPTION
Sort all input rows as specified by the numeric or lexical columns.

Dbsort consumes a fixed amount of memory regardless of input size. (It reverts to temporary files on disk if necessary, based on the -M and -T options.)

The sort should be stable, but this has not yet been verified.

For large inputs (those that spill to disk), dbsort will do some of the merging in parallel, if possible. The --parallel option can control the degree of parallelism, if desired.

OPTIONS
General option:

-M MaxMemBytes
    Specify an approximate limit on memory usage (in bytes). Larger values allow faster sorting because more operations happen in-memory, provided you have enough memory.

-T TmpDir
    where to put tmp files. Also uses environment variable TMPDIR, if -T is not specified. Default is /tmp.

--parallelism N or -j N
    Allow up to N merges to happen in parallel. Default is the number of CPUs in the machine.

Sort specification options (can be interspersed with column names):

-r or --descending
    sort in reverse order (high to low)

-R or --ascending
    sort in normal order (low to high)

-n or --numeric
    sort numerically

-N or --lexical
    sort lexicographically

This module also supports the standard fsdb options:

-d  Enable debugging output.

-i or --input InputSource
    Read from InputSource, typically a file name, or "-" for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

-o or --output OutputDestination
    Write to OutputDestination, typically a file name, or "-" for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

--autorun or --noautorun
    By default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The "--(no)autorun" option controls that behavior within Perl.

--header H
    Use H as the full Fsdb header, rather than reading a header from then input.

--help
    Show help.

--man
    Show full manual.

SAMPLE USAGE

  Input:
    #fsdb cid cname
    10 pascal
    11 numanal
    12 os

  Command:
    cat data.fsdb | dbsort cname

  Output:
    #fsdb      cid     cname
    11 numanal
    12 os
    10 pascal
    #  | dbsort cname

SEE ALSO
dbmerge(1), dbmapreduce(1), Fsdb(3)

CLASS FUNCTIONS

  new
    $filter = new Fsdb::Filter::dbsort(@arguments);

Create a new object, taking command-line arguments.

  set_defaults
    $filter->set_defaults();

Internal: set up defaults.

  parse_options
    $filter->parse_options(@ARGV);

Internal: parse command-line arguments.

  setup
    $filter->setup();

Internal: setup, parse headers.

  segment_start
    $self->segment_start(\@rows);

Sorting happens internally, to handle large things in pieces if necessary.

call "$self-"segment_start> to init things and to restart after an overflow "$self-"segment_overflow> to close one segment and start the next, and "$self-"segment_merge_finish> to put them back together again.

Note that we don't invoke the merge code unless the data exceeds some threshold size, so small sorts happen completely in memory.

Once we give up on memory, all the merging happens by making passes over the disk files.

  segment_next_output
    $out = $self->segment_next_output($input_finished)

Internal: return a Fsdb::IO::Writer as $OUT that either points to our output or a temporary file, depending on how things are going.

  segment_overflow
    $self->segment_overflow(\@rows, $input_finished)

Called to sort @ROWS, writing them to the appropriate place. $INPUT_FINISHED is set if all input has been read.

  segment_merge_start
    $self->segment_merge_start($fn);

Start merging on file $FN. Fork off a merge thread, if necessary.

  segment_merge_finish
    $self->segment_merge_finish();

Merge queued files, if any. Just call dbmerge(1) to do all the real work.

  run
    $filter->run();

Internal: run over each rows.

AUTHOR and COPYRIGHT
Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.