QuickStart Guide - Header File Syntax: Keyword Statements

## 5.3 Header File Syntax: Keyword Statements

The header of a General Array Importer file contains two or more of the keyword statements in the list that follows. The statements from `block` through `type` are listed in alphabetical order for convenient reference (they can be listed in any order in a header file). The placement of the first two statements in the list (`file` and `grid|points`) and the last two (`positions` and `end`) reflect syntactic requirements. The descriptions that follow this list occur in the same partly alphabetized order.

Notes:

1. A statement or part of a statement enclosed in brackets ([ ]) is optional.
2. Except for `positions`, no keyword statement can exceed one line in length.
3. Any line beginning with a pound sign (#) is interpreted as a comment and ignored.

> file = filename

> grid = numx x numy x numz x ...

> or

> points = n

> [ block = skip1, elements1, width1, skip2, elements2, width2, ..., skipf, elementsf, widthf ]

> [ dependency = dependency1, dependency2, ..., dependencyf ]

> [ field = name1, name2, ..., namef ]

> [ format = ( msb | lsb ] [ ascii | text | binary | ieee )]

> [ header = ( bytes | lines | marker "string" )]

> [ interleaving = ( field | record | record-vector | series-vector )]

> [ layout = skip1, width1, skip2, width2, ..., skipf, widthf ]

> [ majority = ( row | column )]

> [ recordseparator = ( bytes n | lines n | marker "string" ), ( bytes n | lines n | marker "string" ), ... ]

> [ series = t [ , start, delta ] [ , separator = ( bytes n | lines n | marker "string" ) ]]

> [ structure = structure1, structure2, ..., structuref ]

> [ type = type1, type2, ..., typef ]

> [ positions = ( ( origin1, delta1, ..., origind, deltad ) | ( positiontype1, positiontype2, ..., positiontyped, position1, position2, ..., positiond ) | ( position1, position2, ..., positiond ) ) ]

> [ end ]

### file

file = filename

Function: | Specifies the name of the file (including the path, if any) containing the data to be imported. The Importer searches the directory where the header file was found and any paths specified with the DXDATA environment variable (see the appropriate appendix in IBM Visualization Data Explorer User's Guide for information on environment variables).
Use: | Required unless the header file contains an `end` statement. If this statement is omitted, the Importer assumes that the data are contained in the same file as the header and that they begin on the line immediately after the `end` statement or at the point specified by a `header` statement.

### grid

grid = numx x numy x numz x ...

Function: | Specifies the size and dimensions of the grid containing the data to be imported.
Use: | Required unless the header file contains a `points` statement.

Notes:

1. The num parameter is an integer specifying the number of coordinate points for a particular dimension (e.g., numx). The number of dimensions is implicitly specified by the number of such values provided. For example,

       grid = 2 x 2

   specifies a regular grid of 4 (four) points (the `x` between integers is required, but the blank spaces are optional.)
2. The `grid` keyword (in contrast to `points`) implies connections between data points. An n-dimensional cuboid is assumed for connections. For example, a 1-dimensional grid generates a line connecting the positions.
3. You can also specify that the number of grid elements are to be found in the data file. For the syntax, see B.1 , "General Array Importer: Keyword Information from Data Files" in IBM Visualization Data Explorer User's Guide.

### points

points = n

Function: | Specifies the number of data points to be imported.
Use: | Required unless the header file contains a `grid` statement.

Notes:

1. The `points` keyword (unlike `grid`) implies an absence of connections between data points.
2. Unless the `locations` reserved word is used in the `field` keyword statement (see "field"), the positions are 1-dimensional. If this reserved word is not used, 1-dimensional positions can be specified with the `positions` keyword (see "positions"). Otherwise, positions are assumed to be regular, with an origin of 0 (zero) and a delta of 1 (one).
3. You can also specify that the number of points is to be found in the data file. For the syntax, see B.1 , "General Array Importer: Keyword Information from Data Files" in IBM Visualization Data Explorer User's Guide.

### block

[ block = skip1, elements1, width1, skip2, elements2, width2, ..., skipf, elementsf, widthf ]

Function: | Specifies characteristics of each data field being imported. This keyword applies only to fixed-format ASCII data with record, record-vector, or series-vector interleaving.
Use: | Optional.

Note: All three parameters take integer values. (Comma separators are optional.)

- `skip` specifies the number of leading characters (in a line) to be passed over before reading the data in a line or in a record.
- `elements` specifies the number of data values stored in each line or in an entire record.
- `width` specifies the number of characters to be read for each element of a given field.

### dependency

[ dependency = dependency1, dependency2, ..., dependencyf ]

Function: | Specifies the dependencies of the data fields being imported.
Use: | Optional. By default, data are assumed to be position dependent. Only if the header file also contains a `grid` statement (see "grid") may you specify connections (cell-centered) dependency (a `points` statement implies positions dependency).

Notes:

1. For field-interleaved data, all fields must have the same dependency.
2. The `locations` field (see "field Keyword Statement," Note 4) must be specified as position dependent.
3. Comma separators are optional.

### field

[ field = name1, name2, ..., namef ]

Function: | Specifies the name and number of individual fields in a data file.
Use: | Optional. If this keyword is not used, the number of fields is derived from other keywords (e.g., `structure` or `type`).

Notes:

1. The values for the `name` parameter are strings separated by commas and without quotation marks.
2. You must name all the fields in the data file.
3. The field names can be used later to refer to individual fields. For example, if you use the Import module to import the data all at once, you can use the Select module to separate out each field by name. Or you can specify the names of the fields you want to import as the `variable` parameter to the Import module. For more information, see Import and Select in IBM Visualization Data Explorer User's Reference.
4. If the position values are intermixed with the data in a file, you must specify the positions as a field (instead of using the `positions` keyword). Use the reserved word `locations` for the field name. The corresponding value for the `structure` keyword should specify the dimensionality of the positions (e.g., "2-vector").

### format

[ format = ( msb | lsb ] [ ascii | text | binary | ieee )]

Function: | Specifies the format and byte order of the data.
Use: | Optional.

Notes:

1. The accepted values for byte order are `msb` (most significant byte first) and `lsb` (least significant byte first). If the format is specified as binary, then the default byte order is the host byte order (i.e., the byte order of the machine on which the Import module is executing).
2. For specifying the data format, `ascii` and `text` are synonymous, as are `binary` and `ieee`. The default is `ascii`. The supported binary form is IEEE.

### header

[ header = ( bytes | lines | marker "string" )]

Function: | Specifies how much material the Importer must skip before it begins reading data from a data file.
Use: | Optional. By default, the Importer assumes that the data begin at the start of the file. `bytes` n: the Importer will pass over n bytes (in a binary file) or n characters (in an ASCII file) before it begins to read data. `lines` n: the Importer will pass over n lines before it begins to read data. `marker` "string": the Importer will begin at the first character after the specified string. Quotation marks are required for a string containing blank spaces or commas.

Notes:

1. If the data begins on the line following a marker, be sure to specify a new-line character ("\n") as part of the string. Note also that the spacing between the marker and the new-line character must be the same as that in the actual file. This spacing is easily determined in the Data Browser by placing the cursor at the end of the marker and then at the end of the line and reading the corresponding offset values (see Figure 18).
2. If the marker itself contains quotation marks or special characters, use the escape character ("\") to indicate them, as shown in the following table.

       backslash                    \          \\
       backspace                    BS         \b
       bit pattern                  ddd        ddd
       carriage return              CR         \r
       double quote                 "          \"
       form feed                    FF         \f
       horizontal tab               HT         \t
       newline                      NL (LF)    \n
       Note: An octal value (ddd) can be used to specify special
             characters other than those shown here.

### interleaving

[ interleaving = ( field | record | record-vector | series-vector )]

Function: | Specifies to the Importer how the data in a data file are interleaved.
Use: | Optional. By default, the Importer assumes that the interleaving is record-vector.

Note: The examples presented here are based on a 1-dimensional grid with 10 elements and two series members:

    ·
    grid = 10
    series = 2
    field = t, v
    structure = scalar, 3-vector
    ·

where `t` is a scalar value and `v` is a vector with three components (vx, vy, and vz). In the examples themselves, `s` represents a series member (0 or 1), and `g` represents a grid element number (0 through 9). The `interleaving` options are as follows:

`field` | Specifies column-oriented data such as that generated by a spreadsheet or by data-listing software. There is a separate "column" for each of the two fields with one element of each field per "row." (For non-scalar data, as here, there is a column for each vector component.) The number of rows corresponds to the size of the grid multiplied by the number of series members.
    |- Field t -|--------------- Field v ---------------|
    t(s=0,g=0),  vx(s=0,g=0),  vy(s=0,g=0),  vz(s=0,g=0),
    t(s=0,g=1),  vx(s=0,g=1),  vy(s=0,g=1),  vz(s=0,g=1),
    ·
    t(s=0,g=9),  vx(s=0,g=9),  vy(s=0,g=9),  vz(s=0,g=9),
    t(s=1,g=0),  vx(s=1,g=0),  vy(s=1,g=0),  vz(s=1,g=0),
    t(s=1,g=1),  vx(s=1,g=1),  vy(s=1,g=1),  vz(s=1,g=1),
    ·
    t(s=1,g=9),  vx(s=1,g=9),  vy(s=1,g=9),  vz(s=1,g=9)
`record` | Specifies block or record-oriented data, where the values of all the elements of all the fields corresponding to one member (e.g., a time step) are listed before the elements and fields of the next member. For non-scalar fields, all the values of each vector component (e.g., all values of x) are listed in a separate record rather than in tuples (as they are in `record-vector` data; see below).
Note: | For scalar fields, `record` and `record-vector` are the same.
    t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9),    ] Field t  *
    vx(s=0,g=0), vx(s=0,g=1), ..., vx(s=0,g=9), *          |  Member
    vy(s=0,g=0), vy(s=0,g=1), ..., vy(s=0,g=9), | Field v  |    s0
    vz(s=0,g=0), vz(s=0,g=1), ..., vz(s=0,g=9), *          *
    t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9),    ] Field t  *
    vx(s=1,g=0), vx(s=1,g=1), ..., vx(s=1,g=9), *          |  Member
    vy(s=1,g=0), vy(s=1,g=1), ..., vy(s=1,g=9), | Field v  |    s1.
    vz(s=1,g=0), vz(s=1,g=1), ..., vz(s=1,g=9)  *          *

The remaining two options (`record-` and `series-vector`) apply to cases in which vector components are stored together:

`record-vector` | Here the values of all the elements of all the components of all the fields corresponding to each member (e.g., a time step) are listed before those corresponding to the next member (e.g., all the data for `s`0 are listed first, followed by all the data for `s`1). In addition, the components of each vector are stored as tuples (in contrast to the way they are stored in `record` data).
    t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9), ] Field t *
    vx(s=0,g=0), vy(s=0,g=0), vz(s=0,g=0),   *         |
    vx(s=0,g=1), vy(s=0,g=1), vz(s=0,g=1),   | Field   |  Member s0
    ...                                      |   v     |
    vx(s=0,g=9), vy(s=0,g=9), vz(s=0,g=9),   *         *
    t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9), ] Field t *
    vx(s=1,g=0), vy(s=1,g=0), vz(s=1,g=0),   *         |
    vx(s=1,g=1), vy(s=1,g=1), vz(s=1,g=1),   | Field   |  Member s1
    ...                                      |   v     |
    vx(s=1,g=9), vy(s=1,g=9), vz(s=1,g=9)    *         *
`series-vector` | Here the values of all the elements of all the members (e.g., time steps) are listed for one field before those of the next field (e.g., all the data for field `t` are listed first, followed by all the data for field `v`). In addition, the components of a vector are stored as tuples rather than in separate records (as they are stored in `record` data; see above).
    t(s=0,g=0), t(s=0,g=1), ..., t(s=0,g=9),  * Field  ] Member s0
    t(s=1,g=0), t(s=1,g=1), ..., t(s=1,g=9),  *   t    ] Member s1
    vx(s=0,g=0), vy(s=0,g=0), vz(s=0,g=0),    *        *
    vx(s=0,g=1), vy(s=0,g=1), vz(s=0,g=1),    |        | Member
    ...                                       |        |   s0
    vx(s=0,g=9), vy(s=0,g=9), vz(s=0,g=9),    | Field  *
    vx(s=1,g=0), vy(s=1,g=0), vz(s=1,g=0),    |   v    *
    vx(s=1,g=1), vy(s=1,g=1), vz(s=1,g=1),    |        | Member
    ...                                       |        |   s1
    vx(s=1,g=9), vy(s=1,g=9), vz(s=1,g=9)     *        *

### layout

[ layout = skip1, width1, skip2, width2, ..., skipf, widthf ]

Function: | Specifies the number of bytes (characters) the Importer must skip before it begins to read a field's data and then the number of bytes it should read (i.e., the "width" of the data item).
Use: | Optional. This keyword applies only to ASCII, field-interleaved data. If the data is in ASCII format but the keyword is not used, the Importer assumes a default of one or more blank spaces (space, tab, new line, or form feed) as the delimiter between fields.

Notes:

1. The components of a vector must each be represented in the same number of characters. Comma separators are optional.

   The following statement tells the Importer to skip 10 characters, read one field of 6 characters, skip 10 characters, and read another field of 4 characters.

       layout = 10, 6, 10, 4

2. Because skip specifies the number of characters to be passed over before each field, it does not apply to single elements of a vector field but to the field as a whole. However, width applies to each element of the vector. For example, the following statement tells the Importer to skip 10 characters, then read a field 6 characters long:

       layout = 10, 6

   If the field v is a 2-vector, then each component (vx and vy) is 6 characters long, for a total width of 12 characters.
3. If there are no spaces between two fields, specify the skip value as 0 (zero).

### majority

[ majority = ( row | column )]

Function: | Specifies the organization of multidimensional arrays composing a data field.
Use: | Optional. The default is `row` (last dimension varies fastest, as in the C programming language). Column majority means that the first dimension varies fastest, as in the FORTRAN programming language.

Note: The maximum number of dimensions supported for column majority is 4.

### recordseparator

[ recordseparator = ( bytes n | lines n | marker "string" ), ( bytes n | lines n | marker "string" ), ... ]

Function: | Specifies the separation between records.
Use: | Optional. This keyword applies only to record and record-vector interleaving.

Notes:

1. With differences to be noted here, the specification of separation is very much like that of the `header` keyword (see "header").
2. If all records are separated by the same amount, a single separator value should be specified. For example, if each pair of successive records is separated by two lines, then

       recordseparator = lines 2

3. If the records are separated by different amounts, a value must be specified for each of the separators:
   - For record-vector data the number of separator values must equal the number of fields minus one. For example, if

         structure = scalar, scalar, 2-vector

     then two separator values must be specified: one between the two scalar fields and one between the second scalar field and the vector field.
   - For record data, the number of separator values must equal the number of dimensions minus one. For the preceding example, three separator values must be specified: the first between the two scalar fields; the second between the second scalar and the first vector component; and the third between the two vector components.

### series

[ series = t [ , start, delta ] [ , separator = ( bytes n | lines n | marker "string" ) ]]

Function: | Specifies, to the Importer, information about a series.
- Use: | Optional. This keyword is required only for the importation of series data. The default assumes no separation between series sections. Descriptive information between series sections is described with the `separator` parameter. If descriptive information precedes the first member of the series, it can be skipped by means of a `header` statement. `t` is a required parameter that specifies the number of series elements in the data file. Its default value is 1. `start` and `delta` are optional parameters that specify the series positions. The position values are defined as: < start, start plus d e l t a, start plus (2 times d e l t a), ellip, start plus (t minus 1) times d e l t a> The defaults for `start` and `delta` are 0 (zero) and 1 (one) respectively. The following example specifies that there are four series members, with positions 0.8, 1.2, 1.6, and 2.0.
      series = 4, 0.8, 0.4
The specification of `separator` is very much like that of the `header` keyword (see "header").

### structure

[ structure = structure1, structure2, ..., structuref ]

Function: | Specifies the structure of each field in a data file.
Use: | Optional. The default is `scalar`. Accepted values are `scalar`, `string[ n ]`, and `2-vector , ..., 9-vector` for each field. However, `5-vector, ..., 9-vector` cannot be specified for column-majority arrays. In `string[ n ]`, n specifies the length of the longest string.

Notes:

1. Use of this keyword requires specifying the structure of all fields in the file.
2. The following example specifies that the first two fields have a scalar structure, while the third is a vector with three components:

       structure = scalar, scalar, 3-vector

3. Since the default is scalar, the statement is not required if, say, six scalar fields are to be imported. But if one of these fields is vector, the statement is required; for example:

       structure = scalar, scalar, 3-vector, scalar, scalar, scalar

4. If string data contain embedded blanks, you must use the `layout` or `block` keyword to specify how the string is to be read.

### type

[ type = type1, type2, ..., typef ]

Function: | Specifies the data type for each specified field.
Use: | Optional. The default is `float`. The accepted values are:
    double      byte               int               short
    float       signed byte        signed int        signed short
    string      unsigned byte      unsigned int      unsigned short

Notes:

1. Use of this keyword requires specifying the type for all fields in the file.
2. The following are pairs of equivalent types:
   - `byte` and `unsigned byte`.
   - `short` and `signed short`.
   - `int` and `signed int`.

### positions

[ positions = ( ( origin1, delta1, ..., origind, deltad ) | ( positiontype1, positiontype2, ..., positiontyped, position1, position2, ..., positiond ) | ( position1, position2, ..., positiond ) ) ]

Each syntax line in this diagram is discussed separately in Note 4 below.

Function: | Defines the positions component of the fields in a data file.
Use: | Optional. The default is regular positions in compact notation, where the origin and delta in each dimension are 0.0 and 1.0 respectively, unless the `locations` keyword has been used.

Notes:

1. This keyword must be placed at the end of the header file or immediately preceding the `end` keyword.
2. The numbers specified in a `positions` statement can span any number of uninterrupted lines (including carriage returns, which is not true of other keyword statements).
3. Positions can also be specified with the `field` statement and the `locations` reserved word (see "field"). This alternative is important in cases where:
   - The positions are irregular and can be interleaved with data themselves.
   - The positions vary with a series member.
   - The positions are not stored in row-majority order in ASCII.
4. There are four ways to specify positions with the `positions` keyword:

- `Regular positions:` < "positions" = origin sub 1 , d e l t a sub 1 , ellip, origin sub d , d e l t a sub d > Specifies a regular grid, using the origin of a dimension and the spacing (delta) of the positions in that dimension. The dimensions must be specified in the same order as that in the `grid` keyword statement (see "grid").

  The statement

      positions = 0.0, 1.0, 0.0, 0.5, 0.0, 1.5

  specifies the origin-delta pairs of a 6 × 6 × 3 grid, with origins of 0.0 in all three dimensions and deltas of 1.0, 0.5, and 1.5.

- `Partially regular positions:` < "positions" = positiontype sub 1 , positiontype sub 2 , ellip , positiontype sub d , position sub 1 , position sub 2 , ellip , position sub k > Specifies an array (regular or irregular) for each dimension from which a product is to be formed. For example, one array with the values 1, 2, 3, and a second with the values 1, 4, 8 will generate the following set of positions:
  (1, 1) (1, 4) (1, 8)
  (2, 1) (2, 4) (2, 8)
  (3, 1) (3, 4) (3, 8)

  For each dimension, specify a string indicating the type of positions for the dimension--accepted values are `regular` and `irregular`. Specify the strings for all dimensions first, then follow with the position values.

  For compact specification, the position values are two numbers: an origin and the delta value for the spacing. Specifying irregular positions requires an explicit listing of each position value, with the same number of positions as specified by the `grid` keyword statement. The order in which positions are specified must correspond to the order in which dimensions are specified in that statement.

  The following example specifies the positions for a 6 × 6 × 3 grid, where the first two dimensions are regular, and the third is irregular:

      position = regular, regular, irregular, 0.0, 1.0, 0.0, 0.5, 0.0, 0.5, 1.5

  Here the first dimension is regular, with positions 0, 1, 2, 3, 4, 5.
  The second dimension is regular, with positions 0.0, 0.5, 1.0, 1.5, 2.0, 2.5.
  The third dimension is irregular, with positions 0, 0.5, 1.5.

  The first few positions are (0, 0, 0) (0, .5, .5) (0, .5, 1.5) (1, 0, 0)....

- `Completely irregular positions:` < "positions" = position sub 1 , position sub 2 , ellip , position sub g > Specifies fully irregular positions: you must list all the position values.

  Note: The requirement that all positions must be listed is what distinguishes a "completely irregular" from a "partially regular" grid (discussed above). For example, the keyword statement

      positions = irregular, irregular, irregular,...

  still defines a partially regular grid, even though each array specified is irregular.

  The number of values you provide corresponds to the product of all the dimension specifications (i.e., the num values) in the `grid` keyword statement. The position values must be listed in row majority order, and they can be delimited by commas.

  You must specify g numbers, where g is the product of the num values of the dimensions all multiplied together, along with the number of dimensions (e.g., m×n×o×d, where m, n, and o are the grid dimensions, or num values, and d is the number of dimensions).

  The following example specifies the six positions for an irregular 2 × 3 grid:

      positions = 0, 0, 0, 2, 0, 6, 2, 1, 5, 4, 7, 7

  The first and last positions are (0, 0) and (7, 7) respectively.

- `Position information from the data file`

  You can specify that the information for the `positions` keyword is to be found in the data file. For the syntax, see B.1 , "General Array Importer: Keyword Information from Data Files" in IBM Visualization Data Explorer User's Guide.

  Note: The `positions` keyword may give a compact encoding of the position. In that respect, this function differs from the `locations` reserved word when used with the `field` keyword (see "field").

### end

[ end ]

Function: | Causes the Importer to stop processing header statements.
Use: | Optional unless the data are in the same file with the header statements. By default, the Importer stops processing at the end of the header file.

[ OpenDX Home at IBM | OpenDX.org ]