- adddim
- aggregate
- allversions
- apply
- attributes
- average
- bernoulli
- beetwen
- build
- build_sparse
- cancel
- cast
- concat
- count
- create array
- cross
- cross_join
- deldim
- dimensions
- diskinfo
- echo
- explain_logical
- explain_physical
- input
- filter
- redimension
- redimension_store
- join
- list
- load library
- load
- lookup
- match
- materialize
- max
- merge
- min
- multiply
- norm
- project
- regrid
- remove
- rename
- repart
- reshape
- reverse
- sample
- save
- scan
- setopt
- scatter/gather
- slice
- sort
- store
- subarray
- substitute
- sum
- thin
- transpose
- unpack
- versions
- window
- xgrid
- mstat

## adddim

Description
  Add dimension to array. Prepend dimensions list with new dimension with given name and size equal to one and chunk interval equal to one.
Syntax
  adddim ( input-array, new-dimension-name-1 )
Example
  `iquery -aq "adddim(matrix, timestamp)"`

## aggregate

Description
  Calculate aggregates for the specified dimensions. This operator takes a list of aggregate calls and a list of dimension names by which grouping is performed and calculates the aggregates for the remaining dimensions. Result is an array with shape corresponding to the specified dimensions and one attribute for each specified aggregate.
Syntax
  aggregate ( input-array, aggregate-1,...aggregate-N, dimension-1,... dimension-N )
Example
  `iquery -aq "aggregate(Reads, POSITION, sum(a), count(*) as num_values)"`

## allversions

Description
  Get all versions of updatable array using version number as additional (first) dimension.
Syntax
  allversions ( updatable-array-name )
Example
  `iquery -aq "allversions(updarr)"`

## apply

Description
  Evaluate expression for each element of the input array. The result is represented as a separate attribute added to the input array.
Syntax
  apply ( input-array, result-attribute, applied-expression )
  input-array { result-attribute-1 : applied-expression-1 , result-attribute-2 : applied-expression-2 , ... }
Example
  `iquery -aq "apply(testarr, sum, a+b)"`

## attributes

Description
  Get list of array attributes. Returns array with the following attributes: name, type.
Syntax
  attributes ( array-reference )
Example
  `iquery -aq "attributes(testarr)"`

## average

Description
  Calculate average value of the specified attribute in the array. The result is an array with single element containing average value. If input array contains only one attribute, then attribute name can be omitted.
Syntax
  average ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
  `iquery -aq "average(testarr, A)"`
  `iquery -aq "average(matrix)"`
  `iquery -aq "average(Reads, Read, OFFSET)"`

## bernoulli

Description
  Sample input array by selecting random number of cells from it. The marginal probability that a cell is included in the sample is provided in the second argument. If users desire a repeatable sample, they can provide an initializing seed value for the internal random number genertor.
Syntax
  bernoulli ( input-array , probability [ , seed ] )
Example
  `iquery -aq "max(bernoulli(testarr, 0.01))"`

## between

Description
  Get subarray of the input array specified by low and high boundaries preserving shape of the input array.
Syntax
  between ( input-array, low-boundary-coordinate-1, ..., low-boundary-coordinate-N, high-boundary-coordinate-1, ..., high-boundary-coordinate-N )
Example
  `iquery -aq "between(testarr, 0, 1, 1, 2)"`

## build

Description
  Construct single-attribute array, assigning values to its elements using a specified expression. This expression can access any of the array coordinates. First example below initializes a matrix with sequential values 1, 2, 3, ... Second example creates identity matrix.
Syntax
  build ( array-reference, expression )
Example
  `iquery -q "create array M3x3 <val:float> [x=0:2,3,0, y=0:2,3,0]"`
  `iquery -aq "build(M3x3, 0)"`
  `iquery -aq "build(M3x3, y*3 + x)"`
  `iquery -aq "build(M3x3, if x = y then 1 else 0)"`

## build_sparse

Description
  Construct single-attribute sparse array, assigning values to its elements selected by predicate using specified expression. This expression can access any of the array coordinates. The following example initializes spars diagonal matrix:
Syntax
  build_sparse ( array-reference, expression, predicate )
Example
  `iquery -q "create array M3x3 <val:float> [x=0:2,3,0, y=0:2,3,0]"`
  `iquery -aq "build_sparse(M3x3, 1, x = y)"`

## cast

Description
  Cast array type. This operator converts input array to the specified array type. The input array and target array descriptor should have the same number of attributes with the same types and also shape of the input and output arrays should be the same. So this operator just allows to rename array and its attributes and dimensions.
Syntax
  cast ( input-array, output-array )
Example
  `iquery -aq "cast(testarr1, M4x4)"`

## concat

Description
  Concatenate two arrays. Array should have the the name number of dimensions. Concatenation is performed by the left-most coordinate. All other dimensions should be the same. And left-most dimension of both arrays should have fixed size and same chunking schema. Also arrays should have the same attributes.
Syntax
  concat ( left-array, right-array )
Example
  `iquery -aq "concat(testarr1, testarr2)"`

## count

Description
  Count non-empty elements of the input array. Group-by dimensions are optional in the count operator. If one or more group-by dimensions are specified, the input array elements are first grouped by these dimensions and the count is performed on the resulting groups.
Syntax
  count ( input-array [ , group-by-dimension-1, group-by-dimension-2, ... ] )
Example
  `iquery -aq "count(testarr)"`
  `iquery -aq "count(Reads, OFFSET)"`

## create array

Description
  Create an array.
Syntax
  create array array-name < attribute-name : type, ... > [ dimension-name = from : till, chunk-interval, chunk-overlap , ... ]
Example
  `iquery -q "CREATE ARRAY test <a: int32, b: int32> [x=0:2,3,0, y=0:2,3,0]"`

## cross

Description
  Produces cross product of two arrays
Syntax
  cross ( left-array, right-array )
Example
  `iquery -aq "cross(matrix, vector)"`

## cross_join

Description
  Performs cross join on specified dimensions. Pairs of dimension names (one from the left array, another from the right array) are specified as parameter of JOIN operator. Number of dimensions of result array is equals to sum of dimensions of input arrays minus number of pairs of matched coordinates.

  In case the attribute name is not unique it can be qualified using array name (A.x) or operand number ($2.y). In the first case attribute "x" of input array "A" is taken. And second case refer to the attribute "y" of second input array.

Syntax
  cross_join ( input-array-1, input-array-2, dim-array-1, dim-array-2, ... )
Example
  `iquery -aq "cross_join(testarr1, testarr2, i, j)"`

## deldim

Description
  Delete left-most dimension from the array. Deleted dimension should have size 1.
Syntax
  deldim ( input-array )
Example
  `iquery -aq "deldim(subarray(matrix, 1, 100, 1, 200))"`

## dimensions

Description
  Get list of array dimensions. Returns array with the following attributes: name, start, length, chunk_interval, chunk-overlap.
Syntax
  dimensions ( array-reference )
Example
  `iquery -aq "dimensions(testarr)"`

## diskinfo

Description
  Get information about storage space. Returns array with the following attributes: used, available, clusterSize, nFreeClusters, nSegments.
Syntax
  diskinfo ()
Example
  `iquery -aq "diskinfo()"`

## echo

Description
  Print text to the output and to the log
Syntax
  echo ( any-text )
Example
  `iquery -aq "echo('Hello world')"`

## explain_logical

Description
  Output a textual representation of the logical plan for the given query.
Syntax
  explain_logical ( query-string [ , language-string ] )
Example
  `iquery -aq "explain_logical('select * from foo','aql')"`

## explain_physical

Description
  Output a textual representation of the physical plan for the given query.
Syntax
  explain_physical ( query-string [ , language-string ] )
Example
  `iquery -aq "explain_physical('select * from foo','aql')"`

## input

Description
  Incremental input of array from input text file. Unlike LOAD operator, input doesn't store result in the database
Syntax
  input ( array-name, input-file-path [ [ [ [ , instance-id [ , input-format [ , max-errors [ , shadow-array ] ] ] ] )
  instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
  input-format - "opaque", "text" or binary loader template, for example: "(int32,dummy(4),string)". Default "text"
  max-errors - maximal number of conversion error which can be ignored. Default 0
  shadow-array - name of shadow array containing information about conversion errors. It contains the same attributes as original array + position of error in imported file. Attributes of shadow array contains text of error message or null if correspondent attribute was successfully loaded
Example
  `iquery -aq "store(input(testarr, 'data.dmp'), testarr2)"`
  `iquery -aq "store(input(testarr, 'data.binary', -2, '(int32 null,float null,string null)', 10, foo_shadow), testarr2)"`

## filter

Description
  Filter array elements not matching specified condition. Mark as empty the elements of the array for which filter-expression is not evaluated to true. If input array already has empty-tag attribute, then it is used, otherwise it will be added.
Syntax
  filter ( input-array, filter-expression )
  input-array ? filter-expression
Example
  `iquery -aq "filter(testarr, A > B)"`

## redimension

Description
  Exchange specified dimensions and attributes of the source array. Mapping between attributes and dimensions is specified using definition of the destination array.
Syntax
  redimension ( source-array, destination-array )
Example
  `iquery -aq "join(A, redimension(B, A))"`

## redimension_store

Description
  Exchange specified dimensions and attributes of the source array and store result to the disk. Mapping between attributes and dimensions is specified using definition of the destination array.
Syntax
  redimension ( source-array, destination-array { ',' aggregation-expression } )
Example
  `iquery -aq "redimension_store(B, A)"`

## join

Description
  Perfroms natural join of two arrays with the same shape. Array should have the same dimensions and the same chunks size. Result is array with the same shape containing attributes from both arrays.
Syntax
  join ( input-array-1, input-array-2 )
Example
  `iquery -aq "join(testarr1, testarr2)"`

## list

Description
  Get list of operators, functions, aggregates, types, arrays names, current query IDs or loaded libraries. Returns array with single attribute name. For queries returns array with fields (query: string, id: int64). For libraries returns array with fields (name: string, major: int32, minor: int32, patch: int32, build: int32). When called without parameters returns list of arrays.
Syntax
  list ( `'arrays'` | `'types'` | `'operators'` | `'functions'` | `'queries'` | `'libraries'`)
Example
  `iquery -aq "list()"`
  `iquery -aq "list('arrays')"`
  `iquery -aq "list('operators')"`
  `iquery -aq "list('types')"`
  `iquery -aq "list('functions')"`
  `iquery -aq "list('aggregates')"`
  `iquery -aq "list('queries')"`
  `iquery -aq "list('libraries')"`

## load

Description
  Load an array from input text file. This is shorthand for store( input(array-name, input-file-name), array-name).
Syntax
  load ( array-name, input-file-path [ [ [ [ , instance-id [ , input-format [ , max-errors [ , shadow-array ] ] ] ] )
  instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
  input-format - "opaque", "text" or binary loader template, for example: "(int32,dummy(4),string)". Default "text"
  max-errors - maximal number of conversion error which can be ignored. Default 0
  shadow-array - name of shadow array containing information about conversion errors. It contains the same attributes as original array + position of error in imported file. Attributes of shadow array contains text of error message or null if correspondent attribute was successfully loaded
Example
  `iquery -aq "load(testarr, 'data.dmp')"`
  `iquery -aq "load(testarr, 'data.binary', -2, '(int32 null,float null,string null)', 10, foo_shadow)"`

## lookup

Description
  Map elements from the second array using attributes of the first array as coordinates in the second array. Result array has the same shape as first array and the same attributes as second array.
Syntax
  lookup ( pattern-array, source-array )
Example
  `iquery -aq "lookup(pattern, dense)"`

## match

Description
  Match pattern array with catalog array. For each cell in pattern array find closest non-empty cell in catalog array distance to which doesn't exceed specified error. Result array has the same shape as pattern array and combined list of attributes pattern and catalog attrbiutes + dimensiones of matched catalog element. Attributes and coordinates of matched catalog entry have "match_" prefix.
Syntax
  match ( pattern-array, catalog-array, error )
Example
  `iquery -aq "match(sky_snaphot, stars_catalog, 50)"`

## materialize

Description
  Materialize chunks of input array, converting them to the specified format (0 - preserve format of the input chunk, 1 - RLE format, 2 - dense format)
Syntax
  materialize ( input-array, format )
Example
  `iquery -aq "materilize(apply(A, x2, x*x), 0)"`

## max

Description
  Calculate maximum of the specified attribute in the array. Result is an array with single element containing maximum of specified attribute. If input array contains only one attribute, then attribute name can be omitted.
Syntax
  max ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
  `iquery -aq "max(testarr, A)"`
  `iquery -aq "max(matrix)"`
  `iquery -aq "max(Reads, Read, POS)"`

## merge

Description
  Merge two or more arrays. All arrays should have the same attributes and dimensions. Merge operator combines elements from the input array in such way: if element of first array is non empty, then it is taken, otherwise correspondent element of the second array is inspected and so on.
Syntax
  merge ( input-array-1, input-array-2, ... )
Example
  `iquery -aq "merge(main_results, supplementary_results)"`

## min

Description
  Calculate minimum of the specified attribute in the array. Result is an array with single element containing minimum of specified attribute. If input array contains only one attribute, then attribute name can be missed.
Syntax
  min ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
  `iquery -aq "min(testarr, A)"`
  `iquery -aq "min(matrix)"`
  `iquery -aq "min(Reads, Read, OFFSET)"`

## multiply

Description
  Multiply two matrices. Matrices should have two dimensions and single attribute.
Syntax
  multiply ( left-matrix, right-matrix )
Example
  `iquery -aq "multiply(matrix1, matrix2)"`

## norm

Description
  Normalize vector (divide each element on length of the vector)
Syntax
  norm ( input-array )
Example
  `iquery -aq "norm(testarr, A)"`

## project

Description
  Project specified attributes of the input array
Syntax
  project ( input-array, attribute-1, ..., attribute-N )
Example
  `iquery -aq "project(testarr, A)"`

## regrid

Description
  Split the input array into grids of specified dimensions and calculate one or more aggregates for each grid. Results array has the same number of dimensions as in the input array but the length of each dimension is divided by the size of the corresponding grid dimension. Result array contains one attribute for every specified aggregate.
Syntax
  regrid ( input-array, grid-interval-1, ... grid-interval-N, aggregate-1,... aggregate-N )
Example
  `iquery -aq "regrid(testarr, 2, 2, sum(A) as sum, avg(A), count(*) as numvalues)"`

## remove

Description
  Remove array from the storage. Currently only metadata of array is removed, but its chunks are still present in repository. So it can be considered as a resource leak.
Syntax
  remove ( array-reference )
Example
  `iquery -aq "remove(testarr)"`

## rename

Description
  Rename existing array in the database.
Syntax
  rename ( old-array-reference, new-array-reference )
Example
  `iquery -aq "rename(testarr, testarr_v1)"`

## repart

Description
  Change partitioning (chunking) of the array. Target array should have the same number of attributes and dimensions, but chunk size may be different. Target array should be previously created using "create array" operator. Repart takes definition of attributes from the source array and definition of dimensions from the target array.
Syntax
  repart ( input-array, target-array )
Example
  `iquery -aq "repart(testarr, testarr2)"`

## reshape

Description
  Change array shape. Target array should have the same number of attributes and fixed size dimensions (unbounded arrays are not allowed). Size of the input and target array should be the same. So it is possible to reshape 2x2x2 array to 4x2, but not 3x3 to 2x2. Target array should be previously created using "create array" operator. Reshape takes definition of attributes from the source array and definition of dimensions from the target array.
Syntax
  reshape ( input-array, target-array )
Example
  `iquery -aq "reshape(testarr, testarr2)"`

## reverse

Description
  Invert order of array elements.
Syntax
  reverse ( input-array )
Example
  `iquery -aq "reverse(testarr)"`

## sample

Description
  Sample input array by selecting random number of chunks from it
Syntax
  sample ( input-array , probability [ , seed ] )
Example
  `iquery -aq "max(sample(testarr, 0.01))"`

## save

Description
  Save array in specified file in text format. This operator is the opposite of the load() operator.
Syntax
  save ( array-reference , file-path [ [ , instance-id [ , store-format ] ] )
  instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
  store-format - "opaque", "store", "sparse", "dense", "csv", "csv+", "tsv", "tsv+" or binary loader template, for example: "(int32,dummy(4),string)". Default "store"
Example
  `iquery -aq "save(testarr, '/data/testarr.dat')"`
  `iquery -aq "save(testarr, '/data/testarr.dat', -2, '(int32,float,string)')"`

## scan

Description
  Read array. Array with the specified name should be previously created and loaded in the database using `load` operator.
Syntax
  scan ( array-reference [ , timestamp ] )
Example
  `iquery -aq "scan(testarr)"`
  `iquery -aq "scan(testarr, '10/28/2010')"`
  `iquery -aq "scan(testarr, now() - 5*24*3600)"`

## setopt

Description
  Set/get configuration option value at runtime. Option value should be specified as string. If new value is not specified, then values of this configuration option at all nodes are printed. If new value is specified, then value of option is updated at all odes and result array contains old and new values of the option at all nodes.
Syntax
  setopt ( option-name [ , new-option-value ] )
Example
  `iquery -aq "setopt('threads', '4')"`

## slice

Description
  Get slice of the array. Result is slice of the input array corresponding to the given coordinates. Number of dimensions of the result array is equal to the number of dimension of input array minus number of specified coordinates (which should be less than number of dimensions of the input array).
Syntax
  slice ( input-array, dimension-1, coordinate-value-1 , ... , dimension-N, coordinate-value-N )
Example
  `iquery -aq "slice(testarr, x, 2)"`

## sort

Description
  Sort one-dimensional array by one or more attributes. Attributes are specified either by name, either by position (1-based). To sort in descending order prepend attribute name with - or use negative value of attribute position.
Syntax
  sort ( input-array, { attribute-name | [-] attribute-position } )
Example
  `iquery -aq "sort(testarr, A)"`
  `iquery -aq "sort(testarr, -1)"`

## store

Description
  Store input array in the database under specified name.
Syntax
  store ( input-array, destination )
  destination-array = input-array
Example
  `iquery -aq "store(multiply(arr1, arr2), result)"`

## subarray

Description
  Get subarray of the input array specified by low and high boundaries.
Syntax
  subarray ( input-array, low-boundary-coordinate-1, ..., low-boundary-coordinate-N, high-boundary-coordinate-1, ..., high-boundary-coordinate-N )
Example
  `iquery -aq "subarray(testarr, 0, 1, 1, 2)"`

## substitute

Description
  Substitute null vakues in the input array, using thier missing reasons as index in the second (substituion) array.
Syntax
  substitute ( input-array , substitute-array )
Example
  `iquery -aq "substitute(nullableArray, substNulls)"`

## sum

Description
  Calculate sum of the specified attribute in the array. Result is an array with single element containing sum of specified attribute. If input array contains only one attribute, then attribute name can be omitted.
Syntax
  sum ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
  `iquery -aq "sum(testarr, A)"`
  `iquery -aq "sum(matrix)"`
  `iquery -aq "sum(Reads, Read, OFFSET)"`

## thin

Description
  Operator inverse to xgrid: leave only each k-th element of input array.
Syntax
  thin ( input-array, from-1, step-1, from-2, step-2, ..., from-N, step-N, )
Example
  `iquery -aq "thin(window(A, 1, 2, 1, 2)"`

## transpose

Description
  Transpose matrix. Input array should have two dimensions.
Syntax
  transpose ( input-array )
Example
  `iquery -aq "transpose(testarr)"`

## unpack

Description
  Unpack array to single dimensional array treating coordinates as attributes. Result array has single zero-based dimension and arguments combining attributes of the input array.
Syntax
  unpack ( input-array, dimension )
Example
  `iquery -aq "unpack(testarr, no)"`

## versions

Description
  Get list of updatable array versions. Returns array with the following attributes: version_id, time_stamp.
Syntax
  versions ( array-reference )
Example
  `iquery -aq "versions(updarr)"`

## window

Description
  Move specified window through the input array and calculate one or more aggregates. The shape of of result array is the same as shape of input array. Result array contains one attribute for each specified aggregate.
Syntax
  window ( input-array, window-size-1, ... widow-size-N, aggregate-1,... aggregate-N )
Example
  `iquery -aq "window(testarr, 3, 3, sum(A), stdev(A) as standard_deviation)"`

## xgrid

Description
  Scale input array by repeating cells of the original array specified number of times. This operator can be considered as inverse to regrid operator: regrid splits input array in "hypercubes" and calculates aggregate function for them and xgrid produces hypercube from the single element of the input array.
Syntax
  xgrid ( input-array, scale-1, ..., scale-N )
Example
  `iquery -aq "xgrid(matrix, 2, 2)"`

## mstat

Description
  Collect mallinfo outputs on every instance. Contains 10 attributes with parameters.
  - arena - non-mmapped space allocated from system
  - ordblks - number of free chunks
  - smblks - number of fastbin blocks
  - hblks - number of mmapped regions
  - hblkhd - space in mmapped regions
  - usmblks - maximum total allocated space
  - fsmblks - space available in freed fastbin blocks
  - uordblks - total allocated space
  - fordblks - total free space
  - keepcost - top-most, releasable (via malloc_trim) space
  Read `man mallinfo` for more details.
Syntax
  mstat ()
Example
  `iquery -aq "mstat()"`

## scatter/gather

Description
  scatter/gather operator redistributes chunks of the input array across cluster nodes according to a specified policy. sg is not a user-visible operator, but it is heavily used by the system internally. It is a physical operator for data movement within the system and users are not expected to invoke it directly. The scidb optimizer inserts an sg operator after load() to redistribute array data using round robin distribution. The optimizer may also insert sg operators to assist with aggregations, operators that don't preserve shape or operators that don't preserve distribution.
Syntax
  sg ( input-array, [ partitioning schema [, node number [, result array name [, store_flag [,offset_x, offset_y,... ] ] ] ] ])
  input-array - array to be redistributed. May also be the output of another operator like scan, filter, etc.
  partitioning schema - integer constant specifying partitioning schema. Possible values:
  0 - psReplication - replicate array on every node of cluster,
  1 - psRoundRobin - distribute chunks by round robin,
  2 - psLocalNode - send every local chunk to specified node,
  3 - psByRow - distribute every chunk to node according to chunk coordinate of dimension 0,
  4 - psByCol - distribute every chunk to node according to chunk coordinate of dimension 1
  node number - a node number to send all chunks. Used only with psLocalNode. Ignored for other partitioning schemas.
  result array name - a name of result array. If omitted, the result of sg is nameless and temporary with query duration. If present, and store_flag is either omitted or true, the result of sg is persisted as a new array with given name. If present, and store_flag is false, the result is temporary but may be aliased by this given name elsewhere in the query.
  store_flag - a boolean indicator used to specify whether sg should store the result as a new array. By default, true if result array name is given.
  offset - a distribution offset vector. For internal use.
Example
  Query to distribute array by round robin onto every node:
  `iquery -aq "sg(seq_info, 1, -1, seq_info_sg)"`
  Query to gather whole array on node 0 in memory array (will not be stored on disk):
  `iquery -aq "sg(seq_info_sg, 2, 0)"`

## load library

Description
Syntax
  AQL: load library library name
  AFL: load_library(library name)
  library name - a string constant with name of .so library that should be loaded
SciDB instance will search library in plugins folder of current directory of instance. Example of point library will be linked there authomatically if you build it from the sources.
Example
  Loading library from examples folder - libpoint.so:
  `iquery -aq "load library 'libpoint.so'"`
  The use of user defined objects from libpoint.so:
  `iquery -aq "apply(dense, b, point2str( str2point('(1,1)') + str2point('(2,2)') ))"`

## cancel

Description
Syntax
  AQL: Not available
  AFL: cancel(query id)
  query id - a int64 number with query id you want to cancel. Use list('queries') operator to know current query ids.
Example
  `iquery -aq "cancel(1299016933)"`