Class LibMatrixReorg
- java.lang.Object
- 
- org.apache.sysds.runtime.matrix.data.LibMatrixReorg
 
- 
 public class LibMatrixReorg extends Object MB: Library for selected matrix reorg operations including special cases and all combinations of dense and sparse representations. Current list of supported operations: - reshape, - r' (transpose), - rdiag (diagV2M/diagM2V), - rsort (sorting data/indexes) - rmempty (remove empty) - rexpand (outer/table-seq expansion)
- 
- 
Field SummaryFields Modifier and Type Field Description static longPAR_NUMCELL_THRESHOLDstatic intPAR_NUMCELL_THRESHOLD_SORTstatic booleanSHALLOW_COPY_REORGstatic booleanSPARSE_OUTPUTS_IN_CSR
 - 
Method SummaryAll Methods Static Methods Concrete Methods Modifier and Type Method Description static voidcheckRexpand(MatrixBlock in, boolean ignore)Quick check if the input is valid for rexpand, this check does not guarantee that the input is valid for rexpandstatic int[]countNNZColumns(MatrixBlock in, int k, ExecutorService pool)static List<Future<int[]>>countNNZColumnsFuture(MatrixBlock in, int k, ExecutorService pool)static int[]countNnzPerColumn(MatrixBlock in)static int[]countNnzPerColumn(MatrixBlock in, int rl, int ru)static MatrixBlockdiag(MatrixBlock in, MatrixBlock out)static booleanisSupportedReorgOperator(ReorgOperator op)static int[]mergeNnzCounts(int[] cnt, int[] cnt2)static MatrixBlockreorg(MatrixBlock in, MatrixBlock out, ReorgOperator op)static MatrixBlockreorgInPlace(MatrixBlock in, ReorgOperator op)static List<IndexedMatrixValue>reshape(IndexedMatrixValue in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks)MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.static MatrixBlockreshape(MatrixBlock in, MatrixBlock out, int rows, int cols, boolean rowwise)CP reshape operation (single input, single output matrix) NOTE: In contrast to R, the rowwise parameter specifies both the read and write order, with row-wise being the default, while R uses always a column-wise read, rowwise specifying the write order and column-wise being the default.static voidrev(IndexedMatrixValue in, long rlen, int blen, ArrayList<IndexedMatrixValue> out)static MatrixBlockrev(MatrixBlock in, MatrixBlock out)static voidrexpand(IndexedMatrixValue data, double max, boolean rows, boolean cast, boolean ignore, long blen, ArrayList<IndexedMatrixValue> outList)MR/Spark rexpand operation (single input, multiple outputs incl empty blocks)static MatrixBlockrexpand(MatrixBlock in, MatrixBlock ret, double max, boolean rows, boolean cast, boolean ignore, int k)CP rexpand operation (single input, single output), the classic example of this operation is one hot encoding of a column to multiple columns.static MatrixBlockrexpand(MatrixBlock in, MatrixBlock ret, int max, boolean rows, boolean cast, boolean ignore, int k)CP rexpand operation (single input, single output), the classic example of this operation is one hot encoding of a column to multiple columns.static voidrmempty(IndexedMatrixValue data, IndexedMatrixValue offset, boolean rmRows, long len, long blen, ArrayList<IndexedMatrixValue> outList)MR rmempty interface - for rmempty we cannot view blocks independently, and hence, there are different CP and MR interfaces.static MatrixBlockrmempty(MatrixBlock in, MatrixBlock ret, boolean rows, boolean emptyReturn, MatrixBlock select)CP rmempty operation (single input, single output matrix)static MatrixBlocksort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret)static MatrixBlocksort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret, int k)static MatrixBlocktranspose(MatrixBlock in)static MatrixBlocktranspose(MatrixBlock in, int k)static MatrixBlocktranspose(MatrixBlock in, MatrixBlock out)static MatrixBlocktranspose(MatrixBlock in, MatrixBlock out, int k)static MatrixBlocktranspose(MatrixBlock in, MatrixBlock out, int k, boolean allowCSR)static MatrixBlocktransposeInPlace(MatrixBlock in, int k)
 
- 
- 
- 
Field Detail- 
PAR_NUMCELL_THRESHOLDpublic static long PAR_NUMCELL_THRESHOLD 
 - 
PAR_NUMCELL_THRESHOLD_SORTpublic static final int PAR_NUMCELL_THRESHOLD_SORT - See Also:
- Constant Field Values
 
 - 
SHALLOW_COPY_REORGpublic static final boolean SHALLOW_COPY_REORG - See Also:
- Constant Field Values
 
 - 
SPARSE_OUTPUTS_IN_CSRpublic static final boolean SPARSE_OUTPUTS_IN_CSR - See Also:
- Constant Field Values
 
 
- 
 - 
Method Detail- 
isSupportedReorgOperatorpublic static boolean isSupportedReorgOperator(ReorgOperator op) 
 - 
reorgpublic static MatrixBlock reorg(MatrixBlock in, MatrixBlock out, ReorgOperator op) 
 - 
reorgInPlacepublic static MatrixBlock reorgInPlace(MatrixBlock in, ReorgOperator op) 
 - 
transposepublic static MatrixBlock transpose(MatrixBlock in) 
 - 
transposepublic static MatrixBlock transpose(MatrixBlock in, MatrixBlock out) 
 - 
transposepublic static MatrixBlock transpose(MatrixBlock in, int k) 
 - 
transposepublic static MatrixBlock transpose(MatrixBlock in, MatrixBlock out, int k) 
 - 
transposepublic static MatrixBlock transpose(MatrixBlock in, MatrixBlock out, int k, boolean allowCSR) 
 - 
countNNZColumnspublic static int[] countNNZColumns(MatrixBlock in, int k, ExecutorService pool) 
 - 
countNNZColumnsFuturepublic static List<Future<int[]>> countNNZColumnsFuture(MatrixBlock in, int k, ExecutorService pool) throws InterruptedException - Throws:
- InterruptedException
 
 - 
transposeInPlacepublic static MatrixBlock transposeInPlace(MatrixBlock in, int k) 
 - 
revpublic static MatrixBlock rev(MatrixBlock in, MatrixBlock out) 
 - 
revpublic static void rev(IndexedMatrixValue in, long rlen, int blen, ArrayList<IndexedMatrixValue> out) 
 - 
diagpublic static MatrixBlock diag(MatrixBlock in, MatrixBlock out) 
 - 
sortpublic static MatrixBlock sort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret) 
 - 
sortpublic static MatrixBlock sort(MatrixBlock in, MatrixBlock out, int[] by, boolean desc, boolean ixret, int k) - Parameters:
- in- Input matrix to sort
- out- Output matrix where the sorted input is inserted to
- by- The Ordering parameter
- desc- A boolean, specifying if it should be descending order.
- ixret- A boolean, specifying if the return should be the sorted indexes.
- k- Number of parallel threads
- Returns:
- The sorted out matrix.
 
 - 
reshapepublic static MatrixBlock reshape(MatrixBlock in, MatrixBlock out, int rows, int cols, boolean rowwise) CP reshape operation (single input, single output matrix) NOTE: In contrast to R, the rowwise parameter specifies both the read and write order, with row-wise being the default, while R uses always a column-wise read, rowwise specifying the write order and column-wise being the default.- Parameters:
- in- input matrix
- out- output matrix
- rows- number of rows
- cols- number of columns
- rowwise- if true, reshape by row
- Returns:
- output matrix
 
 - 
reshapepublic static List<IndexedMatrixValue> reshape(IndexedMatrixValue in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks) MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.- Parameters:
- in- indexed matrix value
- mcIn- input matrix characteristics
- mcOut- output matrix characteristics
- rowwise- if true, reshape by row
- outputEmptyBlocks- output blocks with nnz=0
- Returns:
- list of indexed matrix values
 
 - 
rmemptypublic static MatrixBlock rmempty(MatrixBlock in, MatrixBlock ret, boolean rows, boolean emptyReturn, MatrixBlock select) CP rmempty operation (single input, single output matrix)- Parameters:
- in- input matrix
- ret- output matrix
- rows- ?
- emptyReturn- return row/column of zeros for empty input
- select- ?
- Returns:
- matrix block
 
 - 
rmemptypublic static void rmempty(IndexedMatrixValue data, IndexedMatrixValue offset, boolean rmRows, long len, long blen, ArrayList<IndexedMatrixValue> outList) MR rmempty interface - for rmempty we cannot view blocks independently, and hence, there are different CP and MR interfaces.- Parameters:
- data- ?
- offset- ?
- rmRows- ?
- len- ?
- blen- block length
- outList- list of indexed matrix values
 
 - 
rexpandpublic static MatrixBlock rexpand(MatrixBlock in, MatrixBlock ret, double max, boolean rows, boolean cast, boolean ignore, int k) CP rexpand operation (single input, single output), the classic example of this operation is one hot encoding of a column to multiple columns.- Parameters:
- in- Input matrix
- ret- Output matrix
- max- Number of rows/cols of the output
- rows- If the expansion is in rows direction
- cast- If the values contained should be cast to double (rounded up and down)
- ignore- Ignore if the input contain values below zero that technically is incorrect input.
- k- Degree of parallelism
- Returns:
- Output matrix rexpanded
 
 - 
rexpandpublic static MatrixBlock rexpand(MatrixBlock in, MatrixBlock ret, int max, boolean rows, boolean cast, boolean ignore, int k) CP rexpand operation (single input, single output), the classic example of this operation is one hot encoding of a column to multiple columns.- Parameters:
- in- Input matrix
- ret- Output matrix
- max- Number of rows/cols of the output
- rows- If the expansion is in rows direction
- cast- If the values contained should be cast to double (rounded up and down)
- ignore- Ignore if the input contain values below zero that technically is incorrect input.
- k- Degree of parallelism
- Returns:
- Output matrix rexpanded
 
 - 
checkRexpandpublic static void checkRexpand(MatrixBlock in, boolean ignore) Quick check if the input is valid for rexpand, this check does not guarantee that the input is valid for rexpand- Parameters:
- in- Input matrix block
- ignore- If zero valued cells should be ignored
 
 - 
rexpandpublic static void rexpand(IndexedMatrixValue data, double max, boolean rows, boolean cast, boolean ignore, long blen, ArrayList<IndexedMatrixValue> outList) MR/Spark rexpand operation (single input, multiple outputs incl empty blocks)- Parameters:
- data- Input indexed matrix block
- max- Total nrows/cols of the output
- rows- If the expansion is in rows direction
- cast- If the values contained should be cast to double (rounded up and down)
- ignore- Ignore if the input contain values below zero that technically is incorrect input.
- blen- The block size to slice the output up into
- outList- The output indexedMatrixValues (a list to add all the output blocks to / modify)
 
 - 
countNnzPerColumnpublic static int[] countNnzPerColumn(MatrixBlock in) 
 - 
countNnzPerColumnpublic static int[] countNnzPerColumn(MatrixBlock in, int rl, int ru) 
 - 
mergeNnzCountspublic static int[] mergeNnzCounts(int[] cnt, int[] cnt2)
 
- 
 
-