Class MatrixBlockDictionary
- java.lang.Object
- 
- org.apache.sysds.runtime.compress.colgroup.dictionary.ADictionary
- 
- org.apache.sysds.runtime.compress.colgroup.dictionary.MatrixBlockDictionary
 
 
- 
- All Implemented Interfaces:
- Serializable
 
 public class MatrixBlockDictionary extends ADictionary - See Also:
- Serialized Form
 
- 
- 
Constructor SummaryConstructors Constructor Description MatrixBlockDictionary(MatrixBlock data, int nCol)
 - 
Method SummaryAll Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddToEntry(double[] v, int fr, int to, int nCol)Copies and adds the dictionary entry from this dictionary to the d dictionaryvoidaddToEntry(double[] v, int fr, int to, int nCol, int rep)copies and adds the dictonary entry from this dictionary yo the d dictionary rep times.voidaddToEntryVectorized(double[] v, int f1, int f2, int f3, int f4, int f5, int f6, int f7, int f8, int t1, int t2, int t3, int t4, int t5, int t6, int t7, int t8, int nCol)doubleaggregate(double init, Builtin fn)Aggregate all the contained values, useful in value only computations where the operation is iterating through all values contained in the dictionary.voidaggregateCols(double[] c, Builtin fn, int[] colIndexes)Aggregates the columns into the target double array provided.voidaggregateColsWithReference(double[] c, Builtin fn, int[] colIndexes, double[] reference, boolean def)Aggregates the columns into the target double array provided.double[]aggregateRows(Builtin fn, int nCol)Aggregate all entries in the rows.double[]aggregateRowsWithDefault(Builtin fn, double[] defaultTuple)Aggregate all entries in the rows of the dictionary with a extra cell in the end that contains the aggregate of the given defaultTuple.double[]aggregateRowsWithReference(Builtin fn, double[] reference)Aggregate all entries in the rows with an offset value reference added.doubleaggregateWithReference(double init, Builtin fn, double[] reference, boolean def)Aggregate all the contained values, with a reference offset.ADictionaryapplyScalarOp(ScalarOperator op)Allocate a new dictionary and applies the scalar operation on each cell of the to then return the new dictionary.ADictionaryapplyScalarOpWithReference(ScalarOperator op, double[] reference, double[] newReference)Allocate a new dictionary and apply the scalar operation on each cell to then return a new dictionary.ADictionaryapplyUnaryOp(UnaryOperator op)Allocate a new dictionary and apply the unary operator on each cell.ADictionaryapplyUnaryOpWithReference(UnaryOperator op, double[] reference, double[] newReference)Allocate a new dictionary and apply the scalar operation on each cell to then return a new dictionary.ADictionarybinOpLeft(BinaryOperator op, double[] v, int[] colIndexes)Apply binary row operation on the left side in placeDictionarybinOpLeftWithReference(BinaryOperator op, double[] v, int[] colIndexes, double[] reference, double[] newReference)Apply the binary operator such that each value is offset by the reference before application.MatrixBlockDictionarybinOpRight(BinaryOperator op, double[] v)Apply binary row operation on the right side.MatrixBlockDictionarybinOpRight(BinaryOperator op, double[] v, int[] colIndexes)Apply binary row operation on the right side.DictionarybinOpRightWithReference(BinaryOperator op, double[] v, int[] colIndexes, double[] reference, double[] newReference)Apply the binary operator such that each value is offset by the reference before application.CM_COV_ObjectcentralMoment(CM_COV_Object ret, ValueFunction fn, int[] counts, int nRows)Central moment function to calculate the central moment of this column group.CM_COV_ObjectcentralMomentWithReference(CM_COV_Object ret, ValueFunction fn, int[] counts, double reference, int nRows)Central moment function to calculate the central moment of this column group with a reference offset on each tuple.ADictionaryclone()Returns a deep clone of the dictionary.voidcolSum(double[] c, int[] counts, int[] colIndexes)Get the column sum of the values contained in the dictionaryvoidcolSumSq(double[] c, int[] counts, int[] colIndexes)Get the column sum of the values contained in the dictionaryvoidcolSumSqWithReference(double[] c, int[] counts, int[] colIndexes, double[] reference)Get the column sum of the values contained in the dictionary with an offset reference value added to each cell.booleancontainsValue(double pattern)Detect if the dictionary contains a specific value.booleancontainsValueWithReference(double pattern, double[] reference)Detect if the dictionary contains a specific value with reference offset.static MatrixBlockDictionarycreateDictionary(double[] values, int nCol)longgetExactSizeOnDisk()Calculate the space consumption if the dictionary is stored on disk.longgetInMemorySize()Returns the memory usage of the dictionary.static longgetInMemorySize(int numberValues, int numberColumns, double sparsity)MatrixBlockgetMatrixBlock()MatrixBlockDictionarygetMBDict(int nCol)Get this dictionary as a MatrixBlock dictionary.longgetNumberNonZeros(int[] counts, int nCol)Calculate the number of non zeros in the dictionary.longgetNumberNonZerosWithReference(int[] counts, double[] reference, int nRows)Calculate the number of non zeros in the dictionary.intgetNumberOfValues(int ncol)Get the number of distinct tuples given that the column group has n columnsdoublegetSparsity()Get the sparsity of the dictionary.StringgetString(int colIndexes)Get a string representation of the dictionary, that considers the layout of the data.doublegetValue(int i)Get Specific value contained in the dictionary at index.double[]getValues()Get all the values contained in the dictionary as a linearized double array.ADictionaryinplaceScalarOp(ScalarOperator op)Applies the scalar operation on the dictionary.booleanisLossy()Specify if the Dictionary is lossy.voidmultiplyScalar(double v, double[] ret, int off, int dictIdx, int[] cols)Multiply the v value with the dictionary entry at dictIdx and add it to the ret matrix at the columns specified in the int array.MatrixBlockDictionarypreaggValuesFromDense(int numVals, int[] colIndexes, int[] aggregateColumns, double[] b, int cut)Pre Aggregate values for Right Matrix Multiplication.voidproduct(double[] ret, int[] counts, int nCol)Calculate the product of the dictionary weighted by counts.voidproductWithDefault(double[] ret, int[] counts, double[] def, int defCount)Calculate the product of the dictionary weighted by counts with a default value added .voidproductWithReference(double[] ret, int[] counts, double[] reference, int refCount)Calculate the product of the dictionary weighted by counts and offset by referencestatic MatrixBlockDictionaryread(DataInput in)ADictionaryreplace(double pattern, double replace, int nCol)Make a copy of the values, and replace all values that match pattern with replacement value.ADictionaryreplaceWithReference(double pattern, double replace, double[] reference)Make a copy of the values, and replace all values that match pattern with replacement value.ADictionaryrexpandCols(int max, boolean ignore, boolean cast, int nCol)Rexpand the dictionary (one hot encode)ADictionaryrexpandColsWithReference(int max, boolean ignore, boolean cast, double reference)Rexpand the dictionary (one hot encode)ADictionaryscaleTuples(int[] scaling, int nCol)Scale all tuples contained in the dictionary by the scaling factor given in the int list.ADictionarysliceOutColumnRange(int idxStart, int idxEnd, int previousNumberOfColumns)Modify the dictionary by removing columns not within the index range.ADictionarysubtractTuple(double[] tuple)Allocate a new dictionary where the tuple given is subtracted from all tuples in the previous dictionary.doublesum(int[] counts, int ncol)Get the sum of the values contained in the dictionarydouble[]sumAllRowsToDouble(int nrColumns)Method used as a pre-aggregate of each tuple in the dictionary, to single double values.double[]sumAllRowsToDoubleSq(int nrColumns)Method used as a pre-aggregate of each tuple in the dictionary, to single double values.double[]sumAllRowsToDoubleSqWithDefault(double[] defaultTuple)Method used as a pre-aggregate of each tuple in the dictionary, to single double values.double[]sumAllRowsToDoubleSqWithReference(double[] reference)Method used as a pre-aggregate of each tuple in the dictionary, to single double values.double[]sumAllRowsToDoubleWithDefault(double[] defaultTuple)Do exactly the same as the sumAllRowsToDouble but also sum the array given to a extra index in the end of the array.double[]sumAllRowsToDoubleWithReference(double[] reference)Method used as a pre-aggregate of each tuple in the dictionary, to single double values with a reference.doublesumSq(int[] counts, int ncol)Get the square sum of the values contained in the dictionarydoublesumSqWithReference(int[] counts, double[] reference)Get the square sum of the values contained in the dictionary with a reference offset on each value.StringtoString()voidwrite(DataOutput out)Write the dictionary to a DataOutput.- 
Methods inherited from class org.apache.sysds.runtime.compress.colgroup.dictionary.ADictionarycentralMoment, centralMomentWithReference, colProduct
 
- 
 
- 
- 
- 
Constructor Detail- 
MatrixBlockDictionarypublic MatrixBlockDictionary(MatrixBlock data, int nCol) 
 
- 
 - 
Method Detail- 
createDictionarypublic static MatrixBlockDictionary createDictionary(double[] values, int nCol) 
 - 
getMatrixBlockpublic MatrixBlock getMatrixBlock() 
 - 
getValuespublic double[] getValues() Description copied from class:ADictionaryGet all the values contained in the dictionary as a linearized double array.- Specified by:
- getValuesin class- ADictionary
- Returns:
- linearized double array
 
 - 
getValuepublic double getValue(int i) Description copied from class:ADictionaryGet Specific value contained in the dictionary at index.- Specified by:
- getValuein class- ADictionary
- Parameters:
- i- The index to extract the value from
- Returns:
- The value contained at the index
 
 - 
getInMemorySizepublic long getInMemorySize() Description copied from class:ADictionaryReturns the memory usage of the dictionary.- Specified by:
- getInMemorySizein class- ADictionary
- Returns:
- a long value in number of bytes for the dictionary.
 
 - 
getInMemorySizepublic static long getInMemorySize(int numberValues, int numberColumns, double sparsity)
 - 
aggregatepublic double aggregate(double init, Builtin fn)Description copied from class:ADictionaryAggregate all the contained values, useful in value only computations where the operation is iterating through all values contained in the dictionary.- Specified by:
- aggregatein class- ADictionary
- Parameters:
- init- The initial Value, in cases such as Max value, this could be -infinity
- fn- The Function to apply to values
- Returns:
- The aggregated value as a double.
 
 - 
aggregateWithReferencepublic double aggregateWithReference(double init, Builtin fn, double[] reference, boolean def)Description copied from class:ADictionaryAggregate all the contained values, with a reference offset.- Specified by:
- aggregateWithReferencein class- ADictionary
- Parameters:
- init- The initial value, in cases such as Max value this could be -infinity.
- fn- The function to apply to the values
- reference- The reference offset to each value in the dictionary
- def- If the reference should be treated as an instance of only as reference
- Returns:
- The aggregated value as a double.
 
 - 
aggregateRowspublic double[] aggregateRows(Builtin fn, int nCol) Description copied from class:ADictionaryAggregate all entries in the rows.- Specified by:
- aggregateRowsin class- ADictionary
- Parameters:
- fn- The aggregate function
- nCol- The number of columns contained in the dictionary.
- Returns:
- Aggregates for this dictionary tuples.
 
 - 
aggregateRowsWithDefaultpublic double[] aggregateRowsWithDefault(Builtin fn, double[] defaultTuple) Description copied from class:ADictionaryAggregate all entries in the rows of the dictionary with a extra cell in the end that contains the aggregate of the given defaultTuple.- Specified by:
- aggregateRowsWithDefaultin class- ADictionary
- Parameters:
- fn- The aggregate function
- defaultTuple- The default tuple to aggregate in last cell
- Returns:
- Aggregates for this dictionary tuples.
 
 - 
aggregateRowsWithReferencepublic double[] aggregateRowsWithReference(Builtin fn, double[] reference) Description copied from class:ADictionaryAggregate all entries in the rows with an offset value reference added.- Specified by:
- aggregateRowsWithReferencein class- ADictionary
- Parameters:
- fn- The aggregate function
- reference- The reference offset to each value in the dictionary
- Returns:
- Aggregates for this dictionary tuples.
 
 - 
aggregateColspublic void aggregateCols(double[] c, Builtin fn, int[] colIndexes)Description copied from class:ADictionaryAggregates the columns into the target double array provided.- Specified by:
- aggregateColsin class- ADictionary
- Parameters:
- c- The target double array, this contains the full number of columns, therefore the colIndexes for this specific dictionary is needed.
- fn- The function to apply to individual columns
- colIndexes- The mapping to the target columns from the individual columns
 
 - 
aggregateColsWithReferencepublic void aggregateColsWithReference(double[] c, Builtin fn, int[] colIndexes, double[] reference, boolean def)Description copied from class:ADictionaryAggregates the columns into the target double array provided.- Specified by:
- aggregateColsWithReferencein class- ADictionary
- Parameters:
- c- The target double array, this contains the full number of columns, therefore the colIndexes for this specific dictionary is needed.
- fn- The function to apply to individual columns
- colIndexes- The mapping to the target columns from the individual columns
- reference- The reference offset values to add to each cell.
- def- If the reference should be treated as a tuple as well
 
 - 
applyScalarOppublic ADictionary applyScalarOp(ScalarOperator op) Description copied from class:ADictionaryAllocate a new dictionary and applies the scalar operation on each cell of the to then return the new dictionary.- Specified by:
- applyScalarOpin class- ADictionary
- Parameters:
- op- The operator.
- Returns:
- The new dictionary to return.
 
 - 
applyUnaryOppublic ADictionary applyUnaryOp(UnaryOperator op) Description copied from class:ADictionaryAllocate a new dictionary and apply the unary operator on each cell.- Specified by:
- applyUnaryOpin class- ADictionary
- Parameters:
- op- the operator.
- Returns:
- The new dictionary to return.
 
 - 
applyScalarOpWithReferencepublic ADictionary applyScalarOpWithReference(ScalarOperator op, double[] reference, double[] newReference) Description copied from class:ADictionaryAllocate a new dictionary and apply the scalar operation on each cell to then return a new dictionary. outValues[j] = op(this.values[j] + reference[i]) - newReference[i]- Specified by:
- applyScalarOpWithReferencein class- ADictionary
- Parameters:
- op- The operator to apply to each cell.
- reference- The reference value to add before the operator.
- newReference- The reference value to subtract after the operator.
- Returns:
- A New Dictionary.
 
 - 
applyUnaryOpWithReferencepublic ADictionary applyUnaryOpWithReference(UnaryOperator op, double[] reference, double[] newReference) Description copied from class:ADictionaryAllocate a new dictionary and apply the scalar operation on each cell to then return a new dictionary. outValues[j] = op(this.values[j] + reference[i]) - newReference[i]- Specified by:
- applyUnaryOpWithReferencein class- ADictionary
- Parameters:
- op- The unary operator to apply to each cell.
- reference- The reference value to add before the operator.
- newReference- The reference value to subtract after the operator.
- Returns:
- A New Dictionary.
 
 - 
inplaceScalarOppublic ADictionary inplaceScalarOp(ScalarOperator op) Description copied from class:ADictionaryApplies the scalar operation on the dictionary. Note that this operation modifies the underlying data, and normally require a copy of the original Dictionary to preserve old objects.- Specified by:
- inplaceScalarOpin class- ADictionary
- Parameters:
- op- The operator to apply to the dictionary values.
- Returns:
- this dictionary with modified values.
 
 - 
binOpLeftpublic ADictionary binOpLeft(BinaryOperator op, double[] v, int[] colIndexes) Description copied from class:ADictionaryApply binary row operation on the left side in place- Specified by:
- binOpLeftin class- ADictionary
- Parameters:
- op- The operation to this dictionary
- v- The values to use on the left hand side.
- colIndexes- The column indexes to consider inside v.
- Returns:
- A new dictionary containing the updated values.
 
 - 
binOpLeftWithReferencepublic Dictionary binOpLeftWithReference(BinaryOperator op, double[] v, int[] colIndexes, double[] reference, double[] newReference) Description copied from class:ADictionaryApply the binary operator such that each value is offset by the reference before application. Then put the result into the new dictionary, but offset it by the new reference. outValues[j] = op(v[colIndexes[i]], this.values[j] + reference[i]) - newReference[i]- Specified by:
- binOpLeftWithReferencein class- ADictionary
- Parameters:
- op- The operation to apply on the dictionary values.
- v- The values to use on the left side of the operator.
- colIndexes- The column indexes to use.
- reference- The reference value to add before operator.
- newReference- The reference value to subtract after operator.
- Returns:
- A new dictionary.
 
 - 
binOpRightpublic MatrixBlockDictionary binOpRight(BinaryOperator op, double[] v, int[] colIndexes) Description copied from class:ADictionaryApply binary row operation on the right side.- Specified by:
- binOpRightin class- ADictionary
- Parameters:
- op- The operation to this dictionary
- v- The values to use on the right hand side.
- colIndexes- The column indexes to consider inside v.
- Returns:
- A new dictionary containing the updated values.
 
 - 
binOpRightpublic MatrixBlockDictionary binOpRight(BinaryOperator op, double[] v) Description copied from class:ADictionaryApply binary row operation on the right side.- Specified by:
- binOpRightin class- ADictionary
- Parameters:
- op- The operation to this dictionary
- v- The values to apply on the dictionary (same number of cols as the dictionary)
- Returns:
- A new dictionary containing the updated values.
 
 - 
binOpRightWithReferencepublic Dictionary binOpRightWithReference(BinaryOperator op, double[] v, int[] colIndexes, double[] reference, double[] newReference) Description copied from class:ADictionaryApply the binary operator such that each value is offset by the reference before application. Then put the result into the new dictionary, but offset it by the new reference. outValues[j] = op(this.values[j] + reference[i], v[colIndexes[i]]) - newReference[i]- Specified by:
- binOpRightWithReferencein class- ADictionary
- Parameters:
- op- The operation to apply on the dictionary values.
- v- The values to use on the right side of the operator.
- colIndexes- The column indexes to use.
- reference- The reference value to add before operator.
- newReference- The reference value to subtract after operator.
- Returns:
- A new dictionary.
 
 - 
clonepublic ADictionary clone() Description copied from class:ADictionaryReturns a deep clone of the dictionary.- Specified by:
- clonein class- ADictionary
 
 - 
isLossypublic boolean isLossy() Description copied from class:ADictionarySpecify if the Dictionary is lossy.- Specified by:
- isLossyin class- ADictionary
- Returns:
- A boolean
 
 - 
getNumberOfValuespublic int getNumberOfValues(int ncol) Description copied from class:ADictionaryGet the number of distinct tuples given that the column group has n columns- Specified by:
- getNumberOfValuesin class- ADictionary
- Parameters:
- ncol- The number of Columns in the ColumnGroup.
- Returns:
- the number of value tuples contained in the dictionary.
 
 - 
sumAllRowsToDoublepublic double[] sumAllRowsToDouble(int nrColumns) Description copied from class:ADictionaryMethod used as a pre-aggregate of each tuple in the dictionary, to single double values. Note if the number of columns is one the actual dictionaries values are simply returned.- Specified by:
- sumAllRowsToDoublein class- ADictionary
- Parameters:
- nrColumns- The number of columns in the ColGroup to know how to get the values from the dictionary.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
sumAllRowsToDoubleWithDefaultpublic double[] sumAllRowsToDoubleWithDefault(double[] defaultTuple) Description copied from class:ADictionaryDo exactly the same as the sumAllRowsToDouble but also sum the array given to a extra index in the end of the array.- Specified by:
- sumAllRowsToDoubleWithDefaultin class- ADictionary
- Parameters:
- defaultTuple- The default row to sum in the end index returned.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
sumAllRowsToDoubleWithReferencepublic double[] sumAllRowsToDoubleWithReference(double[] reference) Description copied from class:ADictionaryMethod used as a pre-aggregate of each tuple in the dictionary, to single double values with a reference.- Specified by:
- sumAllRowsToDoubleWithReferencein class- ADictionary
- Parameters:
- reference- The reference values to add to each cell.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
sumAllRowsToDoubleSqpublic double[] sumAllRowsToDoubleSq(int nrColumns) Description copied from class:ADictionaryMethod used as a pre-aggregate of each tuple in the dictionary, to single double values. Note if the number of columns is one the actual dictionaries values are simply returned.- Specified by:
- sumAllRowsToDoubleSqin class- ADictionary
- Parameters:
- nrColumns- The number of columns in the ColGroup to know how to get the values from the dictionary.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
sumAllRowsToDoubleSqWithDefaultpublic double[] sumAllRowsToDoubleSqWithDefault(double[] defaultTuple) Description copied from class:ADictionaryMethod used as a pre-aggregate of each tuple in the dictionary, to single double values. But adds another cell to the return with an extra value that is the sum of the given defaultTuple.- Specified by:
- sumAllRowsToDoubleSqWithDefaultin class- ADictionary
- Parameters:
- defaultTuple- The default row to sum in the end index returned.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
sumAllRowsToDoubleSqWithReferencepublic double[] sumAllRowsToDoubleSqWithReference(double[] reference) Description copied from class:ADictionaryMethod used as a pre-aggregate of each tuple in the dictionary, to single double values.- Specified by:
- sumAllRowsToDoubleSqWithReferencein class- ADictionary
- Parameters:
- reference- The reference values to add to each cell.
- Returns:
- a double array containing the row sums from this dictionary.
 
 - 
colSumpublic void colSum(double[] c, int[] counts, int[] colIndexes)Description copied from class:ADictionaryGet the column sum of the values contained in the dictionary- Specified by:
- colSumin class- ADictionary
- Parameters:
- c- The output array allocated to contain all column groups output.
- counts- The counts of the individual tuples.
- colIndexes- The columns indexes of the parent column group, this indicate where to put the column sum into the c output.
 
 - 
colSumSqpublic void colSumSq(double[] c, int[] counts, int[] colIndexes)Description copied from class:ADictionaryGet the column sum of the values contained in the dictionary- Specified by:
- colSumSqin class- ADictionary
- Parameters:
- c- The output array allocated to contain all column groups output.
- counts- The counts of the individual tuples.
- colIndexes- The columns indexes of the parent column group, this indicate where to put the column sum into the c output.
 
 - 
colSumSqWithReferencepublic void colSumSqWithReference(double[] c, int[] counts, int[] colIndexes, double[] reference)Description copied from class:ADictionaryGet the column sum of the values contained in the dictionary with an offset reference value added to each cell.- Specified by:
- colSumSqWithReferencein class- ADictionary
- Parameters:
- c- The output array allocated to contain all column groups output.
- counts- The counts of the individual tuples.
- colIndexes- The columns indexes of the parent column group, this indicate where to put the column sum into the c output.
- reference- The reference values to add to each cell.
 
 - 
sumpublic double sum(int[] counts, int ncol)Description copied from class:ADictionaryGet the sum of the values contained in the dictionary- Specified by:
- sumin class- ADictionary
- Parameters:
- counts- The counts of the individual tuples
- ncol- The number of columns contained
- Returns:
- The sum scaled by the counts provided.
 
 - 
sumSqpublic double sumSq(int[] counts, int ncol)Description copied from class:ADictionaryGet the square sum of the values contained in the dictionary- Specified by:
- sumSqin class- ADictionary
- Parameters:
- counts- The counts of the individual tuples
- ncol- The number of columns contained
- Returns:
- The square sum scaled by the counts provided.
 
 - 
sumSqWithReferencepublic double sumSqWithReference(int[] counts, double[] reference)Description copied from class:ADictionaryGet the square sum of the values contained in the dictionary with a reference offset on each value.- Specified by:
- sumSqWithReferencein class- ADictionary
- Parameters:
- counts- The counts of the individual tuples
- reference- The reference value
- Returns:
- The square sum scaled by the counts and reference.
 
 - 
sliceOutColumnRangepublic ADictionary sliceOutColumnRange(int idxStart, int idxEnd, int previousNumberOfColumns) Description copied from class:ADictionaryModify the dictionary by removing columns not within the index range.- Specified by:
- sliceOutColumnRangein class- ADictionary
- Parameters:
- idxStart- The column index to start at.
- idxEnd- The column index to end at (not inclusive)
- previousNumberOfColumns- The number of columns contained in the dictionary.
- Returns:
- A dictionary containing the sliced out columns values only.
 
 - 
containsValuepublic boolean containsValue(double pattern) Description copied from class:ADictionaryDetect if the dictionary contains a specific value.- Specified by:
- containsValuein class- ADictionary
- Parameters:
- pattern- The value to search for
- Returns:
- true if the value is contained else false.
 
 - 
containsValueWithReferencepublic boolean containsValueWithReference(double pattern, double[] reference)Description copied from class:ADictionaryDetect if the dictionary contains a specific value with reference offset.- Specified by:
- containsValueWithReferencein class- ADictionary
- Parameters:
- pattern- The pattern/ value to search for
- reference- The reference double array.
- Returns:
- true if the value is contained else false.
 
 - 
getNumberNonZerospublic long getNumberNonZeros(int[] counts, int nCol)Description copied from class:ADictionaryCalculate the number of non zeros in the dictionary. The number of non zeros should be scaled with the counts given. This gives the exact number of non zero values in the parent column group.- Specified by:
- getNumberNonZerosin class- ADictionary
- Parameters:
- counts- The counts of each dictionary entry
- nCol- The number of columns in this dictionary
- Returns:
- The nonZero count
 
 - 
getNumberNonZerosWithReferencepublic long getNumberNonZerosWithReference(int[] counts, double[] reference, int nRows)Description copied from class:ADictionaryCalculate the number of non zeros in the dictionary. Each value in the dictionary should be added to the reference value. The number of non zeros should be scaled with the given counts.- Specified by:
- getNumberNonZerosWithReferencein class- ADictionary
- Parameters:
- counts- The Counts of each dict entry.
- reference- The reference vector.
- nRows- The number of rows in the input.
- Returns:
- The NonZero Count.
 
 - 
addToEntrypublic void addToEntry(double[] v, int fr, int to, int nCol)Description copied from class:ADictionaryCopies and adds the dictionary entry from this dictionary to the d dictionary- Specified by:
- addToEntryin class- ADictionary
- Parameters:
- v- the target dictionary (dense double array)
- fr- the from index
- to- the to index
- nCol- the number of columns
 
 - 
addToEntrypublic void addToEntry(double[] v, int fr, int to, int nCol, int rep)Description copied from class:ADictionarycopies and adds the dictonary entry from this dictionary yo the d dictionary rep times.- Specified by:
- addToEntryin class- ADictionary
- Parameters:
- v- the target dictionary (dense double array)
- fr- the from index
- to- the to index
- nCol- the number of columns
- rep- the number of repetitions to apply (simply multiply do not loop)
 
 - 
addToEntryVectorizedpublic void addToEntryVectorized(double[] v, int f1, int f2, int f3, int f4, int f5, int f6, int f7, int f8, int t1, int t2, int t3, int t4, int t5, int t6, int t7, int t8, int nCol)- Specified by:
- addToEntryVectorizedin class- ADictionary
 
 - 
subtractTuplepublic ADictionary subtractTuple(double[] tuple) Description copied from class:ADictionaryAllocate a new dictionary where the tuple given is subtracted from all tuples in the previous dictionary.- Specified by:
- subtractTuplein class- ADictionary
- Parameters:
- tuple- a double list representing a tuple, it is given that the tuple with is the same as this dictionaries.
- Returns:
- a new instance of dictionary with the tuple subtracted.
 
 - 
getMBDictpublic MatrixBlockDictionary getMBDict(int nCol) Description copied from class:ADictionaryGet this dictionary as a MatrixBlock dictionary. This allows us to use optimized kernels coded elsewhere in the system, such as matrix multiplication. Return null if the matrix is empty.- Specified by:
- getMBDictin class- ADictionary
- Parameters:
- nCol- The number of columns contained in this column group.
- Returns:
- A Dictionary containing a MatrixBlock.
 
 - 
getStringpublic String getString(int colIndexes) Description copied from class:ADictionaryGet a string representation of the dictionary, that considers the layout of the data.- Specified by:
- getStringin class- ADictionary
- Parameters:
- colIndexes- The number of columns in the dictionary.
- Returns:
- A string that is nicer to print.
 
 - 
scaleTuplespublic ADictionary scaleTuples(int[] scaling, int nCol) Description copied from class:ADictionaryScale all tuples contained in the dictionary by the scaling factor given in the int list.- Specified by:
- scaleTuplesin class- ADictionary
- Parameters:
- scaling- The amount to multiply the given tuples with
- nCol- The number of columns contained in this column group.
- Returns:
- A New dictionary (since we don't want to modify the underlying dictionary)
 
 - 
writepublic void write(DataOutput out) throws IOException Description copied from class:ADictionaryWrite the dictionary to a DataOutput.- Specified by:
- writein class- ADictionary
- Parameters:
- out- the output sink to write the dictionary to.
- Throws:
- IOException- if the sink fails.
 
 - 
readpublic static MatrixBlockDictionary read(DataInput in) throws IOException - Throws:
- IOException
 
 - 
getExactSizeOnDiskpublic long getExactSizeOnDisk() Description copied from class:ADictionaryCalculate the space consumption if the dictionary is stored on disk.- Specified by:
- getExactSizeOnDiskin class- ADictionary
- Returns:
- the long count of bytes to store the dictionary.
 
 - 
preaggValuesFromDensepublic MatrixBlockDictionary preaggValuesFromDense(int numVals, int[] colIndexes, int[] aggregateColumns, double[] b, int cut) Description copied from class:ADictionaryPre Aggregate values for Right Matrix Multiplication.- Specified by:
- preaggValuesFromDensein class- ADictionary
- Parameters:
- numVals- The number of values contained in this dictionary
- colIndexes- The column indexes that is associated with the parent column group
- aggregateColumns- The column to aggregate, this is preprocessed, to find remove consideration for empty columns
- b- The values in the right hand side matrix
- cut- The number of columns in b.
- Returns:
- A new dictionary with the pre aggregated values.
 
 - 
replacepublic ADictionary replace(double pattern, double replace, int nCol) Description copied from class:ADictionaryMake a copy of the values, and replace all values that match pattern with replacement value. If needed add a new column index.- Specified by:
- replacein class- ADictionary
- Parameters:
- pattern- The value to look for
- replace- The value to replace the other value with
- nCol- The number of columns contained in the dictionary.
- Returns:
- A new Column Group, reusing the index structure but with new values.
 
 - 
replaceWithReferencepublic ADictionary replaceWithReference(double pattern, double replace, double[] reference) Description copied from class:ADictionaryMake a copy of the values, and replace all values that match pattern with replacement value. If needed add a new column index. With reference such that each value in the dict is considered offset by the values contained in the reference.- Specified by:
- replaceWithReferencein class- ADictionary
- Parameters:
- pattern- The value to look for
- replace- The value to replace the other value with
- reference- The reference tuple to add to all entries when replacing
- Returns:
- A new Column Group, reusing the index structure but with new values.
 
 - 
productpublic void product(double[] ret, int[] counts, int nCol)Description copied from class:ADictionaryCalculate the product of the dictionary weighted by counts.- Specified by:
- productin class- ADictionary
- Parameters:
- ret- The result dense double array (containing one value)
- counts- The count of individual tuples
- nCol- Number of columns in the dictionary.
 
 - 
productWithDefaultpublic void productWithDefault(double[] ret, int[] counts, double[] def, int defCount)Description copied from class:ADictionaryCalculate the product of the dictionary weighted by counts with a default value added .- Specified by:
- productWithDefaultin class- ADictionary
- Parameters:
- ret- The result dense double array (containing one value)
- counts- The count of individual tuples
- def- The default tuple
- defCount- The count of the default tuple
 
 - 
productWithReferencepublic void productWithReference(double[] ret, int[] counts, double[] reference, int refCount)Description copied from class:ADictionaryCalculate the product of the dictionary weighted by counts and offset by reference- Specified by:
- productWithReferencein class- ADictionary
- Parameters:
- ret- The result dense double array (containing one value)
- counts- The counts of each entry in the dictionary
- reference- The reference value.
- refCount- The number of occurences of the ref value.
 
 - 
centralMomentpublic CM_COV_Object centralMoment(CM_COV_Object ret, ValueFunction fn, int[] counts, int nRows) Description copied from class:ADictionaryCentral moment function to calculate the central moment of this column group. MUST be on a single column dictionary.- Specified by:
- centralMomentin class- ADictionary
- Parameters:
- ret- The Central Moment object to be modified and returned
- fn- The value function to apply
- counts- The weight of individual tuples
- nRows- The number of rows in total of the column group
- Returns:
- The central moment Object
 
 - 
centralMomentWithReferencepublic CM_COV_Object centralMomentWithReference(CM_COV_Object ret, ValueFunction fn, int[] counts, double reference, int nRows) Description copied from class:ADictionaryCentral moment function to calculate the central moment of this column group with a reference offset on each tuple. MUST be on a single column dictionary.- Specified by:
- centralMomentWithReferencein class- ADictionary
- Parameters:
- ret- The Central Moment object to be modified and returned
- fn- The value function to apply
- counts- The weight of individual tuples
- reference- The reference values to offset the tuples with
- nRows- The number of rows in total of the column group
- Returns:
- The central moment Object
 
 - 
rexpandColspublic ADictionary rexpandCols(int max, boolean ignore, boolean cast, int nCol) Description copied from class:ADictionaryRexpand the dictionary (one hot encode)- Specified by:
- rexpandColsin class- ADictionary
- Parameters:
- max- the tuple width of the output
- ignore- If we should ignore zero and negative values
- cast- If we should cast all double values to whole integer values
- nCol- The number of columns in the dictionary already (should be 1)
- Returns:
- A new dictionary
 
 - 
rexpandColsWithReferencepublic ADictionary rexpandColsWithReference(int max, boolean ignore, boolean cast, double reference) Description copied from class:ADictionaryRexpand the dictionary (one hot encode)- Specified by:
- rexpandColsWithReferencein class- ADictionary
- Parameters:
- max- the tuple width of the output
- ignore- If we should ignore zero and negative values
- cast- If we should cast all double values to whole integer values
- reference- A reference value to add to all tuples before expanding
- Returns:
- A new dictionary
 
 - 
getSparsitypublic double getSparsity() Description copied from class:ADictionaryGet the sparsity of the dictionary.- Specified by:
- getSparsityin class- ADictionary
- Returns:
- a sparsity between 0 and 1
 
 - 
multiplyScalarpublic void multiplyScalar(double v, double[] ret, int off, int dictIdx, int[] cols)Description copied from class:ADictionaryMultiply the v value with the dictionary entry at dictIdx and add it to the ret matrix at the columns specified in the int array.- Specified by:
- multiplyScalarin class- ADictionary
- Parameters:
- v- Value to multiply
- ret- Output dense double array location
- off- Offset into the ret array that the "row" output starts at
- dictIdx- The dictionary entry to multiply.
- cols- The columns to multiply into of the output.
 
 
- 
 
-