Package org.apache.sysds.runtime.data
Class TensorBlock
- java.lang.Object
- 
- org.apache.sysds.runtime.data.TensorBlock
 
- 
- All Implemented Interfaces:
- Externalizable,- Serializable,- org.apache.hadoop.io.Writable,- CacheBlock
 
 public class TensorBlock extends Object implements CacheBlock, Externalizable ATensorBlockis the most top level representation of a tensor. There are two types of data representation which can be used: Basic/Homogeneous and Data/Heterogeneous Basic supports only oneValueType, while Data supports multipleValueTypes along the column axis. The format determines if theTensorBlockuses aBasicTensorBlockor aDataTensorBlockfor storing the data.- See Also:
- Serialized Form
 
- 
- 
Field SummaryFields Modifier and Type Field Description static int[]DEFAULT_DIMSstatic Types.ValueTypeDEFAULT_VTYPE
 - 
Constructor SummaryConstructors Constructor Description TensorBlock()Create aTensorBlockwith [0,0] dimension and homogeneous representation (aka.TensorBlock(double value)Create a [1,1] basic FP64TensorBlockcontaining the given value.TensorBlock(int[] dims, boolean basic)Create aTensorBlockwith the given dimensions and the given data representation (basic/data).TensorBlock(Types.ValueType[] schema, int[] dims)Create a dataTensorBlockwith the given schema and the given dimensions.TensorBlock(Types.ValueType vt, int[] dims)Create a basicTensorBlockwith the givenValueTypeand the given dimensions.TensorBlock(BasicTensorBlock basicTensor)Wrap the givenBasicTensorBlockinside aTensorBlock.TensorBlock(DataTensorBlock dataTensor)Wrap the givenDataTensorBlockinside aTensorBlock.TensorBlock(TensorBlock that)Copy constructor
 - 
Method SummaryAll Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description TensorBlockallocateBlock()If data is not yet allocated, allocate.TensorBlockbinaryOperations(BinaryOperator op, TensorBlock thatValue, TensorBlock result)voidcompactEmptyBlock()Free unnecessarily allocated empty block.TensorBlockcopy(int[] lower, int[] upper, TensorBlock src)Copy a part of anotherTensorBlockTensorBlockcopy(TensorBlock src)TensorBlockcopyExact(int[] lower, int[] upper, TensorBlock src)Copy a part of anotherTensorBlock.Objectget(int[] ix)doubleget(int r, int c)BasicTensorBlockgetBasicTensor()DataCharacteristicsgetDataCharacteristics()DataTensorBlockgetDataTensor()intgetDim(int i)int[]getDims()doublegetDouble(int r, int c)Returns the double value at the passed row and column.doublegetDoubleNaN(int r, int c)Returns the double value at the passed row and column.longgetExactBlockDataSerializedSize(BasicTensorBlock bt)Get the exact serialized size of aBasicTensorBlockif written byTensorBlock.writeBlockData(DataOutput,BasicTensorBlock).longgetExactSerializedSize()Get the exact serialized size in bytes of the cache block.longgetInMemorySize()Get the in-memory size in bytes of the cache block.longgetLength()long[]getLongDims()voidgetNextIndexes(int[] ix)Calculates the next index array.static voidgetNextIndexes(int[] dims, int[] ix)Calculates the next index array.longgetNonZeros()intgetNumColumns()intgetNumDims()intgetNumRows()Types.ValueType[]getSchema()Get the schema if thisTensorBlockis heterogeneous.StringgetString(int r, int c)Returns the string of the value at the passed row and column.Types.ValueTypegetValueType()Get theValueTypeif thisTensorBlockis homogeneous.booleanisAllocated()booleanisBasic()booleanisEmpty()booleanisEmpty(boolean safe)booleanisMatrix()booleanisShallowSerialize()Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.booleanisShallowSerialize(boolean inclConvert)Indicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.booleanisVector()voidmerge(CacheBlock that, boolean appendOnly)Merge the given block into the current block.voidreadExternal(ObjectInput in)voidreadFields(DataInput in)voidreset()Reset all cells to 0.voidreset(int[] dims)Reset data with new dimensions.static Types.ValueTyperesultValueType(Types.ValueType in1, Types.ValueType in2)voidset(int[] ix, Object v)Set a cell to the value given as an `Object`.voidset(int r, int c, double v)Set a cell in a 2-dimensional tensor.voidset(Object v)voidset(MatrixBlock other)TensorBlockslice(int[] offsets, TensorBlock outBlock)Slice the current block and write into the outBlock.CacheBlockslice(int rl, int ru, int cl, int cu, boolean deep, CacheBlock block)Slice a sub block out of the current block and write into the given output block.CacheBlockslice(int rl, int ru, int cl, int cu, CacheBlock block)Slice a sub block out of the current block and write into the given output block.voidtoShallowSerializeBlock()Converts a cache block that is not shallow serializable into a form that is shallow serializable.voidwrite(DataOutput out)voidwriteBlockData(DataOutput out, BasicTensorBlock bt)Write aBasicTensorBlock.voidwriteExternal(ObjectOutput out)
 
- 
- 
- 
Field Detail- 
DEFAULT_DIMSpublic static final int[] DEFAULT_DIMS 
 - 
DEFAULT_VTYPEpublic static final Types.ValueType DEFAULT_VTYPE 
 
- 
 - 
Constructor Detail- 
TensorBlockpublic TensorBlock() Create aTensorBlockwith [0,0] dimension and homogeneous representation (aka. basic).
 - 
TensorBlockpublic TensorBlock(int[] dims, boolean basic)Create aTensorBlockwith the given dimensions and the given data representation (basic/data).- Parameters:
- dims- dimensions
- basic- if true then basic- TensorBlockelse a data type of- TensorBlock.
 
 - 
TensorBlockpublic TensorBlock(Types.ValueType vt, int[] dims) Create a basicTensorBlockwith the givenValueTypeand the given dimensions.- Parameters:
- vt- value type
- dims- dimensions
 
 - 
TensorBlockpublic TensorBlock(Types.ValueType[] schema, int[] dims) Create a dataTensorBlockwith the given schema and the given dimensions.- Parameters:
- schema- schema of the columns
- dims- dimensions
 
 - 
TensorBlockpublic TensorBlock(double value) Create a [1,1] basic FP64TensorBlockcontaining the given value.- Parameters:
- value- value to put inside
 
 - 
TensorBlockpublic TensorBlock(BasicTensorBlock basicTensor) Wrap the givenBasicTensorBlockinside aTensorBlock.- Parameters:
- basicTensor- basic tensor block
 
 - 
TensorBlockpublic TensorBlock(DataTensorBlock dataTensor) Wrap the givenDataTensorBlockinside aTensorBlock.- Parameters:
- dataTensor- basic tensor block
 
 - 
TensorBlockpublic TensorBlock(TensorBlock that) Copy constructor- Parameters:
- that-- TensorBlockto copy
 
 
- 
 - 
Method Detail- 
resetpublic void reset() Reset all cells to 0.
 - 
resetpublic void reset(int[] dims) Reset data with new dimensions.- Parameters:
- dims- new dimensions
 
 - 
isBasicpublic boolean isBasic() 
 - 
isAllocatedpublic boolean isAllocated() 
 - 
allocateBlockpublic TensorBlock allocateBlock() If data is not yet allocated, allocate.- Returns:
- this TensorBlock
 
 - 
getBasicTensorpublic BasicTensorBlock getBasicTensor() 
 - 
getDataTensorpublic DataTensorBlock getDataTensor() 
 - 
getValueTypepublic Types.ValueType getValueType() Get theValueTypeif thisTensorBlockis homogeneous.- Returns:
- ValueTypeif homogeneous, null otherwise
 
 - 
getSchemapublic Types.ValueType[] getSchema() Get the schema if thisTensorBlockis heterogeneous.- Returns:
- value type if heterogeneous, null otherwise
 
 - 
getNumDimspublic int getNumDims() 
 - 
getNumRowspublic int getNumRows() - Specified by:
- getNumRowsin interface- CacheBlock
 
 - 
getNumColumnspublic int getNumColumns() - Specified by:
- getNumColumnsin interface- CacheBlock
 
 - 
getDataCharacteristicspublic DataCharacteristics getDataCharacteristics() - Specified by:
- getDataCharacteristicsin interface- CacheBlock
 
 - 
getInMemorySizepublic long getInMemorySize() Description copied from interface:CacheBlockGet the in-memory size in bytes of the cache block.- Specified by:
- getInMemorySizein interface- CacheBlock
- Returns:
- in-memory size in bytes of cache block
 
 - 
isShallowSerializepublic boolean isShallowSerialize() Description copied from interface:CacheBlockIndicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.- Specified by:
- isShallowSerializein interface- CacheBlock
- Returns:
- true if shallow serialized
 
 - 
isShallowSerializepublic boolean isShallowSerialize(boolean inclConvert) Description copied from interface:CacheBlockIndicates if the cache block is subject to shallow serialized, which is generally true if in-memory size and serialized size are almost identical allowing to avoid unnecessary deep serialize.- Specified by:
- isShallowSerializein interface- CacheBlock
- Parameters:
- inclConvert- if true report blocks as shallow serialize that are currently not amenable but can be brought into an amenable form via- toShallowSerializeBlock.
- Returns:
- true if shallow serialized
 
 - 
toShallowSerializeBlockpublic void toShallowSerializeBlock() Description copied from interface:CacheBlockConverts a cache block that is not shallow serializable into a form that is shallow serializable. This methods has no affect if the given cache block is not amenable.- Specified by:
- toShallowSerializeBlockin interface- CacheBlock
 
 - 
compactEmptyBlockpublic void compactEmptyBlock() Description copied from interface:CacheBlockFree unnecessarily allocated empty block.- Specified by:
- compactEmptyBlockin interface- CacheBlock
 
 - 
slicepublic CacheBlock slice(int rl, int ru, int cl, int cu, CacheBlock block) Description copied from interface:CacheBlockSlice a sub block out of the current block and write into the given output block. This method returns the passed instance if not null.- Specified by:
- slicein interface- CacheBlock
- Parameters:
- rl- row lower
- ru- row upper
- cl- column lower
- cu- column upper
- block- cache block
- Returns:
- sub-block of cache block
 
 - 
slicepublic CacheBlock slice(int rl, int ru, int cl, int cu, boolean deep, CacheBlock block) Description copied from interface:CacheBlockSlice a sub block out of the current block and write into the given output block. This method returns the passed instance if not null.- Specified by:
- slicein interface- CacheBlock
- Parameters:
- rl- row lower
- ru- row upper
- cl- column lower
- cu- column upper
- deep- enforce deep-copy
- block- cache block
- Returns:
- sub-block of cache block
 
 - 
mergepublic void merge(CacheBlock that, boolean appendOnly) Description copied from interface:CacheBlockMerge the given block into the current block. Both blocks needs to be of equal dimensions and contain disjoint non-zero cells.- Specified by:
- mergein interface- CacheBlock
- Parameters:
- that- cache block
- appendOnly- ?
 
 - 
getDoublepublic double getDouble(int r, int c)Description copied from interface:CacheBlockReturns the double value at the passed row and column. If the value is missing 0 is returned.- Specified by:
- getDoublein interface- CacheBlock
- Parameters:
- r- row of the value
- c- column of the value
- Returns:
- double value at the passed row and column
 
 - 
getDoubleNaNpublic double getDoubleNaN(int r, int c)Description copied from interface:CacheBlockReturns the double value at the passed row and column. If the value is missing NaN is returned.- Specified by:
- getDoubleNaNin interface- CacheBlock
- Parameters:
- r- row of the value
- c- column of the value
- Returns:
- double value at the passed row and column
 
 - 
getStringpublic String getString(int r, int c) Description copied from interface:CacheBlockReturns the string of the value at the passed row and column. If the value is missing or NaN, null is returned.- Specified by:
- getStringin interface- CacheBlock
- Parameters:
- r- row of the value
- c- column of the value
- Returns:
- string of the value at the passed row and column
 
 - 
getDimpublic int getDim(int i) 
 - 
getDimspublic int[] getDims() 
 - 
getLongDimspublic long[] getLongDims() 
 - 
getNextIndexespublic static void getNextIndexes(int[] dims, int[] ix)Calculates the next index array. Note that if the given index array was the last element, the next index will be the first one.- Parameters:
- dims- the dims array for which we have to decide the next index
- ix- the index array which will be incremented to the next index array
 
 - 
getNextIndexespublic void getNextIndexes(int[] ix) Calculates the next index array. Note that if the given index array was the last element, the next index will be the first one.- Parameters:
- ix- the index array which will be incremented to the next index array
 
 - 
isVectorpublic boolean isVector() 
 - 
isMatrixpublic boolean isMatrix() 
 - 
getLengthpublic long getLength() 
 - 
isEmptypublic boolean isEmpty() 
 - 
isEmptypublic boolean isEmpty(boolean safe) 
 - 
getNonZerospublic long getNonZeros() 
 - 
getpublic Object get(int[] ix) 
 - 
getpublic double get(int r, int c)
 - 
setpublic void set(Object v) 
 - 
setpublic void set(MatrixBlock other) 
 - 
setpublic void set(int[] ix, Object v)Set a cell to the value given as an `Object`.- Parameters:
- ix- indexes in each dimension, starting with 0
- v- value to set
 
 - 
setpublic void set(int r, int c, double v)Set a cell in a 2-dimensional tensor.- Parameters:
- r- row of the cell
- c- column of the cell
- v- value to set
 
 - 
slicepublic TensorBlock slice(int[] offsets, TensorBlock outBlock) Slice the current block and write into the outBlock. The offsets determines where the slice starts, the length of the blocks is given by the outBlock dimensions.- Parameters:
- offsets- offsets where the slice starts
- outBlock- sliced result block
- Returns:
- the sliced result block
 
 - 
copypublic TensorBlock copy(TensorBlock src) 
 - 
copypublic TensorBlock copy(int[] lower, int[] upper, TensorBlock src) Copy a part of anotherTensorBlock- Parameters:
- lower- lower index of elements to copy (inclusive)
- upper- upper index of elements to copy (exclusive)
- src- source- TensorBlock
- Returns:
- the shallow copy of the src TensorBlock
 
 - 
copyExactpublic TensorBlock copyExact(int[] lower, int[] upper, TensorBlock src) Copy a part of anotherTensorBlock. The difference tocopy()is that this allows for exact sub-blocks instead of taking all consecutive data elements from lower to upper.- Parameters:
- lower- lower index of elements to copy (inclusive)
- upper- upper index of elements to copy (exclusive)
- src- source- TensorBlock
- Returns:
- the deep copy of the src TensorBlock
 
 - 
getExactSerializedSizepublic long getExactSerializedSize() Description copied from interface:CacheBlockGet the exact serialized size in bytes of the cache block.- Specified by:
- getExactSerializedSizein interface- CacheBlock
- Returns:
- exact serialized size in bytes of cache block
 
 - 
getExactBlockDataSerializedSizepublic long getExactBlockDataSerializedSize(BasicTensorBlock bt) Get the exact serialized size of aBasicTensorBlockif written byTensorBlock.writeBlockData(DataOutput,BasicTensorBlock).- Parameters:
- bt-- BasicTensorBlock
- Returns:
- the size of the block data in serialized form
 
 - 
writepublic void write(DataOutput out) throws IOException - Specified by:
- writein interface- org.apache.hadoop.io.Writable
- Throws:
- IOException
 
 - 
writeBlockDatapublic void writeBlockData(DataOutput out, BasicTensorBlock bt) throws IOException Write aBasicTensorBlock.- Parameters:
- out- output stream
- bt- source- BasicTensorBlock
- Throws:
- IOException- if writing with the output stream fails
 
 - 
readFieldspublic void readFields(DataInput in) throws IOException - Specified by:
- readFieldsin interface- org.apache.hadoop.io.Writable
- Throws:
- IOException
 
 - 
writeExternalpublic void writeExternal(ObjectOutput out) throws IOException - Specified by:
- writeExternalin interface- Externalizable
- Throws:
- IOException
 
 - 
readExternalpublic void readExternal(ObjectInput in) throws IOException - Specified by:
- readExternalin interface- Externalizable
- Throws:
- IOException
 
 - 
binaryOperationspublic TensorBlock binaryOperations(BinaryOperator op, TensorBlock thatValue, TensorBlock result) 
 - 
resultValueTypepublic static Types.ValueType resultValueType(Types.ValueType in1, Types.ValueType in2) 
 
- 
 
-