Class ColumnEncoderComposite
- java.lang.Object
- 
- org.apache.sysds.runtime.transform.encode.ColumnEncoder
- 
- org.apache.sysds.runtime.transform.encode.ColumnEncoderComposite
 
 
- 
- All Implemented Interfaces:
- Externalizable,- Serializable,- Comparable<ColumnEncoder>,- Encoder
 
 public class ColumnEncoderComposite extends ColumnEncoder Simple composite encoder that applies a list of encoders in specified order. By implementing the default encoder API it can be used as a drop-in replacement for any other encoder.- See Also:
- Serialized Form
 
- 
- 
Nested Class Summary- 
Nested classes/interfaces inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoderColumnEncoder.EncoderType
 
- 
 - 
Field Summary- 
Fields inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoderAPPLY_ROW_BLOCKS_PER_COLUMN, BUILD_ROW_BLOCKS_PER_COLUMN
 
- 
 - 
Constructor SummaryConstructors Constructor Description ColumnEncoderComposite()ColumnEncoderComposite(List<ColumnEncoder> columnEncoders)ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta)ColumnEncoderComposite(ColumnEncoder columnEncoder)
 - 
Method SummaryAll Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddEncoder(ColumnEncoder other)voidallocateMetaData(FrameBlock meta)Pre-allocate a FrameBlock for metadata collection.MatrixBlockapply(CacheBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)voidbuild(CacheBlock in)Build the transform meta data for the given block input.voidbuild(CacheBlock in, Map<Integer,double[]> equiHeightMaxs)voidbuildPartial(FrameBlock in)Partial build of internal data structures (e.g., in distributed spark operations).voidcomputeRCDMapSizeEstimate(CacheBlock in, int[] sampleIndices)booleanequals(Object o)List<DependencyTask<?>>getApplyTasks(CacheBlock in, MatrixBlock out, int outputCol)List<DependencyTask<?>>getBuildTasks(CacheBlock in)<T extends ColumnEncoder>
 TgetEncoder(Class<T> type)List<ColumnEncoder>getEncoders()FrameBlockgetMetaData(FrameBlock out)Construct a frame block out of the transform meta data.Set<Integer>getSparseRowsWZeros()<T extends ColumnEncoder>
 booleanhasBuild()<T extends ColumnEncoder>
 booleanhasEncoder(Class<T> type)inthashCode()voidinitMetaData(FrameBlock out)Sets up the required meta data for a subsequent call to apply.booleanisEncoder(int colID, Class<?> type)voidmergeAt(ColumnEncoder other)Merges another encoder, of a compatible type, in after a certain position.voidprepareBuildPartial()Allocates internal data structures for partial build.voidreadExternal(ObjectInput in)Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.voidsetNumPartitions(int nBuild, int nApply)voidshiftCol(int columnOffset)StringtoString()voidupdateAllDCEncoders()voidupdateIndexRanges(long[] beginDims, long[] endDims, int colOffset)Update index-ranges to after encoding.voidwriteExternal(ObjectOutput out)Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.- 
Methods inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoderapply, build, compareTo, getBuildTask, getColID, getColMapping, getEstMetaSize, getEstNumDistincts, getPartialBuildTask, getPartialMergeBuildTask, isApplicable, isApplicable, setColID, setEstMetaSize, setEstNumDistincts
 
- 
 
- 
- 
- 
Constructor Detail- 
ColumnEncoderCompositepublic ColumnEncoderComposite() 
 - 
ColumnEncoderCompositepublic ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta) 
 - 
ColumnEncoderCompositepublic ColumnEncoderComposite(List<ColumnEncoder> columnEncoders) 
 - 
ColumnEncoderCompositepublic ColumnEncoderComposite(ColumnEncoder columnEncoder) 
 
- 
 - 
Method Detail- 
getEncoderspublic List<ColumnEncoder> getEncoders() 
 - 
getEncoderpublic <T extends ColumnEncoder> T getEncoder(Class<T> type) 
 - 
isEncoderpublic boolean isEncoder(int colID, Class<?> type)
 - 
buildpublic void build(CacheBlock in) Description copied from interface:EncoderBuild the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.- Parameters:
- in- input frame block
 
 - 
buildpublic void build(CacheBlock in, Map<Integer,double[]> equiHeightMaxs) - Overrides:
- buildin class- ColumnEncoder
 
 - 
getApplyTaskspublic List<DependencyTask<?>> getApplyTasks(CacheBlock in, MatrixBlock out, int outputCol) - Overrides:
- getApplyTasksin class- ColumnEncoder
 
 - 
getBuildTaskspublic List<DependencyTask<?>> getBuildTasks(CacheBlock in) - Overrides:
- getBuildTasksin class- ColumnEncoder
 
 - 
prepareBuildPartialpublic void prepareBuildPartial() Description copied from class:ColumnEncoderAllocates internal data structures for partial build.- Specified by:
- prepareBuildPartialin interface- Encoder
- Overrides:
- prepareBuildPartialin class- ColumnEncoder
 
 - 
buildPartialpublic void buildPartial(FrameBlock in) Description copied from class:ColumnEncoderPartial build of internal data structures (e.g., in distributed spark operations).- Specified by:
- buildPartialin interface- Encoder
- Overrides:
- buildPartialin class- ColumnEncoder
- Parameters:
- in- input frame block
 
 - 
applypublic MatrixBlock apply(CacheBlock in, MatrixBlock out, int outputCol, int rowStart, int blk) - Overrides:
- applyin class- ColumnEncoder
 
 - 
mergeAtpublic void mergeAt(ColumnEncoder other) Description copied from class:ColumnEncoderMerges another encoder, of a compatible type, in after a certain position. Resizes as necessary.ColumnEncodersare compatible with themselves andEncoderCompositeis compatible with every otherColumnEncoders.MultiColumnEncodersare compatible with every encoder- Overrides:
- mergeAtin class- ColumnEncoder
- Parameters:
- other- the encoder that should be merged in
 
 - 
updateAllDCEncoderspublic void updateAllDCEncoders() 
 - 
addEncoderpublic void addEncoder(ColumnEncoder other) 
 - 
updateIndexRangespublic void updateIndexRanges(long[] beginDims, long[] endDims, int colOffset)Description copied from class:ColumnEncoderUpdate index-ranges to after encoding. Note that only Dummycoding changes the ranges.- Specified by:
- updateIndexRangesin interface- Encoder
- Overrides:
- updateIndexRangesin class- ColumnEncoder
- Parameters:
- beginDims- begin dimensions of range
- endDims- end dimensions of range
- colOffset- is applied to begin and endDims
 
 - 
allocateMetaDatapublic void allocateMetaData(FrameBlock meta) Description copied from interface:EncoderPre-allocate a FrameBlock for metadata collection.- Parameters:
- meta- frame block
 
 - 
getMetaDatapublic FrameBlock getMetaData(FrameBlock out) Description copied from interface:EncoderConstruct a frame block out of the transform meta data.- Parameters:
- out- output frame block
- Returns:
- output frame block?
 
 - 
initMetaDatapublic void initMetaData(FrameBlock out) Description copied from interface:EncoderSets up the required meta data for a subsequent call to apply.- Parameters:
- out- frame block
 
 - 
writeExternalpublic void writeExternal(ObjectOutput out) throws IOException Description copied from class:ColumnEncoderRedirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.- Specified by:
- writeExternalin interface- Externalizable
- Overrides:
- writeExternalin class- ColumnEncoder
- Parameters:
- out- object output
- Throws:
- IOException- if IOException occurs
 
 - 
readExternalpublic void readExternal(ObjectInput in) throws IOException Description copied from class:ColumnEncoderRedirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.- Specified by:
- readExternalin interface- Externalizable
- Overrides:
- readExternalin class- ColumnEncoder
- Parameters:
- in- object input
- Throws:
- IOException- if IOException occur
 
 - 
hasEncoderpublic <T extends ColumnEncoder> boolean hasEncoder(Class<T> type) 
 - 
hasBuildpublic <T extends ColumnEncoder> boolean hasBuild() 
 - 
computeRCDMapSizeEstimatepublic void computeRCDMapSizeEstimate(CacheBlock in, int[] sampleIndices) 
 - 
setNumPartitionspublic void setNumPartitions(int nBuild, int nApply)
 - 
shiftColpublic void shiftCol(int columnOffset) - Overrides:
- shiftColin class- ColumnEncoder
 
 - 
getSparseRowsWZerospublic Set<Integer> getSparseRowsWZeros() - Overrides:
- getSparseRowsWZerosin class- ColumnEncoder
 
 
- 
 
-