Class MatrixIndexingSPInstruction
- java.lang.Object
- 
- org.apache.sysds.runtime.instructions.Instruction
- 
- org.apache.sysds.runtime.instructions.spark.SPInstruction
- 
- org.apache.sysds.runtime.instructions.spark.ComputationSPInstruction
- 
- org.apache.sysds.runtime.instructions.spark.UnarySPInstruction
- 
- org.apache.sysds.runtime.instructions.spark.IndexingSPInstruction
- 
- org.apache.sysds.runtime.instructions.spark.MatrixIndexingSPInstruction
 
 
 
 
 
 
- 
- All Implemented Interfaces:
- LineageTraceable
 
 public class MatrixIndexingSPInstruction extends IndexingSPInstruction This class implements the matrix indexing functionality inside CP.
- 
- 
Nested Class Summary- 
Nested classes/interfaces inherited from class org.apache.sysds.runtime.instructions.spark.SPInstructionSPInstruction.SPType
 - 
Nested classes/interfaces inherited from class org.apache.sysds.runtime.instructions.InstructionInstruction.IType
 
- 
 - 
Field Summary- 
Fields inherited from class org.apache.sysds.runtime.instructions.spark.ComputationSPInstructioninput1, input2, input3, output
 - 
Fields inherited from class org.apache.sysds.runtime.instructions.InstructionDATATYPE_PREFIX, FEDERATED_INST_PREFIX, GPU_INST_PREFIX, INSTRUCTION_DELIM, LITERAL_PREFIX, OPERAND_DELIM, SP_INST_PREFIX, VALUETYPE_PREFIX
 
- 
 - 
Method SummaryAll Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description org.apache.commons.lang3.tuple.Pair<String,LineageItem>getLineageItem(ExecutionContext ec)Obtain lineage trace of an instruction with a single output.static MatrixBlockinmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)static booleanisMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).static booleanisSingleBlockLookup(DataCharacteristics mcIn, IndexRange ixrange)Indicates if the given index range only covers a single blocks of the inputs matrix.voidprocessInstruction(ExecutionContext ec)This method should be used to execute the instruction.- 
Methods inherited from class org.apache.sysds.runtime.instructions.spark.IndexingSPInstructionparseInstruction
 - 
Methods inherited from class org.apache.sysds.runtime.instructions.spark.ComputationSPInstructiongetOutputVariableName
 - 
Methods inherited from class org.apache.sysds.runtime.instructions.spark.SPInstructiongetGraphString, getSPInstructionType, getType, postprocessInstruction, preprocessInstruction, requiresLabelUpdate
 - 
Methods inherited from class org.apache.sysds.runtime.instructions.InstructiongetBeginColumn, getBeginLine, getEndColumn, getEndLine, getExtendedOpcode, getFilename, getInstID, getInstructionString, getLineNum, getOpcode, getOperator, getPrivacyConstraint, printMe, setInstID, setLocation, setLocation, setLocation, setLocation, setPrivacyConstraint, setPrivacyConstraint, toString, updateInstructionThreadID
 - 
Methods inherited from class java.lang.Objectequals, getClass, hashCode, notify, notifyAll, wait, wait, wait
 - 
Methods inherited from interface org.apache.sysds.runtime.lineage.LineageTraceablegetLineageItems, hasSingleLineage
 
- 
 
- 
- 
- 
Method Detail- 
processInstructionpublic void processInstruction(ExecutionContext ec) Description copied from class:InstructionThis method should be used to execute the instruction.- Specified by:
- processInstructionin class- SPInstruction
- Parameters:
- ec- execution context
 
 - 
inmemoryIndexingpublic static MatrixBlock inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange) 
 - 
isSingleBlockLookuppublic static boolean isSingleBlockLookup(DataCharacteristics mcIn, IndexRange ixrange) Indicates if the given index range only covers a single blocks of the inputs matrix. In this case, we perform a key lookup which is very efficient in case of existing partitioner, especially for out-of-core datasets.- Parameters:
- mcIn- matrix characteristics
- ixrange- index range
- Returns:
- true if index range covers a single block of the input matrix
 
 - 
isMultiBlockLookuppublic static boolean isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).- Parameters:
- in- input matrix
- mcIn- input matrix characteristics
- mcOut- output matrix characteristics
- ixrange- index range
- Returns:
- true if index range requires a multi-block lookup
 
 - 
getLineageItempublic org.apache.commons.lang3.tuple.Pair<String,LineageItem> getLineageItem(ExecutionContext ec) Description copied from interface:LineageTraceableObtain lineage trace of an instruction with a single output.- Specified by:
- getLineageItemin interface- LineageTraceable
- Overrides:
- getLineageItemin class- ComputationSPInstruction
- Parameters:
- ec- execution context w/ live variables
- Returns:
- pair of (output variable name, output lineage item)
 
 
- 
 
-