Class SparkParamservUtils
- java.lang.Object
- 
- org.apache.sysds.runtime.controlprogram.paramserv.SparkParamservUtils
 
- 
 public class SparkParamservUtils extends Object 
- 
- 
Constructor SummaryConstructors Constructor Description SparkParamservUtils()
 - 
Method SummaryAll Methods Static Methods Concrete Methods Modifier and Type Method Description static org.apache.spark.api.java.JavaPairRDD<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>assembleTrainingData(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> featuresRDD, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> labelsRDD)Assemble the matrix of features and labels according to the rowIDstatic org.apache.spark.api.java.JavaPairRDD<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>doPartitionOnSpark(SparkExecutionContext sec, MatrixObject features, MatrixObject labels, Statement.PSScheme scheme, int workerNum)
 
- 
- 
- 
Method Detail- 
assembleTrainingDatapublic static org.apache.spark.api.java.JavaPairRDD<Long,scala.Tuple2<MatrixBlock,MatrixBlock>> assembleTrainingData(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> featuresRDD, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> labelsRDD) Assemble the matrix of features and labels according to the rowID- Parameters:
- featuresRDD- indexed features matrix block
- labelsRDD- indexed labels matrix block
- Returns:
- Assembled rdd with rowID as key while matrix of features and labels as value (rowID -> features, labels)
 
 - 
doPartitionOnSparkpublic static org.apache.spark.api.java.JavaPairRDD<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>> doPartitionOnSpark(SparkExecutionContext sec, MatrixObject features, MatrixObject labels, Statement.PSScheme scheme, int workerNum) 
 
- 
 
-