Package org.apache.sysds.api.mlcontext
Class MLContextConversionUtil
- java.lang.Object
- 
- org.apache.sysds.api.mlcontext.MLContextConversionUtil
 
- 
 public class MLContextConversionUtil extends Object Utility class containing methods to perform data conversions.
- 
- 
Constructor SummaryConstructors Constructor Description MLContextConversionUtil()
 - 
Method SummaryAll Methods Static Methods Concrete Methods Modifier and Type Method Description static FrameObjectbinaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks)Convert aJavaPairRDD<Long, FrameBlock>to aFrameObject.static FrameObjectbinaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata)Convert aJavaPairRDD<Long, FrameBlock>to aFrameObject.static MatrixBlockbinaryBlocksToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks, MatrixMetadata matrixMetadata)Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixBlockstatic MatrixObjectbinaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks)Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixObject.static MatrixObjectbinaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks, MatrixMetadata matrixMetadata)Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixObject.static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>dataFrameToFrameBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)Convert aDataFrameto aJavaPairRDD<Long, FrameBlock>binary-block frame.static FrameObjectdataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)Convert aDataFrameto aFrameObject.static FrameObjectdataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)Convert aDataFrameto aFrameObject.static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)Convert aDataFrameto aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix.static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata)Convert aDataFrameto aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix.static MatrixObjectdataFrameToMatrixObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)Convert aDataFrameto aMatrixObject.static MatrixObjectdataFrameToMatrixObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata)Convert aDataFrameto aMatrixObject.static voiddetermineFrameFormatIfNeeded(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)If the FrameFormat of the DataFrame has not been explicitly specified, attempt to determine the proper FrameFormat.static voiddetermineMatrixFormatIfNeeded(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata)If the MatrixFormat of the DataFrame has not been explicitly specified, attempt to determine the proper MatrixFormat.static MatrixObjectdoubleMatrixToMatrixObject(String variableName, double[][] doubleMatrix)Convert a two-dimensional double array to aMatrixObject.static MatrixObjectdoubleMatrixToMatrixObject(String variableName, double[][] doubleMatrix, MatrixMetadata matrixMetadata)Convert a two-dimensional double array to aMatrixObject.static FrameObjectframeBlockToFrameObject(String variableName, FrameBlock frameBlock, FrameMetadata frameMetadata)Convert aFrameBlockto aFrameObject.static String[][]frameObjectTo2DStringArray(FrameObject frameObject)Convert aFrameObjectto a two-dimensional string array.static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>frameObjectToBinaryBlocks(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)Convert aFrameObjectto aJavaPairRDD<Long, FrameBlock>.static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>frameObjectToDataFrame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext)Convert aFrameObjectto aDataFrame.static org.apache.spark.api.java.JavaRDD<String>frameObjectToJavaRDDStringCSV(FrameObject frameObject, String delimiter)Convert aFrameObjectto aJavaRDD<String>in CSV format.static org.apache.spark.api.java.JavaRDD<String>frameObjectToJavaRDDStringIJV(FrameObject frameObject)Convert aFrameObjectto aJavaRDD<String>in IJV format.static List<String>frameObjectToListStringCSV(FrameObject frameObject, String delimiter)Convert aFrameObjectto aList<String>in CSV format.static List<String>frameObjectToListStringIJV(FrameObject frameObject)Convert aFrameObjectto aList<String>in IJV format.static org.apache.spark.rdd.RDD<String>frameObjectToRDDStringCSV(FrameObject frameObject, String delimiter)Convert aFrameObjectto aRDD<String>in CSV format.static org.apache.spark.rdd.RDD<String>frameObjectToRDDStringIJV(FrameObject frameObject)Convert aFrameObjectto aRDD<String>in IJV format.static booleanisDataFrameWithIDColumn(FrameMetadata frameMetadata)Return whether or not the DataFrame has an ID column.static booleanisDataFrameWithIDColumn(MatrixMetadata matrixMetadata)Return whether or not the DataFrame has an ID column.static booleanisVectorBasedDataFrame(MatrixMetadata matrixMetadata)Return whether or not the DataFrame is vector-based.static FrameObjectjavaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD)Convert aJavaRDD<String>in CSV format to aFrameObjectstatic FrameObjectjavaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, FrameMetadata frameMetadata)Convert aJavaRDD<String>in CSV format to aFrameObjectstatic MatrixObjectjavaRDDStringCSVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD)Convert aJavaRDD<String>in CSV format to aMatrixObjectstatic MatrixObjectjavaRDDStringCSVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, MatrixMetadata matrixMetadata)Convert aJavaRDD<String>in CSV format to aMatrixObjectstatic FrameObjectjavaRDDStringIJVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, FrameMetadata frameMetadata)Convert aJavaRDD<String>in IJV format to aFrameObject.static MatrixObjectjavaRDDStringIJVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, MatrixMetadata matrixMetadata)Convert aJavaRDD<String>in IJV format to aMatrixObject.static org.apache.spark.api.java.JavaSparkContextjsc()Obtain JavaSparkContext from MLContextProxy.static MatrixObjectmatrixBlockToMatrixObject(String variableName, MatrixBlock matrixBlock, MatrixMetadata matrixMetadata)Convert aMatrixBlockto aMatrixObject.static double[][]matrixObjectTo2DDoubleArray(MatrixObject matrixObject)Convert aMatrixObjectto a two-dimensional double array.static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>matrixObjectToBinaryBlocks(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext)Convert aMatrixObjectto aJavaPairRDD<MatrixIndexes, MatrixBlock>.static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>matrixObjectToDataFrame(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext, boolean isVectorDF)Convert aMatrixObjectto aDataFrame.static org.apache.spark.api.java.JavaRDD<String>matrixObjectToJavaRDDStringCSV(MatrixObject matrixObject)Convert aMatrixObjectto aJavaRDD<String>in CSV format.static org.apache.spark.api.java.JavaRDD<String>matrixObjectToJavaRDDStringIJV(MatrixObject matrixObject)Convert aMatrixObjectto aJavaRDD<String>in IJV format.static List<String>matrixObjectToListStringCSV(MatrixObject matrixObject)Convert aMatrixObjectto aList<String>in CSV format.static List<String>matrixObjectToListStringIJV(MatrixObject matrixObject)Convert aMatrixObjectto aList<String>in IJV format.static org.apache.spark.rdd.RDD<String>matrixObjectToRDDStringCSV(MatrixObject matrixObject)Convert aMatrixObjectto aRDD<String>in CSV format.static org.apache.spark.rdd.RDD<String>matrixObjectToRDDStringIJV(MatrixObject matrixObject)Convert aMatrixObjectto aRDD<String>in IJV format.static FrameObjectrddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd)Convert aRDD<String>in CSV format to aFrameObjectstatic FrameObjectrddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd, FrameMetadata frameMetadata)Convert aRDD<String>in CSV format to aFrameObjectstatic MatrixObjectrddStringCSVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd)Convert aRDD<String>in CSV format to aMatrixObjectstatic MatrixObjectrddStringCSVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd, MatrixMetadata matrixMetadata)Convert aRDD<String>in CSV format to aMatrixObjectstatic FrameObjectrddStringIJVToFrameObject(org.apache.spark.rdd.RDD<String> rdd, FrameMetadata frameMetadata)Convert aRDD<String>in IJV format to aFrameObject.static MatrixObjectrddStringIJVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd, MatrixMetadata matrixMetadata)Convert aRDD<String>in IJV format to aMatrixObject.static org.apache.spark.SparkContextsc()Obtain SparkContext from MLContextProxy.static org.apache.spark.sql.SparkSessionspark()Obtain SparkSession from MLContextProxy.static MatrixObjecturlToMatrixObject(URL url, MatrixMetadata matrixMetadata)Convert a matrix at a URL to aMatrixObject.
 
- 
- 
- 
Method Detail- 
doubleMatrixToMatrixObjectpublic static MatrixObject doubleMatrixToMatrixObject(String variableName, double[][] doubleMatrix) Convert a two-dimensional double array to aMatrixObject.- Parameters:
- variableName- name of the variable associated with the matrix
- doubleMatrix- matrix of double values
- Returns:
- the two-dimensional double matrix converted to a
         MatrixObject
 
 - 
doubleMatrixToMatrixObjectpublic static MatrixObject doubleMatrixToMatrixObject(String variableName, double[][] doubleMatrix, MatrixMetadata matrixMetadata) Convert a two-dimensional double array to aMatrixObject.- Parameters:
- variableName- name of the variable associated with the matrix
- doubleMatrix- matrix of double values
- matrixMetadata- the matrix metadata
- Returns:
- the two-dimensional double matrix converted to a
         MatrixObject
 
 - 
urlToMatrixObjectpublic static MatrixObject urlToMatrixObject(URL url, MatrixMetadata matrixMetadata) Convert a matrix at a URL to aMatrixObject.- Parameters:
- url- the URL to a matrix (in CSV or IJV format)
- matrixMetadata- the matrix metadata
- Returns:
- the matrix at a URL converted to a MatrixObject
 
 - 
matrixBlockToMatrixObjectpublic static MatrixObject matrixBlockToMatrixObject(String variableName, MatrixBlock matrixBlock, MatrixMetadata matrixMetadata) Convert aMatrixBlockto aMatrixObject.- Parameters:
- variableName- name of the variable associated with the matrix
- matrixBlock- matrix as a MatrixBlock
- matrixMetadata- the matrix metadata
- Returns:
- the MatrixBlockconverted to aMatrixObject
 
 - 
frameBlockToFrameObjectpublic static FrameObject frameBlockToFrameObject(String variableName, FrameBlock frameBlock, FrameMetadata frameMetadata) Convert aFrameBlockto aFrameObject.- Parameters:
- variableName- name of the variable associated with the frame
- frameBlock- frame as a FrameBlock
- frameMetadata- the frame metadata
- Returns:
- the FrameBlockconverted to aFrameObject
 
 - 
binaryBlocksToMatrixObjectpublic static MatrixObject binaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks) Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixObject.- Parameters:
- binaryBlocks-- JavaPairRDD<MatrixIndexes, MatrixBlock>representation of a binary-block matrix
- Returns:
- the JavaPairRDD<MatrixIndexes, MatrixBlock>matrix converted to aMatrixObject
 
 - 
binaryBlocksToMatrixObjectpublic static MatrixObject binaryBlocksToMatrixObject(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks, MatrixMetadata matrixMetadata) Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixObject.- Parameters:
- binaryBlocks-- JavaPairRDD<MatrixIndexes, MatrixBlock>representation of a binary-block matrix
- matrixMetadata- the matrix metadata
- Returns:
- the JavaPairRDD<MatrixIndexes, MatrixBlock>matrix converted to aMatrixObject
 
 - 
binaryBlocksToMatrixBlockpublic static MatrixBlock binaryBlocksToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> binaryBlocks, MatrixMetadata matrixMetadata) Convert aJavaPairRDD<MatrixIndexes, MatrixBlock>to aMatrixBlock- Parameters:
- binaryBlocks-- JavaPairRDD<MatrixIndexes, MatrixBlock>representation of a binary-block matrix
- matrixMetadata- the matrix metadata
- Returns:
- the JavaPairRDD<MatrixIndexes, MatrixBlock>matrix converted to aMatrixBlock
 
 - 
binaryBlocksToFrameObjectpublic static FrameObject binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks) Convert aJavaPairRDD<Long, FrameBlock>to aFrameObject.- Parameters:
- binaryBlocks-- JavaPairRDD<Long, FrameBlock>representation of a binary-block frame
- Returns:
- the JavaPairRDD<Long, FrameBlock>frame converted to aFrameObject
 
 - 
binaryBlocksToFrameObjectpublic static FrameObject binaryBlocksToFrameObject(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> binaryBlocks, FrameMetadata frameMetadata) Convert aJavaPairRDD<Long, FrameBlock>to aFrameObject.- Parameters:
- binaryBlocks-- JavaPairRDD<Long, FrameBlock>representation of a binary-block frame
- frameMetadata- the frame metadata
- Returns:
- the JavaPairRDD<Long, FrameBlock>frame converted to aFrameObject
 
 - 
dataFrameToMatrixObjectpublic static MatrixObject dataFrameToMatrixObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame) Convert aDataFrameto aMatrixObject.- Parameters:
- dataFrame- the Spark- DataFrame
- Returns:
- the DataFramematrix converted to a converted to aMatrixObject
 
 - 
dataFrameToMatrixObjectpublic static MatrixObject dataFrameToMatrixObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata) Convert aDataFrameto aMatrixObject.- Parameters:
- dataFrame- the Spark- DataFrame
- matrixMetadata- the matrix metadata
- Returns:
- the DataFramematrix converted to a converted to aMatrixObject
 
 - 
dataFrameToFrameObjectpublic static FrameObject dataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame) Convert aDataFrameto aFrameObject.- Parameters:
- dataFrame- the Spark- DataFrame
- Returns:
- the DataFramematrix converted to a converted to aFrameObject
 
 - 
dataFrameToFrameObjectpublic static FrameObject dataFrameToFrameObject(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata) Convert aDataFrameto aFrameObject.- Parameters:
- dataFrame- the Spark- DataFrame
- frameMetadata- the frame metadata
- Returns:
- the DataFrameframe converted to a converted to aFrameObject
 
 - 
dataFrameToMatrixBinaryBlockspublic static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame) Convert aDataFrameto aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix.- Parameters:
- dataFrame- the Spark- DataFrame
- Returns:
- the DataFramematrix converted to aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix
 
 - 
dataFrameToMatrixBinaryBlockspublic static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> dataFrameToMatrixBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata) Convert aDataFrameto aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix.- Parameters:
- dataFrame- the Spark- DataFrame
- matrixMetadata- the matrix metadata
- Returns:
- the DataFramematrix converted to aJavaPairRDD<MatrixIndexes, MatrixBlock>binary-block matrix
 
 - 
dataFrameToFrameBinaryBlockspublic static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> dataFrameToFrameBinaryBlocks(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata) Convert aDataFrameto aJavaPairRDD<Long, FrameBlock>binary-block frame.- Parameters:
- dataFrame- the Spark- DataFrame
- frameMetadata- the frame metadata
- Returns:
- the DataFramematrix converted to aJavaPairRDD<Long, FrameBlock>binary-block frame
 
 - 
determineMatrixFormatIfNeededpublic static void determineMatrixFormatIfNeeded(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, MatrixMetadata matrixMetadata)If the MatrixFormat of the DataFrame has not been explicitly specified, attempt to determine the proper MatrixFormat.- Parameters:
- dataFrame- the Spark- DataFrame
- matrixMetadata- the matrix metadata, if available
 
 - 
determineFrameFormatIfNeededpublic static void determineFrameFormatIfNeeded(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, FrameMetadata frameMetadata)If the FrameFormat of the DataFrame has not been explicitly specified, attempt to determine the proper FrameFormat.- Parameters:
- dataFrame- the Spark- DataFrame
- frameMetadata- the frame metadata, if available
 
 - 
isDataFrameWithIDColumnpublic static boolean isDataFrameWithIDColumn(MatrixMetadata matrixMetadata) Return whether or not the DataFrame has an ID column.- Parameters:
- matrixMetadata- the matrix metadata
- Returns:
- trueif the DataFrame has an ID column,- falseotherwise.
 
 - 
isDataFrameWithIDColumnpublic static boolean isDataFrameWithIDColumn(FrameMetadata frameMetadata) Return whether or not the DataFrame has an ID column.- Parameters:
- frameMetadata- the frame metadata
- Returns:
- trueif the DataFrame has an ID column,- falseotherwise.
 
 - 
isVectorBasedDataFramepublic static boolean isVectorBasedDataFrame(MatrixMetadata matrixMetadata) Return whether or not the DataFrame is vector-based.- Parameters:
- matrixMetadata- the matrix metadata
- Returns:
- trueif the DataFrame is vector-based,- falseotherwise.
 
 - 
javaRDDStringCSVToMatrixObjectpublic static MatrixObject javaRDDStringCSVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD) Convert aJavaRDD<String>in CSV format to aMatrixObject- Parameters:
- javaRDD- the Java RDD of strings
- Returns:
- the JavaRDD<String>converted to aMatrixObject
 
 - 
javaRDDStringCSVToMatrixObjectpublic static MatrixObject javaRDDStringCSVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, MatrixMetadata matrixMetadata) Convert aJavaRDD<String>in CSV format to aMatrixObject- Parameters:
- javaRDD- the Java RDD of strings
- matrixMetadata- matrix metadata
- Returns:
- the JavaRDD<String>converted to aMatrixObject
 
 - 
javaRDDStringCSVToFrameObjectpublic static FrameObject javaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD) Convert aJavaRDD<String>in CSV format to aFrameObject- Parameters:
- javaRDD- the Java RDD of strings
- Returns:
- the JavaRDD<String>converted to aFrameObject
 
 - 
javaRDDStringCSVToFrameObjectpublic static FrameObject javaRDDStringCSVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, FrameMetadata frameMetadata) Convert aJavaRDD<String>in CSV format to aFrameObject- Parameters:
- javaRDD- the Java RDD of strings
- frameMetadata- frame metadata
- Returns:
- the JavaRDD<String>converted to aFrameObject
 
 - 
javaRDDStringIJVToMatrixObjectpublic static MatrixObject javaRDDStringIJVToMatrixObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, MatrixMetadata matrixMetadata) Convert aJavaRDD<String>in IJV format to aMatrixObject. Note that metadata is required for IJV format.- Parameters:
- javaRDD- the Java RDD of strings
- matrixMetadata- matrix metadata
- Returns:
- the JavaRDD<String>converted to aMatrixObject
 
 - 
javaRDDStringIJVToFrameObjectpublic static FrameObject javaRDDStringIJVToFrameObject(org.apache.spark.api.java.JavaRDD<String> javaRDD, FrameMetadata frameMetadata) Convert aJavaRDD<String>in IJV format to aFrameObject. Note that metadata is required for IJV format.- Parameters:
- javaRDD- the Java RDD of strings
- frameMetadata- frame metadata
- Returns:
- the JavaRDD<String>converted to aFrameObject
 
 - 
rddStringCSVToMatrixObjectpublic static MatrixObject rddStringCSVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd) Convert aRDD<String>in CSV format to aMatrixObject- Parameters:
- rdd- the RDD of strings
- Returns:
- the RDD<String>converted to aMatrixObject
 
 - 
rddStringCSVToMatrixObjectpublic static MatrixObject rddStringCSVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd, MatrixMetadata matrixMetadata) Convert aRDD<String>in CSV format to aMatrixObject- Parameters:
- rdd- the RDD of strings
- matrixMetadata- matrix metadata
- Returns:
- the RDD<String>converted to aMatrixObject
 
 - 
rddStringCSVToFrameObjectpublic static FrameObject rddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd) Convert aRDD<String>in CSV format to aFrameObject- Parameters:
- rdd- the RDD of strings
- Returns:
- the RDD<String>converted to aFrameObject
 
 - 
rddStringCSVToFrameObjectpublic static FrameObject rddStringCSVToFrameObject(org.apache.spark.rdd.RDD<String> rdd, FrameMetadata frameMetadata) Convert aRDD<String>in CSV format to aFrameObject- Parameters:
- rdd- the RDD of strings
- frameMetadata- frame metadata
- Returns:
- the RDD<String>converted to aFrameObject
 
 - 
rddStringIJVToMatrixObjectpublic static MatrixObject rddStringIJVToMatrixObject(org.apache.spark.rdd.RDD<String> rdd, MatrixMetadata matrixMetadata) Convert aRDD<String>in IJV format to aMatrixObject. Note that metadata is required for IJV format.- Parameters:
- rdd- the RDD of strings
- matrixMetadata- matrix metadata
- Returns:
- the RDD<String>converted to aMatrixObject
 
 - 
rddStringIJVToFrameObjectpublic static FrameObject rddStringIJVToFrameObject(org.apache.spark.rdd.RDD<String> rdd, FrameMetadata frameMetadata) Convert aRDD<String>in IJV format to aFrameObject. Note that metadata is required for IJV format.- Parameters:
- rdd- the RDD of strings
- frameMetadata- frame metadata
- Returns:
- the RDD<String>converted to aFrameObject
 
 - 
matrixObjectToJavaRDDStringCSVpublic static org.apache.spark.api.java.JavaRDD<String> matrixObjectToJavaRDDStringCSV(MatrixObject matrixObject) Convert aMatrixObjectto aJavaRDD<String>in CSV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aJavaRDD<String>
 
 - 
frameObjectToJavaRDDStringCSVpublic static org.apache.spark.api.java.JavaRDD<String> frameObjectToJavaRDDStringCSV(FrameObject frameObject, String delimiter) Convert aFrameObjectto aJavaRDD<String>in CSV format.- Parameters:
- frameObject- the- FrameObject
- delimiter- the delimiter
- Returns:
- the FrameObjectconverted to aJavaRDD<String>
 
 - 
matrixObjectToJavaRDDStringIJVpublic static org.apache.spark.api.java.JavaRDD<String> matrixObjectToJavaRDDStringIJV(MatrixObject matrixObject) Convert aMatrixObjectto aJavaRDD<String>in IJV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aJavaRDD<String>
 
 - 
frameObjectToJavaRDDStringIJVpublic static org.apache.spark.api.java.JavaRDD<String> frameObjectToJavaRDDStringIJV(FrameObject frameObject) Convert aFrameObjectto aJavaRDD<String>in IJV format.- Parameters:
- frameObject- the- FrameObject
- Returns:
- the FrameObjectconverted to aJavaRDD<String>
 
 - 
matrixObjectToRDDStringIJVpublic static org.apache.spark.rdd.RDD<String> matrixObjectToRDDStringIJV(MatrixObject matrixObject) Convert aMatrixObjectto aRDD<String>in IJV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aRDD<String>
 
 - 
frameObjectToRDDStringIJVpublic static org.apache.spark.rdd.RDD<String> frameObjectToRDDStringIJV(FrameObject frameObject) Convert aFrameObjectto aRDD<String>in IJV format.- Parameters:
- frameObject- the- FrameObject
- Returns:
- the FrameObjectconverted to aRDD<String>
 
 - 
matrixObjectToRDDStringCSVpublic static org.apache.spark.rdd.RDD<String> matrixObjectToRDDStringCSV(MatrixObject matrixObject) Convert aMatrixObjectto aRDD<String>in CSV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aRDD<String>
 
 - 
frameObjectToRDDStringCSVpublic static org.apache.spark.rdd.RDD<String> frameObjectToRDDStringCSV(FrameObject frameObject, String delimiter) Convert aFrameObjectto aRDD<String>in CSV format.- Parameters:
- frameObject- the- FrameObject
- delimiter- the delimiter
- Returns:
- the FrameObjectconverted to aRDD<String>
 
 - 
matrixObjectToListStringCSVpublic static List<String> matrixObjectToListStringCSV(MatrixObject matrixObject) Convert aMatrixObjectto aList<String>in CSV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aList<String>
 
 - 
frameObjectToListStringCSVpublic static List<String> frameObjectToListStringCSV(FrameObject frameObject, String delimiter) Convert aFrameObjectto aList<String>in CSV format.- Parameters:
- frameObject- the- FrameObject
- delimiter- the delimiter
- Returns:
- the FrameObjectconverted to aList<String>
 
 - 
matrixObjectToListStringIJVpublic static List<String> matrixObjectToListStringIJV(MatrixObject matrixObject) Convert aMatrixObjectto aList<String>in IJV format.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to aList<String>
 
 - 
frameObjectToListStringIJVpublic static List<String> frameObjectToListStringIJV(FrameObject frameObject) Convert aFrameObjectto aList<String>in IJV format.- Parameters:
- frameObject- the- FrameObject
- Returns:
- the FrameObjectconverted to aList<String>
 
 - 
matrixObjectTo2DDoubleArraypublic static double[][] matrixObjectTo2DDoubleArray(MatrixObject matrixObject) Convert aMatrixObjectto a two-dimensional double array.- Parameters:
- matrixObject- the- MatrixObject
- Returns:
- the MatrixObjectconverted to adouble[][]
 
 - 
matrixObjectToDataFramepublic static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> matrixObjectToDataFrame(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext, boolean isVectorDF) Convert aMatrixObjectto aDataFrame.- Parameters:
- matrixObject- the- MatrixObject
- sparkExecutionContext- the Spark execution context
- isVectorDF- is the DataFrame a vector DataFrame?
- Returns:
- the MatrixObjectconverted to aDataFrame
 
 - 
frameObjectToDataFramepublic static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> frameObjectToDataFrame(FrameObject frameObject, SparkExecutionContext sparkExecutionContext) Convert aFrameObjectto aDataFrame.- Parameters:
- frameObject- the- FrameObject
- sparkExecutionContext- the Spark execution context
- Returns:
- the FrameObjectconverted to aDataFrame
 
 - 
matrixObjectToBinaryBlockspublic static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> matrixObjectToBinaryBlocks(MatrixObject matrixObject, SparkExecutionContext sparkExecutionContext) Convert aMatrixObjectto aJavaPairRDD<MatrixIndexes, MatrixBlock>.- Parameters:
- matrixObject- the- MatrixObject
- sparkExecutionContext- the Spark execution context
- Returns:
- the MatrixObjectconverted to aJavaPairRDD<MatrixIndexes, MatrixBlock>
 
 - 
frameObjectToBinaryBlockspublic static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> frameObjectToBinaryBlocks(FrameObject frameObject, SparkExecutionContext sparkExecutionContext) Convert aFrameObjectto aJavaPairRDD<Long, FrameBlock>.- Parameters:
- frameObject- the- FrameObject
- sparkExecutionContext- the Spark execution context
- Returns:
- the FrameObjectconverted to aJavaPairRDD<Long, FrameBlock>
 
 - 
frameObjectTo2DStringArraypublic static String[][] frameObjectTo2DStringArray(FrameObject frameObject) Convert aFrameObjectto a two-dimensional string array.- Parameters:
- frameObject- the- FrameObject
- Returns:
- the FrameObjectconverted to aString[][]
 
 - 
jscpublic static org.apache.spark.api.java.JavaSparkContext jsc() Obtain JavaSparkContext from MLContextProxy.- Returns:
- the Java Spark Context
 
 - 
scpublic static org.apache.spark.SparkContext sc() Obtain SparkContext from MLContextProxy.- Returns:
- the Spark Context
 
 - 
sparkpublic static org.apache.spark.sql.SparkSession spark() Obtain SparkSession from MLContextProxy.- Returns:
- the Spark Session
 
 
- 
 
-