Class RemoteParForSpark
- java.lang.Object
- 
- org.apache.sysds.runtime.controlprogram.parfor.RemoteParForSpark
 
- 
 public class RemoteParForSpark extends Object This class serves two purposes: (1) isolating Spark imports to enable running in environments where no Spark libraries are available, and (2) to follow the same structure as the parfor remote_mr job submission. NOTE: currently, we still exchange inputs and outputs via hdfs (this covers the general case if data already resides in HDFS, in-memory data, and partitioned inputs; also, it allows for pre-aggregation by overwriting partial task results with pre-paggregated results from subsequent iterations) TODO reducebykey on variable names
- 
- 
Constructor SummaryConstructors Constructor Description RemoteParForSpark()
 - 
Method SummaryAll Methods Static Methods Concrete Methods Modifier and Type Method Description static RemoteParForJobReturnrunJob(long pfid, String prog, HashMap<String,byte[]> clsMap, List<Task> tasks, ExecutionContext ec, Set<String> brVars, List<ParForStatementBlock.ResultVar> resultVars, boolean cpCaching, int numMappers, boolean topLevelPF)
 
- 
- 
- 
Method Detail- 
runJobpublic static RemoteParForJobReturn runJob(long pfid, String prog, HashMap<String,byte[]> clsMap, List<Task> tasks, ExecutionContext ec, Set<String> brVars, List<ParForStatementBlock.ResultVar> resultVars, boolean cpCaching, int numMappers, boolean topLevelPF) 
 
- 
 
-