spark

Type Members

1. class AssembleFeatures extends Estimator[AssembleFeaturesModel] with HasFeaturesCol with MMLParams

   

   Creates a vector column of features from a collection of feature columns

2. class AssembleFeaturesModel extends Model[AssembleFeaturesModel] with Params with ConstructorWritable[AssembleFeaturesModel]

   

   Model produced by AssembleFeatures.

3. abstract class AsyncClient extends BaseClient

   

4. abstract class AsyncHTTPClient extends AsyncClient with HTTPClient

   

5. class BasicDatasetGenerationConstraints extends HasDatasetGenerationConstraints

   

   Basic constraints for generating a dataset.

6. class BatchIterator[T] extends Iterator[List[T]]

   

7. case class Benchmark(name: String, value: Double, precision: Double, higherIsBetter: Boolean = true) extends Product with Serializable

   

8. abstract class Benchmarks extends TestBase

   

9. class BestModel extends Model[BestModel] with ConstructorWritable[BestModel]

   

   Model produced by FindBestModel.

10. class Blur extends ImageTransformerStage

    

    Blurs the image using a box filter.

    Blurs the image using a box filter. The params are a map of the dimensions of the blurring box. Please refer to OpenCV for more information.

11. class BrainScriptBuilder extends AnyRef

    

    Utility methods for manipulating the BrainScript and overrides configs output to disk.

12. case class BrainScriptConfig(name: String, text: Seq[String]) extends Product with Serializable

    

13. class CNTKCommandBuilder extends CNTKCommandBuilderBase

    

14. abstract class CNTKCommandBuilderBase extends AnyRef

    

15. class CNTKConfig extends MMLConfig

    

16. class CNTKFunctionParam extends ComplexParam[SerializableFunction]

    

    Param for ByteArray.

    Param for ByteArray. Needed as spark has explicit params for many different types but not ByteArray.

17. class CNTKLearner extends Estimator[CNTKModel] with CNTKParams

    

    Annotations

    @InternalWrapper()

18. class CNTKModel extends Model[CNTKModel] with ComplexParamsWritable with HasMiniBatcher with Wrappable

    

    Annotations

    @InternalWrapper()

19. trait CNTKParams extends MMLParams

    

20. class Cacher extends Transformer with DefaultParamsWritable

    

21. class CheckpointData extends Transformer with CheckpointDataParams

    

    Cache the dataset at this point to memory or memory and disk

22. trait CheckpointDataParams extends MMLParams

    

23. class ClassBalancer extends Estimator[ClassBalancerModel] with DefaultParamsWritable with HasInputCol with HasOutputCol

    

    An estimator that calculates the weights for balancing a dataset.

    An estimator that calculates the weights for balancing a dataset. For example, if the negative class is half the size of the positive class, the weights will be 2 for rows with negative classes and 1 for rows with positive classes. these weights can be used in weighted classifiers and regressors to correct for heavilty skewed datasets. The inputCol should be the labels of the classes, and the output col will be the requisite weights.

24. class ClassBalancerModel extends Model[ClassBalancerModel] with ConstructorWritable[ClassBalancerModel]

    

25. case class ClassifierTrainParams(parallelism: String, numIterations: Int, learningRate: Double, numLeaves: Int, maxBin: Int, baggingFraction: Double, baggingFreq: Int, baggingSeed: Int, earlyStoppingRound: Int, featureFraction: Double, maxDepth: Int, minSumHessianInLeaf: Double, numMachines: Int, objective: String) extends TrainParams with Product with Serializable

    

    Defines the Booster parameters passed to the LightGBM classifier.

26. class CleanMissingData extends Estimator[CleanMissingDataModel] with HasInputCols with HasOutputCols with MMLParams

    

    Removes missing values from input dataset.

    Removes missing values from input dataset. The following modes are supported: Mean - replaces missings with mean of fit column Median - replaces missings with approximate median of fit column Custom - replaces missings with custom value specified by user For mean and median modes, only numeric column types are supported, specifically: Int, Long, Float, Double For custom mode, the types above are supported and additionally: String, Boolean

27. class CleanMissingDataModel extends Model[CleanMissingDataModel] with ConstructorWritable[CleanMissingDataModel]

    

    Model produced by CleanMissingData.

28. class ColorFormat extends ImageTransformerStage

    

    Converts an image from one color space to another, eg COLOR_BGR2GRAY.

    Converts an image from one color space to another, eg COLOR_BGR2GRAY. Refer to OpenCV for more information.

29. class ColumnNamesToFeaturize extends Serializable

    

    Class containing the list of column names to perform special featurization steps for.

    Class containing the list of column names to perform special featurization steps for. colNamesToHash - List of column names to hash. colNamesToDuplicateForMissings - List of column names containing doubles to duplicate so we can remove missing values from them. colNamesToTypes - Map of column names to their types. colNamesToCleanMissings - List of column names to clean missing values from (ignore). colNamesToVectorize - List of column names to vectorize using FastVectorAssembler. categoricalColumns - List of categorical columns to pass through or turn into indicator array. conversionColumnNamesMap - Map from old column names to new. addedColumnNamesMap - Map from old columns to newly generated columns for featurization.

    Annotations

    @SerialVersionUID()

30. class ComputeModelStatistics extends Transformer with ComputeModelStatisticsParams

    

    Evaluates the given scored dataset.

31. trait ComputeModelStatisticsParams extends MMLParams with HasLabelCol with HasScoresCol with HasScoredLabelsCol with HasEvaluationMetric

    

32. class ComputePerInstanceStatistics extends Transformer with ComputePerInstanceStatisticsParams

    

    Evaluates the given scored dataset with per instance metrics.

    Evaluates the given scored dataset with per instance metrics.

    The Regression metrics are: - L1_loss - L2_loss

    The Classification metrics are: - log_loss

33. trait ComputePerInstanceStatisticsParams extends MMLParams with HasLabelCol with HasScoresCol with HasScoredLabelsCol with HasScoredProbabilitiesCol with HasEvaluationMetric

    

34. abstract class Configuration extends AnyRef

    

35. class Consolidator[T] extends AnyRef

    

36. trait ConstructorReadable[T <: ConstructorWritable[_]] extends MLReadable[T]

    

37. class ConstructorReader[T] extends MLReader[T]

    

38. trait ConstructorWritable[S <: PipelineStage] extends MLWritable

    

    This trait allows you to easily add serialization to your Spark Models, assuming that they are completely parameterized by their constructor.

    This trait allows you to easily add serialization to your Spark Models, assuming that they are completely parameterized by their constructor. The main two fields required ate the TypeTag that allows the writer to inspect the constructor to get the types that need to be serialized, the actual objects that are serialized need to be defined in the field objectsToSave.

39. abstract class ConstructorWriter[S <: PipelineStage] extends MLWriter

    

40. class ContextObjectInputStream extends ObjectInputStream

    

41. class CropImage extends ImageTransformerStage

    

    Crops the image for processing.

    Crops the image for processing. The parameters are: "x" - First dimension; start of crop "y" - second dimension - start of crop "height" -height of cropped image "width" - width of cropped image "stageName" - "crop"

42. class CustomInputParser extends HTTPInputParser with ComplexParamsWritable

    

43. class CustomOutputParser extends HTTPOutputParser with ComplexParamsWritable

    

44. class DataConversion extends Transformer with MMLParams

    

    Converts the specified list of columns to the specified type.

    Converts the specified list of columns to the specified type. Returns a new DataFrame with the converted columns

45. trait DataFrameEquality extends TestBase

    

46. class DataFrameSugars extends AnyRef

    

47. case class DataStreamReaderExtensions(dsr: DataStreamReader) extends Product with Serializable

    

48. case class DataStreamWriterExtensions[T](dsw: DataStreamWriter[T]) extends Product with Serializable

    

49. abstract class DataWriter extends AnyRef

    

50. case class DatasetMissingValuesGenerationOptions(percentMissing: Double, columnTypesWithMissings: ColumnOptions.ValueSet, dataTypesWithMissings: DataOptions.ValueSet) extends Product with Serializable

    

51. case class DatasetOptions(columnTypes: ColumnOptions.ValueSet, dataTypes: DataOptions.ValueSet, missingValuesOptions: DatasetMissingValuesGenerationOptions) extends Product with Serializable

    

    Options used to specify how a dataset will be generated.

    Options used to specify how a dataset will be generated. This contains information on what the data and column types (specified as flags) for generating a dataset will be limited to. It also contain options for all possible missing values generation and options for how values will be generated.

52. class DiscreteHyperParam[T] extends Dist[T]

    

53. abstract class Dist[T] extends AnyRef

    

    Represents a distribution of values.

    Represents a distribution of values.

    T

    The type T of the values generated.

54. class DoubleRangeHyperParam extends RangeHyperParam[Double]

    

55. class DropColumns extends Transformer with MMLParams

    

    DropColumns takes a dataframe and a list of columns to drop as input and returns a dataframe comprised of only those columns not listed in the input list.

    DropColumns takes a dataframe and a list of columns to drop as input and returns a dataframe comprised of only those columns not listed in the input list.

56. class DynamicBufferedBatcher[T] extends Iterator[List[T]]

    

57. class DynamicMiniBatchTransformer extends Transformer with MiniBatchBase

    

58. class EnsembleByKey extends Transformer with MMLParams

    

59. case class EntityData(content: Array[Byte], contentEncoding: Option[HeaderData], contentLength: Long, contentType: Option[HeaderData], isChunked: Boolean, isRepeatable: Boolean, isStreaming: Boolean) extends Product with Serializable

    

60. class Explode extends Transformer with HasInputCol with HasOutputCol with MMLParams

    

61. class Featurize extends Estimator[PipelineModel] with MMLParams

    

    Featurizes a dataset.

    Featurizes a dataset. Converts the specified columns to feature columns.

62. class FindBestModel extends Estimator[BestModel] with FindBestModelParams

    

    Evaluates and chooses the best model from a list of models.

63. trait FindBestModelParams extends Wrappable with ComplexParamsWritable with HasEvaluationMetric

    

64. class FixedBatcher[T] extends Iterator[List[T]]

    

65. class FixedBufferedBatcher[T] extends Iterator[List[T]]

    

66. class FixedMiniBatchTransformer extends Transformer with MiniBatchBase

    

67. class FlattenBatch extends Transformer with Wrappable with DefaultParamsWritable

    

68. class Flip extends ImageTransformerStage

    

    Flips the image

69. class FloatRangeHyperParam extends RangeHyperParam[Float]

    

70. class GaussianKernel extends ImageTransformerStage

    

    Applies gaussian kernel to blur the image.

    Applies gaussian kernel to blur the image. Please refer to OpenCV for detailed information about the parameters and their allowable values.

71. class GenerateDataType extends Serializable

    

    Generates the specified random data type.

72. class GridSpace extends ParamSpace

    

    Represents a parameter grid for tuning with discrete values.

    Represents a parameter grid for tuning with discrete values. Can be generated with the ParamGridBuilder.

73. abstract class HTTPInputParser extends Transformer with HasOutputCol with HasInputCol

    

74. abstract class HTTPOutputParser extends Transformer with HasInputCol with HasOutputCol

    

75. trait HTTPParams extends Wrappable

    

76. case class HTTPRequestData(requestLine: RequestLineData, headers: Array[HeaderData], entity: Option[EntityData]) extends Product with Serializable

    

77. case class HTTPResponseData(headers: Array[HeaderData], entity: EntityData, statusLine: StatusLineData, locale: String) extends Product with Serializable

    

78. class HTTPTransformer extends Transformer with HTTPParams with HasInputCol with HasOutputCol with ComplexParamsWritable

    

79. trait Handler extends AnyRef

    

80. trait HandlingUtils extends AnyRef

    

81. trait HasDatasetGenerationConstraints extends AnyRef

    

    Specifies the trait for constraints on generating a dataset.

82. trait HasErrorCol extends Params

    

83. trait HasEvaluationMetric extends Wrappable

    

84. trait HasFeaturesCol extends Wrappable

    

85. trait HasInputCol extends Wrappable

    

86. trait HasInputCols extends Wrappable

    

87. trait HasLabelCol extends Wrappable

    

88. trait HasMiniBatcher extends Params

    

89. trait HasOutputCol extends Wrappable

    

90. trait HasOutputCols extends Wrappable

    

91. trait HasScoredLabelsCol extends Wrappable

    

92. trait HasScoredProbabilitiesCol extends Wrappable

    

93. trait HasScoresCol extends Wrappable

    

94. trait HasURL extends Wrappable

    

95. class HdfsWriter extends SingleFileResolver

    

96. case class HeaderData(name: String, value: String) extends Product with Serializable

    

97. class HyperparamBuilder extends AnyRef

    

    Specifies the search space for hyperparameters.

98. class ImageFeaturizer extends Transformer with HasInputCol with HasOutputCol with Wrappable with ComplexParamsWritable

    

    The ImageFeaturizer relies on a CNTK model to do the featurization, one can set this model using the modelLocation parameter.

    The ImageFeaturizer relies on a CNTK model to do the featurization, one can set this model using the modelLocation parameter. To map the nodes of the CNTK model onto the standard "layers" structure of a feed forward neural net, one needs to supply a list of node names that range from the output node, back towards the input node of the CNTK Function. This list does not need to be exhaustive, and is provided to you if you use a model downloaded from the ModelDownloader, one can find this layer list in the schema of the downloaded model.

    The ImageFeaturizer takes an input column of images (the type returned by the ImageReader), and automatically resizes them to fit the CMTKModel's inputs. It then feeds them through a pre-trained CNTK model. One can truncate the model using the cutOutputLayers parameter that determines how many layers to truncate from the output of the network. For example, layer=0 means that no layers are removed, layer=2 means that the image featurizer returns the activations of the layer that is two layers from the output layer.

    Annotations

    @InternalWrapper()

99. class ImageSetAugmenter extends Transformer with HasInputCol with HasOutputCol with DefaultParamsWritable

    

100. class ImageTransformer extends Transformer with HasInputCol with HasOutputCol with MMLParams

     

     Image processing stage.

     Image processing stage. Please refer to OpenCV for additional information

     Annotations

     @InternalWrapper()

101. abstract class ImageTransformerStage extends Serializable

     

     Image processing stage.

102. class IndexToValue extends Transformer with HasInputCol with HasOutputCol with MMLParams

     

     This class takes in a categorical column with MML style attibutes and then transforms it back to the original values.

     This class takes in a categorical column with MML style attibutes and then transforms it back to the original values. This extends MLLIB IndexToString by allowing the transformation back to any types of values.

103. case class InputData(format: String, path: String, shapes: Map[String, InputShape]) extends Product with Serializable

     

104. case class InputShape(dim: Int, form: String) extends Product with Serializable

     

105. class IntRangeHyperParam extends RangeHyperParam[Int]

     

106. class InternalWrapper extends Annotation with StaticAnnotation

     

     Generate the internal wrapper for a given class.

     Generate the internal wrapper for a given class. Used for complicated wrappers, where the basic functionality is auto-generated, and the rest is added in the inherited wrapper.

107. class JSONInputParser extends HTTPInputParser with HasURL with ComplexParamsWritable

     

108. class JSONOutputParser extends HTTPOutputParser with ComplexParamsWritable

     

109. class Lambda extends Transformer with Wrappable with ComplexParamsWritable

     

110. class LightGBMBooster extends Serializable

     

     Represents a LightGBM Booster learner

111. class LightGBMClassificationModel extends ProbabilisticClassificationModel[Vector, LightGBMClassificationModel] with ConstructorWritable[LightGBMClassificationModel]

     

     Model produced by LightGBMClassifier.

     Model produced by LightGBMClassifier.

     Annotations

     @InternalWrapper()

112. class LightGBMClassifier extends ProbabilisticClassifier[Vector, LightGBMClassifier, LightGBMClassificationModel] with LightGBMParams

     

     Trains a LightGBM Binary Classification model, a fast, distributed, high performance gradient boosting framework based on decision tree algorithms.

     Trains a LightGBM Binary Classification model, a fast, distributed, high performance gradient boosting framework based on decision tree algorithms. For more information please see here: https://github.com/Microsoft/LightGBM. For parameter information see here: https://github.com/Microsoft/LightGBM/blob/master/docs/Parameters.rst

     Annotations

     @InternalWrapper()

113. trait LightGBMParams extends MMLParams

     

     Defines common parameters across all LightGBM learners.

114. class LightGBMRegressionModel extends RegressionModel[Vector, LightGBMRegressionModel] with ConstructorWritable[LightGBMRegressionModel]

     

     Model produced by LightGBMRegressor.

115. class LightGBMRegressor extends BaseRegressor[Vector, LightGBMRegressor, LightGBMRegressionModel] with LightGBMParams

     

     Trains a LightGBM Regression model, a fast, distributed, high performance gradient boosting framework based on decision tree algorithms.

     Trains a LightGBM Regression model, a fast, distributed, high performance gradient boosting framework based on decision tree algorithms. For more information please see here: https://github.com/Microsoft/LightGBM. For parameter information see here: https://github.com/Microsoft/LightGBM/blob/master/docs/Parameters.rst Note: The application parameter supports the following values:

     • regression_l2, L2 loss, alias=regression, mean_squared_error, mse, l2_root, root_mean_squared_error, rmse
     • regression_l1, L1 loss, alias=mean_absolute_error, mae
     • huber, Huber loss
     • fair, Fair loss
     • poisson, Poisson regression
     • quantile, Quantile regression
     • mape, MAPE loss, alias=mean_absolute_percentage_error
     • gamma, Gamma regression with log-link. It might be useful, e.g., for modeling insurance claims severity, or for any target that might be gamma-distributed
     • tweedie, Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any target that might be tweedie-distributed

116. trait LinuxOnly extends TestBase

     

117. class LocalWriter extends SingleFileResolver

     

118. class LongRangeHyperParam extends RangeHyperParam[Long]

     

119. class MMLConfig extends Configuration

     

120. trait MMLParams extends Wrappable with DefaultParamsWritable

     

121. class MPICommandBuilder extends CNTKCommandBuilderBase with MPIConfiguration

     

122. trait MPIConfiguration extends AnyRef

     

123. class MetricsLogger extends AnyRef

     

     Helper class for logging metrics to log4j.

124. trait MiniBatchBase extends Transformer with DefaultParamsWritable with Wrappable

     

125. class ModelDownloader extends Client

     

     Class for downloading models from a server to Local or HDFS

126. class ModelNotFoundException extends FileNotFoundException

     

     Exception returned if a repo cannot find the file

127. case class ModelSchema(name: String, dataset: String, modelType: String, uri: URI, hash: String, size: Long, inputNode: Int, numLayers: Int, layerNames: Array[String]) extends Schema with Product with Serializable

     

     Class representing the schema of a CNTK model

     Class representing the schema of a CNTK model

     name

     name of the model architecture

     dataset

     dataset the model was trained on

     modelType

     type of problem the model is suited for eg: (image, text, sound, sentiment etc)

     uri

     location of the underlying file (local, HDFS, or HTTP)

     hash

     sha256 hash of the underlying file

     size

     size in bytes of the underlying file

     inputNode

     the node which represents the input

     numLayers

     the number of layers of the model

     layerNames

     the names nodes that represent layers in the network

128. class MultiColumnAdapter extends Estimator[PipelineModel] with Wrappable with ComplexParamsWritable

     

     The MultiColumnAdapter takes a unary pipeline stage and a list of input output column pairs and applies the pipeline stage to each input column after being fit

     The MultiColumnAdapter takes a unary pipeline stage and a list of input output column pairs and applies the pipeline stage to each input column after being fit

129. class MultiNGram extends Transformer with HasInputCol with HasOutputCol with MMLParams

     

     Extracts several ngrams

130. class MultiVectorAssembler extends VectorAssembler

     

131. class NativeLoader extends Serializable

     

132. case class NetworkParams(executorIdToHost: Map[Int, String], defaultListenPort: Int, addr: String, port: Int) extends Product with Serializable

     

133. class NullOrdering[T] extends Ordering[T]

     

134. class PartitionConsolidator extends Transformer with HTTPParams with HasInputCol with HasOutputCol with ComplexParamsWritable

     

135. sealed class PartitionSample extends Transformer with PartitionSampleParams

     

136. trait PartitionSampleParams extends MMLParams

     

137. case class ProtocolVersionData(protocol: String, major: Int, minor: Int) extends Product with Serializable

     

138. class RandomDatasetGenerationConstraints extends HasDatasetGenerationConstraints

     

     Contraints on generating a dataset where all parameters are randomly generated.

139. abstract class RandomMMLGenerator[T] extends RandomDataGenerator[T]

     

     Base abstract class for random generation of data.

     Base abstract class for random generation of data.

     T

     The data to generate.

140. class RandomRowGenerator extends RandomMMLGenerator[Row]

     

     Randomly generates a row given the set space of data, column options.

141. class RandomRowGeneratorCombiner extends RandomMMLGenerator[Row]

     

     Combines an array of row generators into a single row generator.

142. class RandomSpace extends ParamSpace

     

     Represents a generator of parameters with specified distributions added by the HyperparamBuilder.

143. abstract class RangeHyperParam[T] extends Dist[T]

     

144. case class RegressorTrainParams(parallelism: String, numIterations: Int, learningRate: Double, numLeaves: Int, objective: String, alpha: Double, tweedieVariancePower: Double, maxBin: Int, baggingFraction: Double, baggingFreq: Int, baggingSeed: Int, earlyStoppingRound: Int, featureFraction: Double, maxDepth: Int, minSumHessianInLeaf: Double, numMachines: Int) extends TrainParams with Product with Serializable

     

     Defines the Booster parameters passed to the LightGBM regressor.

145. class RenameColumn extends Transformer with MMLParams with HasInputCol with HasOutputCol

     

     RenameColumn takes a dataframe with an input and an output column name and returns a dataframe comprised of the original columns with the input column renamed as the output column name.

     RenameColumn takes a dataframe with an input and an output column name and returns a dataframe comprised of the original columns with the input column renamed as the output column name.

146. class Repartition extends Transformer with MMLParams

     

     Partitions the dataset into n partitions

147. case class RequestLineData(method: String, uri: String, protoclVersion: Option[ProtocolVersionData]) extends Product with Serializable

     

148. class ResizeImage extends ImageTransformerStage

     

     Resizes the image.

     Resizes the image. The parameters of the ParameterMap are: "height" - the height of the image "width" "stageName" Please refer to OpenCV for more information

149. abstract class Schema extends AnyRef

     

     Abstract representation of a schema for an item that can be held in a repository

150. class SelectColumns extends Transformer with MMLParams

     

     SelectColumns takes a dataframe and a list of columns to select as input and returns a dataframe comprised of only those columns listed in the input list.

     SelectColumns takes a dataframe and a list of columns to select as input and returns a dataframe comprised of only those columns listed in the input list.

     The columns to be selected is a list of column names

151. class SharedSingleton[T] extends Serializable

     

     Holds a variable shared among all workers that behaves like a local singleton.

     Holds a variable shared among all workers that behaves like a local singleton. Useful to use non-serializable objects in Spark closures that maintain state across tasks.

152. class SharedVariable[T] extends Serializable

     

     Holds a variable shared among all workers.

     Holds a variable shared among all workers. Useful to use non-serializable objects in Spark closures.

     Note this code has been borrowed from: https://www.nicolaferraro.me/2016/02/22/using-non-serializable-objects-in-apache-spark/

153. class SimpleHTTPTransformer extends Transformer with HTTPParams with HasMiniBatcher with HasInputCol with HasOutputCol with ComplexParamsWritable with HasErrorCol

     

154. abstract class SingleFileResolver extends DataWriter

     

155. abstract class SingleThreadedHTTPClient extends HTTPClient with SingleThreadedClient

     

156. abstract class SingleTypeReducer extends Transformer with TypeConversion

     

157. class SingleVectorAssembler extends VectorAssembler

     

158. case class StatusLineData(protocolVersion: ProtocolVersionData, statusCode: Int, reasonPhrase: String) extends Product with Serializable

     

159. class SummarizeData extends Transformer with SummarizeDataParams

     

     Compute summary statistics for the dataset.

     Compute summary statistics for the dataset. The following statistics are computed: - counts - basic - sample - percentiles - errorThreshold - error threshold for quantiles

160. trait SummarizeDataParams extends MMLParams

     

161. class TLCConfig extends MMLConfig

     

162. abstract class TestBase extends FunSuite with BeforeAndAfterEachTestData with BeforeAndAfterAll

     

163. class TextFeaturizer extends Estimator[TextFeaturizerModel] with TextFeaturizerParams with HasInputCol with HasOutputCol

     

     Featurize text.

164. class TextFeaturizerModel extends Model[TextFeaturizerModel] with ConstructorWritable[TextFeaturizerModel]

     

165. trait TextFeaturizerParams extends Wrappable with DefaultParamsWritable

     

166. class TextPreprocessor extends Transformer with HasInputCol with HasOutputCol with Wrappable with ComplexParamsWritable

     

     TextPreprocessor takes a dataframe and a dictionary that maps (text -> replacement text), scans each cell in the input col and replaces all substring matches with the corresponding value.

     TextPreprocessor takes a dataframe and a dictionary that maps (text -> replacement text), scans each cell in the input col and replaces all substring matches with the corresponding value. Priority is given to longer keys and from left to right.

167. class Threshold extends ImageTransformerStage

     

     Applies a threshold to each element of the image.

     Applies a threshold to each element of the image. Please refer to threshold for more information

168. class TimeIntervalBatcher[T] extends Iterator[List[T]]

     

169. class TimeIntervalMiniBatchTransformer extends Transformer with MiniBatchBase

     

170. class Timer extends Estimator[TimerModel] with TimerParams with ComplexParamsWritable

     

171. class TimerModel extends Model[TimerModel] with TimerParams with ConstructorWritable[TimerModel]

     

172. trait TimerParams extends Wrappable

     

173. class TrainClassifier extends Estimator[TrainedClassifierModel] with HasLabelCol with ComplexParamsWritable with HasFeaturesCol

     

     Trains a classification model.

     Trains a classification model. Featurizes the given data into a vector of doubles.

     Note the behavior of the reindex and labels parameters, the parameters interact as:

     reindex -> false labels -> false (Empty) Assume all double values, don't use metadata, assume natural ordering

     reindex -> true labels -> false (Empty) Index, use natural ordering of string indexer

     reindex -> false labels -> true (Specified) Assume user knows indexing, apply label values. Currently only string type supported.

     reindex -> true labels -> true (Specified) Validate labels matches column type, try to recast to label type, reindex label column

174. abstract class TrainParams extends Serializable

     

     Defines the common Booster parameters passed to the LightGBM learners.

175. class TrainRegressor extends Estimator[TrainedRegressorModel] with HasLabelCol with MMLParams with ComplexParamsWritable

     

     Trains a regression model.

176. class TrainedClassifierModel extends Model[TrainedClassifierModel] with ConstructorWritable[TrainedClassifierModel]

     

     Model produced by TrainClassifier.

177. class TrainedRegressorModel extends Model[TrainedRegressorModel] with ConstructorWritable[TrainedRegressorModel]

     

     Model produced by TrainRegressor.

178. class Trie extends Serializable

     

179. class TuneHyperparameters extends Estimator[TuneHyperparametersModel] with Wrappable with ComplexParamsWritable with HasEvaluationMetric

     

     Tunes model hyperparameters

     Tunes model hyperparameters

     Allows user to specify multiple untrained models to tune using various search strategies. Currently supports cross validation with random grid search.

     Annotations

     @InternalWrapper()

180. class TuneHyperparametersModel extends Model[TuneHyperparametersModel] with ConstructorWritable[TuneHyperparametersModel]

     

     Model produced by TuneHyperparameters.

     Model produced by TuneHyperparameters.

     Annotations

     @InternalWrapper()

181. trait TypeConversion extends AnyRef

     

182. class UDFTransformer extends Transformer with Wrappable with ComplexParamsWritable with HasInputCol with HasInputCols with HasOutputCol

     

     UDFTransformer takes as input input column, output column, and a UserDefinedFunction returns a dataframe comprised of the original columns with the output column as the result of the udf applied to the input column

     UDFTransformer takes as input input column, output column, and a UserDefinedFunction returns a dataframe comprised of the original columns with the output column as the result of the udf applied to the input column

     Annotations

     @InternalWrapper()

183. class UnrollImage extends Transformer with HasInputCol with HasOutputCol with MMLParams

     

     Converts the representation of an m X n pixel image to an m * n vector of Doubles

     Converts the representation of an m X n pixel image to an m * n vector of Doubles

     The input column name is assumed to be "image", the output column name is "<uid>_output"

184. class ValueIndexer extends Estimator[ValueIndexerModel] with ValueIndexerParams

     

     Fits a dictionary of values from the input column.

     Fits a dictionary of values from the input column. Model then transforms a column to a categorical column of the given array of values. Similar to StringIndexer except it can be used on any value types.

185. class ValueIndexerModel extends Model[ValueIndexerModel] with ValueIndexerParams with ComplexParamsWritable

     

     Model produced by ValueIndexer.

186. trait ValueIndexerParams extends MMLParams with HasInputCol with HasOutputCol

     

187. abstract class VectorAssembler extends AnyRef

     

188. trait Wrappable extends Params