AddDocuments

class AddDocuments.AddDocuments(actionCol='@search.action', batchSize=100, concurrency=1, concurrentTimeout=100.0, errorCol=None, handler=None, indexName=None, outputCol=None, serviceName=None, subscriptionKey=None, timeout=60.0, url=None)

Bases: mmlspark.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaTransformer

Parameters:

• actionCol (str) – You can combine actions, such as an upload and a delete, in the same batch.upload: An upload action is similar to an ‘upsert’where the document will be inserted if it is new and updated/replacedif it exists. Note that all fields are replaced in the update case.merge: Merge updates an existing document with the specified fields.If the document doesn’t exist, the merge will fail. Any fieldyou specify in a merge will replace the existing field in the document.This includes fields of type Collection(Edm.String). For example, ifthe document contains a field ‘tags’ with value [‘budget’] and you executea merge with value [‘economy’, ‘pool’] for ‘tags’, the final valueof the ‘tags’ field will be [‘economy’, ‘pool’]. It will not be [‘budget’, ‘economy’, ‘pool’].mergeOrUpload: This action behaves like merge if a document with the given key already exists in the index. If the document does not exist, it behaves like upload with a new document.delete: Delete removes the specified document from the index. Note that any field you specify in a delete operation, other than the key field, will be ignored. If you want to remove an individual field from a document, use merge instead and simply set the field explicitly to null. (default: @search.action)
• batchSize (int) – The max size of the buffer (default: 100)
• concurrency (int) – max number of concurrent calls (default: 1)
• concurrentTimeout (double) – max number seconds to wait on futures if concurrency >= 1 (default: 100.0)
• errorCol (str) – column to hold http errors (default: [self.uid]_error)
• handler (object) – Which strategy to use when handling requests (default: UserDefinedFunction(<function2>,StringType,None))
• indexName (str) –
• outputCol (str) – The name of the output column (default: [self.uid]_output)
• serviceName (str) –
• subscriptionKey (object) – the API key to use
• timeout (double) – number of seconds to wait before closing the connection (default: 60.0)
• url (str) – Url of the service

getActionCol()

Returns:You can combine actions, such as an upload and a delete, in the same batch.upload: An upload action is similar to an ‘upsert’where the document will be inserted if it is new and updated/replacedif it exists. Note that all fields are replaced in the update case.merge: Merge updates an existing document with the specified fields.If the document doesn’t exist, the merge will fail. Any fieldyou specify in a merge will replace the existing field in the document.This includes fields of type Collection(Edm.String). For example, ifthe document contains a field ‘tags’ with value [‘budget’] and you executea merge with value [‘economy’, ‘pool’] for ‘tags’, the final valueof the ‘tags’ field will be [‘economy’, ‘pool’]. It will not be [‘budget’, ‘economy’, ‘pool’].mergeOrUpload: This action behaves like merge if a document with the given key already exists in the index. If the document does not exist, it behaves like upload with a new document.delete: Delete removes the specified document from the index. Note that any field you specify in a delete operation, other than the key field, will be ignored. If you want to remove an individual field from a document, use merge instead and simply set the field explicitly to null. (default: @search.action) Return type:str

getBatchSize()

Returns:The max size of the buffer (default: 100) Return type:int

getConcurrency()

Returns:max number of concurrent calls (default: 1) Return type:int

getConcurrentTimeout()

Returns:max number seconds to wait on futures if concurrency >= 1 (default: 100.0) Return type:double

getErrorCol()

Returns:column to hold http errors (default: [self.uid]_error) Return type:str

getHandler()

Returns:Which strategy to use when handling requests (default: UserDefinedFunction(<function2>,StringType,None)) Return type:object

getIndexName()

Returns: Return type:str

static getJavaPackage()

Returns package name String.

getOutputCol()

Returns:The name of the output column (default: [self.uid]_output) Return type:str

getServiceName()

Returns: Return type:str

getSubscriptionKey()

Returns:the API key to use Return type:object

getTimeout()

Returns:number of seconds to wait before closing the connection (default: 60.0) Return type:double

getUrl()

Returns:Url of the service Return type:str

classmethod read()

Returns an MLReader instance for this class.

setActionCol(value)

Parameters:actionCol (str) – You can combine actions, such as an upload and a delete, in the same batch.upload: An upload action is similar to an ‘upsert’where the document will be inserted if it is new and updated/replacedif it exists. Note that all fields are replaced in the update case.merge: Merge updates an existing document with the specified fields.If the document doesn’t exist, the merge will fail. Any fieldyou specify in a merge will replace the existing field in the document.This includes fields of type Collection(Edm.String). For example, ifthe document contains a field ‘tags’ with value [‘budget’] and you executea merge with value [‘economy’, ‘pool’] for ‘tags’, the final valueof the ‘tags’ field will be [‘economy’, ‘pool’]. It will not be [‘budget’, ‘economy’, ‘pool’].mergeOrUpload: This action behaves like merge if a document with the given key already exists in the index. If the document does not exist, it behaves like upload with a new document.delete: Delete removes the specified document from the index. Note that any field you specify in a delete operation, other than the key field, will be ignored. If you want to remove an individual field from a document, use merge instead and simply set the field explicitly to null. (default: @search.action)

setBatchSize(value)

Parameters:batchSize (int) – The max size of the buffer (default: 100)

setConcurrency(value)

Parameters:concurrency (int) – max number of concurrent calls (default: 1)

setConcurrentTimeout(value)

Parameters:concurrentTimeout (double) – max number seconds to wait on futures if concurrency >= 1 (default: 100.0)

setErrorCol(value)

Parameters:errorCol (str) – column to hold http errors (default: [self.uid]_error)

setHandler(value)

Parameters:handler (object) – Which strategy to use when handling requests (default: UserDefinedFunction(<function2>,StringType,None))

setIndexName(value)

Parameters:indexName (str) –

setOutputCol(value)

Parameters:outputCol (str) – The name of the output column (default: [self.uid]_output)

setParams(actionCol='@search.action', batchSize=100, concurrency=1, concurrentTimeout=100.0, errorCol=None, handler=None, indexName=None, outputCol=None, serviceName=None, subscriptionKey=None, timeout=60.0, url=None)

Set the (keyword only) parameters

Parameters:

• actionCol (str) – You can combine actions, such as an upload and a delete, in the same batch.upload: An upload action is similar to an ‘upsert’where the document will be inserted if it is new and updated/replacedif it exists. Note that all fields are replaced in the update case.merge: Merge updates an existing document with the specified fields.If the document doesn’t exist, the merge will fail. Any fieldyou specify in a merge will replace the existing field in the document.This includes fields of type Collection(Edm.String). For example, ifthe document contains a field ‘tags’ with value [‘budget’] and you executea merge with value [‘economy’, ‘pool’] for ‘tags’, the final valueof the ‘tags’ field will be [‘economy’, ‘pool’]. It will not be [‘budget’, ‘economy’, ‘pool’].mergeOrUpload: This action behaves like merge if a document with the given key already exists in the index. If the document does not exist, it behaves like upload with a new document.delete: Delete removes the specified document from the index. Note that any field you specify in a delete operation, other than the key field, will be ignored. If you want to remove an individual field from a document, use merge instead and simply set the field explicitly to null. (default: @search.action)
• batchSize (int) – The max size of the buffer (default: 100)
• concurrency (int) – max number of concurrent calls (default: 1)
• concurrentTimeout (double) – max number seconds to wait on futures if concurrency >= 1 (default: 100.0)
• errorCol (str) – column to hold http errors (default: [self.uid]_error)
• handler (object) – Which strategy to use when handling requests (default: UserDefinedFunction(<function2>,StringType,None))
• indexName (str) –
• outputCol (str) – The name of the output column (default: [self.uid]_output)
• serviceName (str) –
• subscriptionKey (object) – the API key to use
• timeout (double) – number of seconds to wait before closing the connection (default: 60.0)
• url (str) – Url of the service

setServiceName(value)

Parameters:serviceName (str) –

setSubscriptionKey(value)

Parameters:subscriptionKey (object) – the API key to use

setSubscriptionKeyCol(value)

Parameters:subscriptionKey (object) – the API key to use

setTimeout(value)

Parameters:timeout (double) – number of seconds to wait before closing the connection (default: 60.0)

setUrl(value)

Parameters:url (str) – Url of the service