salt.modules.solr

Apache Solr Salt Module

Author: Jed Glazner Version: 0.2.1 Modified: 12/09/2011

This module uses HTTP requests to talk to the apache solr request handlers to gather information and report errors. Because of this the minion doesn't necessarily need to reside on the actual slave. However if you want to use the signal function the minion must reside on the physical solr host.

This module supports multi-core and standard setups. Certain methods are master/slave specific. Make sure you set the solr.type. If you have questions or want a feature request please ask.

Coming Features in 0.3

  1. Add command for checking for replication failures on slaves
  2. Improve match_index_versions since it's pointless on busy solr masters
  3. Add additional local fs checks for backups to make sure they succeeded

Override these in the minion config

solr.cores
A list of core names e.g. ['core1','core2']. An empty list indicates non-multicore setup.
solr.baseurl
The root level URL to access solr via HTTP
solr.request_timeout
The number of seconds before timing out an HTTP/HTTPS/FTP request. If nothing is specified then the python global timeout setting is used.
solr.type
Possible values are 'master' or 'slave'
solr.backup_path
The path to store your backups. If you are using cores and you can specify to append the core name to the path in the backup method.
solr.num_backups
For versions of solr >= 3.5. Indicates the number of backups to keep. This option is ignored if your version is less.
solr.init_script
The full path to your init script with start/stop options
solr.dih.options
A list of options to pass to the DIH.

Required Options for DIH

clean : False
Clear the index before importing
commit : True
Commit the documents to the index upon completion
optimize : True
Optimize the index after commit is complete
verbose : True
Get verbose output
salt.modules.solr.abort_import(handler, host=None, core_name=None, verbose=False)

MASTER ONLY Aborts an existing import command to the specified handler. This command can only be run if the minion is configured with solr.type=master

handler : str
The name of the data import handler.
host : str (None)
The solr host to query. __opts__['host'] is default.
core : str (None)
The core the handler belongs to.
verbose : boolean (False)
Run the command with verbose output.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.abort_import dataimport None music {'clean':True}
salt.modules.solr.backup(host=None, core_name=None, append_core_to_path=False)

Tell solr make a backup. This method can be mis-leading since it uses the backup API. If an error happens during the backup you are not notified. The status: 'OK' in the response simply means that solr received the request successfully.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.
append_core_to_path : boolean (False)
If True add the name of the core to the backup path. Assumes that minion backup path is not None.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.backup music
salt.modules.solr.core_status(host=None, core_name=None)

MULTI-CORE HOSTS ONLY Get the status for a given core or all cores if no core is specified

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str
The name of the core to reload

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.core_status None music
salt.modules.solr.delta_import(handler, host=None, core_name=None, options=None, extra=None)

Submits an import command to the specified handler using specified options. This command can only be run if the minion is configured with solr.type=master

handler : str
The name of the data import handler.
host : str (None)
The solr host to query. __opts__['host'] is default.
core : str (None)
The core the handler belongs to.
options : dict (__opts__)
A list of options such as clean, optimize commit, verbose, and pause_replication. leave blank to use __opts__ defaults. options will be merged with __opts__
extra : dict ([])
Extra name value pairs to pass to the handler. e.g. ["name=value"]

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.delta_import dataimport None music {'clean':True}
salt.modules.solr.full_import(handler, host=None, core_name=None, options=None, extra=None)

MASTER ONLY Submits an import command to the specified handler using specified options. This command can only be run if the minion is configured with solr.type=master

handler : str
The name of the data import handler.
host : str (None)
The solr host to query. __opts__['host'] is default.
core : str (None)
The core the handler belongs to.
options : dict (__opts__)
A list of options such as clean, optimize commit, verbose, and pause_replication. leave blank to use __opts__ defaults. options will be merged with __opts__
extra : dict ([])
Extra name value pairs to pass to the handler. e.g. ["name=value"]

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.full_import dataimport None music {'clean':True}
salt.modules.solr.import_status(handler, host=None, core_name=None, verbose=False)

Submits an import command to the specified handler using specified options. This command can only be run if the minion is configured with solr.type: 'master'

handler : str
The name of the data import handler.
host : str (None)
The solr host to query. __opts__['host'] is default.
core : str (None)
The core the handler belongs to.
verbose : boolean (False)
Specifies verbose output

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.import_status dataimport None music False
salt.modules.solr.is_replication_enabled(host=None, core_name=None)

SLAVE CALL Check for errors, and determine if a slave is replicating or not.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.is_replication_enabled music
salt.modules.solr.lucene_version(core_name=None)

Gets the lucene version that solr is using. If you are running a multi-core setup you should specify a core name since all the cores run under the same servlet container, they will all have the same version.

core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return: dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.lucene_version
salt.modules.solr.match_index_versions(host=None, core_name=None)

SLAVE CALL Verifies that the master and the slave versions are in sync by comparing the index version. If you are constantly pushing updates the index the master and slave versions will seldom match. A solution to this is pause indexing every so often to allow the slave to replicate and then call this method before allowing indexing to resume.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.match_index_versions music
salt.modules.solr.optimize(host=None, core_name=None)

Search queries fast, but it is a very expensive operation. The ideal process is to run this with a master/slave configuration. Then you can optimize the master, and push the optimized index to the slaves. If you are running a single solr instance, or if you are going to run this on a slave be aware than search performance will be horrible while this command is being run. Additionally it can take a LONG time to run and your HTTP request may timeout. If that happens adjust your timeout settings.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.optimize music
salt.modules.solr.ping(host=None, core_name=None)

Does a health check on solr, makes sure solr can talk to the indexes.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.ping music
salt.modules.solr.reload_core(host=None, core_name=None)

MULTI-CORE HOSTS ONLY Load a new core from the same configuration as an existing registered core. While the "new" core is initializing, the "old" one will continue to accept requests. Once it has finished, all new request will go to the "new" core, and the "old" core will be unloaded.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str
The name of the core to reload

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.reload_core None music

Return data is in the following format:

{'success':bool, 'data':dict, 'errors':list, 'warnings':list}
salt.modules.solr.reload_import_config(handler, host=None, core_name=None, verbose=False)

MASTER ONLY re-loads the handler config XML file. This command can only be run if the minion is a 'master' type

handler : str
The name of the data import handler.
host : str (None)
The solr host to query. __opts__['host'] is default.
core : str (None)
The core the handler belongs to.
verbose : boolean (False)
Run the command with verbose output.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.reload_import_config dataimport None music {'clean':True}
salt.modules.solr.replication_details(host=None, core_name=None)

Get the full replication details.

host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.replication_details music
salt.modules.solr.set_is_polling(polling, host=None, core_name=None)

SLAVE CALL Prevent the slaves from polling the master for updates.

polling : boolean
True will enable polling. False will disable it.
host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.set_is_polling False
salt.modules.solr.set_replication_enabled(status, host=None, core_name=None)

MASTER ONLY Sets the master to ignore poll requests from the slaves. Useful when you don't want the slaves replicating during indexing or when clearing the index.

status : boolean
Sets the replication status to the specified state.
host : str (None)
The solr host to query. __opts__['host'] is default.
core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to set the status on all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.set_replication_enabled false, None, music
salt.modules.solr.signal(signal=None)

Signals Apache Solr to start, stop, or restart. Obviously this is only going to work if the minion resides on the solr host. Additionally Solr doesn't ship with an init script so one must be created.

signal : str (None)
The command to pass to the apache solr init valid values are 'start', 'stop', and 'restart'

CLI Example:

salt '*' solr.signal restart
salt.modules.solr.version(core_name=None)

Gets the solr version for the core specified. You should specify a core here as all the cores will run under the same servlet container and so will all have the same version.

core_name : str (None)
The name of the solr core if using cores. Leave this blank if you are not using cores or if you want to check all cores.

Return : dict<str,obj>:

{'success':boolean, 'data':dict, 'errors':list, 'warnings':list}

CLI Example:

salt '*' solr.version