SPLK-2002 Premium Exam Questions

According to the Indexer Clustering Administration Guide, configuration bundles pushed from the Cluster Manager (Master Node) overwrite the contents of the $SPLUNK_HOME/etc/slave-apps/ directory on each search peer (indexer). However, Splunk provides a special persistent location — the _cluster app’s local directory — for files that must survive bundle redeployments.

Specifically, any configuration files placed in:

$SPLUNK_HOME/etc/slave-apps/_cluster/local/

will persist after future bundle pushes because this directory is excluded from the automatic overwrite process.

This is particularly useful for maintaining local overrides or custom configurations that should not be replaced by the Cluster Manager, such as environment-specific inputs, temporary test settings, or monitoring configurations unique to that peer.

Other directories under slave-apps are overwritten each time a configuration bundle is pushed, ensuring consistency across the cluster. Likewise, master-apps exists only on the Cluster Manager and is used for deployment, not persistence.

Thus, the _cluster/local folder is the only safe, Splunk-documented location for configurations that need to survive bundle redeployment.

References (Splunk Enterprise Documentation):

• Indexer Clustering: How Configuration Bundles Work

• Maintaining Local Configurations on Clustered Indexers

• slave-apps and _cluster App Structure and Behavior

• Splunk Enterprise Admin Manual – Cluster Configuration Management Best Practices

The Splunk Enterprise Security and Encryption documentation specifies that the primary mechanism for securing data in motion within a Splunk environment is to enable TLS/SSL encryption between forwarders and indexers. This ensures that log data transmitted from Universal Forwarders or Heavy Forwarders to Indexers is fully encrypted and protected from interception or tampering.

The correct configuration involves setting up signed SSL certificates on both forwarders and indexers:

On the forwarder, TLS settings are defined in outputs.conf, specifying parameters like sslCertPath, sslPassword, and sslRootCAPath.

On the indexer, TLS is enabled in inputs.conf and server.conf using the same shared CA for validation.

Splunk’s documentation explicitly states that this configuration protects data-in-transit between the collection (forwarder) and indexing (storage) tiers — which is the critical link where sensitive log data is most vulnerable.

Other communication channels (e.g., deployment server to clients or browser to Splunk Web) can also use encryption but do not secure the ingestion pipeline that handles the indexed data stream. Therefore, TLS should be implemented between Splunk forwarders and indexers.

References (Splunk Enterprise Documentation):

• Securing Data in Transit with SSL/TLS

• Configure Forwarder-to-Indexer Encryption Using SSL Certificates

• Server and Forwarder Authentication Setup Guide

• Splunk Enterprise Admin Manual – Security and Encryption Best Practices

 The following is a good practice for a search head cluster deployer: The deployer must be used to distribute non-replicable configurations to search head cluster members. Non-replicable configurations are the configurations that are not replicated by the search factor, such as the apps and the server.conf settings. The deployer is the Splunk server role that distributes these configurations to the search head cluster members, ensuring that they have the same configuration. The deployer does not only distribute configurations to search head cluster members when they “phone home”, as this would cause configuration inconsistencies and delays. The deployer does not distribute configurations to search head cluster members to be valid configurations, as this implies that the configurations are invalid without the deployer. The deployer does not only distribute configurations to search head cluster members with splunk apply shcluster-bundle, as this would require manual intervention by the administrator. For more information, see Use the deployer to distribute apps and configuration updates in the Splunk documentation.

Adding more search peers and making sure forwarders distribute data evenly across all indexers will provide the most search performance improvement when the distributed deployment is approaching its capacity. Adding more search peers will increase the search concurrency and reduce the load on each indexer. Distributing data evenly across all indexers will ensure that the search workload is balanced and no indexer becomes a bottleneck. Replacing the indexer storage to SSD will improve the search performance, but it is a costly and time-consuming option. Adding more search heads will not improve the search performance if the indexers are the bottleneck. Rescheduling slow searches to run during an off-peak time will reduce the search contention, but it will not improve the search performance for each individual search. For more information, see [Scale your indexer cluster] and [Distribute data across your indexers] in the Splunk documentation.