Troubleshooting list indexing issues

  • The List Manager application creates and manages lists that other applications and modules use, for example, EXM. List Manager uses sitecore_analytics_index as a data source. During processing, the sitecore_analytics_index populates from the analytics database. List locking ensures that all contacts in the list are indexed and ready. Lists lock after every add ,remove, or clear operation. A special agent checks if all operations have finished and unlocks the lists.

    In Sitecore XP 8.2 Update-2 and later versions, you can disable list locking by enabling the \App_Config\Include\ListManagement\Sitecore.ListManagement.DisableListLocking.config.disabled file.

    This article describes the most common issues that can make list indexing stuck, describes how the list can be re-indexed, and contains the information that Sitecore Support requires to troubleshoot issues with the List Manager.

    Note: The ability to disable List Locking was introduced in Sitecore XP 8.2 and List Locking has been deprecated in Sitecore XP 9.0. However, you must still be aware of the way that lists behave when they are edited concurrently. For more information, see the following article: List Locking.

  • Scenario: List indexing does not finish.

    What can be checked and fixed:

    • Ensure that at least one server performs the processing role. Configuration files with Sitecore.Analytics.Processing.* names are enabled on the server.
    • If there are several environments (testing/development/production, and so on), each environment must have its own set of Mongo databases.
    • Check that the server did not restart during the list upload process.
    • Make sure that the CD servers are configured properly and the processing role is disabled for them. The easiest way to check it is to make sure that the Sitecore.Analytics.Processing.Aggregation.Services.config file is disabled. If the file is enabled, please follow this article to configure the servers properly.
    • Make sure that the sitecore_analytics_index uses the Sitecore.ContentSearch.Maintenance.Strategies.TimedIndexRefreshStrategy strategy.
    • Make sure that your dedicated EXM dispatch server does not perform the processing role. The Sitecore.Analytics.Processing.Aggregation.Services.config file must be disabled.
    • If you use Lucene as your search provider for the sitecore_analytics_index, there must be only one Content Management server. This server must perform the processing role. You cannot have any dedicated processing servers.
    • If there is a dedicated processing server or more than one Content Management instance, you must use SOLR as your search provider.
    • Make sure that the Analytics.ClusterName setting value is set on the Content Delivery servers as described in this article.
    • If you have more than one Content Delivery server, make sure that the Shared Session state is stored in an out-of-process storage. for example MongoDB, SQL, or Redis. Refer to this documentation section for details.

    After checking the listed suggestions, try to re-index the list as described in the following How To Re-Index The Affected List section. If re-indexing does not help or the issue happens frequently, please refer to the Troubleshooting Data section.

  • To re-index the list:

    1. Install the patch that corresponds to your Sitecore XP version:
    2. Open the /sitecore/admin/listmanager.aspx page.
    3. Select the affected list from the drop-down menu and click Check.
    4. Depending on the results:
      • If the number of contacts in the index is smaller than the number in the analytics database, click the Re-index button and wait for the process to finish.
      • If the number of contacts in the analytics database is smaller than the number in the index, it may mean that some contacts were removed manually from the analytics database or the data was not re-indexed after changing. If the data wasn't removed from the database manually, click the Remove obsolete contacts button. If the data was removed manually, you will need to rebuild the reporting database.
      • If the number of contacts in the analytics database and index is the same, but differs from the number on the list item, locate the list item under the /sitecore/system/List Manager/All Lists one and change the Recipients field to the correct value. In most cases, the list must be unlocked after 15 seconds. Otherwise, you can unlock it manually from List Manager dashboard.

    Note: If the number of contacts in the analytics database differs from the number of contacts on the list item, the issue is most likely caused by incorrect code that was used to add contacts to the list. If a contact is added to a list using the analytics API (by modifying contact tags), it is important to increment the Recipient field value of the list item in the master database. This action can be difficult to perform on Content Delivery servers where the Master database is not available. For scaled environments, you must use the EXM API, which performs all needed operations.

  • If you performed the previous actions and the issue still exists and is reproducible, contact Sitecore Support and provide the following information:

    • The way that you add contacts to the list (List Manager UI/custom code/EXM API, and so on).
    • Is the issue reproducible for all lists?
    • Does the number of indexed contacts always remain at 0 of n, or do some contacts get processed?
    • The output of the /sitecore/admin/listmanager.aspx page after you execute the Check operation for the affected list (see the previous How To Re-Index The Affected List section).
    • What servers are present in the environment and their roles.
    • Basic information from each server.
    • The code that you use to add contacts to a list and an archive of the \bin folder if the contacts are added to the list using custom code.
  • Sometimes, when troubleshooting list indexing issues, you must enable extended logging to find the root cause of the problem. This section is optional and you can perform the following steps, if a support engineer requests this from you:

    1. In the \App_Config\Sitecore.config file (for XP 8.1 and later) or the \Web.config file (for XP 8.0), change the priority value to DEBUG for the <root> node:
        <priority value="DEBUG" />
        <appender-ref ref="LogFileAppender" />
    2. In the \App_Config\Include\Sitecore.ContentSearch.config file, change the level value to DEBUG for the Crawling logger:
      <logger name="Sitecore.Diagnostics.Crawling" additivity="false">
        <level value="DEBUG"/>
        <appender-ref ref="CrawlingLogFileAppender"/>
    3. Enable the \App_Config\Include\Sitecore.ContentSearch.VerboseLogging.config file.

Applies to:

CMS 8.0 - 8.2

November 13, 2017
December 06, 2018


  • Search and Indexing