Troubleshooting xDB data issues

This article describes the processes for troubleshooting the most frequent issues with Analytics data in the Sitecore Experience Database installation.

To use the troubleshooting instructions in this article, perform the test described in the Verify a connection to the collection database document and then choose one of the following scenarios according to the results:

Scenario

Contact or interaction data does not appear in the collection database after a contact visits the website.

Note that data does not appear in the collection database until after the session has timed out.

Refer to the How to check interactions stored in the collection database section to check if the new data is saved to the database.

To troubleshoot when data is not collected in the collection database:

For Sitecore XP 8.0 Initial Release or higher, check the Internet connection and network-related restrictions on the MongoDB server. Ensure that the firewall allows you to connect to the MongoDB server. For 9.0 Initial Release or higher, make sure that xConnect services are accessible from CD servers.
Ensure that both private and shared session states are configured according to the session state configuration scenarios.
Ensure that no CDN/cache services, such as Akamai, are running. All requests must be processed by the Sitecore instance to ensure that Sitecore Analytics tracks them.
Using a non-Sitecore session state provider for any private or shared session states requires you to ensure that the service supports the Session_End ASP.NET event.
Check whether the Global.asax file exists in the Website folder. If not, copy the default Global.asax file to the Website folder from the appropriate Sitecore installation archive.
If the custom application class is defined in the Global.asax file, ensure that this class is derived from the standard Sitecore.Web.Application class and does not override any standard logic.
Ensure that all the layout pages contain the <sc:VisitorIdentification runat="server" /> tag (@Html.Sitecore().VisitorIdentification() helper for MVC websites) in the <head> section.
Ensure that the ...\layouts\system path or files at this path are not added to the <setting name="IgnoreUrlPrefixes"...> setting in the Web.config file.
When using the Sitecore MVC, ensure that the Sitecore.MvcAnalytics.config file is not disabled.
If you have a multiserver xDB installation, ensure that all the servers are configured according to the configuration documentation.

Scenario

Experience Analytics does not show any data. The fact and dimension tables (Reporting database, SQL Server) do not populate with new Analytics data.

Known issues

[Fixed in XP 9.0.2] The processing might require more time due to the fact that the device detection database cannot be updated. To resolve this issue, see the following article https://kb.sitecore.net/articles/828414.
[Fixed in XP 9.1.0] If exmEnabled:define is set to 'No' in the web.config file, install the following patch https://github.com/SitecoreSupport/Sitecore.Support.215205/releases.

To troubleshoot when Experience Analytics data is out of sync or missing:

Ensure that the Interactions collection of the collection database is populated with new interactions. Refer to the How to check interactions stored in the collection database section to check if new interactions are collected. If new interactions are not added to the collection database, have a look at Scenario 1 of this article.
The Reference Data Service is accessible from CM.
Check the Internet connection and network-related restrictions on the reporting SQL database. Ensure that the firewall allows you to connect to the SQL server instance.
For Sitecore XP 8.0 Initial Release or higher, ensure that the Segment table of the reporting SQL database is not empty. If this table is empty, populate it with data from a clean Sitecore reporting database or redeploy reporting definitions through the /sitecore/admin/RedeployMarketingData.aspx administrative page.
Check and analyze how many records are in the processing pool by running the sequence of the same queries within a certain time limit. The increasing number of records signals the need for adjusted aggregation speed.

For XP 9.0 Initial Release or higher, run the following query against the Processing.Pools database :
```
SELECT COUNT(*) FROM [xdb_processing_pools].[InteractionLiveProcessingPool]
```
Click here to read more about processing performance optimizations.
For Sitecore XP 8.0 Initial Release or higher, run the following query against the tracking_live mongo database:
```
db.getCollection('ProcessingPool').find({}).count()
```
Click here to read more about configuration adjustments.

Note 1: Experience Analytics application caches report data by date range. This can lead to behavior so that the data does not appear in reports immediately. To force the application to take data from the reporting database, change the date range using the date picker in the top-right corner of the application. Use a date range that has not been previously applied (for example, by changing the upper or lower bound by one day).

Note 2: Experience Analytics "Online interactions by visits and value per visit" report shows data for online channels. If only this report does not show any data, the channel definitions may be not deployed. To address the issue the following steps must be performed:

Go to Launchpad > Control Panel > Deploy marketing definitions > select Taxonomies and run Deploy.
Try to create a new interaction, wait for session ending and aggregation and check the report again.
If the data for the new interaction is present, rebuild the reporting database to get the historical data.

Scenario

The Experience Profile is not updated with new information about visits.
New contacts do not appear in the Experience Profile reports.
The Experience Profile reports show a list of contacts, but you cannot open a contact's profile.
The search does not function properly.

To troubleshoot when Experience Profile data is out of sync or missing:

For Sitecore XP 8.0 Initial Release or higher:

Ensure that the MongoDB server is running and accessible for the Sitecore instance.
Ensure that the Contacts collection of the collection database (MongoDB) is populated with new contacts. To generate a new contact, visit the website using Firefox or Chrome private browsing mode. If the collection database cannot add a new contact, look at Scenario 1 of this article.
Ensure that sitecore_analytics_index data and collection database (MongoDB) data are consistent. If you have inconsistent data, perform a complete rebuild of the reporting database. This also updates the sitecore_analytics_index to match the collected data.

For 9.0 Initial Release or higher:

Ensure that the xConnect service is accessible from the Sitecore instance.
Ensure that the SOLR server used for xDB indexes is running and accessible from xConnect instances.
Ensure that the Contacts collection of the Shard databases is populated with new contacts.

Note 1:

For Sitecore XP 8.2 Update-3 and later, the Experience Profile application displays only identified contacts to avoid performance issues. Enable indexing of anonymous contacts for troubleshooting, if necessary. We recommend that you disable this in the production environment.

To enable anonymous contacts indexing:

[For XP 8.2 Update-3 or higher] Set the ContentSearch.Analytics.IndexAnonymousContacts setting value to "true" in the \App_Config\Include\Sitecore.ContentSearch.Analytics.config file.

[For XP 9.0 Initial Release or higher] Follow the instructions in this article.

Note 2:

In Sitecore 9, PII Sensitive data is not indexed by default. The Email, FirstName, and LastName fields that are used in the Experience Profile are marked as PII Sensitive. As a result, the search using these fields does not work by default. Check this article for details.

Scenario

The Path Analyzer does not display any data.
The Path Analyzer is not populated with new data.
The Path Analyzer data is out of sync with the Experience Analytics data.

To troubleshoot when Path Analyzer data is out of sync or missing:

Ensure that the Interactions collection of the collection database is populated with new interactions. Generate a new interaction by visiting any page on the website. If new interactions are not added to the collection database, look at Scenario 1 of this article.
Check if the TreeDefinitions table of the reporting SQL database has been populated. If this table is empty, populate it with data from a clean Sitecore reporting database or redeploy reporting definitions through the /sitecore/admin/RedeployMarketingData.aspx administrative page. After that, perform a rebuild of the Path Analyzer maps.
If the Path Analyzer data does not match the Experience Analytics data, perform a rebuild of the Path Analyzer maps.

To check that interactions appear correctly in the collection database, for all mentioned releases:

Start InPrivate browsing in Firefox or Chrome and visit a few webpages on the site.
Find the SC_ANALYTICS_GLOBAL_COOKIE cookie and save the value preceding the pipe. For example, retrieve the value "169e920af41d4960ad9eff4d3260f8d9" from the following cookie "169e920af41d4960ad9eff4d3260f8d9|True".
Request the Abandon.aspx page to flush the data immediately or close the window and wait for the end of the session.
[For XP 8.0 Initial Release or higher] Connect to the analytics database and run the following query:

db.getCollection('Interactions').find({ContactId: NUUID("169e920a-f41d-4960-ad9e-ff4d3260f8d9")})
//id retrieved from Cookie should go here. Add dashes to make the value correct GUID.

[For XP 9.0 Initial Release or higher] Connect to shard databases and run the following query against each of them:

SELECT * FROM [xdb_collection].[Interactions] WHERE ContactId IN
(SELECT LastKnownContactId  FROM [xdb_collection].[DeviceProfiles]
WHERE DeviceProfileId = '169e920a-f41d-4960-ad9e-ff4d3260f8d9') 
--id retrieved from Cookie should go here. Add dashes to make the value correct GUID.

One of the databases must contain the value.

Description

Scenario 1. Data is not collected in the Collection database

Scenario 2. Experience Analytics data is out of sync or missing

Scenario 3. Experience Profile Data Is Out Of Sync Or Missing

Scenario 4. Path Analyzer Data Is Out Of Sync Or Missing

How to check interactions Stored in the collection database