Deep dive into Sitecore’s xDB

After I had to deal with xDB and trying to understand what it exactly is I will share you a compact summary of my learning

What is xDB? 

The Experience Database (xDB) is the central big data marketing repository for the customer experience.

xDB translates all collected data from a single user’s visit and interactions across the website into something more meaningful for marketers – and that silently in the background.

xDB is the key for creating customer experiences through techniques like personalization. 

  • It gathers all the interactions of visitors to build a view of the customer’s experience
  • It can collect the data from other platforms like ERP, CRM, customer service etc.
  • It is also available on the cloud to provide the huge capacity to store the data

You can use xDB with Sitecore XP, if you don’t want to take the advantages of xDB or it is not necessary, than Sitecore XM is your choice.

But:

Decision against xDB limits your ability to use the elements of Sitecore’s experience marketing features. However, in-session personalization is working, but personalization based on historical data is unavailable.

So you can personalize some pages based on contact’s current visit. You can also disable xDB and xDB tracking in XP environment via configuration (see Configuration settings)


About data and xDB

To better understand the xDBs approach I summarized the types of personalisation of content. In my opinion there are two ways of data to personalize:

Not collectedCollected
Based on not really known data with scoring, profile/pattern cards, hits on pages will be scored and that lead to profilesUsing collected real information (also historical) about the user has supplied to personalize their experience

When you have the data you can guide the user through a specific path via Marketing automation

Basically everything gets collected by xDB what you can see e.g in Google Analytics.

Also generic page information (when and what pages are hit), Geo Location – where the user is located, the user’s device and even more.

What makes xDB also valuable:

  • Integration of other systems – e.g CRM (Salesforce etc.) with the ability of  bi-directional communication
  • Pushing xDB data to CRM – For example the user which is calling the support was watching Nike shoes on the shop site which helps to advise him
  • Also sales can enrich data like last call and push back to xDB to personalize content again (discount e.g because they know that there person called in)

Also Mobile, IOT, Chatbot interfaces can go to xDB to retrieve and enrich experience data.


Application roles

Generally there are two xDB application roles

  • xDB Processing – analyses and aggregates collected data
  • xDB Reporting – retrieves reporting data from various data sources

xDB Processing

Processing refers to the analysis of collected data, and is performed by Processing servers. Processing includes the following activities:

  • Interaction aggregation
  • Contact processing
  • Historical re-aggregation (Rebuilding the reporting database)
  • Maintenance of the reporting database
  • Distributed processing (for example, allowing scheduled tasks to execute on the processing server)

Aggregation is specific to the grouping of interaction data. Contacts are processed, not aggregated.

xDB Reporting

Includes:

The Reporting database, which is populated by the interaction aggregation pipeline.

A Reporting API, which is used by the Content Management server to populate Experience Analytics reports.

The Reporting Service, which is an optional but recommended core role that sits in between clients and the reporting database.

How you can access the reporting database:

  • Use T-SQL directly
  • Use the Reporting API without a Reporting Service, in which case the request is sent directly to SQL via a provider
  • Use the Reporting API with a Reporting Service, in which case the request is sent over HTTPS

 In a Sitecore context, you should use the Reporting API.


Storage roles

There are five xDB databases which store all the custom experience data:

xDB Collection database

Contains all contacts and interactions, including facets and events

xDB Processing Pools database

Stores work items with IDs for newly created contacts and interactions. Work items are added by the xConnect Collection service role and consumed by the xDB Processing role during live aggregation processing. Acts as a retry mechanism for live aggregation, history aggregation and distributed processing by storing work items with IDs for contacts and interactions that could not be processed and should be retried.
Work items are added by and consumed by xDB Processing role.

xDB Processing Tasks database

Stores processing tasks related to history aggregation and distributed processing

xDB Reference Data database

Contains marketing reference content for all xDB data, such as definitions and taxonomies

xDB Reporting database

Contains data that has been aggregated or reduced by xDB Processing


xDB index

The xDB index contains contact and interaction data and is updated by the xConnect Search Indexer

Contains also personal data when index facets marked [PIISensitive].

This index can be hosted on

  • Solr
  • Azure Search

Session State

Session state is a real important part of the Sitecore XP for two reasons:

  • As contacts browse a website, data about their interaction is stored in session state until the end of the session, when it is committed to xConnect
  • It is possible for a single contact to access a website with two devices concurrently – XP must be able to handle sharing data across two active sessions

When contacts browse a website, data about their interactions with Sitecore is stored in session state until the end of the session, when it is committed to xConnect.

xConnect then updates information about the session, and any changes in the contact’s data, in the analytics and contact database.

In the real life it is possible for a single contact to have two or more sessions on the same website at the same time. This can happen when a contact is using different devices or multiple browsers on the same device.

You must have a session state server to keep track of all of the different sessions for one contact.

Configuring the session state is particularly important if you deploy a multi-server environment with multiple processing servers or clusters of CD servers

During a session, Sitecore writes data to two session state stores:

PrivateShared
Private session state contains information about contact visit information, such as pages viewed, goals converted, or campaigns triggered.Private session state is ‘private’ to the browser being used to access the website.Shared session state contains information that is ‘shared’ across potentially multiple active sessions.This includes any contact information that has been loaded into the tracker at the start of the session.

Use the xConnect Client API to update a contact with an active session.


Configuration settings

SettingNotes
Xdb.EnabledEnable or disable tracking in the xDB on the reporting server.To run in CMS-only mode, you must set this setting to false to prevent the recording, reporting, and aggregation of data in the xDB.
Xdb.Tracking.EnabledEnable or disable tracking on the Content Management and Content Delivery roles.

References

https://doc.sitecore.com/developers/82/sitecore-experience-platform/en/xdb.html

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s