Migrating from Ektron to Sitecore: Media Library

So you wanna migrate from Ektron to Sitecore …

Overview

Your client has chosen to migrate from Ektron to Sitecore. How can the data be migrated?

In this series, I’ll describe the process that our team recently went through for migrating Ektron content to Sitecore without a content migration tool, without the Ektron API, but by diving right into the database.

Be sure to read the first 4 posts of this series before reading this one.  The links are at the bottom of this post.

About Our Project

We worked with Ektron Version: 9.10 SP1 (Build 9.1.0.184).

Ektron Library

The images, videos, and files that make up the Ektron (media) Library are physically stored in two folders at the root of the website, /uploadedImages and /uploadedFiles.

(pic)

There is a reference to these media items in the “Library” area of the Ektron admin tool and it contains some metadata fields.  Below, the Title and Description fields are editable and saved within Ektron.

Ektron media metadata

Migration Options

Once you have the Ektron media files on your local system, you have these options to getting the media into Sitecore:

Alternate Text

Unlike Sitecore, the alt tags are only saved to the reference link when the media is consumed by some content. (Sitecore allows you to save a default alternate text with the media item as well as allowing you to add it as a property when linking to it within a content item).

By default, Sitecore’s Image template has the Alternate Text field set to required so you’ll have the validation errors unless the field is populated.

alt text

Powershell Extensions to the rescue once again! If the Ektron library does not provide a meaningful value in the Title field, it is auto-populated with the filename.  Using Powershell Extensions in Sitecore, you can iterate through your uploaded images and populate the Alt field with the filename.


Check out all posts for this series:

Advertisements

Migrating from Ektron to Sitecore: Suggested Field Mappings

So you wanna migrate from Ektron to Sitecore …

Overview

Your client has chosen to migrate from Ektron to Sitecore. How can the data be migrated?

In this series, I’ll describe the process that our team recently went through for migrating Ektron content to Sitecore without a content migration tool, without the Ektron API, but by diving right into the database.

Be sure to read the first 3 posts of this series before reading this one.  The links are at the bottom of this post.

About Our Project

We worked with Ektron Version: 9.10 SP1 (Build 9.1.0.184).

Content Table

Let’s do a deeper dive into the columns that make up the dbo.content table of Ektron.

Below is a mapping of Ektron fields to potential Sitecore template fields that our project used and a short explanation if needed.

Ektron column Sitecore field Note
content_id “Legacy Content Id” Include on a new _Imported interface template which is referenced on each Sitecore page template
content_language  __language You’ll need to do additional data massaging.  For instance, 1033 = “en”
content_title Item Name
(and possibly Display Name)
 For the item name, make sure to massage out all the characters that your Sitecore settings don’t support.
NOTE: If there is an alias set up for this content_Id (see UrlAliasMapping table in Ektron), use that value for the Item Name).
content_html
(XML field)
*All template-specific fields
(and possibly Display Name)
This field is the xml of the template-specific fields.
date_created __Created Ektron date format:
2016-09-21 03:09:08.000
Sitecore format:
20160921T030908Z
See conversion tip below.
last_edit_lname __ Updated by
__Owner
Concatenate the first and last name to form match a Sitecore userid.
last_edit_fname __ Updated by
__Owner
Concatenate the first and last name to form match a Sitecore userid.
last_edit_date __ Updated Ektron date format:
2016-09-21 03:09:08.000
Sitecore format:
20160921T030908Z
See conversion tip below.
content_teaser Meta Description
and possibly Teaser
This field may or may not be populated by the content author.
go_live_date Published Date Include as an interface template field if desired.
end_date Archive Date As mentioned in a previous post, if this field is populated, upon save, Sitecore moves this item to the Archive table (out of the master table).
image Meta Image This field may or may not be populated by the content author.

Conversion tip for Dates

The Ektron columns with datatype of DateTime will store the datetime in this format:
2016-09-21 03:09:08.000
Sitecore expects the dates to be in this format: 20160921T030908Z.

Assuming you are using Entity Framework to extract the data, the date fields will be available in a DateTime format.  Use the following to convert it to a Sitecore datetime string.

internal string ConvertDateToString(DateTime date)
 {
 var sitecoreDateTime = 
Sitecore.Common.DateTimeExtensions.SpecifyKind(date, DateTimeKind.Utc);
 return Sitecore.DateUtil.ToIsoDate(sitecoreDateTime);
 }

IMPORTANT: Dates stored in the content_html column are typically stored in a string format.  You’ll have to convert them to a datetime format and then use the method above to convert them to a Sitecore datetime string.

Conversion tip for Statistics

Within the Sitecore.Data.Items.ItemEditing API class, you’ll either use EditContext or .Editing.BeginEdit() method.   if you use the statements above without additional parameters, the Updated and Updated By fields will be auto-populated by Sitecore, even if you set them while modifying the item.

You can turn off this functionality by setting the ‘updateStatistics’ optional parameter to false.


Check out all posts for this series:

Migrating from Ektron to Sitecore: Migration Tips

So you wanna migrate from Ektron to Sitecore …

Overview

Your client has chosen to migrate from Ektron to Sitecore. How can the data be migrated?

In this series, I’ll describe the process that our team recently went through for migrating Ektron content to Sitecore without a content migration tool, without the Ektron API, but by diving right into the database.

Be sure to read the intro and Ektron database overview posts first.

About Our Project

Our assignment was to convert two Ektron websites to Sitecore; one ported over with the same look and feel as the existing site (Project A); the other, a complete redesign with the existing content (Project B). Throughout the blog series, I’ll reference the two projects as Project A and Project B.
NOTE:  We worked with Ektron Version: 9.10 SP1 (Build 9.1.0.184).

Dependencies

There is alot of planning necessary when migrating content from one system to another.  After deciding what is going to be migrated to the new system, a migration order must be established.  In our project, much of the content was dependent on three areas:  the media library, taxonomy list and the People business area. (The client is a law firm so its people are its primary asset and they are tied to other business areas of the site like events, articles, and blogs). So, in the case of both of our projects, taxonomy is imported first, followed by media library, followed by People.

Migration Tip #1 – Migrate a Legacy Content Id to each Sitecore item

When building the Sitecore templates, make sure to include a single-line text field to house the Ektron Content ID.  This serves several purposes … it is used in migration to convert an Ektron relationship between two content items to a Sitecore relationship.  Also, with this ID, you can confidently re-run your migration script as many times as necessary without concern of having duplicate content within Sitecore (by always searching for the content ID before creating a new Sitecore item).

Migration Tip #2 – Retain the Ektron database indefinitely

This is important if the client chooses not to migrate 100% of the content.  Later, there may be a request to retrieve some unmigrated data.  Create several screenshots of the Ektron content tree so that the client has a familiar view for picking out content that wasn’t migrated so you can target it by folder path.  Then, use the content_folder_tbl’s FolderPath column to find the content.

Migration Tip #3 – Create a 301 redirect list while migrating

If your project is like mine, where several of the urls are changing, have an output of your migration be a mapping of old Ektron website url to new Sitecore website url and add these lists to your Sitecore Redirect Manager.

Migration Tip #4 – Update the Sitecore statistics with Ektron information

When saving imported content from Ektron, be sure to grab the admin fields for the Ektron content and update the __Created, __Created by, __Updated, and __Updated by fields.  See the next blog post for details on the Ektron to Sitecore mapping.

Migration Tip #5 – Add the alternate text to images and the title text to links at migration time

Like Sitecore, Ektron allows you to set the alternate text of an image when the image is referenced on a page.  Make sure to bring this over and if it’s not populated, suggest to the client that this is populated at content migration time with the image name (without the extension).

Migration Tip #6 – Fix the internal links in the rich text fields

Run your migration scripts a second time targeting the internal links that are in the rich text field.  You can use the 301 redirect table from tip #3 to assist.

Migration and Helix

As mentioned in the previous post, we opted to access the database directly within our Visual Studio project.

I started with a new project in the Foundation solution folder to house the the Entity Framework connection and some base classes that would change infrequently.

Namespace:  Foundation.ContentMigration.General

In this solution, I installed Entity Framework via the NuGet Package.

Install-Package EntityFramework

This Foundation project contains the EktronContentEntities model and a BaseRespository.cs class, a base class from which all business unit specific repositories inherit.  This contains the following:

  • instantiation of the EktronEntities class
  • pre-loading of the taxonomies from Sitecore
  • optional pre-loading of People from Sitecore
  • a simple method that pulled a list of items from Sitecore based on template from the index
  • methods to get taxonomies and people by Ektron content id

Each Feature project that has migrated content has a Content Migration repository class that inherits from this BaseRepository class.

I outline the contents of this feature project in the next blog.


Check out all posts for this series:

Migrating from Ektron to Sitecore: Ektron Database Tables

So you wanna migrate from Ektron to Sitecore …

Overview

Your client has chosen to migrate from Ektron to Sitecore. How can the data be migrated?

In this series, I’ll describe the process that our team recently went through for migrating Ektron content to Sitecore without a content migration tool, without the Ektron API, but by diving right into the database.

Be sure to read the intro post first.

About Our Project

Our assignment was to convert two Ektron websites to Sitecore; one ported over with the same look and feel as the existing site (Project A); the other, a complete redesign with the existing content (Project B). Throughout the blog series, I’ll reference the two projects as Project A and Project B.
NOTE:  We worked with Ektron Version: 9.10 SP1 (Build 9.1.0.184).

The Decision on Content Extraction Method

There are four primary ways to extract data from Ektron:

  • Third party migration mapping tool
  • Ektron’s Localization – Extract for Translation tool
  • Ektron Framework API
  • Extraction from SQL database

In my experience with a migration mapping tool, they come with a bit of a learning curve and tend to be a bit inflexible.  As a developer who likes to get her hands dirty, the cons outweighed the pros.

The Extract for Translation tool provides a third party with content for easy language translation and keeping the HTML in tact.  The XML format was a bit overwhelming and frankly not usable for content migration.  Here’s an example of the export:

translation tool

That left the two options that provide the most freedom and flexibility to the developer.  We evaluated using the Ektron Framework API; it seemed that GetItem() may meet the needs but ultimately ended up going with the extraction from the database because Entity Framework makes working with relational database tables a breeze.

Ektron Database Fundamentals

Below I discuss important fields and primary tables of the database where we extracted data.

Ektron Content Id

In Sitecore, the item’s guid is the unique identifier for an Item.  In Ektron, it is the Content Id.  Having the parent folder’s Content Id for each major business unit (e.g. products, orders, articles, contacts, etc)  of the site is the key to accessing the content from the database.  The admin tool is useful for this purpose.  In Ektron, navigate to the parent folder of the business unit and view the properties to retrieve the parent folder’s content id.

folder properties content id

 Content Table

The Ektron Content table is the center of the Ektron universe, much like the Sitecore Item table.  It contains the base content that you want to extract and import into Sitecore.

Let’s take a look at the table:

content table columns

Some of the fields are self-explanatory and their usage in  Sitecore greatly depends on how they were used in Ektron.  For instance, there two columns called “content_teaser” and “content_title” … these may or may not be consumed by the Ektron website code.

In the image above, I’ve highlighted some important columns.

content_html – This field contains most of the content of the page, in XML format that was built in Ektron with a Smart Form.
Example:
xml example

folder_id – This field is populated if the content is logically nested underneath a folder in the Ektron content tree.  Using the content id of the folder of a business unit (as described above), you can simply query for all children to get the content.

SELECT *
 FROM [EktronDB].[dbo].[content] 
 where folder_id = 17179882623 -- the parent folder's content id
 order by content_title desc

end_date – This field is populated when the content has been archived within Ektron. Eventually, it can be mapped to the Archive Date field Careful with this one … If this field is mapped to the Archive date field in Sitecore right away, the content will be moved to the Archived database.

Taxonomy Tables

The Ektron Taxonomy Tables hold the relationships between the content and the taxonomy structure.  For instance, if a piece of content contains a select list field of US States, this is where you’re going to find all states related to that content.

taxonomy tables

Here’s an example query:

SELECT c.*, ti.*, tx.*
 FROM [EktronDB].[dbo].[content] c 
 JOIN [EktronDB].[dbo].[taxonomy_item_tbl] ti on ti.taxonomy_item_id = c.content_id
 JOIN [EktronDB].[dbo].[taxonomy_tbl] tx on ti.taxonomy_id = tx.taxonomy_id
 where c.content_id = 10737421044 -- insert your content id
 and tx.taxonomy_language_id = 1033 -- 1033 = English in Ektron

Metadata Tables

The Ektron Metadata tables have the potential to hold a variety of data for a piece of content.  In our project, the metadata stored the checkbox value for which callouts were displayed in the sidebar for a piece of content. It also was used to store some standard content fields that were eventually displayed on the website.

meta tables

Example query:

SELECT c.content_id, t.meta_name, m.meta_value
 FROM [EktronDb].[dbo].[content] c
 JOIN [EktronDb].[dbo].[content_meta_tbl] m on c.content_id = m.content_id
 JOIN [EktronDb].[dbo].[metadata_type] t on t.meta_type_id = m.meta_type_id 
 where folder_id = 1719881363 -- insert your content id
 and m.meta_value <> ''
 and m.active=1

UrlAliasMapping Table

Since we are dealing with webforms and most Ektron detail pages used this url convention, url.com/?id=171881363, we opted to take advantage of the alias table which the client used to provide friendly urls in marketing materials.

urlalias.png

Example query:

SELECT *
 FROM [EktronDb].[dbo].[content] c
 JOIN [EktronDb].[dbo].[UrlAliasMapping] u on c.content_id = u.targetid
 where c.folder_id = 17179869710

Content Folder table

The content_folder_tbl is handy to look up the folder structure that the Ektron admin has.  It contains the folder IDs, folder paths, event the folder ID paths for each folder.

content folder table

Example query:

SELECT * [folder_id]
 ,[folder_name]
 ,[FolderPath]
 ,[FolderIdPath]
 FROM [EktronDB].[dbo].[content_folder_tbl]
 where FolderPath like 'Lists%'
 order by FolderPath asc

 


Check out all posts for this series:

Migrating from Ektron to Sitecore: Introduction

So you wanna migrate content from Ektron to Sitecore?

Overview

Ektron has merged with Episerver and clients of Ektron may find themselves at a crossroads when their Ektron contract is up for renewal.  Do they stay with Episerver or move on to another CMS?

Sitecore, a leading web content management and experience management platform, is certainly a candidate to consider in this decision, with its smart Experience Platform.  However, what is involved in content migration?

There are content migration tools on the market (like GatherContent) but is there an automated way of migrating over content without a tool?

In this series, I’ll describe the process that our team recently went through for migrating Ektron content to Sitecore without a content migration tool, without the Ektron API, but by diving right into the database head first.

The Project

Our assignment was to convert two Ektron websites to Sitecore; one ported over with the same look and feel as the existing site (Project A); the other, a complete redesign with the existing content (Project B). Throughout the blog series, I’ll reference the two projects as Project A and Project B.
NOTE:  We worked with Ektron Version: 9.10 SP1 (Build 9.1.0.184).

We were given access to the following (all necessary):

  • Ektron administration interface
  • Ektron code (webforms)
  • Ektron SQL database.

Because we were switching from Ektron web forms to Sitecore MVC, the code was valuable only to obtain the HTML and existing business logic for Project A.  The code would be even more useful if we were required to convert to webforms but, as a dying technology, we opted to not go with webforms.

Let’s start with getting to know Ektron.

Ektron Components

Ektron admin tool

Ryan Bailey has written an overview of Ektron’s structure and key concepts to consider when migrating from Ektron to Sitecore.  I recommend that you read it before continuing.

http://blog.ryanbailey.co.nz/2015/07/migrating-from-ektron-to-sitecore.html

To elaborate a bit on what Bailey has written in his “From the top down” section of Ektron components, I’ve broken down how we used each section.  NOTE: Because the site was being overhauled in Project B, most of the sublayouts and templates from Ektron were not useful for Project B.

Pagebuilder template

For Project A, we used the pagebuilder templates HTML markup in the code to build renderings that supported placeholders (i.e. One column rendering, two column rendering).

Widgets and Smart Forms

For Project A, as Bailey mentions, these were handy to port over the HTML, replacing the ASP.NET controls with MVC razor, utilizing Glass Mapper.   The templates themselves were not useful.

Menus, Taxonomies & Collections

For the simple menu for Project A, we simply observed the menu directly on the website and replicated it within Sitecore.

The taxonomy structure in Ektron was replicated (via content migration) as it is where values for dropdowns and content relationships in Ektron.  For instance, this is where the list of US States would be stored if needed by the website.

Collections are curated lists of existing content.  This functionality was used only for Project B (the complete overhaul) so it wasn’t a candidate for automated content migration. If this is used extensively, the content id could be mapped to the Sitecore guid and the lists be recreated in Sitecore.

Library

The library is similar to Sitecore’s media library but also stores Hyperlinks (internal links between two pieces of Ektron content). The media for Ektron is physically stored in the website root within two folders:  /uploadedFiles and /uploadedImages.  The downfall of Ektron is that references to the library items is path-based, not Id-based so re-organizing media content in Ektron is not friendly.


Check out all posts for this series:

SUGCON 2017 – My Highlights

I had the most fortunate opportunity to fly over from the US to Amsterdam to attend SUGCON 2017 (Thank you Verndale!). Wow, do the Dutch and Danes put on a great conference!  Although I’m still curious about whole walnuts and the shot of juice? served in the late afternoon on the last day, I learned a boatload about Sitecore and upcoming hot topics.  Here’s a summary of some of the sessions that I attended:

Keynote

Lars Nielson and Pieter Brinkman opened the conference with some inspiring words and asking us all to push the boundaries of Sitecore.

OData Content API

There was talk on Sitecore Services Client and its OData Content API; consolidate and reduce the number of technologies (and security risks) to handle HTTP calls.

SXA overview and HoloLens and Robbie and Alexa

hololens

We were given a demo on how quickly SXA can be used to spin up a website; in this case, a website that visually shows all user groups throughout the world, on a map.  But that’s not all … how about throwing in some HoloLens integration and Sitecore Cloud and the map becomes a globe and the globe can be spun, user groups can be chosen and opened for more detail.  Next, let’s add a connection (via a JSON Sitecore device) to a robot named Robbie and integrate it with Amazon’s Alexa … “Alexa, Ask Sitecore to find all user group events.”

What else?  Well, Robbie can see things; Robbie can see your face … Robbie has learned how to read emotions via facial recognition.  The future says to set up Sitecore personalization based on the user’s emotion.  Cool.

This spawned possible answers to questions like:

  • What factors are driving desirable behavior most strongly?
  • How can we prevent the loss of existing customers?
  • What audience behavioral segments are we unaware of? We could target them, or banish them.
  • Can we optimize the sequence of tactics in our lead nurturing program?
  • Can we automate up-selling and cross-selling?
  • Can we make our content authoring processes more efficient?

Sitecore Integration with Microsoft Cognitive Services

mcs

This is the session I was most excited about.  Back in March, for the Sitecore Hackathon, our group used Data Exchange Framework to tap into Microsoft Cognitive Services and pull in tags for images.  It wasn’t until after we were done that I learned that Mark Stiles had already been hard at work creating a layer to easily integrate Sitecore and these services … and he presented its possibilities at the conference.

Within his “Sitecore Cognitive Services,” there are three bits of functionality: the Bot Framework, Sitecore Cognitive Services Service, and the Sitecore Cognitive Services module.

The Bot Framework is machine learning at its best.  Program a chatbot to gradually learn how to properly respond to your customers questions.

The Service is a fast way to integrate your website with the plethora of features Microsoft Cognitive Services offers, like Computer Vision, Emotion Recognition, Facial Analysis,  Video Analysis, even Speech Analysis.  There’s so much to tap into.

The Module is taking this a step further and integrating all of this into the Sitecore Admin.  Faceted image searches, textual analysis of text within images, facial analysis of people within Sitecore images.  Yes, machine-generate the alt text for your images, recognize if an image is too, ahem, adult, what colors are used in the image?

The possibilities seem endless.  By the way, Mark hasn’t tapped into all of the possible Microsoft Cognitive Services functionality; if you’re interested in helping out, contact him.

xDB Contacts and Concurrency Control

Ohhhh … lots of yummy take-aways from this session as well.

For instance, did you know that there are four sets of APIs for accessing the xDB contact data? … which is in a number of locations/states, I may add …

  • Tracker.Current.Contact (in-memory)
  • ContactRepository (access the data in the Collection database)
  • SharedSessionStateManager (access the data in the Shared Session database)
  • ContactManager (utilize both the ContactRepository and SharedSessionStateManager)

Dmytro Shevchenko also discussed the need for a shared session (i.e. for being able to maintain a single contact from interactions coming from multiple private sessions).

A great illustration offered was the summary of Contact Lock stages.

contact lock states

In the initial state, the latest Contact data exists in the Collection database.

When the Session starts, the Contact is locked in the Collection database and copied and loaded  (and unlocked) in the Shared Session database.

On Page Request, the Contact is still locked in the Collection database, and also locked in the Shared Session database but loaded into the Tracker API at this time.

When the Page Request ends, the Contact is removed from the Tracker, it is saved and unlocked in the Shared Session database, and still locked (and not updated) in the Collection database.

At the end of the session, the Contact is no longer available in the Shared Session (of course) and is saved and unlocked in the Collection database.

Well, that’s my summary of highlights from Day 1.  I’ll be posting Day 2 highlights after my extended trip to Paris.

Sitecore, SolrCloud and ZooKeeper Server Relationship

solrcloudzk

Recently there have been conversations online regarding the proper configuration of a SolrCloud/ZooKeeper with Sitecore setup.  This post will walk through the diagram above which illustrates a possible configuration of the ensemble.

ZooKeeper

The purpose of ZooKeeper is to maintain, in memory, the current state of each Solr server, the index configuration information stored in ‘z-nodes’, and also ensure that there is always a ZooKeeper leader via a leader election process.  Using in-memory to store this information allows for low latency and high throughput when accessing.

In the diagram above, there are 3 ZooKeepers to maintain server requests.  A primary principle of ZooKeeper is to maintain a majority, or quorum.  That is the recommended minimum as 3 zk servers tolerates 1 zk server being down; 5 zk servers tolerates 2 zk servers being down at the same time.  An odd number of servers (greater than 1) is recommended.

All ZooKeeper servers are known each Solr node and stored in the solr.xml file in the zkhost node, comma-separated. Example of 3 zk nodes:

<str name="zkHost">55.55.55.1:2181,55.55.55.2:2181,55.55.55.3:2181</str>

A good detailed reference on ZooKeeper (not in conjunction with Solr/Sitecore) is: https://www.hakkalabs.co/articles/apache-zookeeper-introduction.

SolrCloud

The number of SolrCloud servers (aka nodes) in the example of above (2) is demonstrating two replica servers.  In SolrCloud, there is no master or slave(s). Instead, a server is a replica and/or shard, which means the data on the server is duplicated on another node (replica) and/or the entire index is divided amongst the servers and a server contains a portion, or shard, of the data.  Typically sharding is reserved for indexes that are too large to fit on a single server, however, sharding without necessarily maxing out the capacity of the servers may be a performance strategy as well.

Load balancer

Contrary to some information out there, a load balance between SolrCloud/Zookeeper and Sitecore would not ‘load balance’ the ZooKeeper nodes.  The load balancer address would be in Sitecore’s configuration file, however, the load balancer is balancing the index and query requests into the Solr nodes.

In comments below, please feel free to put in additional relevant high-level information and I can add it to this post.