Over the years we have been involved in various Data Migration projects
either as a stand alone exercise or as part of a larger piece of project work.
Migration has often been treated as a Cinderella part of a larger project but in
reality it is crucial to the success of any project. Also interface projects to transfer data between systems. The underlying
principles are true in all projects and over the years success has come from the
- Planning - detailed
data mapping is so important and an understanding of all transformations
- Cleansing - within
existing systems it is best to ensure the data's integrity prior to migration.
Also to ensure that data is kept in a good state it is sometimes important to
tighten up validation on the legacy systems.
- Change Control - if
an existing system has to be maintained during the Migration project then it
is important to ensure that the migration project is kept up to date with what
is happening in the other systems.
- Testing - A large
number of test migrations should be performed and validated. An automated
migration procedure can be defined to ensure a reliable process.
- Validation - both the
target system has to be tested with migrated data and construction of reports
to ensure that all data is transferred.
performed over the year include the following:-
- Livingston Rental
1999 - Oracle using automated unix scripts to transfer and transform data to a
new database structure. With automated scripts tests were run at least 3 times
each week to create test data for the main project.
- BUPA 2000 - Data
transfers between a cqcs system and mainframe and also a Point CRM system.
- Hays Mail Services -
various projects to transfer data to SAP out of a system (written by Peter
Chadburn in cqcs) also data links to CRM Point system. Also data transfer and
re-structures into Sales management system.
- Express Dairies -
various database re-design projects to improve performance which required data
migration. The result was a reduction in weekend batch processing from 36 to
- NetSuite projects -
formatting of data given by clients to ensure that their existing data was
loaded in the correct format. Also the development using ASP/VBScript sub
system to process XML exports from NetSuite and also to create XML import
files that can be imported into NetSuite.
- Public Guardianship
Office - Data Cleansing of existing old database to ensure quality addresses
using the Quick Address with programs written to re-format data.