Migrating to a Document Management system: What to Take into Account and How to Do It

Any document management project goes through a series of phases that are more or less difficult, depending on the capacity of the implemented product and the experience of the team that sets it in motion.

Throughout our more than 10 years of experience specialising in helping our clients with their document management problems, at Athento, we have found that one of the most critical steps when implementing a document management system (DMS) or an ECM software platform is the migration of the documents and their metadata.

On this matter, we come across different situations:

1. The client already has a document management system implemented.

2. The client does not have a DMS or ECM, but has the documents or their metadata in files within a shared file system or spreadsheets with their metadata.

Based on our experience, the first situation is certainly the more complicated one, though it may seem quite the opposite. This is because it is necessary to ensure backward compatibility of everything there is. That’s to say, we have to ensure that the client can continue to access their old data—without losing information—from the new tool.

To migrate documents to a ECM platform, a series of key tasks must be done:

1.- An initial analysis to assess the situation of the documentation, what metadata it has, the format of the original files, and the volume of documentation.

On this point, it is key to carry out the analysis as if it were a business intelligence project. It is necessary to obtain information on the volume of documents from different perspectives: by document type, by location, by author, and by date. This makes the subsequent quality process easier. Afterwards, we can check using different means that all of the information has been migrated, as the volume indicated in the analysis will coincide the volume in the new software. Likewise, it is necessary to have “master tables” that are used to feed information into the document metadata, especially if some type of transformation of this data will be done afterwards.

2.- The design and development of the migration scripts, which will be in charge of completing the following operations:

• Obtain information from the original source.
• Store it in an intermediate point to be able to use it and not depend on the original source.
• Carry out the transformation processes on the metadata.
• And finally, load the documents.

3.- The design and development of the scripts that will measure the quality of this migration.

4.- Necessary measures must be taken to control the speed of the migration, the transformation of metadata, and quality processes. Normally, all projects have a tight deadline. In large migration processes, this becomes even more critical. The migration of 2 million documents can last for a minimum of 4 weeks, depending as always on the available infrastructure. Network speeds and disc reading and writing speeds are key in this process.

5.- Quality testing in “UAT” mode.

Other considerations of interest when it comes to migrating a document management system are:

• It is advisable to choose the size of the document packets to migrate carefully. This will depend on the total volume of documents, though a good size is 50,000 documents per packet.
• Explore methods for large-scale uploading of documents and metadata to the document manager. In the series of articles “How to Upload Documents and Their Metadata to the DMS on a Large Scale”, we told you how the CSV uploading method and the unattended CSV uploading method work, the advantages of each method, and practical cases that each one is applicable to.
• Make a realistic plan after the analysis of the data. It is important to know the real circumstances, risks, and challenges that you are going to face in the migration in order to be able to plan it with certainty. It is difficult to make a realistic migration plan knowing only the volume because there are other factors, such as transfer speeds, that are going to directly impact the migration.
• It is recommendable to “phase” the migration. For example, start with the most current documentation, open files, current contracts, etc. That’s to say, place priority on the information that is critical to the business. If there is a lot of information to migrate, the best strategy might be to migrate the newest documentation first. This way, the system can begin to be used in production before the migration is complete and without impacting users’ work.
• Save the original source (or a copy of it) for at least one month. This is a contingency management measure in case users are missing some document or information.

I hope this post helps all of you who are considering migrating documents to a new system. Although migrations are complex, by following best practices like what we have outlined above, we can carry it out successfully and within the estimated time frame.