What are the steps of migrating telecommunication network data into an inventory system and how can you prepare the data? How to plan this process and what should you look out for? Our experts shared some good practices of data migration. They will help you avoid errors and smoothly complete the migration so that you can make use of all network information as fast as possible. Learn how to effectively migrate network data from different sources into an inventory system.
Along with the growing network, there is an increasing amount of related data. At some point in the network development, information is so vast that storing it in different, scattered sources is no longer feasible and constricts everyday work.
- Planning connections of new customers to the network
- Handling outages
- Selling services
- Updating reports to the Office of Electronic Communications (UKE)
The answer to this problem is implementing a telecommunication network inventory system. Buying such a system is only the beginning, as it also has to be powered by data.
This task often poses a challenge for network operators. Entering collected data into a network inventory system isn’t complicated but it is tedious. The main challenge is finding a way to smoothly transfer data and ensure that no files are missing. It’s also important to keep the database up-to-date.
The key is creating an effective plan as well as being systematic and accurate. Knowing this, what should be the steps in the data migration process?
1. Organizing and preparing data
2. Maintaining
data
3. Loading
data
4. Verifying
data
5. Supplementing
data
Step 1 – Organize and prepare data
Before moving data into a system, it needs to be organized and you have to create its inventory. To make it easier, you can divide this step into 3 smaller chunks:
Collect all data
Gather all information in every format you use (digital files, paper documents). At this point, you can’t know whether the data is complete, accurate, and readable enough to identify network elements and connections. This is only a preparation for creating the inventory.
Make a data inventory – learn what kind of data you have
Create a table to register each file and document. Define the characteristics of each dataset:
- data range,
- network layer,
- area related to data,
- data format,
- document type, e.g. paper documents, digital files,
- the order of data loading,
- migration time (at this stage you don’t have this information, so fill this column in later on. This record will be useful when estimating the costs).
- the planned date for loading data (more about this in the 3rd step),
- files or related documents,
- data completeness,
- data accuracy.
The range of information that needs to be registered depends on your experience in this type of activity.
Verify what data formats were used in the past and whether they’re suitable for automatic loading. Data might need to be entered manually. It can usually be categorized into three groups:
Electronic documentation suitable for automatic migration:
- as-built projects (dxf, dwg, shp)
- tabular data summaries (xls, csv, text)
- maps of internal network structures (kml, kmz)
Electronic documentation that cannot be automatically transformed into objects:
- map scans
- schematic diagrams (vision, corel, graphic files)
- rectified schematics (visio, corel, graphic files)
- coupler connection schemas (visio, corel, graphic files)
Paper documents:
- as-built projects
- network schemas
- coupler connection schemas
- physical inventories
Besides document types mentioned above, consider additional documentation related to particular network elements, e.g. photos, lease agreements, acceptance reports, etc.
Verify data completeness
Verify all data per each network layer. Now it’s also recommended to estimate its completeness and consistency. Answer the following questions:
- Do you have complete data? What is it related to?
- Is all information up to date?
- Do you need to add missing data before loading it or can you do it later?
Fill in the answers in the table for each item. Try to make the verification as accurate as possible.
Step 2 – Create a procedure for data maintenance/update
Your network works non-stop and changes all the time: you connect new users, fix outages, and expand or reconfigure flows. The moment you first enter data into the inventory system, you have to maintain it and keep it up to date. Otherwise, your data will quickly become outdated and useless.
The best practice is to prepare an update procedure and implement it before migration.
Step 3 – Create a plan for loading data into the inventory system
The procedure should define:
- Situations when you need to introduce changes to the inventory system,
- The way in which changes should be reported,
- Persons responsible for introducing changes to the inventory system.
The procedure should be performed at every stage: before, during, and after the data migration as well as during regular network operations. This is why it should be simple and intuitive for every person responsible for reporting network changes. It’s worth it to designate an employee (or several employees) responsible for the control and verification of the inventory system data.
Look through the data you prepared in step 2 and define a plan for its uploading. The order for entering datasets into the system depends on many factors, usually:
- Your data types – first you can load data in electronic format (e.g. external network projects in DWG or DXF formats) because it is easier. Afterward, load the rest of the data.
- Data relevance – load the most needed information first, for example, if you plan to connect new users, data about the related network fragment is the most crucial at the moment.
- Order of entering data defined in the inventory system.
The plan for loading data should be noted in the data inventory table. Now you need to define the order of entering data into the system.
There are many possible scenarios depending on your data quality, business goals, current investments, etc. The most popular model is loading data according to the diagram below.
- Data import from DWG
- Import from tables
- Manual modeling of missing data
- cables, splices and racks
- importing connections from tables
- or manual crossover
If your network is large enough to require dividing into fragments, data migration can also be split into geographic areas. A lot depends on your data type. You might import the entire external network from several DWG files and then enter internal elements. Another option is importing the entire data for a given network fragment and then moving on to the next one.
Step 4 – verify data completeness and accuracy again
After entering network data into the system, you still need to verify its accuracy and compare it to the source data or network status. Errors may occur and if you don’t notice them at this stage, they will negatively impact your inventory system quality and network operations.
Errors may occur for several reasons:
- The data was incorrect or incomplete,
- The network was mostly modeled manually and manual data entry is more error-prone,
- During the migration, network changes appeared and weren’t included in the process.
Plan to perform data verification after every larger data loading process. Select random data samples to get a full overview of the entire network. With time, data may become outdated. Every dataset sample needs to be verified in terms of data accuracy and data completeness.
You should use several statistical samples for accuracy verification. Their number and size depend on the size of the analyzed dataset. You can find methods for statistical accuracy estimation in related literature or on the Internet (search for “statistical quality control”).
Step 5 – Add the missing data
After creating a data inventory you have a full view of all your data. Some of it may be still incomplete or outdated so this is the best time to add the missing information. This way, your system will operate using a fully up-to-date set.
Updating information often requires a visit to the field and assessing the network status. This work can be performed during regular maintenance operations. There are tools that can help you speed up data collection, minimize errors, and increase data quality.
Data collection and updating can be also treated as separate projects. The downside to this approach is that it can get time-consuming in the case of large datasets. Moreover, data collected at the beginning of this process can become outdated before we get to the end. It also requires additional work from technicians.
Regardless of your chosen approach, it’s crucial to include this step in the migration process. This way you will ensure that the data in the inventory system is up-to-date.
What’s next?
This is the last step – remember about careful planning and meticulous implementation.
A well performed migration process will help you use the full potential of the inventory system and your network operations will be much more effective.
