A significant share of the budget and schedule losses in ERP projects comes not from the software, but from dirty data carried over from the legacy system. Three separate customer accounts for the same customer, stock codes that each use a different unit of measure, invalid tax numbers: when these are migrated to the new system as they are, an expensive ERP turns into a tool that speeds up the legacy system's mistakes.
In this guide, we cover ERP data cleansing and master data management step by step. You will learn which data to cleanse first, how to detect quality problems, how to plan your migration strategy, and what to check before go-live, so you can make your project more predictable.
Why ERP Data Cleansing Determines the Fate of the Project
An ERP is a single backbone that all departments use for the same data. In a legacy environment, where each unit works with its own Excel file or small program, errors stay local. In an ERP, a single incorrect record spreads to purchasing, inventory, finance, and reporting at the same time.
Consider a realistic scenario: a distribution company is migrating 12,000 customer and vendor accounts from its old accounting software. The same dealer is registered three times as "ABC Gıda", "A.B.C. Gıda Ltd.", and "ABC GIDA LTD ŞTİ". In the new system, the dealer's balance is split across three accounts, credit limit checks don't work, and the sales report comes out wrong. The problem doesn't come from the ERP, but from the cleanup that wasn't done before migration.
When data quality is neglected, the typical consequences are:
- The go-live date being postponed again and again,
- Users not trusting the new system and continuing to keep parallel Excel files,
- Financial reconciliation discrepancies and closing delays,
- Automations and integrations being fed with faulty data.
Leaving data work to the end of the project is the most common mistake. The right approach is to include data quality in the project scope during the ERP selection and requirements analysis phase and to assign responsibility from the start.
What Is Master Data, and Which Data Should Be Cleansed First?
Master data consists of the core records that are used repeatedly in business processes and change relatively little: customers, vendors, products, chart of accounts, employees, warehouses, and so on. Transactional data (orders, invoices, stock movements), on the other hand, is generated continuously in relation to these records. If master data is wrong, every transaction built on top of it will be wrong too.
Separate master data from transactional data
The most valuable asset in a migration is master data; how much transactional data to migrate is a separate decision. In most projects, open items (unpaid invoices, open orders, current stock balances) are migrated, while historical transactions are archived. This distinction significantly reduces both the migration volume and the cleansing workload.
Priority order and typical problems
| Master data type | Common problems | Cleansing priority |
|---|---|---|
| Accounts (customers/vendors) | Duplicate records, invalid tax numbers, missing address and contact information | Very high |
| Product / stock cards | Multiple codes for the same product, inconsistent units, unused items | Very high |
| Chart of accounts and cost centers | Unused accounts left over from the old structure, breakdowns that don't fit reporting | High |
| Bills of materials (recipes) | Outdated components, incorrect scrap rates | High (for manufacturers) |
| Employee and authorization data | Departed staff still appearing as active, role confusion | Medium |
Customer/vendor and product cards come first in almost every project, because the sales, purchasing, inventory, and finance modules all depend on them.
Data Inventory and Quality Measurement: Where Do You Start Cleansing?
Before you start cleansing, you need to know how dirty what is. A plan based on guesswork leads to surprises in the final week of migration. The steps below are a simple method that works for small and mid-sized projects:
- List your sources: Identify all data sources, including the legacy ERP, accounting software, Excel files, e-commerce panel, CRM, and field apps. Shadow sources (personal Excel files) are usually the dirtiest.
- Get record counts: For each master data type, determine the total number of records, the number of active ones, and the number that have had activity in the last year. Inactive records can be removed from the migration list.
- Write the quality rules: For example, "a tax number must be 10 or 11 digits," "every product must have a single base unit," "an active account must have at least one piece of contact information."
- Run the rules against the data: Count violations with SQL queries, Excel filters, or simple scripts. Report the error rate for each rule.
- Classify the issues: Divide them into three groups: those that can be fixed automatically, those that require a business unit decision, and those to be deleted.
- Set targets: "One hundred percent clean" is not realistic; define acceptable thresholds for critical fields in advance and submit them for management approval.
Practical ways to catch duplicate records
When detecting duplicate accounts, don't rely on the company name alone. Compare multiple fields together, such as tax number, phone, e-mail, and normalized address. Stripping out "LTD," "ŞTİ," "A.Ş." and punctuation differences from names and running a similarity check will surface most pairs in the first pass. However, don't leave the merge decision to automation; accounting and sales should jointly approve which card will be the "master" card.
Master Data Management: Making Cleansing Permanent
Data that has been cleansed once will become dirty again within a few months if it isn't managed. Master data management (MDM) is the governance model that defines who creates data, under which rules, and with which approval. It doesn't require a large software investment; clear rules and responsibilities come first.
Data ownership
Every master data type should have a business owner: finance or sales operations for customer accounts, product management or purchasing for product cards. IT is the party that stores and protects the data; the business unit is responsible for its accuracy. Without this distinction, everyone uses the data but no one cleans it.
Coding and naming standards
Will product codes be meaningful or sequential? Will company names be written in capital letters? In what format will address fields be kept? The answers to these questions should be gathered in a one-page standard and built into the ERP as mandatory fields, dropdown lists, and validation rules. The less free text there is, the higher the quality.
Record creation and approval workflow
When a new customer or product card is created, require an existing-record search first, and then separate the person who creates the card from the person who approves it. This flow can be set up as a workflow inside the ERP. If your standard ERP doesn't support these rules, custom validation and approval steps can be added as part of ERP software development.
In setups where multiple systems (e-commerce, CRM, field sales) use the same customer and product data, it is essential to determine which system is the "source of truth." We covered this topic separately in our article on data integration and data consistency.
Migration Strategy, Transformation, and Reconciliation
How will you migrate the data?
The migration approach is chosen based on data volume, business continuity tolerance, and team capacity. Let's compare three common approaches:
| Approach | How it works | Advantage | Risk |
|---|---|---|---|
| Migrate as-is | Data is transferred to the new system without being cleansed | Starts quickly | Dirty data settles into the new system and becomes harder to fix later |
| Cleanse and migrate (staging layer) | Data is cleansed and validated in a staging area, then transferred | Quality control and repeatability | Requires preparation time up front |
| Phased migration | Master data and open items first, then other data module by module | Risk is distributed, the team learns as it goes | Two systems must be managed during the transition period |
For most businesses, the most balanced option is to cleanse before migrating and, if necessary, to migrate in phases. Data integration work is decisive at this stage for managing the data flow from the legacy system to the target system and making it repeatable.
Mapping and trial migrations
Prepare a mapping document showing which field in the new system corresponds to each field in the old system. Write the transformation rules (for example, converting the old "KG" unit to the new "kilogram," or assigning a default value to an empty VAT rate) into this document so that no rule lives only in someone's head.
Run at least two or three trial migrations before the actual cutover. Reducing the error list with each trial both matures the scripts and shows how long the actual migration night will take. If you need a live data flow between the old and new systems, the system integration architecture should also be part of this plan.
Reconciliation: Do the numbers match?
After migration, saying "the data arrived" is not enough; you need to prove its accuracy. Make the following comparisons:
- Record counts: Do the numbers of records transferred in the source and target match, including rejected records?
- Financial balances: Do account balances and trial balance totals match the old system exactly?
- Stock values: Have quantity and cost balances been verified by warehouse?
- Sample check: Have business unit users checked randomly selected records on screen?
Recording the reconciliation results in a signed report makes the go-live decision objective.
Pre-Go-Live Data Checklist
Before the go-live date, every item below should be a "yes":
- A business owner has been assigned for each master data type.
- Quality rules have been written and error rates have dropped below the accepted thresholds.
- Duplicate customer and product records have been merged, and merge decisions have been documented.
- A decision has been made on whether inactive and unused records will be migrated.
- Field mapping and transformation rules have been documented.
- At least two trial migrations have been completed, and migration time has been measured.
- Financial and stock reconciliation has been signed off.
- The new record creation standard and approval workflow are active in the ERP.
- Read-only archive access to the old system and its backup are secured.
- A channel is ready to collect data correction requests for the first 30 days after go-live.
When migrating customer and employee records that contain personal data, also review your retention and deletion obligations under KVKK (Turkey's Personal Data Protection Law); not migrating unnecessary personal data to the new system reduces both risk and cleansing workload.
Frequently Asked Questions
How long does ERP data cleansing take?
The duration depends less on the number of records than on the quality of the data and how quickly you can make decisions. Tasks that require business unit approval, such as merging duplicate records, often take longer than the technical work. In general, we recommend starting cleansing in the early stages of the ERP project and adding it to the schedule as a separate work package.
Who should do data cleansing: IT or the business unit?
The technical work (queries, scripts, transfer) is carried out by IT or the consultant team; however, the business unit decides which record is correct. Accounting should validate customer accounts, and purchasing or product management should validate product cards. If this division of labor isn't established, cleansing turns into a purely technical exercise.
Do I have to migrate all historical data to the new ERP?
No. Usually master data, open items, and current balances are migrated; historical transactions can be kept in a read-only archive. You should decide by evaluating statutory retention periods and your reporting needs together with your accountant.
Is a separate MDM software required for master data management?
In small and mid-sized businesses, mostly not. Clear ownership, a coding standard, mandatory fields, and an approval workflow can also be set up inside the ERP. If multiple systems share the same master data and synchronization becomes complex, a dedicated MDM solution or an integration layer may come into play.
How do you maintain data quality after go-live?
Periodic quality reports (for example, a monthly check for duplicate records and missing fields), a record creation approval workflow, and regular meetings of data owners keep quality high. Building the rules into the system is far more sustainable than relying on human discipline.
Conclusion: ERP Data Cleansing Protects Your Investment
ERP data cleansing is not a side task of the project; it is a prerequisite for reaching the single source of truth that the software promises. Taking a data inventory, writing quality rules, assigning ownership, cleansing before migrating, and documenting reconciliation make go-live predictable. Master data management, in turn, preserves this gain after the migration.
At Tekno Danışman, we are by your side in ERP migration projects on data analysis, transformation rules, integration, and master data governance. Before deciding whether to move to an off-the-shelf ERP or whether you need a custom solution, you can review our off-the-shelf vs. custom ERP decision matrix. To evaluate the state of your current data together and prepare a migration plan tailored to you, contact us.
