Lesson 3 of 5
ETL and ownership
8 min
Listen to this lesson
Every migration, whatever the tool, does three things: it extracts data from the source system, transforms it into the shape the target needs, and loads it. Each step has different people, risks and checks.
- ExtractSelect the right records from the source (for example open items only, active customers only)
- CleanseFix duplicates, gaps and errors, ideally in the source system
- TransformApply mapping rules: old values to new values, new required fields, new data model
- LoadLoad into the target with validation; record errors
- ValidateReconcile counts and values; business checks by data owners
Extract
Extraction defines what moves. Selection rules matter as much as the technology: active customers only, or every customer with an open item or a sale in the last two years? Open purchase orders only, or also those partially received? Each rule needs a business owner's agreement, because it decides what the new system will know on day one.
Transform
Transformation makes source data fit the target. It includes value mapping (old payment terms code to new code, old plant number to new plant), structural changes (legacy customer and supplier records become business partners in SAP S/4HANA), enrichment (filling fields that are new or newly required) and cleansing rules applied in flight. Write every rule down in a mapping specification that the data owner approves. If a rule is not written down, it cannot be tested.
| Transformation | What it does | Alternadores del Norte example |
|---|---|---|
| Value mapping | Translates old codes to new codes | Legacy payment term CR30 to payment term 0004 in SAP S/4HANA |
| Structural change | Reshapes records for the new data model | Legacy customers and suppliers become business partners |
| Enrichment | Fills fields that are new or newly required | Mandatory business partner fields the legacy ERP never held, completed |
| In-flight cleansing | Applies agreed cleansing rules during the move | Standardized address formats for Flotillas Monterrey S.A. de C.V. |
Load
Loading pushes the transformed data into SAP S/4HANA through a tool that applies the same validations as the normal application, so that loaded data behaves like data created by users. For most objects, that tool is the SAP S/4HANA Migration Cockpit, covered in lesson 5.
Who owns what
The single most important role in migration is the data owner: a business person accountable for the correctness of a data object. Not IT, not the migration specialist. IT can move data; only the business can say whether it is right.
| Role | Responsibilities | Contoso |
|---|---|---|
| Data owner | Accountable for correctness; approves selection, mapping and cleansing rules; signs off validation | Dan for financial balances and open items in company code 1910; Maria for customers and open sales orders |
| Data steward / cleanser | Performs cleansing in the source; fixes data quality issues | Alternadores del Norte's finance and sales operations analysts |
| Migration specialist | Designs and runs extraction, transformation and load; reports errors and statistics | Migration team |
| Functional consultant | Defines target design and required fields; explains load errors | Order-to-cash and finance consultants |
| Test lead | Plans validation and reconciliation; tracks migration defects; owns evidence | Priya |
| Cutover manager | Places loads in the cutover plan; times mocks; runs the go-live sequence | Luis |
Migration roles and responsibilities.
What would you do?
Contoso's migration specialist proposes mapping all of Alternadores del Norte's legacy customer payment terms to a single new term, "Net 30", to simplify the load. Who should decide?
The mapping specification
The mapping specification is the contract between the business, the migration team and the testers. For each migration object it should contain:
- The selection rule: which source records are in scope, and why.
- The field mapping: each target field, its source field or rule, and whether it is required.
- The value mappings: tables of old values to new values, such as payment terms or units of measure.
- The cleansing rules applied in the source or in flight.
- The validation checks that will prove the object loaded correctly, such as counts and totals to reconcile.
- The owner's approval, with a date and a version number.
Testers design migration tests directly from this document, which is why an unapproved or out-of-date specification is a testing problem as much as a migration problem.
- DraftMigration specialist and consultant write selection, mapping, value and cleansing rules
- ApproveData owner, such as Dan or Maria, approves with a date and version number
- Design testsTesters build counts, totals and sample checks from the approved version
- Change controlAny rule change creates a new version, re-approved, and the affected tests are updated
- Extract selects, transform reshapes, load inserts with validation; cleansing and validation surround them.
- Write every selection, mapping and cleansing rule in a specification the data owner approves.
- The business data owner is accountable for correctness; IT moves the data.
- Sign-off records name the data owners who approved rules and validated results.
Check your understanding · not graded
Lesson check
1. Who is accountable for the correctness of migrated customer data?
2. What is value mapping?
3. Why must mapping rules be written down?
4. Where is cleansing ideally done?