Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 1 of 5

In-memory and columnar

11 min

Listen to this lesson

0:00 / 18:51

Every SAP S/4HANA system runs on SAP HANA. For a platform team, that is not just a product name on an architecture slide: it changes how the system is sized, how it behaves under load, how backups work and what "the database is slow" actually means. This course gives you the operating model. This first lesson covers the idea everything else rests on: SAP HANA keeps its working data in memory and stores most business tables by column.

Row storage versus column storage

A traditional database stores a table row by row: all the fields of one sales order item sit next to each other, then the next item, and so on. That is efficient when you read or write whole records one at a time. A column store keeps all the values of one column together instead: every plant, then every material, then every quantity. When a report asks "total quantity by plant for the year", the database reads only the two columns it needs and skips the rest.

AspectRow storeColumn store
Physical layoutAll fields of a record stored togetherAll values of one column stored together
Good atFrequent single-record reads and writesScans, aggregations and filtering across many rows
CompressionLimitedHigh: repeated values in a column are stored once
In SAP HANAUsed for a minority of tables with heavy single-row accessDefault and recommended for the vast majority of tables
Row and column storage compared. SAP training material describes the column store as the default table type as of SAP HANA 2.0 SPS 03.
The Sign-Off Table · thesignofftable.com/academy/sap-bas-102

Column storage compresses very well. SAP HANA keeps a sorted dictionary of the distinct values in a column and stores each row as a small integer that points into it. A column such as plant, with only a handful of values (1710 and 1720 at Contoso), shrinks dramatically. Compression is what makes it affordable to hold large databases in memory, and it is also why the compression rate, and therefore the memory needed, depends heavily on the data itself.

Dictionary compression of the plant column: each distinct value is stored once, and every row keeps only a small integer pointing into the dictionary.
The Sign-Off Table · thesignofftable.com/academy/sap-bas-102

Main storage, delta storage and the delta merge

Highly compressed, sorted data is fast to read but expensive to change. SAP HANA solves this by splitting each column table into a read-optimized main storage and a write-optimized delta storage. New and changed records go into the delta; reads combine both. Periodically a delta merge folds the delta into a new main storage and recompresses it. By default SAP HANA decides when to merge on its own, through a background process called mergedog that looks at delta size, available memory and time since the last merge.

  1. WriteMaria posts a goods issue; the new rows land in the delta storage of the affected column tables
  2. ReadQueries read the main storage and the delta together, so the new rows are visible at once
  3. Merge decisionThe automatic merge process sees the delta has grown and schedules a merge
  4. MergeMain and delta are combined into a new, recompressed main storage; a fresh delta takes new writes meanwhile
  5. PersistThe new main storage is written to disk and the old structures are dropped
How the column store absorbs changes. A merge temporarily needs extra memory because old and new main storage coexist.
The Sign-Off Table · thesignofftable.com/academy/sap-bas-102

Platform teams notice merges in two ways. Merges cost CPU and, for a short time, extra memory, so very large loads (such as Contoso's Wave 2 data loads for company code 1910) can show merge activity spikes. And an alert that a table's delta has grown past a threshold is a sign that merges are not keeping up and deserves a look before it becomes a memory problem.

The life of a column table's delta storage: writes grow it, the automatic merge folds it into a new main storage, and a delta that keeps growing raises an alert.
The Sign-Off Table · thesignofftable.com/academy/sap-bas-102

Why SAP S/4HANA could simplify its data model

Older SAP ERP systems on disk-based databases stored the same information several times: line items, plus totals tables, plus secondary index tables kept only to make reports fast. Because SAP HANA can aggregate line items on the fly, SAP S/4HANA removed many of those redundant structures. The best-known example is the universal journal (table ACDOCA), which holds financial and controlling line items together; inventory movements were similarly consolidated in a single material document table. Many old totals and index tables survive only as compatibility views so that existing reports still read them.

Fiori apps and reportsAsk business questions such as "open items by customer" or "stock by plant"
CDS viewsDefine the data model and push calculations down into the database
Line-item tablesSingle source of truth, for example the universal journal
Column store in memoryCompressed columns make on-the-fly aggregation fast enough to drop most stored totals
Why fewer tables: aggregates are calculated from line items at query time instead of being stored separately.
The Sign-Off Table · thesignofftable.com/academy/sap-bas-102

What would you do?

Dan, the finance process owner, asks Sam why the new CMQ system does not need the nightly job that rebuilt Contoso's custom totals table in ECC. What is the best explanation?

Key points
  • SAP HANA keeps working data in memory and stores most business tables by column.
  • Column storage compresses well and makes scans and aggregations fast.
  • Changes go to a delta storage that is merged into the main storage, usually automatically.
  • Fast aggregation let SAP S/4HANA drop many totals and index tables, for example through the universal journal.

Check your understanding · not graded

Lesson check

  1. 1. Why does a column store make "total quantity by plant" fast?

  2. 2. Where do new and changed records first land in a SAP HANA column table?

  3. 3. What is the universal journal an example of?

  4. 4. What does a delta merge temporarily need?