Data Management and Data Governance
Context: FIT1043_MOC, FIT3003_MOC · the Governance stage of the value chain · protecting data as an asset · frames privacy and compliance
Quick Revision
- 🎯 Objective: manage and govern data as a valuable asset ➔ management = the internal lifecycle; governance = external usage/value.
- 📦 Core Components: management (plans/policies/practices over the lifecycle) | governance (access, protection, compliance, usage).
- ⚡ Key Constraint: the exam split — data management = internal data lifecycle; data governance = usage/value with legal & ethical considerations.
📝 How It Works
1. Why Manage Data
- Value ➔ data is valuable; collection is time-consuming and hard.
- Scale ➔ large volumes generated at high growth rate, from multiple sources (business docs, ERP systems).
- Continuous ➔ management changes over time as new technology/services arrive.
2. Data Management (internal lifecycle)
- Definition (DAMA) ➔ development of architectures, policies, practices, procedures to manage the data lifecycle.
- In short ➔ development/execution/supervision of plans, policies, programs, practices that control, protect, deliver and enhance the value of data assets.
- Strategies ➔ retention period, access mechanism, archive storage, data format across lifecycle phases; plus security risk assessment and mitigation.
3. Data Governance (external usage/value)
- Focus ➔ narrowly on access, protection, compliance, and usage — to bring maximum business benefit.
- Issues it handles ➔ how data is organised/protected/accessed; who may access which portion (privacy/confidentiality); who manages and is accountable; policies for compliance (GDPR, PDPA).
4. At warehouse scale (FIT3003 W12)
- Security ➔ authentication and least-privilege access · encryption at rest and in transit · auditing and monitoring · masking or restricting sensitive data.
- Data quality ➔ completeness and validity checks · duplicate detection · consistency across sources · timely detection of pipeline failures ➔ Data Cleaning (Dirty Data).
- Governance ➔ ownership and accountability · metadata, documentation and lineage · retention and deletion policies · compliance with privacy requirements (the deck cites the Australian Privacy Principles).
- Shared responsibility ➔ managed cloud services provide security features, but the organisation stays responsible for configuring them and for how data is used ➔ Cloud Data Warehousing, Data Lakehouse.
⚖️ Core Decision Matrix
| Data Management | Data Governance | |
|---|---|---|
| Scope | internal data lifecycle | external usage / value |
| Concern | store, protect, deliver, enhance value | access, protection, compliance, accountability |
| Question | how do we handle the data? | who may use it, and under what rules? |
When It Flips: the two are complementary — management runs the lifecycle inside the organisation; governance decides who uses the data and how, balancing business value against legal/ethical constraints (the "conflicting objectives").
🧠 Active Recall
Differentiate data management from data governance (internal vs external perspective).
Answer
- Short answer: Data management is the management of the internal data lifecycle (architectures, policies, practices to control/protect/deliver/enhance value); data governance focuses on usage — access, protection, compliance, and accountability — to maximise business value within legal/ethical limits.
- Why: Lifecycle vs value chain ➔ management is inward (handling the data); governance is outward (who uses it and under what rules).