DataOps Overview

 Concepts Covered

• Continuous Data Delivery (Parellel to CI/CD in DevOps)
• Data Sources
  • Flat Files
  • Spreadsheets
  • Database Types (RDBMS, Graph, TimeSeries, NoSql, In-Memory)
  • Relational Database Management Systems (RDBMS)
    • • Primary / Foreign Keys - 1to1, 1toMany, ManytoMany relationships
      • Lookup Tables and Domains
      • SQL / Stored Procedures
      • Input Forms & Reports
  • APIs (REST)
  • Backend Data Access Programming
• Extract - Transform - Load (ETL)
  • Low Code / No Code Automation Tools for Managing Data Access, Correction, Alteration, Formatting, and Uploading
    • SAFE FME, AWS Glue / Catalog (MDM)
    • SQL, Stored Procedures (RDBMS)
    • Hadoop, Apache Spark (Time Series Analysis), AWS Redshift, Azure Synapse
    • Cognos, RAW, Informatica / Talend, AWS Insights
• Master Data Management (MDM)
  • Beyond Data Lakes of the Past (Scalability / Cost)
  • Data Mesh Architecture (Distributed - Federated, Decentralized Data Architecture)
  • Metadata
    • Access / Use Restrictions
    • Data Quality Metrics
  • Application Connectivity