• Analyze multiple structured and unstructured data sources to understand schemas, data quality, and relationships.
• Assess and document the metadata, formats, and business rules of incoming data sets.
Data Profiling & Matching:
• Perform comprehensive data profiling to assess completeness, accuracy, uniqueness, and consistency of data.
• Develop logic to match and reconcile records across different systems using keys, rules, or fuzzy matching techniques.
Master Data Management (MDM):
• Design and implement processes for building and maintaining golden master records.
• Support creation and maintenance of master data across domains such as customer, product, asset, etc.
• Handle data deduplication and standardization.
• Build scalable data pipelines using Azure Data Factory for ingesting, transforming, and cleansing data from multiple sources.
• Ensure efficient data movement across staging, processing, and analytical layers.
• Leverage Azure Synapse Analytics (optional) for advanced data processing and analytics
Collaboration & Governance:
• Work closely with business analysts, data stewards, and application owners to understand data definitions and ownership.
• Enforce data governance standards and best practices.
Performance & Optimization:
• Monitor data workflows and optimize performance for large-scale processing.
• Troubleshoot data issues and implement robust logging and monitoring.