Build & Data Platform Engineering
Design and implement scalable data platforms and pipelines on whichever of our two core ecosystems matches your strengths — AWS (with Databricks) or Azure/Fabric (with Databricks or native Fabric tooling) — with exposure to other environments (Snowflake, GCP) considered a plus. This includes developing reliable batch, streaming, and near-real-time pipelines using technologies such as Spark and Delta Lake, and building ingestion, transformation, and curation workflows for both structured and unstructured data.
You will implement modern data architectures including lakehouse patterns and medallion layering (bronze, silver, gold) within Databricks or Fabric, ensuring systems are reusable, scalable, and aligned with enterprise needs.
Enable AI, GenAI & Data Products
Deliver high-quality datasets that support analytics, machine learning, causal modeling, and optimization systems. You will enable data pipelines for GenAI use cases (including LLMs, RAG pipelines, and vector-based data flows), as well as agent-based architectures and intelligent workflows, ensuring that data is model-ready and production-grade — leveraging Databricks’ MLflow and Unity Catalog, or the equivalent Azure/Fabric and Azure ML tooling, depending on the platform you work in.
Data Modeling, Orchestration & Automation
Design scalable logical and physical data models for analytical and operational use cases, ensuring consistency across domains. Orchestrate workflows using tools such as Airflow, dbt, Databricks Workflows, Azure Data Factory, or equivalents, with strong focus on automation, reliability, and maintainability of end-to-end pipelines.
Architecture, Governance & Observability
Apply modern architecture patterns including event-driven and streaming architectures, and ensure adherence to best practices in data governance, lineage, quality, and access control (RBAC/ABAC), using tools such as Unity Catalog and AWS Lake Formation, or Microsoft Purview, depending on the platform in use.
Establish strong data observability, including monitoring of data freshness, pipeline reliability, and SLA adherence, ensuring systems remain trustworthy and production-ready.
Data Serving, Integration & Optimization
Enable data serving layers (APIs, feature inputs, analytical endpoints) to support downstream systems, including ML and AI platforms. Continuously monitor and optimize pipelines and infrastructure for performance, scalability, and cost efficiency across our core cloud ecosystems.
Requirements Discovery & Business Partnership
Bring genuine curiosity to every engagement — ask the right questions to understand not just what stakeholders are asking for, but why. Apply common sense thinking, detailed analysis, and experience-based recommendations to help derive and refine business requirements, surfacing gaps or better alternatives where they exist rather than simply executing a brief as written.
Work closely with data scientists, ML engineers, analysts, and business stakeholders to translate requirements into robust data solutions. Support adoption of data products and contribute to best practices across the data and AI ecosystem.