We are looking for an experienced Resident Solution Architect (RSA) – Data & Databricks to lead the design and delivery of scalable, cloud-based data and analytics solutions for enterprise customers. The role requires strong expertise in Databricks, Data Engineering, PySpark, Python, SQL, Lakehouse Architecture, and Cloud platforms.
The ideal candidate will combine deep technical expertise with strong customer-facing and consulting capabilities. You will work closely with enterprise stakeholders to understand complex business and technology requirements, design effective data solutions, lead architecture discussions and POCs, and provide technical direction to engineering teams.
Requirements
Responsibilities
Key Responsibilities
•
Design end-to-end Data, Analytics, and Lakehouse solutions using Databricks.
•
Develop and review scalable ETL/ELT pipelines and enterprise data platforms.
•
Work hands-on with Databricks, Apache Spark, PySpark, Python, and SQL.
•
Design solutions using Delta Lake, Unity Catalog, Databricks SQL, and Databricks Workflows.
•
Develop scalable data architectures across Azure, AWS, and GCP environments.
•
Conduct technical discovery sessions, architecture workshops, and solution discussions with enterprise customers.
•
Understand business and technical requirements and translate them into scalable architecture and implementation approaches.
Lead POCs, technical demonstrations, solution validations, and architecture assessments.
•
Provide technical guidance, mentoring, and architectural direction to Data Engineering teams.
•
Review data pipelines, architecture designs, code, and implementation approaches for scalability and maintainability.
•
Troubleshoot and optimise Databricks and Spark workloads for performance, scalability, reliability, and cost efficiency.
•
Support data platform modernisation, migration, and transformation initiatives.
•
Collaborate with Sales, Pre-Sales, Delivery, Product, and Engineering teams on technical solutioning.
•
Engage with senior customer stakeholders to communicate architecture decisions, technical recommendations, risks, and trade-offs.
•
Identify opportunities to improve data platform architecture, engineering practices, automation, and operational efficiency.
•
Stay current with developments across Databricks, cloud data platforms, distributed computing, and modern data engineering technologies.
What Makes You a Great Fit
•
8+ years of experience in Data Engineering, Data Architecture, Solution Architecture, Big Data, or a closely related field.
•
Strong hands-on expertise in Databricks and enterprise data platforms.
•
Strong knowledge of Apache Spark and PySpark.
•
Advanced programming skills in Python and SQL.
•
Strong understanding of Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks SQL, and Databricks Workflows.
•
Proven experience designing scalable ETL/ELT pipelines, data platforms, data models, and data warehouses.
•
Experience working with at least one major cloud platform: Azure, AWS, or GCP.
•
Strong understanding of distributed data processing, scalability, reliability, and data platform architecture.
•
Hands-on experience with performance tuning and optimisation of Databricks and Spark workloads.
•
Proven experience in technical solution design, architecture workshops, POCs, technical demonstrations, and solution validation.
•
Strong customer-facing consulting experience with the ability to engage Architects, CTOs, CDOs, Engineering Managers, and senior technology stakeholders.
•
Excellent communication, presentation, stakeholder-management, and technical storytelling skills.
•
Ability to provide technical leadership and mentorship to Data Engineering teams.
•
Strong analytical and problem-solving skills with a structured approach to complex technical challenges.
•
Experience with Databricks certification would be an advantage.
•
Exposure to Snowflake, Kafka, Spark Streaming, dbt, Terraform, CI/CD, MLflow, MLOps, GenAI, LLM/RAG, or data migration is desirable.
•
Strong ownership mindset with the ability to work independently in a remote, customer-facing environment.