BR
BrillioDatabricks Data Specialist - R01569707
LocationBangalore, Karnataka, India
Work modehybrid
SeniorityLead
DepartmentAI & Data Engineering, AI & Data Engineering : Data Engineering
Company size5,001–10,000 people
PostedAug 10, 2026 · 2mo ago
Verified live1d ago
At a glanceSummarised by Seekless from the posting.
Hands-on experience with Databricks Platform
Hands-on experience with Delta Lake
Hands-on experience with Delta Live Tables (DLT)
Hands-on experience with Unity Catalog
Hands-on experience with Databricks Workflows
Hands-on experience with PySpark and Apache Spark
Hands-on experience with Structured Streaming
Hands-on experience with Auto Loader
Hands-on experience with SQL
Hands-on experience with Lakehouse Data Modeling
Strong understanding of data engineering best practices and scalable data architectures
Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline
Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines
Strong analytical, troubleshooting, and problem-solving capabilities
Experience working in agile and collaborative environments
Excellent communication and stakeholder management skills
Experience with Azure Data Factory (ADF)
Experience with Azure Synapse Analytics
Experience with Microsoft Purview
Experience with Microsoft Fabric
Experience with AWS Glue
Experience with AWS Lambda
Experience with AWS Step Functions
Experience with Apache Airflow
Experience with DBT
Experience with Fivetran
Experience with Informatica
Experience with Apache Kafka
Experience with Power BI
Experience with Collibra
Experience with Alation
Experience with BigQuery
Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms
Strong understanding of data governance, security, and compliance frameworks
Experience delivering both batch and real-time data processing solutions
Ability to work independently while collaborating effectively across global teams
Skills
Databricks
PySpark
Apache Spark
Delta Lake
Delta Live Tables (DLT)
Unity Catalog
Databricks Workflows
Structured Streaming
Auto Loader
SQL
Azure Data Factory (ADF)
Azure Synapse Analytics
Microsoft Purview
Microsoft Fabric
AWS Glue
AWS Lambda
AWS Step Functions
Apache Airflow
DBT
Fivetran
Informatica
Apache Kafka
Power BI
Collibra
Alation
BigQuery
Data Specialist
Primary Skills
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
•
Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
•
Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
•
Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
•
Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
•
Implement scalable and efficient data ingestion processes using Auto Loader.
•
Develop and manage real-time data processing solutions using Structured Streaming.
•
Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
•
Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
•
Establish and enforce data governance, security, and access controls using Unity Catalog.
•
Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
•
Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
Required Skills (Must Have)
•
Databricks Platform
•
Delta Lake
•
Delta Live Tables (DLT)
•
Unity Catalog
•
Databricks Workflows
•
PySpark and Apache Spark
•
Structured Streaming
•
Auto Loader
•
SQL
•
Lakehouse Data Modeling
•
Strong understanding of data engineering best practices and scalable data architectures
Preferred Skills (Good to Have)
Azure Ecosystem
•
Azure Data Factory (ADF)
•
Azure Synapse Analytics
•
Microsoft Purview
•
Microsoft Fabric
AWS Ecosystem
•
AWS Glue
•
AWS Lambda
•
AWS Step Functions
Data Engineering & Integration
•
Apache Airflow
•
DBT
•
Fivetran
•
Informatica
Streaming & Analytics
•
Apache Kafka
•
Power BI
Data Governance
•
Collibra
•
Alation
GCP
•
BigQuery
Qualifications
•
Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
•
Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
•
Strong analytical, troubleshooting, and problem-solving capabilities.
•
Experience working in agile and collaborative environments.
•
Excellent communication and stakeholder management skills.
Preferred Candidate Profile
•
Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
•
Strong understanding of data governance, security, and compliance frameworks.
•
Experience delivering both batch and real-time data processing solutions.
•
Ability to work independently while collaborating effectively across global teams.
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Specialization
•
Databricks Engineering: Lead Data Engineer
Job requirements
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
•
Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
•
Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
•
Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
•
Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
•
Implement scalable and efficient data ingestion processes using Auto Loader.
•
Develop and manage real-time data processing solutions using Structured Streaming.
•
Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
•
Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
•
Establish and enforce data governance, security, and access controls using Unity Catalog.
•
Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
•
Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
Required Skills (Must Have)
•
Databricks Platform
•
Delta Lake
•
Delta Live Tables (DLT)
•
Unity Catalog
•
Databricks Workflows
•
PySpark and Apache Spark
•
Structured Streaming
•
Auto Loader
•
SQL
•
Lakehouse Data Modeling
•
Strong understanding of data engineering best practices and scalable data architectures
Preferred Skills (Good to Have)
Azure Ecosystem
•
Azure Data Factory (ADF)
•
Azure Synapse Analytics
•
Microsoft Purview
•
Microsoft Fabric
AWS Ecosystem
•
AWS Glue
•
AWS Lambda
•
AWS Step Functions
Data Engineering & Integration
•
Apache Airflow
•
DBT
•
Fivetran
•
Informatica
Streaming & Analytics
•
Apache Kafka
•
Power BI
Data Governance
•
Collibra
•
Alation
GCP
•
BigQuery
Qualifications
•
Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
•
Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
•
Strong analytical, troubleshooting, and problem-solving capabilities.
•
Experience working in agile and collaborative environments.
•
Excellent communication and stakeholder management skills.
Preferred Candidate Profile
•
Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
•
Strong understanding of data governance, security, and compliance frameworks.
•
Experience delivering both batch and real-time data processing solutions.
•
Ability to work independently while collaborating effectively across global teams.
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI