Job Detail

Home / Job Detail

Staff Data Engineer - Job Id 11184

Bangalore (Karnataka-India)
7-10 Yrs
Date Posted: May 26, 2026
Skills Required:

Job Description

We are looking for a Staff Data Engineer to work with our data architecture towards reliability, stability and evolution of our data platform. This is a senior individual contributor role with influence on our data strategy — you will make and defend technical decisions, set the engineering standard for how data flows from source systems to business-critical reporting and deliver towards warehouse/lakehouse production maturity,

You will operate in a regulated fintech environment where data accuracy directly impacts customer outcomes. You are expected to work with high autonomy, pull in stakeholders rather than wait for direction, and leave every system you touch more maintainable than you found it.

This role spans data engineering, data governance and platform architecture. You are the technical owner of the pipeline layer — the person who decides how things are built, not just executes on decisions.

What You'll Own

Platform Architecture & Technical Direction

  • Work in strong collaboration with senior platform engineers and data architects to drive the end-to-end architecture of the data platform. This includes stabilizing ingestion, orchestrating data transformation (example: DBT Medallion: orchestration), and delivery to dashboards (Metabase, Tableau, Redash, Databricks Lakeview)
  • Design and document binding technical decisions for stabilizing schemas,pipeline patterns, cluster strategy, incremental models, and data quality frameworks
  • Drive Databricks Unity Catalog migration to completion including schema sign-off, policy enforcement, and stakeholder communication
  • Define the long-term orchestration strategy: Airflow on Kubernetes as the primary engine, with evaluated adoption of new tools to streamline critical data workflows where necessary
  • Participate and weigh in on a lightweight MDM layer evaluation and tools
Pipeline Reliability & Ownership
  • Own production SLAs for all data pipelines; treat any breach as a P0 and drive root cause to resolution
  • Design for idempotency, re-runnability, and safe backfill from the start — not as an afterthought
  • Build and maintain monitoring, alerting, and observability for pipeline health (Prometheus, Grafana, Graylog)
  • Own incident response for data SLA failures; produce post-mortems with actionable follow-through

Data Quality & Governance
  • Collaborate and drive architecture and building of a data validation framework:
  • Implement anomaly detection and automated flagging at the pipeline level
  • Own and establish RBAC/ABAC policy implementation for data access governance across departments and service accounts
  • Enforce PII masking standards (SHA-256 with salt for PAN and customer identifiers) at the pipeline and catalog layer; audit and remediate legacy MD5-based masking in migrated schemas
  • Maintain 5-year audit log retention standards; all pipeline runs emit structured, queryable audit events.
Engineering Standards & Mentorship
  • Set and enforce coding, testing, and documentation standards for the data team
  • Drive pro-active knowledge transfer across teams to ensure sound and cohesive understanding of the Scripbox Data Platform and create operational playbooks for on-call engineers
  • Review PRs for pipeline changes; treat schema changes without a corresponding dictionary update as a merge blocker
  • Mentor junior staff engineers on data engineering patterns, DAG design, warehouse best practices, and fintech data domain knowledge

Cross-Functional Partnership
  • Partner with product and backend engineering to ensure new features are instrumented for data from day one
  • Work directly with compliance and legal to translate SEBI CSCRF and DPDP requirements into technical guardrails at the platform layer
  • Support analytics and BI teams as a technical enabler — remove bottlenecks proactively


Must Have

  • 7–10 years of hands-on data engineering experience, with at least 3 years in a technical ownership or lead role
  • Deep SQL — window functions, CTEs, incremental patterns, query plan analysis, performance tuning on large datasets
  • Expert-level DBT — models, macros, Jinja, incremental strategies, tests, packages, and manifest-driven orchestration
  • Production Airflow experience — complex DAG authoring, parent/child dependencies, graceful failure handling, backfill strategy
  • Cloud data warehousing architecture experience (Databricks, Redshift, BigQuery, or Snowflake) — not just usage but design decisions
  • Python for data engineering — ingestion scripts, watermarking, validation frameworks, file format handling (CSV, DBF, Parquet, zipped sources)
  • AWS fluency — S3, EC2, EKS, IAM, Secrets Manager; comfortable debugging infrastructure issues without a dedicated DevOps handoff
  • Demonstrated ability to write and defend tech specs for data so that technical decisions are traceable and referenceable

Strong Plus
  • Databricks Unity Catalog — Delta Lake, Workflows, Spark tuning, column masking, external locations
  • ClickHouse — schema design, Spark/S3 ingestion, query optimisation for analytical workloads
  • Fintech / financial services domain — mutual funds, NAV, AUM, RTA data formats, XIRR calculations
  • Familiarity with SEBI/AMFI data formats or BSE StarMF/CAMS/KFintech integration patterns
  • Data governance tooling — OpenMetadata, Alation, or equivalent; metadata lineage and catalog management
  • Experience implementing PII classification, masking, and audit frameworks in a regulated environment
  • Kubernetes — not just log access; comfortable writing Helm values, debugging pod scheduling, managing resource limits
  • Familiarity with DPDP act

Job Detail

  • Type:
    Full Time/Permanent
  • Shift:
    First Shift (Day)
  • Positions:
    2
  • Gender:
    No Preference
  • Degree:
    Graduation
  • Industry:
    Banking / Financial Services / Broking
Apply Now

Share This Job

Related Jobs

Quick Links

Find Jobs By Type



Contact Us

C-64/2, Okhla Industrial Area, Phase II, New Delhi, Delhi 110020
+9176781 87130 | +9195408 31300 | 01141010035
Copyright © 2026 Job24by7. All Rights Reserved.
Close

Raise your Query

Hi! Simply click below and type your query.

Our experts will reply you very soon.

Click Here to Chat JOB24by7 Recruitment Consultan...
WhatsApp Us
Job24by7 Assistant