Senior Data Engineer

//Building Scalable Data Infrastructure As our Senior Data Engineer, you will be the foundation of our data team, designing and implementing scalable, secure, and well-structured data infrastructure. You will lead the adoption of best practices in data engineering, ensuring our pipelines—both real-time and incremental streaming—are efficient, resilient, and aligned with the needs of analysts, data scientists, and external customers.

//Leading Data Governance & Best Practices You will establish and maintain our data catalogs, governance frameworks, and best practices for managing and transforming data. Your expertise in Databricks, Unity Catalog, Terraform (AWS + Databricks), Docker, Python, and DBT or flypipe will drive the team's success. Your role extends beyond just infrastructure-you will ensure that data accessibility, security, and documentation empower analysts and data scientists to extract maximum value.

// Hands-On Technical Leadership We are looking for a hands-on leader who thrives on challenges. As we build our data engineering team, you will mentor and guide engineers in creating high-performance streaming and batch pipelines, optimizing transformations. Your willingness to challenge the status quo, experiment with new technologies, and foster a culture of innovation will be key to our success.

// Within 3 months, you will: Assess & Prioritize Data Needs – Identify key data sources for ingestion, evaluate infrastructure security, setup, and scalability, and define a roadmap for implementation. Improve Data Pipelines & Governance – Ensure pipelines, catalogs, and dictionaries are structured, well-documented, and support both internal and external dashboard accessibility. Identify & Solve Technical Challenges – Quickly raise concerns, propose and implement technical solutions to improve data reliability, performance, and security. Engage with Key Stakeholders – Build strong relationships with engineering and product teams to align data infrastructure with business needs and ensure smooth collaboration.

// Within 6 months, you will: Own the Technical Strategy for Data Engineering – Take full ownership of data infrastructure, pipelines, and governance, ensuring they are secure, scalable, and aligned with best practices. Lead & Mentor the Data Engineering Team – Guide other data engineers, establish coding standards, and drive technical excellence in pipeline development, modeling, and transformation. Adapt Data Systems to Business & Product Needs – Stay informed about product and business changes, proactively adjusting data models and infrastructure to support evolving requirements. Optimize Performance & Cost Efficiency – Continuously refine data pipelines and modeling to balance cost, performance, and scalability, ensuring smooth workflows for data scientists and analysts.

// Within 9 months, you will: Own & Drive Data Engineering Strategy – Oversee the technical direction of data engineering, ensuring infrastructure, pipelines, and models evolve based on how users interact with data. Optimize Costs & Performance Continuously – Implement cost-effective solutions while maintaining high-performance pipelines and scalable infrastructure for analytics and ML workloads. Ensure Reliable & Well-Structured Data Models – Maintain and refine data models and engineering toolsets to provide data scientists, analysts, and engineers with the right foundation for their work. Scale the Team & Maintain Data Operations – Lead onboarding and mentorship of new team members, ensuring smooth development and production operations while adapting to new data sources and evolving business needs.

5+ years as data engineer, working in a fast-paced environment, preferably in Fintech

Expertise in Data Infrastructure & Pipelines – Strong experience in building, optimizing, and maintaining data pipelines (incremental & real-time/streaming) with a focus on scalability, security, and performance.

Deep Knowledge of Databricks & Unity Catalog – Hands-on experience with Databricks, Unity Catalog, and data governance best practices for secure and organized data access (Databricks Data engineer Professional certificate is a plus!)

Proficiency in Terraform & Cloud Infrastructure (AWS) – Strong ability to manage Databricks + AWS setup using Terraform, ensuring a scalable and well-managed infrastructure.

Strong Python & DBT Skills – Extensive experience in Python for data engineering tasks and DBT for transformation and data modeling.

Data Modeling & Governance Expertise – Proven ability to design efficient data models, implement data dictionaries/catalogs, and ensure high-quality, accessible data for analysts and data scientists.

Ability to Guide & Mentor Data Engineers – Leadership in setting best practices, reviewing code, and mentoring engineers, ensuring a high technical standard within the team.

Familiarity with Docker & CI/CD Pipelines – Experience with containerization and CI/CD for managing data engineering workflows.

Experience with External Data Sharing & Security – Understanding of data security and access management for both internal and external stakeholders.

ML Pipeline Deployment Knowledge – Understanding of ML models in production and how to optimize pipelines for data scientists.