Senior Data Engineer
Clera · Remote – worldwide
- Salary
- Not stated
- Posted
- 10d ago
- Source
- Himalayas
Job description
About the Role
Join a data infrastructure and analytics team building a platform that turns information from many sources into reliable, current company intelligence. You will own the platform end to end, from architecture and ingestion through production operations, helping power downstream products with trusted data.
What You'll Do
• Design and build a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
• Ingest data from provider APIs, webhooks, files, and web scraping, with rate limiting and robust failure handling.
• Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.
• Build messaging and task queue workflows with retries, dead-letter queues, and controlled replay.
• Preserve original payloads in Google Cloud Storage and transform data with dbt into raw, standardized, and curated layers.
• Develop idempotent upserts, data contracts, automated quality checks, pipeline tests, and monitoring for freshness and failures.
• Design and optimize PostgreSQL schemas and loading patterns for reliable downstream use.
• Integrate data enrichment and LLM-based structured extraction into the platform.
What We're Looking For
• 5 to 10 or more years of experience, including production ownership of multi-source data platforms from architecture through deployment and ongoing operation.
• Strong hands-on experience with Airflow or similar orchestration tools, dbt, and Google Cloud Platform.
• Experience building event-driven systems with messaging or task queues and asynchronous processing.
• Strong SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading.
• Proficiency in Python and concurrent or asynchronous processing, such as asyncio, aiohttp, or HTTPX.
• Practical web scraping experience with tools such as Scrapy, Playwright, or Selenium.
• Experience with data contracts, storage layers, and automated data quality checks. Entity resolution is valuable; BigQuery, Dataform, and LLM-based structured extraction are helpful.
• Comfort working in a startup or small team with broad ownership and adaptability.
Compensation & Benefits
Visa sponsorship is not available.
Location
Remote.
Originally posted on Himalayas
Saffa.global isn't the employer or recruiter. Never pay anyone for a job offer, visa or "processing fee".
Similar jobs
Engineering Manager - Ubuntu Core
Canonical · Remote – EMEA, LATAM, Canada, USA
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, engineering innovation, and IoT. Our customers include the world's leading public cloud and silicon providers, and industry leaders…
Business Development Specialist
Bybit · Remote – worldwide
About UsEstablished in 2018, Bybit is one of the world’s leading cryptocurrency exchanges and digital financial platforms, serving over 80 million users across more than 200 countries and regions.
(1601) Senior Data Platform and Automation Engineer
Nortal · Remote – worldwide
Join Nortal for your next big adventure. We are seeking a Senior Data Platform and Automation Engineer to join our team, grow with us, and teach us a few things along the way.
Implementation Manager
Self Bill Pro · Remote – worldwide
About Self Bill Pro:Self Bill Pro is a rapidly growing startup dedicated to solving one of the most persistent challenges in the employee benefits ecosystem: the complexity of monthly benefits billing.
Director of Engineering - MetaDefender Email Security
OPSWAT · Remote – incl. South Africa
OPSWAT, a global leader in IT, OT, and ICS critical infrastructure cybersecurity, delivers an end-to-end platform that gives public and private sector organizations and enterprises the critical advantage needed to protect their complex networks, secure their devices, and ensure compliance.
Associate Director, Product Design
Parallel · Remote – worldwide
Associate Director, Product DesignAt Parallel, we partner with companies who care about building great products, and we help them ship work that improves lives and moves markets.