Remote - US
Remote
Senior
Full Time
💰$207,200 - $274,140
remotedata platformarchitectureseniorcloudAWSKubernetesTerraformHelmprivacy compliance
Requirements
- •Bachelor’s or Master’s degree in Computer Science, Engineering, or related field, or equivalent practical experience.
- •15+ years of experience in software development or related field, with at least 8 years in architecture or senior technical leadership role.
- •Proven track record of architecting and delivering large-scale data platforms and setting technical strategy on cloud infrastructure such as AWS, deployed with Kubernetes, Terraform and Helm.
- •Deep expertise in storage and compute for analytical data including columnar and open table formats, lake and lakehouse architecture, partitioning, indexing, query planning, cost and performance at petabyte scale.
- •Breadth across data storage and retrieval technologies with hands-on depth in at least two of analytical/OLAP engines (e.g., ClickHouse), search engines (e.g., ElasticSearch/OpenSearch), and lake or distributed processing engines (e.g., Spark, Trino, Flink).
- •Working knowledge of streaming systems (e.g., Kafka) and their interaction with downstream storage.
- •Practical experience designing data governance at scale including data classification, catalog and metadata management, lineage, and access control integrated with authentication and authorization systems.
- •Practical experience implementing privacy and compliance requirements (deletion, retention, residency) across large heterogeneous data estates.
- •Proficiency in one or more of Python, Java, and Go.
- •Track record of writing and driving architecture decisions across multiple teams and influencing technical roadmap direction.
- •Experience elevating engineering standards and mentoring senior engineers across teams.
- •Demonstrated commitment to operational excellence including architecting for high availability, observability and incident response in distributed systems.
- •Strong written and verbal communication skills including explaining complex tradeoffs to engineers and senior leadership.
What You'll Do
- •Own the architecture and multi-year technical strategy for the data platform, including lake/lakehouse, ClickHouse, Elasticsearch, and governance.
- •Make and document major design decisions on storage, open table formats, catalog strategy, compute engines, tenancy, data placement, and build-versus-adopt tradeoffs.
- •Set reference architectures for core storage and query engines deployment, tuning, scaling, and operation.
- •Define the reliability model for the platform including availability targets, failure domains, disaster recovery, observability, and incident response patterns.
- •Build privacy and compliance into the architecture including deletion, retention, data residency, and PII classification.
- •Drive data governance including classification, catalog and metadata, lineage, access control, data contracts, quality and integrity standards.
- •Review designs and technical direction across the organization and act as escalation point on complex problems.
- •Mentor principal and senior engineers and grow architectural judgment across the organization.
- •Partner with product, security, infrastructure and consuming teams to turn business needs into platform roadmaps.
- •Track emerging data technologies and bring forward those worth adopting with evidence from prototypes or benchmarks.
Nice to Have
- •Experience operating engines like ClickHouse or ElasticSearch at scale including replication, sharding, ingestion behavior, and multi-tenant isolation.
- •Experience with catalog and metadata systems (e.g., Unity Catalog, Glue, Polaris, DataHub) and open table format catalog strategy.
- •Experience with platform migration across domains or ownership models such as moving from central to domain-owned data.
- •Track record of measurable cost optimization on large data platforms.
- •Networking and network security knowledge.
Benefits
- •Competitive pay
- •Generous time off
- •Ample parental and wellness leave
- •Healthcare
- •Retirement savings program
- •Equity plan participation
- •Corporate bonus plan
- •Paid sick time
- •Paid personal time off
- •Paid parental leave
