Description

Our company specializes in developing a cutting-edge search and discovery platform, encompassing search, autocomplete, browse, and recommendations, tailored specifically for online retailers, suppliers, and shops. Our goal is to provide all online stores with the ability to have state-of-the-art discovery powered by AI and modern technologies, optimizing key e-commerce metrics, which differentiates us from our competitors.

Boasting a robust infrastructure, we handle billions of requests daily and manage a vast data ecosystem, including ~400 GB of data intake daily and approximately 2PB stored in our data lake.

The Data Platform is the central piece and the foundation of the data processing at our company. It is the set of tools and infrastructure that every data person uses daily, plus it's used by some of our public-facing API to fetch data from. It is responsible for data landing from external clients (API), providing external visualization tools with data (Cube.js, ClickHouse), storing data in appropriate formats (S3, ClickHouse, Delta, Parquet), data processing (Python, Spark/Databricks, ClickHouse, AWS Athena, AWS Lambda), monitoring (Prometheus, PagerDuty, Sentry, custom internal APIs with FastAPI), automated testing of pipelines and data quality, cost observability, optimizations, and many more things.

We are hiring a Senior Data Engineer to work on our Data Platform. You'll be working in a team of engineers that both own, manage, and develop cloud infrastructure that powers the platform and develop data pipelines, cubes, services, and other tools that make it work. Tens of engineers, top management, and external users will depend on your work.

Requirements

We're looking for a senior engineer with at least 4 years of experience who has solid skills in any programming language (ideally Python), big data engineering, web services, cloud providers (ideally AWS), and is willing to build many diverse things to develop the platform.

You will work on building various data platform components with us, listening to user feedback, and proactively changing things for the better. Here are some of the projects you may be involved with:

• Implement a data model for our tables (bronze, silver, gold) to encapsulate parsing logic, lower costs, and provide better abstractions.
• Power our recommendation system with a fast, global data store.
• Improve the build system in our repositories to speed up test execution, job deployment, and thus improve developer productivity.
• Optimize Spark pipelines, ClickHouse tables/queries, and streaming processing in AWS Lambda.

Benefits

• Compensation: base range of 90-110k USD + stock options + work from home bonus.
• Fully remote team - choose where you live.
• Work from home stipend! We want you to have the resources you need to set up your home office.
• Apple laptops provided for new employees.
• Training and development budget for every employee, refreshed each year.
• Parental leave for qualified employees.
• Work with smart people who will help you grow and make a meaningful impact.
• Company retreat to the Canary Islands if you join before August 1!

Diversity, Equity, and Inclusion

We are committed to cultivating a work environment that is diverse, equitable, and inclusive. As an equal opportunity employer, we welcome individuals of all backgrounds and provide equal opportunities to all applicants regardless of their education, diversity of opinion, race, color, religion, gender, gender expression, sexual orientation, national origin, genetics, disability, age, veteran status, or affiliation in any other protected group. Studies have shown that women and people of color may be less likely to apply for jobs unless they meet every one of the qualifications listed. Our primary interest is in finding the best candidate for the job. We encourage you to apply even if you don’t meet all of our listed qualifications.

4o