Skills You’ll Need To Bring
You are a self-starter and can work directly with the research team to gather and synthesize high-impact needs from teammates, designing and implementing the appropriate technical solutions, and effectively communicating about deliverables, timelines and tradeoffs.
You’ve spent 3+ years as software engineer, with a wide generalist skillset.
You’ve worked with distributed systems, and previously administered a big data toolset (Ray, Spark, Airflow, etc.).
You have hands-on experience shipping scalable data solutions in the cloud (e.g AWS, GCP, Azure, etc.), across multiple data stores (e.g Snowflake, Redshift, Hive, SQL/NoSQL, etc.).
You’re comfortable pulling data from whatever source it may be in, whether that involves writing a complex SQL query to pull a billion rows out of a Bigquery Database, or coordinating the transfer of 10+ PB of data across two S3 regions.
You are highly comfortable with scripting, whether in Typescript, Python, or even Bash when the need arises.
Most importantly, you know when to pull in the complex overengineered tools, and when to instead rely on duct tape.