Lead Data Engineer

Who are we?

UpGuard’s mission is to protect the world’s data. We obsessively seek out elegant, robust ways to enable our customers to find, acknowledge, and remediate cyber risk. With UpGuard, organizations leverage our security expertise and software to automate what was once laborious, spreadsheet-driven processes–whether it's monitoring the attack surface of hundreds of vendors or assessing the security of their own infrastructure. UpGuard is used by some of the world’s largest, fastest-growing, and most innovative companies. 

Our Analytics team at UpGuard supports all of our teams to extract key insights from the data that we collect. We extract data from multiple sources and build insightful dashboards to display key metrics and support leaders to make decisions. We also build integrations between systems to seamlessly collect required data. 

Where does this role fit in?

As UpGuard continues to scale at an increasing rate, each of our teams, from Product to Sales, are becoming more data and insight hungry. As a result, our central Analytics function is bandwidth constrained and needs to scale with our business growth. In essence, with the emergence of automation and AI, we are looking to democratise data insights and analytics and to do this we need to enable our users with access to data, while we still ensure data integrity, security and governance are maintained. This role will be a key enabler  to achieve this vision and accelerate our efforts in this area.

• Data Integration & Pipeline Development: Design, build, and maintain reliable data pipelines to consolidate information from various internal systems (CRM, support platforms, marketing automation, product usage data, etc.) and relevant third-party sources
• Semantic Layer Design & Management: Develop and manage a comprehensive semantic layer (potentially using technologies like LookML or similar) that provides clear definitions and mappings for data concepts to underlying database structures as well as high level business logic.
• Data Quality & Governance: Implement and enforce data quality checks, validation rules, and governance processes to ensure the accuracy, consistency, and reliability of data
• AI & Agent Data Enablement: Ensure AI agents have access to the necessary structured and unstructured data (Knowledge Bases, support tickets, documentation, internal data, pricing info, marketing content, etc.) required for them to perform tasks like answering questions, qualifying leads, preparing for calls, and identifying opportunities
• Documentation & Knowledge Sharing: Create clear documentation for data models, pipelines, and the semantic layer to enable both human users and AI agents to effectively interact with the data warehouse

• Looker is our main BI platform: a candidate should be familiar with Explores, Looks, Dashboards and the Developer interface - maintaining dimensions and measures, create and maintain models, and be able to run and debug raw SQL queries. Be able to edit and deploy custom visualizations using Looker’s extension SDK. Interface with Looker’s API.
• CloudSQL (PostgreSQL) and BigQuery for storage and query: be comfortable with writing complex queries, indices, materialised views, etc.
• Containers, Docker and Kubernetes (via GKE) are used to run our internal automations. Being familiar and confident being a Platform Engineer in our GCP project.
• Be familiar with technologies like n8n to automate processes and potentially Golang as a programming language as our non-standard ETL workers are written in Go.
• Be comfortable interfacing with a variety of APIs from our vendor platforms (either REST+JSON or MCP Server, depending on what a vendor platform throws at you).
• Have experience with version control via GitHub, and a basic review process, like GitHub Flow.
• If all else fails, being comfortable and confident falling back to a Golang+TypeScript+ChartJS solution, just to get data in the right shape or form to enable the Analyst to continue their work.

• Monthly Lifestyle subsidy: use this for financial, physical and mental wellbeing
• WFH set-up allowance: to ensure you have the right environment to work in, we will help you get set-up within your first 3 months at UpGuard
• $1500 USD annual Learning & Development allowance: to support your career development all team members will be able to expense development opportunities against this allowance
• Generous Annual Leave/PTO allowances: time to recharge your batteries
• 18 weeks paid Parental Leave: irrespective of parenting role
• Personal Leave allowance: this includes sick & carer’s leave
• Fully remote working environment: whilst we have physical offices in Sydney & Hobart, we do not mandate compulsory attendance
• Top spec hardware: all team members will be provided with top-spec laptops for their role

#LI-SR1

UpGuard is a Certified Great Place to Work® in the US, Australia, UK and India, establishing its position as a leading global technology employer. 99% of team members agree that UpGuard is a great place to work, apply now to find out why!

As an Equal Employment Opportunity and Affirmative Action Employer, qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status.

For applications to positions in the United States, please note, at this time we can only support hiring in the following US states: CA, MD, MA, IL, OR, WA, CO, TX, FL, PA, LA, MO, or DC.  

Before starting work with us, you will need to undertake a national police history check and reference checks. Also please note that at this time, we cannot support candidates requiring visa sponsorship or relocation.