Staff Data Engineer - Databricks SME (Remote)

Staff Data Engineer - Databricks SME This is a remote position. Ad Hoc is a technology company that empowers organizations to deliver scalable, impactful digital services. Using modern, agile methods, our team creates products that meet people's needs and transform their experience of government. Work on things that matter Our collaborations have shaped some of the defining moments in public-sector service delivery. We've helped build products that connect Veterans to tailored services, help millions access affordable health care, and support important programs like Head Start. As we work with agencies to deliver critical services, we're also changing how the government approaches technology. Built for a remote life Our culture, communications, and tools are built for remote work, enabling us to bring together top talent nationwide. At Ad Hoc, remote life empowers our teams to design work environments that fit their lives and that foster flexibility and collaboration to achieve positive outcomes for our customers. Committed to high expectations and a welcoming culture Ad Hoc values acceptance, accountability, and humility. We aren't heroes. We learn from our mistakes and improve the process for the next time. We build small, inclusive teams to collaborate closely with our partners to solve the right problems and deliver software that works. The Veterans Affairs business unit helps transform the VA into a modern digital services organization where Veteran outcomes are at the center of every effort. We partner with the VA to design and deliver seamless user experiences for Veterans, their families and caregivers, and VA employees. By applying better practices in service design, product management, and technology, we enable the VA to increase the use, quality, and reliability of services and decrease the time Veterans spend waiting for outcomes. Primary Responsibilities: Staff Data Engineer - Databrick expert builds scalable data pipelines, optimizing workloads, and enabling advanced analytics across cloud platforms, collaborates with data scientists and engineers to transform raw data into actionable insights, ensuring high performance, data integrity, and seamless integration within the Databricks environment. The Databricks expert builds robust data lakehouse architectures and advanced analytics and machine learning workflows. This role involves collaborating with various stakeholders to understand data requirements and develop solutions. Primary expectations of a Staff Data Engineer include: • Develop and maintain ETL/ELT pipelines using Databricks notebooks and workflows • Optimize Spark jobs for performance, scalability, and cost-efficiency • Design and implement data lakehouse architectures • Integrate Databricks with external data sources, APIs, and on-prem databases • Collaborate with data scientists, and engineers to support end-to-end data solutions • Ensure data governance, security, and compliance using tools like Unity Catalog and RBAC • Serves as a mentor, supporting junior level team members within the business unit and organization Basic Qualifications: • Bachelor's degree in computer science, information technology, or a related field, and 8+ years of experience • Minimum of 5 years of experience as a data analyst • Strong knowledge of database structures, theories, principles, and practices • 5+ years of experience in big data engineering and analytics • Experience with cloud platforms, ML and data cataloging tools • Understanding of data privacy and protection regulations • Strong analytical and problem-solving skills • Proficiency with SQL, Python, R, or other data analysis languages • Experience with data visualization tools and reporting platforms • Strong understanding of statistics, data modeling, and analytical techniques • Familiarity with databases, data warehouses, and ET processes • Ability to lead and motivate a remote team • Excellent time management and organizational skills • Adaptability and openness to new ideas Preferred Qualifications: • Prior government contracting experience To learn more about working at Ad Hoc, please visit: Benefits: • Company-subsidized health, dental, and vision insurance • Flexible PTO • 401K with employer match • Paid parental leave after one year of service • Employee Assistance Program Ad Hoc LLC is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, national origin, ancestry, sex, sexual orientation, gender identity or expression, religion, age, pregnancy, disability, work-related injury, covered veteran status, political ideology, marital status, or any other factor that the law protects from employment discrimination. In support of various state and city equal pay transparency laws, Ad Hoc job descriptions feature the starting range we reasonably expect to pay to candidates who would join our team with little to no need for training on the responsibilities we've outlined above. Actual compensation is influenced by a wide range of factors including but not limited to skill set, level of experience, and responsibility. The range of starting pay for this role is $150,000 - $160,000. Our recruiters will be happy to answer any questions you may have, and we look forward to learning more about your salary requirements. Apply tot his job