Job title: Data Engineer - Spark | Kafka
Job type: Permanent
Emp type: Full-time
Industry: Scala
Skills: Scala Spark
Salary type: Annual
Salary from: CAD $170,000.00
Salary to: CAD $210,000.00
Location: Toronto Division, ON
Job ID: 32081
Contact name: Mitchell Franklin
Contact email: Mitchell@umatr.io

Job Description

Join a highly skilled team of innovators. Apply big data, artificial intelligence, and machine learning to bring the next generation of financial products and services to global markets.

Build the only fraud and risk management platform that orchestrates data from the entire customer journey, fighting fraud more effectively with configurable risk models in a single, easy-to-use platform.

Our client is a leading provider of cutting-edge software solutions, specializing in fraud risk management. Their innovative SaaS platform helps businesses mitigate fraud risks, protect sensitive data, and maintain trust with their customers.

What You'll Do: 

If working with billions of events, petabytes of data, and optimizing for the last millisecond is something that excites you then read on! We are looking for software engineers who have seen their fair share of messy data sets and have been able to structure them for further fraud detection and prevention; anomaly detection and other AI products.

You will be working on writing frameworks for real-time and batch pipelines to ingest and transform events from 100’s of applications every day. These events will be consumed by both machines and people. The teams ML and Software engineers consume these events to build new and optimize existing models to detect and fight new fraud patterns. You will also help optimize the feature pipelines for fast execution and work with software engineers to build event-driven microservices.

You will get to put cutting-edge tech in production and the freedom to experiment with new frameworks, try new ways to optimize, and resources to build the next big thing in fintech using data!

  • Work directly with the Platform Engineering Team to create reusable experimental and production data pipelines and centralize the data store.
  • Unbox, deep-dive, understand, tune, and master the frameworks and technologies used day-to-day.
  • Adopt problem-solving as a way of life – always go to the root cause.
  • Keep the data whole, safe, and flowing with expertise on high-volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc).
  • Make the data available for online and offline consumption by machines and humans.
  • Maintain and optimize underlying storage systems to perform according to the set SLAs.
  • Shape the data by developing efficient structures and schema for the data in storage and transit.
  • Explore new technology options for data processing, storage, and share them with the team.
  • Develop tools and contribute to open source wherever possible.

 

Who You Are:

  • Degree in Computer Science, Engineering, or a related field
  • You understand how microservices work and are familiar with concepts of domain-driven design and data modeling. Large-scale system design experience is a plus.
  • You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day.  
  • You are passionate about producing clean, maintainable, and testable code as part of a real-time data pipeline.
  • You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues.
  • You have worked with Spark and Kafka before and have experimented with or heard about Spark Streaming/Kafka Streams/Flink and understand when to use one over the other.
  • You have experience implementing offline and online data processing flows and understand how to choose and optimize underlying storage technologies. You have worked or experimented with NoSQL databases such as  Cassandra.
  • On a bad day, maintaining and bringing up a cluster doesn’t bother you.
  • You may not be a networking expert but you understand issues with ingesting data from applications in multiple data centers across geographies, on-premises, and cloud, and will find a way to solve them.
  • Proficient in Java/Scala/Python/Spark.

 

What is in it for you: 

  • For the 5th year in a row, they have been certified as a ‘Great Place to Work’
  • An open work environment that fosters collaboration, ownership, creativity, and urgency
  • Flexibility with work schedule - with office in the heart of Toronto 
  • Enrolment in the Group Health Benefits plan from day 1, no waiting period
  • Team building events on and off-site
  • Fuel for the day: weekly delivery of groceries, and all types of snacks
  • Catered lunches and desserts on a monthly basis
  • Daily fun in the office with our competitive games of Ping Pong, Chess, Pool, FIFA, and more games, etc.
  • An unlimited amount of freshly made coffee! We’re pretty serious about our coffee beans

 

Interested in being part of this rocketship? 

File types (doc, docx, pdf, rtf, png, jpeg, jpg, bmp, jng, ppt, pptx, csv, gif) size up to 5MB