Data Engineer

IMC · Chicago, United States · On-site · Posted 6 days ago

Get 3 free interview questions and apply →Apply on the employer's site ↗

Likely interview questions for this job

Tell me about yourself.

The interviewer is evaluating how clearly you articulate your career progression and whether your background in building scalable, real-time data infrastructure aligns with IMC's trading environment.

Can you describe how you have designed and tuned an Apache Kafka and Flink (or Spark Streaming) pipeline for high-throughput, low-latency data processing, including how you handled exactly-once semantics and backpressure?

The interviewer is testing your hands-on mastery of core streaming architectures and state management under heavy real-time loads, which are critical for market data processing. Focus on specific tuning parameters, partition strategies, and fault-tolerance mechanisms you implemented.

How do you approach designing and partitioning data storage for large-scale datasets using modern formats like Apache Iceberg or HDFS, and how would you troubleshoot an unexpected pipeline bottleneck or data skew issue in production?

This checks your architectural data modeling skills, capacity planning, and your ability to conduct root-cause analysis using metrics and observability tools. Explain your diagnostic workflow from metrics to cluster profiling and how your storage choices optimize query performance.

Upload your résumé for free sample answers →

About the job

We are seeking a dedicated and experienced Data Engineer to join our Chicago team. The ideal candidate is energized by working in a cutting-edge environment that enables IMC to continue to be at the forefront of the evolving global financial markets. Your Core Responsibilities: • Architect, develop and deploy our Big Data environment (Kafka, Hadoop, Dremio, etc.) • Build, deploy, and monitor our data processing pipelines (Java, Python, Spark, Flink) • Collaborate with development teams on data modeling, data ingestion, and capacity planning • Work with users to ensure data integrity and availability • Act as a Big Data SME and consult on a variety of data-related questions from users and developers Your Skills and Experience: • 5+ years experience working in a mature data engineering environment • 3+ years of experience building Kafka streaming applications and/or maintaining Kafka clusters • 2+ years of experience building applications/pipelines with Big Data backends (S3, HDFS, Databricks, Iceberg, etc) • Experience with Apache Spark, Apache Flink or similar tools • Strong Java, Python, and SQL development skills • Experience with common data-science toolkits, especially python-based • Hands-on experience with Kubernetes and Docker • Experience with monitoring tools such as Prometheus/Grafana, Alert Manager, Alerta and OpsGenie • Strong statistical analysis skills • Demonstrated ability to troubleshoot and conduct root-cause analysis • Unix scripting experience (bash, python, etc.) The Base Salary range for the role is included below. Base salary is only one component of total compensation; all full-time, permanent positions are eligible for a discretionary bonus and benefits, including paid leave and insurance. Please visit Benefits - US | IMC Trading for more comprehensive information. Salary Range $175,000—$225,000 USD About Us IMC is a research-driven trading firm where quantitative modeling, machine learning, and engineering shape how modern markets are traded. A stabilizing force in markets since 1989, we provide liquidity across trading venues, delivering the best outcome in value and risk management to investors. Using our own technology and capital, we build proprietary systems and algorithms that operate across global markets. Our researchers, traders, and engineers work as a collective, combining rapid experimentation, advanced infrastructure, and real-time feedback to turn insight into execution and execution into advantage.

Applying is free. Dyalo never charges you to apply. Always check a job on the employer's own site.

More: IT and software jobs →