Software Engineer, Platform Engineering - AI/GPU Infra (On-Prem) - Mercari

JD in Japanese follows. 英文の後に和文JDをご覧いただけます。 Software Engineer, Platform Engineering - AI/GPU Infra (On-Prem) - Mercari • Employment Status: Permanent • Work Hours: Full Flextime (no core time) • Office: Roppongi For more details, see the Overview of Our Positions section on our Careers site. About Mercari Circulate all forms of value to unleash the potential in all people "What can I do to help society thrive with the finite resources we have?" The Mercari marketplace app was born in 2013 out of this thought by our founder Shintaro Yamada as he traveled the world. We believe that by circulating all forms of value, not just physical things and money, we can create opportunities for anyone to realize their dreams and contribute to society and the people around them. Mercari aims to use technology to connect people all over the world and create a world where anyone can unleash their potential. For more information about Mercari Group's mission, see Mercari’s Culture Doc. Organization/Team Mission Mercari Engineering Principles Mercari Engineering Principles are a shared understanding that serves as the foundation of engineering beliefs and behavior at Mercari. The Engineering Principles are designed to complement the organizational identity (Mercari's mission, values, and culture) from an engineering viewpoint. These principles ultimately help us achieve Mercari's mission by defining the ideal state we seek to realize in the long term. • Passion For The Product • Grow Together • Solve Through Mechanisms • Collaborate Openly For more details, please see Engineering Culture. See here for more information about our mission and values. The Platform team helps Mercari engineers deliver more reliable services and improve development efficiency by managing the entire network — from edge areas such as the CDN to the cloud, service mesh (Istio), and physical data centers. As Mercari invests in self-hosted compute for its AI/LLM initiatives, we are building a new on-premises GPU infrastructure capability within the team. This role owns the on-prem side: securing data-center facilities and building the network foundation that enables GPU compute, and integrating it securely with our existing cloud and platform networks. We are primarily looking for a data-center (DC) network engineer — ideally with hands-on GPU installation and maintenance experience — who will also collaborate closely with our Corporate IT team on facilities, connectivity, and on-site operations. Work Responsibilities • Design, build, and operate Mercari's on-prem DC/GPU infrastructure, including data-center facilities (racks, power, cooling) and the supporting network • Design and build the network foundation for GPU workloads: compute fabric (InfiniBand/RoCE), storage/in-band, service network, and isolated OOB management • Design, configure, and operate routing for the on-prem GPU fabric and its cloud/platform interconnects using BGP (and OSPF/IS-IS, VLAN/VRF as needed), including ASN planning, route summarization, and routing policy • Coordinate with data-center providers and SI partners on facility readiness, equipment procurement, lead times, and remote-hands/installation work • Manage Mercari's internal data center end to end, including vendor/SI discussions, commercial negotiations, and contract & procurement handling • Perform hands-on, on-site data-center networking work — running and changing cabling, and configuring routers, switches, firewalls, and other network devices (this role requires regular physical visits to our data center) • Integrate the on-prem DC/GPU environment securely with existing corporate, cloud (GCP/AWS), and platform networks • Establish monitoring, alerting, logging, and operational handoff for GPU systems and network devices • Support GPU PoC execution and help define the path from PoC to production • Automate provisioning and operations, and manage configuration as code (IaC) • Collaborate closely with the Corporate IT team on data-center facilities, connectivity, and on-site operations (this collaboration is required for the role) • Collaborate with ML Platform, Infrastructure, Security, and Procurement stakeholders • Own data-center facility readiness (rack, power, cooling, carry-in routes), including NVIDIA pre-delivery site survey confirmation for DGX B200/B300-class nodes • Drive RFP, quotation, and vendor selection cycles with partners, and own NDA, R&R/RACI, contract, and PO handling together with Procurement • Prepare technical and cost materials for internal approval gates (JB Engineering board, investment committee, Procurement Team), including TCO/ROI comparison against cloud GPU/TPU • Build and operate storage and dataset paths for models and checkpoints, and keep the stack portable between on-prem and GCP/GKE so workloads can fall back to cloud • Define and operate SLOs, on-call, incident response, runbooks, and BCP/DR for the on-prem inference environment, aligned with Mercari SRE practices • Bring on-prem GPU and compute nodes to parity with Mercari's cloud security posture (asset visibility, endpoint/posture tooling, audited bastion, isolated OOB access) together with Enterprise Security • Track and report GPU utilization, power, and token-cost savings with FinOps as capacity scales from the first node to multi-node Unique Challenges • Stand up Mercari's first on-premises GPU infrastructure largely from the ground up, owning facility, network, and operational readiness end to end • Bridge cloud-native networking practices with traditional data-center and on-prem operations • Work with high-density compute systems (e.g., DGX B200-class) and high-speed fabrics (e.g., 400G InfiniBand/RoCE) • Make pragmatic decisions under evolving requirements and procurement timelines, while keeping the management plane isolated and the environment secure and supportable • Scale from a single 8-GPU PoC node to a multi-node environment without redesigning the foundation • Operate a hybrid model where on-prem small language models and external frontier LLMs are routed per use case and cost/sensitivity policy • Establish on-prem operational practice inside a fully cloud-native organization, including compliance expectations such as PCI-DSS and ISMS Qualifications Required Experience/Skills • Strong networking fundamentals (TCP/IP, IP address management, OSI layers 2–7, routing, DNS, TLS, HTTP) • Hands-on experience designing, configuring, and troubleshooting data-center routing and switching, especially BGP (ASN planning, route summarization, and routing policy), as well as OSPF, VLAN/VRF • Hands-on experience designing and operating physical/data-center networking (switches, routers, firewalls, structured cabling) • Production experience operating Linux servers • Hands-on data-center experience, including for GPU/high-density systems: physically installing and changing cabling and configuring routers, switches, and firewalls on-site (this role requires regular in-person visits to our data center) • Experience with data-center operations: racks, power, cooling, cross-connects, and coordinating remote-hands / installation work • Hands-on experience with data-center physical/facility operations, including power equipment and circuit switching (PDUs, breakers, circuit changes), and responding to outages and production incidents • Experience managing data-center vendors and SI partners, including commercial discussions and contract/procurement handling • Experience integrating on-prem and cloud networks (GCP or AWS), including VPN, proxies, and routing • Security mindset in network design and operations: firewall rule design and management, egress/outbound policies, and network segmentation/isolation • Ability to automate workflows using Python or Bash and to manage configuration using Infrastructure as Code (IaC) • Experience writing design documents and working with internal stakeholders and external vendors/SIers • Experience running production infrastructure