IOI: In-network Optical Inference

In-network Optical Inference (IOI) provides low-latency machine learning inference by leveraging programmable switches and optical matrix multiplication. IOI uses a transceiver module, called a Neuro Transceiver, with an optical processor to perform linear operations, such as matrix multiplication, in the optical domain. IOI's transceiver modules can be plugged into programmable packet switches, which are programmed to perform non-linear activations in the electronic domain and to respond to inference queries. Processing inference queries at the programmable packet switches inside the network, without sending them to cloud or edge inference servers, significantly reduces end-to-end inference latency experienced by users.

Researchers

Dirk R Englund / Ryan Hamerly / Liane Sarah Bel Bernstein / Alexander Sludds / Manya Ghobadi / Zhizhen Zhong / Weiyang Wang

Departments: Dept of Electrical Engineering & Computer Science, Research Laboratory of Electronics, Electrical Eng & Computer Sci
Technology Areas: Computer Science: Networking & Signals / Sensing & Imaging: Optical Sensing
Impact Areas: Connected World

  • in-network optical inference
    United States of America | Granted

License this technology

Interested in this technology? Connect with our experienced licensing team to initiate the process.

Sign up for technology updates

Sign up now to receive the latest updates on cutting-edge technologies and innovations.