Fraunhofer Institute for Silicon Technology
Fraunhofer Institute for Silicon Technology

NeurOSmart

Fraunhofer lead project: analog neuromorphic accelerators enabling efficient and safe smart sensors.

NeurOSmart

The challenge

Autonomous Robot transportation in warehouses, Warehouse automation concept. 3D illustration

As mobile robot systems become more autonomous, the number of sensors increases, the effort required to link their data increases, and with it the need for computing power to realize reliable and safe real-time operation. Therefore, scalability of the architecture, sufficient transmission bandwidth between sensor and data processing, and minimization of power requirements are the main challenges for the development of high-performance computers to be used in mobile systems. It is predicted that in less than 10 years, the required computing capacity in the sensor periphery will have to match that of a supercomputer today. This requirement can only be met by a combination of hardware and software components specifically developed for each other.

Our solution: The NeurOSmart project

Autonomy of mobile robot systems

The NeurOSmart project aims to set a new standard for intelligent hybrid computing architectures in autonomous machines and transportation systems. For this purpose, a high-performance sensor system, AI-supported preprocessing and a novel high-performance, neuromorphic, ultra-low-power in-memory accelerator chip are combined.

The perspective is an increase in energy efficiency of data processing by at least two orders of magnitude.

Operations within the framework of the project

Microchip processor

The NeurOSmart approach focuses on the direct integration of data processing intelligence into the sensor system. This reduces a significant portion of the computational load on the part of the HPC system in an environmentally and resource-friendly manner, so that the computational hardware in the sensor system can be adapted to its requirements directly during sensor development in the codesign.

As a pioneer of integration in a competitive sensor system, NeurOSmart uses an open scanning LiDAR system developed by Fraunhofer as the sensor base to provide direct access to the incoming data streams. In addition, a highly scalable neuromorphic HPC chip is coupled with a sophisticated AI-powered pre-processing pipeline to interpret the data directly at the sensor.

In total, NeurOSmart bundles the technical expertise of five Fraunhofer institutes, of which Fraunhofer ISIT is coordinating through Prof. Dr. Holger Kapels. For Fraunhofer ISIT, NeurOSmart results in new exciting requirements and considerable synergies are created by linking the know-how between the five institutes.

Project details

Name

 NeurOSmart: analog neuromorphic accelerators that enable efficient and safe smart sensors.

Project type

 Fraunhofer lead project

Duration

 4 years (January 2022-December 2025)

Coordinator

 Fraunhofer ISIT, Prof. Dr. Holger Kapels

Project partner

 Fraunhofer ISIT, Fraunhofer IPMS, Fraunhofer IMS, Fraunhofer IWU, Fraunhofer IAIS

Funding

 Subsidies

Goal

 Increase energy efficiency of data processing by at least two orders of magnitude.

Five Fraunhofer institutes are participating in the NeurOSmart project, each contributing specific technologies:

© Fraunhofer ISIT/ Eric Shambroom

Fraunhofer ISIT

Fraunhofer ISIT is the coordinator of the NeurOSmart project: research competencies include the development and integration of piezoelectric and ferroelectric materials for microelectronic and electromechanical applications. Within the project, Fraunhofer ISIT is primarily responsible for evaluating AIScN as a revolutionary "next-generation" ferroelectric for use in ferroelectric field-effect transistors.

© Fraunhofer IPMS

Fraunhofer IPMS

One of the primary research competences is the development of memory technologies in advanced node CMOS realizations. For this purpose, Fraunhofer IPMS has an innovative ferroelectric memory technology (FeFET). In the context of NeurOSmart, the memory emulation is used, among other things, for software blocks to control the in-memory hardware blocks and to manage data streams.

© Fraunhofer IWU

Fraunhofer IWU

Fraunhofer IWU's area of competence ranges from machine tools, forming, joining and assembly technology, precision engineering and mechatronics to digitalization in production and virtual reality in the context of mechanical engineering. Specifically, Fraunhofer IWU has many years of experience in the research and development of safe human-robot systems as well as sensitive robots. As part of the project, the institute will evaluate the sensor system for applicability in an industrial environment.

© Fraunhofer IAIS

Fraunhofer IAIS

Fraunhofer IAIS has research expertise in the areas of distributed learning, speech assistance systems, and computer vision for autonomous driving. In this context, the institute has a powerful speech recognition system for automatic transcription and recognition of speech signals. Within the framework of NeurOSmart, the institute bundles its competencies for neural network training, among other things.

© Fraunhofer IMS

Fraunhofer IMS

The core competencies of the Fraunhofer IMS are the development of embedded software and AI, Smart Sensor Systems in the business areas Health, Industry, Mobility and Space and Security. The institute has developed AIfES (Artificial Intelligence for Embedded Systems) - a platform-independent and constantly growing machine learning library in the C programming language. AlfES is also an important prerequisite for NeurOSmart for the development of the hardware platform for sensor and scanner control.