Fraunhofer Institute for Silicon TechnologySmart Window - Augmented Reality for tomorrow's ship management
The precisely transmitted image of the quay wall on the ship's window of the steering bridge, provided with some additional information, for example on current conditions and mooring possibilities - even in fog, darkness and poor visibility - this is what the ship navigation of tomorrow could look like. With the joint project "Smart Window", Fraunhofer ISIT has worked together with partners from industry and the Fraunhofer Institute for Computer Graphics Research (IGD) on an intelligent solution for implementing an augmented reality display on a ship's bridge in order to process, filter and display all relevant data for ship operation in the future. Embedding the AR display in a window of the ship's bridge offers the possibility of displaying a variety of information directly in the skipper's field of vision, thus making the skipper's work easier.
Fraunhofer ISIT has developed a scanning full-colour laser projection system as the central element for the novel AR display. The core of the system is a silicon micromirror. It is suspended on two perpendicular axes. The mirror can oscillate around these axes, excited by electrostatic forces. If laser light is thrown onto the mirror, such a mirror can be used to project any image onto the ship's window that is sufficiently large and well illuminated for the application. To ensure that the mirror achieves large deflection angles and does not become dirty, it is encapsulated with a glass cover at low internal pressure at wafer level.
The project as a whole is very demanding. An augmented reality view is created through the complex interaction of data processing, sensor technology and visualisation, which is usually created specifically for one viewing angle. In order to feed information into the user's field of vision in such a way that it specifically enriches reality, the digital model of the world must be brought into congruence with reality. For example, global data, such as position data from the electronic nautical charting system, and local data from the ship's own sensors, such as radar, must be fused and displayed relative to the viewer. In addition to enormous computing power, this requires information about the user's field of vision in particular, so that the superimposed data can be clearly assigned to real objects from the user's point of view. If the user moves, this must be tracked in real time and the virtual image adjusted accordingly. This tracking function is performed by a 3D camera, which must function robustly and accurately even in the low light conditions on a ship's bridge at night.
Another important component for AR projections on the windows of the steering bridge are suitable glass laminates for the windows. These consist of standard-compliant individual panes of glass bonded with foils to form a laminate, which must meet the strict requirements for suitability in maritime environments.
The Smart Window project has great significance for the increasing demands on the current transport and supply infrastructure in the face of increasing globalisation. Shipping plays a special role here. For while truck drivers and train drivers are tied to the infrastructure of the road and rail network and pilots can rely on electronic monitoring and control worldwide, ships are often still left to their own devices in many places. This requires a high level of decision-making competence and forward-looking data-based planning from the skippers.
"In particular, up-to-date comprehensive information on navigation, the weather and local peculiarities such as crossing ships is available in real time with Smart Window, as is information on the technical status of the ship and the functionality of many subsystems. Smart Window thus offers good approaches to further optimise safety in maritime traffic," explains the head of the project at Fraunhofer ISIT, Dr Shanshan Gu-Stoppel.
