Architecture¶
Kanzi VR consists of these components:
- Streaming Plugin
Streaming Component
Receiver Node (2D/3D)
Streaming Kzb Player
- VR Plugin
VR Render Pass
VR Viewer
The Streaming Plugin brings the HMI content to the virtual environment screens and the VR Plugin makes it possible to experience the environment through virtual reality headsets.
Streaming Plugin¶
With the Streaming Plugin, you can see and interact with the HMI content in the virtual environment. This is done by streaming the contents of each HMI display with usually separate source Kanzi applications to the target virtual environment.
The Streaming Plugin provides Kanzi Studio plugins for both sending and receiving sides. You can start the receivers and senders in any order.
Sending¶
The Streaming Component captures the visuals of the node to which it is attached and offers to send those visuals to interested recipients over the network.
To use the Streaming Component, import the Streaming Plugin to your Kanzi Studio project.
You can attach a Streaming Component to the Screen node of the sending project, or you can attach multiple instances of the Streaming Component to individual nodes.
The streaming works from the Kzb Player or Kanzi Studio Preview. A simplified approach is to use the Streaming Kzb Player that attaches a single Streaming Component to the Screen of any kzb file at runtime. This makes any kzb file streamable.
An option to using the Streaming Component is to stream the whole screen of an Android device using the Android Streaming Application. This way you can also stream parts of a screen that was not created with Kanzi.
With the Streaming Plugin, you can stream input. For example, this enables you to interact with the streamed content using a mouse or VR controllers.
Receiving¶
The receiving Kanzi Studio project, which is usually the virtual environment, such as a car cockpit, uses Receiver Nodes.
The network location of the sender is configured for each Receiver Node. The Receiver Node gets the streamed visuals over the network and mediates any input back to the sending side for processing.
The receiving works in either the Kzb Player or Kanzi Studio Preview. To receive streams to a virtual environment shown in a VR headset, you must use the VR Viewer.
VR Plugin¶
With the VR Plugin, you can view and interact with the virtual environment. The functionality is available through:
A Windows desktop application called VR Viewer
Kanzi Studio Preview
You can use either of these approaches:
Use the virtual environment with a mouse through the VR Viewer or the Kanzi Studio Preview.
Both render the virtual environment in 3D.
Use the virtual environment through a VR headset and VR controllers.
This is an optional mode in the VR Viewer and not available in the Kanzi Studio Preview.
The VR mode of the VR Viewer converts the Render Pass of the scene of the kzb file into a VR Render Pass, making it optional to refer to the VR Plugin in a Kanzi Studio project. The VR Render Pass renders the virtual environment to a format supported by OpenXR that can translate the VR experience to a multitude of prominent VR headsets.
The mouse or VR controller input is translated into input in Kanzi, making it possible to click and drag Kanzi components, such as buttons, and content on the streamed displays.