%2051.png)
Tutorial #9: Audio/Frequency Mapping
In this tutorial, you'll learn how to use the Immersive Lighting System to render audio to light sources as if they based attributes of the audio currently being sampled. You can also create audio volumes to map them in world space.
​
EXPERIMENTAL FEATURE: This tutorial as well as the features are not final and may change. If you have any feedback, we'd love to hear your thoughts!
#1: Create an Immersive Lighting Rig to test rendering (optional)
​
Create a new level and add an Immersive Lighting Rig actor to the level. Set Simulated Light Radius to 8, Simulate Immersive Lighting to true, and Use Layout to true.
In the above example, we've created a test rig that simulates multiple light sources to test out the frequency mapping (we are simply using the example from the video/pixel mapping tutorial). We can now use these Emulated Light Controllers to render our audio against if we'd like to test it or we can instead render it to our real-world lights.
#2: Create a new IL Audio Performance and IL Frequency Map UAsset
​
Start by creating a new folder in your Content Browser called Audio under your IL folder if you've already created it as an example. Right-click in the new folder and under the Immersive Lighting menu, select Immersive Lighting Audio Frequency Map and name the asset DemoFrequencyMap. Open the asset so we can edit it.
Color Curve: The color curve to use for the Immersive Lighting Audio Performance
Min Frequency Range: The minimum frequency range to use for the Immersive Lighting Audio Performance
Max Frequency Range: The maximum frequency range to use for the Immersive Lighting Audio Performance
Frequency Interval: The frequency interval to use for the Immersive Lighting Audio Performance
Magnitude Sensitivity: The sensitivity of the smoothing applied to the magnitude if SmoothMagnitude is enabled
Smooth Magnitude: Whether or not to smooth the magnitude of the Immersive Lighting Audio Performance
​
The Immersive Lighting Audio Frequency Map assets determine how to map a range of audio frequencies to a LinearColorCurve. These can be used across any number of Immersive Lighting Audio Performances.
​
Now, let's create the IL Audio Performance asset that will use this new IL Frequency Map.
Frequency Map: The primary Frequency Map to use for the entire IL Audio Performance or the first layer of multiple frequency layers if 'Enable Frequency Layers' is enabled
Color Curve: The color curve to use for this layer if 'Override Color' is enabled
Height Color Curve: The color curve to use for the normalized height of each controller if 'Apply Height Color' is enabled
Depth Color Curve: The color curve to use for the normalized depth of each controller if 'Apply Depth Color' is enabled
Layer Curve: The float curve (based on the normalized height of the light within a controller array) to use to interpolate between the Frequency Layers if they are enabled. This curve needs to range between 0 and 1 to work properly
Layer Type: The type of frequency layer whether they are interpolated between factors for instance such as the normalized height or depth of the light within the controller array. To apply IL Audio Performances to both a set of lights for the height (on stage) or maybe using the depth (lights beyond the stage in the arena), use multiple assets and apply them to different groups of controllers
Max Magnitude: The maximum magnitude for this layer to use when normalizing the Layer Time if this Audio Performance is set to a Layer Type of 'Magnitude'
Frequency Layers: Frequency layers are optional and but offer more depth when customizing your Immersive Lighting Audio Performance. These are interpolated from the previous (or primary layer) to this one using a value ranging from 0 to 1 determining the normalized height of the light within its controller array
Enabled Frequency Layers: Whether or not to enable using multiple frequency layers based on the height of the Immersive Light Controllers. The way this works is that the layers at the bottom of the array are mapped to the bottom of the range (ie. 0 = floor) while the layers at the top of the array are mapped to the top of the range (ie. 1 = ceiling). This relies on both the ending corresponding Time values of each layer and the Float Curve Unreal Asset on how to interpolate between Frequency Maps
Override Primary Color: Whether or not to override the color curve in the primary Frequency Map
Apply Height Color: Whether or not to apply the 'Height Color Curve' to the overall performance. This multiplies the color gradient against the output based on the normalized height of the controllers within the controller array
Apply Depth Color: Whether or not to apply the 'Depth Color Curve' to the overall performance. This multiplies the color gradient against the output based on the normalized depth of the controllers within the controller array
#3: Setup the Audio Performance to be rendered using multiple Light Controllers
​
Let's setup the audio performance to be rendered starting on BeginPlay. Call Get Immersive Lighting System and then call Start Immersive Lighting. After that, call Set Audio Light Performance to set it as the active one. Call Start Audio Light Performance, Play Sound 2D with your SoundCue and then Start Analyzing Output. If you are testing this using the Emulated Light Controllers, call Set Audio Light Performance With Controllers.
​
For EndPlay, call Stop Audio Light Performance, Stop Analyzing Output and then Stop Immersive Lighting.
​
Start/Stop Immersive Lighting is called since we need it enabled to process audio performance assets. The feature will evolve as it nears a more production-ready state.
