positional constraints over time as a space-time optimization problem in the tangent space of the curves driving the animation controls. Their method has the key properties that it allows for the manipulation of positions and orientations over time. https://www.lost-identification.com
Justen Lazzaro talked about his fan art VR scene Ratchet & Clank and a prepared a detailed breakdown of the ultimate trim technique and animated VR forcefield material. The scene is partly made with the help of Ryan Benno and his mentorship program.
I’ve always been particularly captivated by the environments of my favorite video games. These fantastic places tend to stick with me long after I’ve finished playing the game as if they were real places that still exist somewhere just beyond my reach. If only I could book the right plane ticket, I could show up in some of my favorite game worlds… at least that’s always been a dream of mine.
The first time I put on a VR headset I started to realize that my dream was beginning to come true. I actually could go to these worlds… and stay in the comfort of my own home! Once I realized this, I decided to try my hand at building an environment from one of my favorite game universes, Ratchet and Clank.
Specifically, I decided to build a room from the Nebula G34 space station, from the PS4 remake of the original Ratchet and Clank. At some point during this environment creation journey, I had the privilege of getting mentored by Ryan Benno through his mentorship program. Ryan is currently a senior environment artist at Insomniac Games and worked on the 2016 Ratchet and Clank game as an environment artist.
Ryan helped me with several pieces of the production of this environment, such as ultimate trim texturing, as well as creating an animated force field shader, both of which I’ll be breaking down below. But beyond that, Ryan pushed me to tell a different story with this environment than what was told in the PS4 video game.
Since the PS4 game is actually a remake of the original PS2 Ratchet and Clank, I attempted to incorporate some of the elements that existed in the original game that didn’t make it into the PS4 version, such as the character Fred and the Alien Queen enemy (which was the first boss fight in the entire series).
If you have a Steam VR compatible headset, such as the HTC Vive, Oculus Rift, or a Windows Mixed Reality Headset, you can download and explore this environment in VR!
Ultimate Trim Breakdown
If you’re not familiar with the ultimate trim process, there’s more info available in the GDC presentation The Ultimate Trim: Texturing Techniques of Sunset Overdrive.
Trim Base Creation
The idea behind the Ultimate Trim technique presented by Insomniac games is to create a standardized trim layout so that trim materials can be easily swapped between assets without changing any UV mapping. This layout consists of a series of horizontal trim strips with 45-degree normal map bevels. Each subsequent trim has roughly twice the vertical resolution as the previous strip. A final row at the bottom allows various other texture elements to be included for further flexibility.
My fan-art project for Ratchet and Clank involved using a total of 6 trim strips if you count the bottom section. I decided on the trim heights displayed here, based on 1K texture resolution for mathematical simplicity.
Next, I created a 1024 x 1024 cm square in my 3d app and high-poly modeled the trim segments, making sure to keep a 45-degree angle at the edge and match the trim heights outlined in the image.
Baking and Detailing
This project required 2 normal map patterns, with one of those patterns getting a second, relatively minor variation.
- Painted panels with metallic elements
- Metallic panels
- Painted Metallic Panels.
- The same as variation 2 but with paint chip details around the edges of the 1st and 3rd trim strips.
- Some trim strips from this material weren’t used in this environment, so I didn’t spend time detailing anything that wouldn’t be used.
ID map 1
- Created in photoshop to allow easy separation of painted and metallic materials in Substance Painter.
ID map 2
- Created for easy separation of different base metal materials in Substance Painter.
The chipped paint look was initially created by painting in photoshop using NDO. A mask was exported to allow for easy color variation in Substance Painter between the chipped and unchipped areas.
Later on, I accomplished this step using only Substance Painter by painting a mask over a fill layer and adjusting the height and base color settings to get the results I wanted.
Roughness maps were created in Substance Painter using a combination of procedural map blending and hand-painting.
The painted trim materials for this project have a relatively solid color and each color variation is handled by separate material instances. Textures were kept greyscale so the colors could be set up with color selectors in the material.
Color Selection in Shader
As shown in the image, metallic colors are masked out prior to color selection, then added back in afterward.
While the Ratchet and Clank art style works well with bold, solid colored materials, it’s still useful being able to add secondary color info if necessary.
I created a color mask to allow mixing in secondary colors with color selectors in material instances, very similar to how the primary color selection is accomplished.
Two color selectors allow for replacing the bright values of the mask as well as the dark values. This is then mixed with the primary color info using a “soft light” blend.
Blending happens prior to the metallic color being added back and an opacity slider controls the intensity of secondary colors.
Secondary Color Selection in Shader
- Using this method, three texture sets resulted in nine different material variations.
- 1 All metallic.
- 6 Painted (with metallic elements).
- 2 Painted metallic.
It’s also worth noting that I wrote and release a free add-on for Blender that considerably speeds up the process of UV mapping ultimate trim textures.
Animated Force Field Shader
Translating to Virtual Reality
This material is based on a force field material from the Ratchet and Clank PS4 game.
One of the major lessons to understand when making content for virtual reality is that not everything that looks good on a TV or computer screen will look good in VR and vice versa.
This project was made for the first wave of consumer VR headsets, the HTC Vive, and Oculus Rift. These headsets only have 1080×1200 pixels per eye, which results in a “screen door effect” where you can see blank space between pixels as if you’re looking through a screen door.
The PS4 material has a reasonable amount of fine detail when it comes to the scanlines and noise patterns around the edges.
My first pass at creating this material had a similar level of detail to the material by Insomniac Games and looked great on my computer monitor. Unfortunately, in VR it looked terrible, due to the limited resolution available. So, one of the major challenges was to create a suitable translation for VR.
This problem was solved by taking a “broad strokes” approach. I kept shapes large and avoided making anything too fine in detail.
It was also essential to parameterize critical elements of the material to allow for quick adjustments without needing to re-author textures. This helped with bouncing back and forth between the computer screen and VR to see if my edits helped.
The forcefield mesh for this environment is a very simple planar shape. As you might expect for such a simple mesh, the UVs are also simple, but care was taken to leave some empty texture space for the animated scanline to rest in when it’s not visible.
A total of five grayscale textures were used to create this material. Four images were packed in the RGBA channels of one texture file, while the remaining image was imported as a solo grayscale texture.
The material is set up for translucency and is unlit. Only emissive color and opacity inputs are used.
While I experimented with using texture info for opacity, the final material only ended up using a scalar parameter, so virtually all of the material information is being fed into emissive color.
It’s also worth noting that this project used Unreal Engine’s forward renderer, so results may differ from the default deferred renderer.
Hex Pattern Setup
Every texture in the material was set up with a parameter to control the UV scale using multiplication with a TexCoord.
Here, the “electricity” texture is fed into a panner node so that it scrolls across the hex pattern. Two parameters control how fast it moves in the X or Y direction.
The electricity image is subtracted from the hex pattern so that only bits of the pattern remain.
This “output” image shows a rough approximation of what is actually happening in the final material instance at this stage, with scaled UVs.
To apply color to the grayscale output of the last step, a Lerp is used with two color parameter inputs.
The blacks get replaced by the base hex pattern color, and the whites become the “electricity” color.
Since the center portion of the hex pattern shouldn’t be colored yet, another lerp is used with the original hex texture as alpha and input A as pure black. The colors that were just generated remain and the center portion returns to black.
Again, this “output” image shows a rough approximation of what is actually happening in the final material instance at this stage, with scaled UVs.
The hex pattern color is finalized by adding a color into the gaps of the pattern.
A parameter controls this color, which is multiplied by an inverted version of the original hex texture so that this color won’t have any effect on the hex lines.
This gets added to the results of the last step to finalize the base emissive color.
Note: The “gap color” actually gets added in a later step, but I’ve opted to show it here as a basic illustration of what’s going on.
Base Hex Pattern/Electricity Result
Edge Noise Setup
Edge noise was created with two panners that are essentially the same as the electricity panner setup.
The “noise lines” texture is used in both of these panners so that the texture can be panned in opposite directions.
The results of the panners are added together prior to the color being applied.
Edge Noise Mask
Since the noise is only meant to be seen around the edges of the mesh, a mask was used.
In addition to being able to scale the mask texture, I created parameters to manually adjust its position in X and Y.
Edge Noise Color and Masking
The colors of the noise lines and some additional elements were originally set up so they could have a pulsing pattern by lerping between two colors and using a sine node as the alpha. This ended up being overkill in VR and wasn’t used.
The noise lines were colored by multiplying them with a color parameter. That output was multiplied by the mask to remove the lines from the center of the mesh. Then the hex and gap colors were added in.
Hex with Edge Noise Results
The scanline is set up with a combination all of the methods used on the previous textures: Tiling scale, adjustment parameters for X and Y, panning speed for X and Y, as well as a color selection parameter.
An exclusion blend is used over the top of the results of all the previous operations. This creates a green trailing color as the gradient passes over the “electricity” colors of the hex pattern.
I found basic (non-changing) material animations to be suitable for every part of the material except for the scanline. I wanted to add some irregularity to the scanline movement. I also wanted to use an electricity sound cue that would only be audible when the scanline was visible.
These two things were both accomplished by manually keying animations via a timeline in the blueprint. The material was configured as a dynamic material instance, the scanline location and sound cue volume parameters were exposed, and a looping keyframe animation was created.
About Ryan Benno’s Mentorship
Ryan’s mentorship is an amazing way to get one-on-one feedback from a professional environment artist! This is incredibly valuable for a self-taught artist like myself, but I think it could be just as valuable for somebody with an art school background.
It’s worth mentioning that between working full time and teaching several mentees each week, Ryan stays pretty busy. So ultimately, you’ll get the most out of the mentorship if you come into it with a clear idea of what questions you have and what you’d like help with.
I made sure to write down every question that popped into my head throughout the week so that I could ask them during our session. Sometimes these were directly related to techniques or problems I was having with my project, but I probably asked just as many questions about working in the games industry in general.
If you’re at the point in your art journey where you think getting professional feedback on a weekly basis would be beneficial, then I highly recommend Ryan’s mentorship! It was a tremendous help for me.
Justen Lazzaro, Technical Environment Artist