Building a UE4 Scene from a Picture

My name is Sean Napolitano and I am currently a Third year student at Champlain College.  My family comes from a military background, so I’ve grown up moving across the United States and a few location in Europe before “settling” in Virginia. I’ve been involved with art for quite literally as long as I can remember. I used to be a part of my various schools theater productions, primarily making props and set pieces. I’ve since begun learning how to do the exact same thing, but digitally.

I’m currently studying Game Art and Animation at Champlain, specializing in Environmental Art. Outside of a few Game Jams most of my projects are the various in class assignments, ranging from props, vehicles, environments and actual game productions.

The assignment for the Advanced Seminar class was to recreate a piece of concept are or a painting within 10 weeks. I recreated the piece “Placidity”, by Evgeny Lushpin in that time. One of the hardest challenges with recreating a painting such as Placidity is correctly managing the extreme perspective of the environment inside the engine – in this case Unreal 4. Placidity was able to bend and break some aspects of perspective due to its nature as a painting, and was able to create some impossible or at least very hard to reproduce angles. In the painting, for example, the green building on the right has floors about half the height of the red building in front of it, despite being joined together. Reconciling the painting with what Unreal 4 can manage was a constant challenge, mostly circumvented with lots of perspective tricks, such as with the railing and bridge on the left, and general smoke and mirrors. Another challenge for me was not getting overwhelmed with the scope of the project, having never attempted anything of this scale. Breaking it down into sections – left of canal, right of canal, far canal, background – and only focusing on one task at a time – “today I texture X part, Today I work on lighting” – was the backbone of my approach. 

My workflow consisted primarily of modeling of modular pieces and props in 3ds MAX, textures created in Substance Designer with some supporting props textured in Substance Painter, and the fully textured pieces assembled and presented in Unreal Engine 4. I started the project by bringing the painting into Photoshop, cutting out the different modules, identifying different base materials and trim textures, and isolating different props and elements to recreate. Early production consisted of the modeling of the modules that would compose the foreground structures, then onwards to the mid-ground, reusing modules where applicable or creating new ones if needed. Materials were added in passes after the initial structures where laid. Once the buildings where completed additional details where added (plants, larger props).  

The buildings are primarily made of 2 by 3 meter modules that contain various types of windows and frames. This way I was able to quickly make a majority of my scene using relatively few pieces, reducing the overall memory impact of the scene and helping it render and load faster.

One of the last passes I did was to enact the practice of “Breaking the 3D”. The goal was to break the strait, clean lines that I had left over from modeling, such as the hard line created by the end of the roofs or on the edges of the buildings. Sometimes this entailed the creation of larger more elaborate props, such as the various pulleys or the entirety of the steeple in the background. Other times it involved the creation of a small piece of texture to replicate across a module. For example, I made a singular brink and roof shingle, and placed them along edges of their respective modules to make them more random and organic.

Making the props

*I’m unsure what you mean by elements, so I’m assuming props and other additions like the water or the birds

Most of the props were created later in the project. Initially larger scale props such as chimneys or the tables were created and textured, with smaller and smaller props coming later. The various crates and Barrels where the last props created, primarily to fill the left most walkway and add some more interest to the scene.

For some of the more unique elements of my scene namely the water and the plants. The Water relies on translating two separate normal maps at different speeds in a shader in Unreal. The red and green channels where multiplied together, while the blue was kept constant. This was I was able to create ripples of varying height and in a semi-random pattern. In order to create the reflection I used a flat plane with no roughness within a planar reflection capture, allowing it to be distorted. The plants and birds rely on an opacity map, and in the case of the birds, an emissive so that they stand out from the background.

Materials

I made the materials for this scene in Substances Designer. This project was the first time I had used Substance Designer to create realistic materials, and relied heavily on the Allegorithmic tutorials and some of the tutorials by Sharpstance. One thing I tried to focus on was making sure that the materials where easy to instance within Unreal, and that the damaged, chipped versions of materials flowed nicely across the modules with and without vertex painting.

Lighting

Lighting this scene was an adventure. In the end I had two major lighting set-ups. One for the buildings and elements, and one for the overall scene. For the buildings I used mainly emissive lighting from the lit windows themselves, with a point light over top of each window. The windows used their base colour map to dictate the emissive lighting, so I ensured that the texture had variations in intensity (ranging from yellow-orange to a dull brown) so that it would look like the light was emerging from “within” the window. That ensured that the window itself was lit. The point lights provided the more ambient illumination of the structures immediately around the window frame, so that each section of the building felt more open.

For the more general lighting, I relied primarily of directional lighting, with a total of 3 directional lights to illuminate the left foreground, right foreground and the background respectively. To ensure that they did not illuminate the incorrect sections of the scene (the background light would adversely affect everywhere else) I made large blocks hidden behind the various buildings and the camera so that their shadows would shield the other parts of the scene.

Challenges

What wasn’t a big thing I learned during this project? The demands made upon me for this project where higher and stricter that anything I had made before. Lighting (of this variety) and shaders where completely new concepts for me to learn and understand. But the most significant skill for me was the planning, both before the work began, and during the project itself. Blocking out the time to properly make each asset and texture. Looking at the references and understanding (or trying to) what needs to be done and how. Breaking down a seemingly insurmountable (for me) assignment into smaller tasks that I can achieve.

Most of the problems I ran into where of my own making: trying to light the scene without researching how that process works, or having materials that simply look incorrect because I tried to re-invent the wheel making them instead of watching tutorials and learning how people have done things in the past. One significant example was the water. I was having difficulty getting the reflections to A) appear, and B) go in the right direction. I lost nearly 2 days of work just to fiddling with the water and trying to correct it before I finally sought help. Within 20 minutes it was fixed.

The most useful trick I was taught was that of breaking the 3D that had mentioned earlier. It helped bring the scene to life, made it feel more real. All I had to do was spend 3-4 hours placing bricks and shingles by hand. The only other thing (that isn’t even really a trick, just really nice to have learned) is using the Fractal Sum Base and the Slope Blur nodes in Substance Designer to create really nice looking stone chipping. For the professionals these trick might be mundane, but for me they were eye-opening.

Most of these “revelations” came from my professor Vincet Joyal – who pushed me each week to improve the scene – and my classmates. Wouldn’t have done it without them.