Creating a Fantasy Medieval Environment in UE4

Introduction

Hi, my name is Raveen Pathirana and I'm a self-taught 3D artist looking to get into an environment art position. Currently, I'm chipping away at developing my portfolio with new works every few months or so. CGMA, Youtube, Gumroad, LevelUp Digital were my primary sources for learning – and of course, reading articles of breakdowns.

Medieval Environment: Concept and Reference

I've always admired fantasy-type games and environments with a focus around the middle ages – works such as Skyrim, Dragon Age, Witcher, Valhalla, and even Game of Thrones. There's a ton of concepts that are inspired by these but Ede László's work was very unique; it imbued an alien presence through the fungus, and that mix was what really inspired me.

Wood is obviously one, then stone, fungus, snow, props, and plants. To help me with producing the materials and assets needed I gathered references and grouped them into categories.

Each contains a PureRef file so that it's easier to view. These folders grow in content along with the project.

Blockout

The main house is interesting because it's not exactly a modular building. Of course, you can make it modular with certain methods, but I think that goes against the uniqueness of the concept. At first, I wanted to make the whole village but I knew that was difficult so I just focused on what the artwork illustrated. To breakdown the main building, I separated it into smaller parts.

Proportion and scale were one of the challenges with this scene and it's something that required a bit of time to get it right, but having a character reference, and the callout for the building made it easier to visualise. Scattering the wood was a little tedious but along the way, I did utilize the match transform modification to speed up the transform properties, which I'll show later in the article. 

For the fungus and its roots, it's the same process; start simple but still retain the rough silhouette. I made some simple shapes to block in the other assets and ensured its pivot point was on the center. With the blockouts for what was initially planned were done, I finally put them in the engine.

When doing blockouts, I really try to put a decent effort into creating the silhouette and not worry too much about topology. I'm not going to go in-depth about lighting right now since it will change; for now I added an exponential height fog with a few spot lights to block in lighting.

Sculpting the Fungus

Now that we've discussed blockouts, let's dive into the sculpting phase! I'll start with the fungus purely because it's the most complicated part. Finding the references was quite a challenge since there's no accurate description you can google. So I decided to look into bark and even abstract art and discovered some neat shapes, directions, and forms. The idea wasn't to get every aspect here projected into 3D but to be inspired by the way it flows.

Overall, I only made two sculpts of fungus which saved time as you can rotate them and get about three variations from one. Utilizing the fast 3DCoat retopo tools, I made a low-mid poly object, to get more control over vert painting and deformation in Maya.

Looking at the callout from Ede's work helped a lot with adding details and texturing the bake. It was a simple process for the texturing, I layered the fungus with subtle color variations such as red, pink, purple, and blue, added some micro bump detail, then with a dirt brush and a mask I painted in the exposed part in a muted yellow. Finally, to set it up for UE4, I added an emissive mask biasing towards the concave areas.

I grouped the fungi together in Maya and used the lattice tool to taper the objects as a whole. To visualise how it will look in the engine, I applied the AO on a new lambert material.

Now that we've discussed the main body of the fungus, I'll touch on the roots. This part again took a few iterations because the issue I was facing was not to make it too high poly but enough to retain the density. That's where I used SpeedTree.

I imported it in UE4, rotated it, and then added a basic material. For the main house with the wood surrounding the fungus, I exported it from UE4 and made adjustments with the lattice deformer in Maya. 

All other props including foliage were modeled and textured by me and here is a selection rendered in Marmoset Toolbag:

Most are unique assets but some use tileables and a few use trims (from baked assets). The cover and sacks were modeled in Marvelous Designer, here's the tutorial those were inspired by:

Vegetation

Vegetation was mainly grouped inside of Maya with the exception of one set, which was done in SpeedTree. First I looked at my references and concept to see what type of foliage would work best. These were my main refs but there were a lot more variations. 

I tried multiple methods to create leaves like building them off Substance Designer and scanning, but I personally stuck with sculpting them in ZBrush and then texturing with Designer. The main brushes were Inflate, DamStandard, and Move.

Compared to the stone brick, I used symmetry a lot to speed up the process, then tweaked the shape and silhouette in certain parts and made variations like that. 

As mentioned earlier, I textured the leaves with Designer. To do so, I started out with importing the PSD height files extracted from ZBrush (use psd, not jpg or png) and worked with one of the heights to slowly build the material. The same principle as the fungus – start with subtle color variations and add micro bumps, then work on roughness, then AO. Finally, I exposed certain parameters, especially for the color, selected inputs, and grouped them as an atlas using Material Blend.

To project the texture onto a mesh, I first cut the silhouette out with the multi cut tool and added a few edges to curl the shape. I lined them up to create a circle, scaled the group down, and duplicated it to create an arrangement similar to the concept.

Using the same texture as for the leaves, I made a leaf mesh inside of SpeedTree that stems from a big branch node. I duplicated the leaf mesh to create some variation and that's it. Again having the character and concept reference really helped in making the clusters.

Texturing

All the assets were textured inside of Painter. The general workflow to texture the props was similar to the fungus: looking at references and slowly building the layers. Watching tutorials on how to make a solid wood texture really helped in developing my own wood. One of the better ones was Derek Elsof's tutorial at LevelUp Digital. After making a nice wood everything else becomes easier since that foundation is there.

Prioritising the wood was crucial for me – if you look at the concept it's dominated by wood. For the planks that support the main building, a lot of the work comes from the material rather than the sculpt. Once I've textured an asset, I usually make a UVMask input for UE4. Hamish Ames' free PDF on UVMasks explains it more in-depth. Assets that required more detail and control in variations had a UVMask associated with them.

Materials that were created in Designer were the hay, wood, wood plank (which was generated easily from the wood), rope, mortar, fungus, and wood. There were variations of these materials but generally speaking the base was created in Designer. For the snowy hay, I mixed a photo and hay in Alchemist.

Personally, I think the stone brick was the trickiest surface to imitate, especially in the engine. Again having good reference really helped in slowly defining the material. At first, I wanted to see how far I can take it in Designer. Creating believable stone bricks is much harder than it seems, so I decided to go with the sculpt to Designer to Alchemist workflow. Also making it in ZBrush helped in extracting individual bricks to place around the scene.

To start, I again looked at the reference, broke down a section of the photo, made a rough tileable mockup, and added necessary changes. From here it's nothing too special, just looking at the reference, understanding how to break up the bricks, and most importantly staying patient! Flipped Normals goes through the process of making tileable geometry here:

I didn't sculpt in the mortar since the ref had a relatively flat exposure, so I made and blended it in Designer. To break down the material, I identified the major changes in pattern from the ref. Stone, mortar, lichen, and fungus seemed like the substances that made up the overall structure. In Designer, I textured the stone wall as with everything else, slowly building up the details and variation in colour. I ended up exposing a few parameters to get more control over the lichen. This is where I used Alchemist to emulate the "moss + lichen" effect. The gif below should explain it better. Alchemist can be a little laggy so down res the texture to 1k and when you're happy with the result, go back to whichever scale you started with.

To set up the main building, I brought in my baked wood and roof tiles, sculpted in ZBrush and textured in Painter. I applied different vertex colors to respective UV sets. The last three wood pieces were modeled in Maya with the sculpt tools and used the tileable wood made earlier.   

Since the blockout was established and the materials made, modeling the main building was a fairly seamless transition. To demonstrate the process, here's how I combined everything together. I utilised the match transform to quickly get the position of the target, instead of having to manually rotate, scale, and move meshes. For this to work, both selections need to have the same pivot point, so if it's in the middle for one it should be so for the second. Also, make sure not to freeze transformations.

To check the tiling of the stone, I added a new lambert material with the AO of the stone brick.

Once the meshes were in the engine, I finalised my master material after some experimentation. I found that POM was a better option given that I wanted the stone to peak out more than a regular vert painted material. POM is a little expensive but it's cheaper than tessellation; of course, bump offset would be an alternative but for a personal piece like this I stuck with POM. Most of what you see here is based on Lincoln Hughs's Free Parallax Occlusion Materials, and it's not as complicated as it seems. I blended the layers with a height lerp with the option for a third layer – an optional puddle layer, an optional dither, and optional tessellated snow. Static switch parameters are great in giving the option if something's not required.

For the landscape material, I tried out four layers but really ended up using only two. The landscape also uses a POM node. I've added that inside a material function along with a few input parameters regarding color, height, roughness, and normal.

The ground blends in action, including landscape and POM. 

Having outstanding materials means you can easily create variations and hook them up in the engine with few issues. For example, the mud that was mainly created in Designer gave me the opportunity  to expose nodes and make similar materials. Furthermore, adding details such as water was easily achieved inside of Alchemist with its default water filter.

To further add details along the landscape, I used simple planes, one flat and the other had a small bump. The larger flat one has two layers with the puddle and tessellated snow. With the smaller ones, I made multiple instances with different textures, with a regular lerp instead of parallax. To blend in the planes with the ground, I used a simple dither.

Props don't need parallax, so I made a regular vert painted material using a height lerp with a switch for a UVMask.

The fungus material took a few iterations; I tried out SSS but ended up with emissive to make it cheaper to render. I added blend masks to isolate areas that needed to be vert painted for color, emissive, and the time node. This node had a simple panner associated with it to create a slow-moving pattern. I had an option to switch between either the tessellated snow or detailed z-up snow.

For portfolio renders, optimization wasn't at the forefront, but it's worth being mindful of shader complexity. In the test level, you can see that the terrain is the most complex one, then it's the stone areas – that shows how expensive POM can be, so use it with caution!

Composition

Composition was a smooth phase since it was set up during the blockout. Having a nice concept like Ede's helps in figuring out how to fill the scene with details. However, after taking an hour of feedback from Kyle Bromley at the TMC, I wanted to see if I could push or change a few things. That's where I did some paintovers in Photoshop including a rough color grade. 

Not everything was added in, for example, the hanging was a cool idea but ended up being too distracting. However, the majority of it was, which helped in filling the scene.

Models such as the snow clump on the roof and ground, sacks, stone trims on the edges of the walls, buckets, ladder, rope, nails, candle, and even clovers really help in creating a sense of scale and add a bit of story as well. To approach their placement, I like to think of clumps or density and areas of rest. So the sides are generally quite busy with props whereas the ground is fairly simple, the textures are doing the heavy lifting.

Preparing the Renders

Before I added in any lights I played around with the exponential height fog to get a good base. There was an atmospheric fog but it didn't end up doing much. Here're the settings for the height fog:

This was the most crucial step for the lighting to look right along with fog cards. For lighting, I used a few spot lights to start off with, then a directional light with a low intensity to bring in a global volumetric fog, then point lights to fake some indirect lighting and accompany the fire. All lights were set to moveable.

Settings for spot lights. Point lights were secondary so they were at a low intensity.

After setting up the lighting, I moved onto gradually adding fog cards. The main building is the main focal point so making the background lighter in value helps create contrast. To strengthen the focus, I added a torchlight near the centre to highlight the door. 

For the final renders I had these settings:

Kyle gave some great insight into how to change the color around using the camera raw filter in Photoshop. There's some great filters to play around with and the final renders were the result of this. Pushing the greens and reds accentuated the alien mood and gave it a toxic haze. The original color wasn't all that bad but I quite liked the pop the new colors gave. To start, I changed the cyans, then moved on to adjust the whole image inside of the camera raw filter. Once I got a color grade that I liked I copied the smart object filters into a LUT taken from the UE4 page. The result wasn't perfect in the engine so I had to adjust a few areas in the contrast and shadows.

Challenges

The main challenges behind the project were finding excellent reference, making things feel believable, and coming up with technical solutions to creative ideas. There were things that could've been done a lot better, for example, optimization is something that took a backseat along with architecture. But given that the focus was on texturing, lighting, and composition, I'm pretty happy with the result. 

The piece of advice I'd give is to stay patient. If you can hit a high visual quality you can do things faster in the future, but if you keep rushing work you'll find yourself restarting a lot. Of course, you need to call it done at some point, and that's where asking for feedback in Discord groups, on Reddit, Polycount, or anywhere else can really help! This project took around four months, a little on and off towards the end though. Without the patience, I wouldn't have pulled this off. Also when choosing an artwork to recreate in 3D (not just an environment), choose something you'd like to see in a game or cinematic, and set a goal you'd like to accomplish. It could be texturing, composition, level design, modularity, lighting, modeling, foliage, terrain, or even FX.

If you've come this far, thanks for reading and I hope it helped in some way!

Raveen Pathirana, 3D Artist 

Interview conducted by Arti Sergeev