EasySky: Breakdown of a Procedural Skybox for UE4

Introduction & Career

My name is Ron Kamphuis, I was born and raised in the Netherlands. Games have always been my passion since I got my first PC. I remember my first steps in Doom and my first experience with SimCity. I've played these games for more hours than I'm proud to admit. But it was only until Unreal Tournament that I really started to play around with level editors. Unreal 99 is the first Game Engine that I familiarized myself with. I guess this was around the year 2000. First, I only added existing props to existing levels, but soon I started to create my own levels. Back then, levels were made with BSP volumes. It was a very easy but tedious job to add all of those volumes and move the vertices one by one.

In the years 2002-2006, I joined a modification team for a game called Tactical Operations: Crossfire. The game was a modification for UT2004. I joined as an environment artist and made a level for it. It was a fun experience for me, one that taught me a lot about game development and eventually got me into the professional games industry.

In 2007, I started working professionally in the games industry for the first time. I was an Environment Artist intern at The Chronicles of Spellborn. This game was an MMORPG based on a heavily modified Unreal 2.5 engine. I worked there for almost two and a half years before the company closed doors, and I moved on. I helped out on numerous levels throughout the game. To this day, those are still some of my most memorable years in the games industry.

After working at The Chronicles of Spellborn, I moved to some smaller companies and even had to leave the games industry for 2 years before I joined Vanguard Games in August 2013 as an Environment Artist. I worked on several games there, including Halo Spartan Assault and Halo Spartan Strike.

At the start of 2016, the company started to shift focus to VR. It was something I was unfamiliar with, but the first time I tried it, I loved it.

The company re-branded to Force Field Entertainment, and I started to gain more interest in the technical aspects of game art. During the first VR game that we made, Landfall, I took on the role of Technical Artist. I learned a lot about performance optimization and shaders. I also showed interest in the general art workflow and pipeline in the studio.

Nowadays, I'm the Principal Technical Artist at Force Field, responsible for the GPU performance, shaders, and art workflow of all of our release titles. We recently released several titles for the Oculus Quest, including National Geographic – Explore VR, Time Stall, and the Anne Frank House VR.

Sky Production: How it Developed

Skies have always played a huge part in games, as they add a lot of immersion to it. The implementation of skies has developed over time. Back in the early days, skyboxes were mostly created by cubemaps. This can be seen as an unfolded cube with a sky material on there. It's basically a picture of a sky. The cube is then attached to the player camera, so it will always move with the player, creating the illusion of something that is very far away.

But just using a cubemap had a couple of downsides. It was all non-animated, so skies were very static, and it had no parallax between different background elements.

Unreal Tournament did some clever shader work to render moving clouds on top of this. And, in some levels, they rotated the skybox to create an illusion of a rotating sky, but it was still very basic.

As hardware continued to improve, skies got more realistic and better looking. The source engine allowed developers to break up the skybox in separate elements. The skybox was not attached to the player, but moved along with him, at a slower speed. This created a better depth illusion since the player was able to experience depth between some of the skybox elements. (Far away buildings, for example).

Another technique that was becoming more popular was using skydomes. These had a different approach than using cubemaps. It was made by a hemisphere and contained just one texture. Directions were easier to predict in a shader. This allowed developers to more easily pan clouds over them for example.

In the Last of Us, the people from Naughty Dog used these skydomes to create very cheap but realistic-looking moving skies with deforming clouds. For this, they used a technique called flow maps. A flow map is a very simple technique, that's being used in the games industry for years now. It warps a texture map based on a flow map. The 3 channels of the flow map control the direction where the pixels warp to. They loop this effect by blending 2 flow maps together.

After that, hardware developed even more rapidly, allowing developers to create even more realistic skies. One of the best procedural skies I have seen so far is the one from Horizon Zero Dawn. The clouds in the game are completely procedural and can create almost every weather scenario you can imagine. During Siggraph 2015, they presented their real-time volumetric clouds for the first time. It took the team years to develop these skies, but the results are quite stunning.

Article source: Guerrilla

About EasySky

The idea behind EasySky is to make a procedural skybox accessible to anyone. Creating a skybox is very time consuming, and in the case of Horizon Zero Dawn, it took the team years to complete.

My goal was to create something that looks good, and performs well on a lot of platforms, including mobile. My years of VR and mobile VR experience helped me a lot here because I have to deal with those hardware restrictions there as well.

Before I started, I defined some keywords that described the focus points of the project. Every decision I made afterward had to be in line with these keywords.

• Easy to use
• Should work out of the box
• Professional looking
• Multiplayer friendly
• Well documented and/or self-explanatory

The package itself contains a single blueprint. Anyone could just place it into a level and have a working skybox. Of course, parameters could be tweaked to make it look unique for your level.

I think what makes EasySky unique is the fact that it's very performance-friendly. The whole sky, including clouds, sun, moon, and atmosphere, is rendered in 1 shader. This eliminates overdraw and makes the sky perform very well. The shader has a little over 300 pixel shader instructions, and +- 30 vertex instructions. 

It contains several features that a sky should have. All features can be modified to the user's liking.

It includes features like:

• Day / night cycle
• Dynamic clouds
• Sun and moonlight
• Atmospheric light
• Fog
• And more

The full list of features can be found on the documentation website.

Day & Night Cycle

The day and night cycle was the first thing I implemented. I took some time to do some proper research first because these things can become quite complex. I had to learn things about northern latitude and moon phases. I watched a couple of videos that explain the rotation of the moon, earth, and sun in detail. After that, I started implementing the Northern latitude.

In real life, the earth is tilted a little bit, This influences the amount of Sun that you will see based on the time of the year. I thought about implementing this, but figured it wasn't worth the extra calculations needed to do that. I kept it simple. Most of the users are only interested in a generic day/night cycle. For example, the sun comes up at 6 AM and goes down at 6 PM. I figured it was more important to set the maximum angle of the sun. (at noon that would be right above you or in front of you). This was far easier to implement and doesn't cost a lot of processing power.

The calculation was rather simple. 

In the figure above you can see how I calculate 3 vectors: The sun rotation at 0 degrees and the sun rotation at 90 and -90 degrees. If the Northern Latitude setting is negative, I take the negative rotation value, if it's positive, I take the positive value.

The input for this function is something that I called SunWeight. It defines the relative location of the sun. If the sun is down (not visible, night time), SunWeight is 0. If the sun is at its highest point, the weight is 1. This sunweight is stored in a simple curve. Users are free to change the curve to their liking. 

Two extra points were added to make the sun rise and set faster.

The moon is done in a similar but slightly different way because it doesn't match the same speed as the Sun. A full moon cycle around the earth takes +- 27 days. This mismatches the sun cycle, so I had to implement this a little bit different. The principles are the same though.

Lighting

The sky contains 2 directional lights and a skylight. All of these are dynamic because they need to change the scene over time. The sun's color and intensity are also determined by a curve. This offers users an easy but flexible way of adjusting the colors.

Besides the sun color, there is also the sky color that can be altered. At first, I extracted the sun color and created a smooth gradient over the entire sky with it. But it made the whole sky look very bland. This is because a sky doesn't always have the same color as the sun. There is usually a glow around the sun, but the particles in the air that you see on the other side of the sky, are usually just blue. That's why I added another curve to be used as sky color. After some testing, I figured that one value for the entire sky was enough. The fog would blend the sky with the ground, and I used a simple gradient calculation in the shader to mimic a sky gradient.

Last but not least, I needed some ambient light. Just using 2 dynamic directional lights wasn't enough. It creates ugly black shadows, so I needed something to fill this up. Using a skylight was the most obvious choice here, but it gave me a problem. The skylight uses a cubemap to fill in the ambient light. But since my skybox was fully dynamic, I either had to update this cubemap every tick or deal with the fact that the same bounce light would be used during the night and day time.

I tried it, but it looked bad. Day time was getting too dark, or night time too bright. I decided that I pre-rendered 2 cubemaps. One is used during night time and the other one during the day time. The cubemaps blend seamlessly during their transition time. After some tests, I figured that 1 hour in advance of the sunset or 1 hour after the sunrise looked best because the dark cubemap of the night would emphasize the bright sun intensity and color during these times.

A huge part of the atmosphere is also defined by fog. Fog creates mood in a scene, and I used it to blend the ground with the skybox. What's even better, the exponential height fog from Unreal is pretty cheap, so performance won't be affected too much.

Sun & Moon

The moon and sun share the same function to project a texture on a surface. In order to do this, I did some projection math. First, I calculate the projection plane. This plane's normal is the Sun Vector multiplied with -1. In the image below it is shown in orange.

Now, all we need to do is calculate if and where a vector would hit the plane normal. If we extend any given vector, we can check if we ever hit the plane. If it would, then we know which pixel of the texture we need to draw. Of course, this needs to happen in 3d, but this picture explains rather well what we are trying to achieve. 

Now let's have a look at how I implemented this in the material itself.

Inputs (left), are the absolute world position (the world position of the pixel), WorldCenterPosition (Center point of the sky, this is the point where we project from, and ObjectVec (The sun or moon vector).

The calculation after the ObjectVec calculates the normal of the projected plane. With a simple Cross product with the up vector (0,0,1), we can extract the side vector of the Sun. After that, we can do another cross vector with both the forward and side vector to get the up vector. This should give us the plane normals that we were looking for (Up, side and forward). After this, we can check all incoming vectors and see where they would project on this plane.

The projection itself can be done with a function that Unreal already provides. It's called InverseTransformMatrix, but it's really just 3 dot products. The vector to Transform is the vector we want to project onto the plane. The BasisX, Y and Z vectors are the 3 direction vectors of the plane that we just calculated. The final part on the right is to scale the texture. 

The last thing we need to do is make sure we only get the positive side of the Sun/Moon vector, so we do a simple dot product to calculate this. This can be seen at the top of the screenshot.

In a future update, I will move the plane calculation to the blueprint, because there is no need to calculate this for every pixel. 

The cloud system is something I spent most of the time on. Clouds are very complicated, and one could spend months trying to create convincing, good looking clouds. I've tried multiple things myself here as well. First, I went for the flow map approach that Naughty Dog showed in the Last of Us. But it was not flexible enough. There was no way to increase or decrease the number of clouds, and it relied on a sky texture rather than procedural clouds.

I then started to experiment with panning textures. As I've written before, panning textures are nothing new. They were already there in the '90s, and have always worked well. But just a panning texture does not give you nice clouds. It's very static because the clouds never change.

One way to break this repetition is to mask away the clouds with a second panning texture, or even a third one. Although this works rather well, you can still see the texture tiling and repetition if you look at it for a longer period of time.

I ended up using a mixture of techniques. I was very intrigued by the flow map technique and started to use this for the cloud deformation. The end result looked great. Clouds were deforming in a natural way, and I was onto a great start here. 

But then I faced the biggest challenge when doing clouds. How to make them look 3D rather than flat/ And how do I make sure that the lighting is correct based on the Sun and Moon location? I've tried several things but ended up using a technique that involves the cloud's normal maps. This is cheap but convincing enough to make a cloud look 3D. I transformed the normal map to world space coordinates. This gave me information about the direction clouds were facing in the shader. With some simple dot products, I was able to read which sides of the cloud faced the light. 

The final piece was to increase and decrease the number of clouds, so you could create an overcast of clear sky. This was a lot easier, because I could subtract or add values to the cloud map, and this would then remove or add clouds to the cloud layer.

Afterword

EasySky is very lightweight but very good looking. It runs good on mobile as well. A good skybox can really push your game environments, but it can be very time consuming to create one. With EasySky, artists can spend more time on the rest of the scene. 

The sky can be heavily customized, so not one skybox is identical.

Exclusively for 80 Lvl, EasySky will be available at a discount of 30% starting October the 7th until October the 13th. Keep an eye on the Unreal Store or join our Discord for more information on EasySky.

Links

• EasySky on the UE4 Marketplace
• EasySky Documentation
• EasySky Discord Channel

Ron Kamphuis, Principal Technical Artist at Force Field Entertainment

Interview conducted by Kirill Tokarev