Digitizing Materials With Photometric Stereo and ArtEngine

Piotr Bieryt, 3D Artist at Forte Digital shares how he leveraged a technique called photometric stereo, along with Unity ArtEngine, to create digital materials.

Introduction

My name is Piotr Bieryt and I’m a 3D artist at Forte Digital, a digital solutions consulting firm. Before consulting, I worked as a character artist for CD Projekt Red, the Polish game studio best known for developing The Witcher franchise. My expertise is primarily in hard surface modeling and texturing. 

Recently, in my personal time, I began exploring automatic photogrammetry scanning techniques for digitizing real-world objects. I owe a huge thanks to Tomek Radziewicz for sharing his incredible handmade sculptures with me, as he provided me with a beautiful subject matter as I refined my scanning approach.

Throughout my years working in gaming, I’ve found it interesting to observe how 3D workflows and tools are increasingly merging across industries. For example, for a recent project with furniture manufacturer Flokk, I leveraged a technique called photometric stereo, along with Unity ArtEngine to create digital materials. Photometric stereo is industry agonistic and can be used not only for product visualization (as was the case for Flokk), but in gaming, VFX, animation, archviz, and beyond. Given the workflow’s wide applicability, I thought I’d share a bit about my approach.

My work with Flokk was part of a larger transformation to redesign the company’s website with better product visuals and self-serve checkout capabilities. Flokk’s products (which largely consist of office chairs) are bespoke, meaning customers can choose among dozens of fabrics, specify characteristics about the seat size, lift, foot base, and wheels, and add additional accessories such as a neck rest and armrests – an experience not so dissimilar to buying a car. 

As such, one of my main tasks was to digitize Flokk’s hundreds of chair fabrics and create renders for a web-based 3D configurator so customers could design a chair in the comfort of their browser. Scan clean up is usually a long and tedious process, however, assisted creation tools such as ArtEngine helped me create these assets in a fraction of the time it once took.

Photometric Stereo and Hardware Setup

To digitize the fabrics, I used a scanning workflow called photometric stereo, a technique that allows for the capture of a subject’s surface properties using several photographs taken with different light conditions. Using photometric stereo, one can extract data on not only albedo (i.e., color – just as a typical flatbed scanner can do) but normals (i.e., a surface’s relief pattern), and sometimes specularity and roughness, which are key inputs into creating a PBR material.

The technique is similar to photogrammetry, but it can be a lot faster, given you only need to take a few photos. It’s great for relatively flat surfaces like fabrics. For more complex surfaces and objects, you’ll want to use photogrammetry. Here’s an animation of what the scanning process looks like.

Though a photometric stereo setup can be as simple as a camera, tripod, and flash, given the number of materials I had to digitize and the need for consistency, I quickly realized that a manual scanning process would be very tedious. I was reminded of a blog post by Allegorithmic, which had illustrated the general idea behind material scanning, as well as prototypes built by Dave Riganelli. Inspired by these resources, I decided to build a custom, fully automated rig from scratch. 

As a proof-of-concept, I built a simple prototype out of cardboard and then verified the scan quality. As it was working very promisingly, I began designing a newer version with a much larger scanning area and other automation functionality, such as switching cross-polarization.

The entire design process took about a month, plus another month for 3D printing, soldering, assembly, and coding all functionalities. The heart of the scanner was based on Arduino due to the platform’s simplicity and programming ease.

After assembling the rig using laser-cut plywood and 3D-printed elements, I covered the prototype’s interior with black velour to prevent discoloration and light reflections and installed a removable black plate on the bottom to capture transparency using illumination from below. 

The combination of laser-cut plywood, 3D printed parts and Arduino is a great choice for scanner prototypes, as you can rapidly iterate through different ideas. I love building things and automating processes, so I had a lot of fun during this part of the process too.

I used a mirrorless Olympus 16 MP camera with a 60mm macro lens (Micro Four Thirds system) to capture the fine details on Flokk’s fabrics, and shot in RAW to ensure accurate white balance and colors. After color correction, I began my work in ArtEngine.

Quick Scan Cleanup with ArtEngine

My typical workflow inside ArtEngine was as follows: compile the 8 photos into the desired maps, remove seams and tiling artifacts, and then output to Blender Cycles.

In the example I’ll go through below, the sample was a 10x10 cm swatch of a semitransparent fabric from one of Flokk’s chairs. When digitizing materials, transparency requires an additional transparency channel, which can create complexity. To address this, I scanned the fabric twice, once with it lit from the sides (a standard photometric stereo capture process), and once lit from the bottom (to capture transparency).

After importing into ArtEngine, I plugged each of the two image sets into a Multi-Angle to Texture node to combine the 16 images into three Maps – Albedo, Normal, and Transparency:

1 of 3

I then applied Gradient Removal (similar to the High Pass filter in Photoshop) to both the albedo and normal maps to remove visible gradients and enable better tileability. 

Before and after images:

After applying Compose Material to merge the three maps into a single material, I used Pattern Unwarp to correct for natural distortions in the fabric. It’s worth spending time straightening a sample before scanning to reduce time spent post-processing, but if you can’t get all the kinks out, ArtEngine has great tools for correcting after the fact. 

Before and after images:

1 of 2

I then used Crop to frame a portion of the straightened texture with a 1:1 aspect ratio. Below is the node graph.

Next came Mutation Structure, a node used to further improve tiling by using AI to recognize and eliminate repetitive patterns, while keeping the structure of the underlying pattern intact.

Mutation Structure is ArtEngine’s magic. It was a huge game-changer for us that allowed us to focus more on the artistic side of scanning, rather than fight with software or algorithms.

After adjusting several parameters, including the world scale factor and output dimensions, I arrived at a highly detailed 8K texture that was six times larger than the 10x10cm scanned sample and had no obvious tiling artifacts.

Finally, I used Height Generation and Roughness/Gloss Generation to create Height and Roughness Maps, as well as a final Compose Material to compile everything for easy export.

Below is the final node graph.

Pulling It All Together

For materials with the same underlying structure that differed only by color, I scanned them only once and then used Blender together with the rendering engine Cycles to create different color variations and do the final renders as well. 

1 of 4

You can see my work on Flokk.com. After selecting a chair of interest, users can now customize nearly every aspect, view their configuration up close and different various angles, and understand the costs associated with changing certain features before deciding to place an online order. The Forte team did an incredible job with this very complex project!

I hope you enjoyed learning about my workflows. If you’d like to learn more about my work, you can visit my ArtStation portfolio, connect with me on LinkedIn or shoot me an email. Also, right now, you can try ArtEngine for $19/mo (vs. regular price of $95/mo) – so if you’re interested in scanning, I’d encourage you to check it out.

Piotr Bieryt, Hard Surface Artist

Join discussion

Comments 0

    You might also like

    We need your consent

    We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more