Capturing Guidelines

Creating 3D radiance fields is done by shooting regular 2D video or a set of 2D photos. Postshot then trains a 3D radiance field model that reproduces these input images as accurately as possible. This means that the model can at most be as good as the input images. It is therefore important to capture the best possible images of the scene.

The goal of this document is to give you practical guidelines for shooting images that work well for radiance field training.

Below we will use the term capture shot to describe the process of shooting images for radiance field training.

Beware that this document is a work in progress. We will improve it as we explore this new technology and the workflows around it. As with any other part of Postshot, your feedback is much appreciated.

The Scene

Avoid moving objects, lights and shadows

Postshot's current radiance field profiles assume strictly static scenes. Regarding motion, capture shots are somewhat similar to long exposure photography. Anything that moves on the images will create blur or ghosting artefacts in the radiance field. Of course you could also use motion to create these effects on purpose.

Lens flairs are also a variant of this issue. While they are often static, lens flairs are am extreme form case of view-dependent lighting that is likely to cause problems for the radiance field. If there are angles in your capture shot in which flairs arise, you want to get rid of them quickly through appropriate movement.

Do not use a camera flash. As the camera moves, the flash will move too, resulting in different lighting in each image. While static studio flashes do not have this issue, differences in illumination may still occur due to the accuracy of the shutter synchronisation. Therefore, it is recommended that you only use continuous lights.

Camera Settings

Image count or video length

The recommended image count or video length depends on the size of your scene. Aim for about 30-50% overlap between images. Less overlap may cause unstable reconstruction, while more overlap may not improve the reconstruction while still taking time to process.

When shooting video, try to move at a speed that prevents motion blur, while otherwise moving through the scene as quickly as possible. Use the lowest available framerate. Postshot will typically use only 2-3 FPS of imported video files.

In some studio settings, image sets as small as 40 images can work. In other settings several hundreds or thousands of images are required for a good reconstruction.

Try not to include poor images with artefacts like blur, lens flares, occluders or moving objects. Including such images tends to hurt the quality more than their additional information may help.

Focal length

While Postshot can handle any focal length, the most stable reconstruction is usually obtained from images that have a good amount of context rather than close-up shots of details. For hand-held recordings, 24mm full frame (or an equivalent on smaller sensors) is a good starting point. For outdoor drone shots you can use longer lenses if the distance to the object is already large.

Camera path

Shoot from many angles

The goal of a capture shot is to record as many points of the scene as possible, each from as many different angles as possible. At the same time, we want to keep the number of images low.

To achieve this, a good scheme for the camera path are a set of 2-5 rings around the scene, each with a different tilt (vertical viewing angle) and pedestal (height above ground). Shoot upward when the camera is low, and downward when it's high.

When capturing a room, position yourself with your back to the wall and shoot along the longest line of sight into the room.

When capturing objects, it's often helpful to include one or more large rings for context.

Unless you want to reconstruct an isolated object, remember to include shots of the floor and ceiling.

Always move the camera

Don't pan, tilt or roll without moving the camera at the same time.

The reason for this is that camera tracking is based on the triangulation of points. In order to form a triangle for any point in the scene, it takes two camera positions that see the point. Therefore, the extreme failure case is a camera fixed at one position while rotating (pan, tilt or roll).

Avoid blur

Prefer short exposure times and small apertures to avoid both motion blur and defocus blur. If the lighting conditions don't allow you to turn both down, prefer using a small aperture. You can compensate for a longer exposure time by moving the camera more slowly.

Using higher ISO values is also an option, as radiance fields tend to tolerate noise better than blur.

Example

Here is an example video that you can use to get an idea of the capturing process and to test training a radiance field in Postshot.

As you may see, all of the above is more what you'd call 'guidelines' than actual rules. There are several things about this shot that are not ideal.

There are lens flairs at one point and moving shadows at another. The moving shadows create holes in the ground of the radiance field, while the flairs are tolerated better. They still wash out the radiance field around their viewing direction.

The cars in the background and the leaves on the trees are also moving in the scene. But this has comparatively little influence.

And lastly, the shot could be sharper. It's a smartphone video with H265 compression. So there is both some motion blur and loss of details from the compression.