In this tutorial, we will introduce some methods about the design of data pipelines, and how to customize and extend your own data pipelines for the project.
Data Pipeline is a modular form for data process. We make common data processing operations into python class, which named pipeline
.
The following code block shows how to define a pipeline class to calculate viewdirs from rays' direction.
@PIPELINES.register_module()
class GetViewdirs:
"""get viewdirs from rays_d
"""
def __init__(self, enable=True, **kwargs):
self.enable = enable
def __call__(self, results):
"""get viewdirs
Args:
results (dict): The resulting dict to be modified and passed
to the next transform in pipeline.
"""
if self.enable:
viewdirs = results['rays_d'].clone()
viewdirs = viewdirs / torch.norm(viewdirs, dim=-1, keepdim=True)
viewdirs = torch.reshape(viewdirs, [-1, 3]).float()
results['viewdirs'] = viewdirs
return results
To use the GetViewdirs
, we can simply add dict(type='GetViewdirs')
to train_pipeline
in config file.
We logically divide data process pipeline into 4 python files:
creat.py
create or calculate new variables.augment.py
data augmentation operations.transforms.py
convert data type or change coordinate system.compose.py
Combine various data processing operations into a pipeline.A complete data pipeline configuration is shown below.
train_pipeline = [
dict(type='Sample'),
dict(type='DeleteUseless', keys=['images', 'poses', 'i_data', 'idx']),
dict(type='ToTensor', keys=['pose', 'target_s']),
dict(type='GetRays'),
dict(type='SelectRays',
sel_n=N_rand_per_sampler,
precrop_iters=500,
precrop_frac=0.5), # in the first 500 iter, select rays inside center of image
dict(type='GetViewdirs', enable=use_viewdirs),
dict(type='ToNDC', enable=(not no_ndc)),
dict(type='GetBounds'),
dict(type='GetZvals', lindisp=lindisp,
N_samples=N_samples), # N_samples: number of coarse samples per ray
dict(type='PerturbZvals', enable=is_perturb),
dict(type='GetPts'),
dict(type='DeleteUseless', keys=['pose', 'iter_n']),
]
In this case, the input data is a dict, created in _fetch_train_data()
data = {'poses': self.poses, 'images': self.images, 'i_data': self.i_train, 'idx': idx}
In data pipeline, the data processing flow is as follows:
Sample
select one image or pose via idx
, create pose
and target_s
DeleteUseless
delete 'images', 'poses', 'i_data', 'idx'
in dict, they are already uselessToTensor
convert 'pose', 'target_s'
in dictGetRays
calculate 'rays_d', 'rays_o'
from camera parameter and images shapeSelectRays
select a batchsize raysGetViewdirs
calculate viewdirs from rays' directionToNDC
Coordinate system transformationGetBounds
get near and farGetZvals
samples points along rays between near point and far pointPerturbZvals
data augmentationGetPts
get points' position此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。