Detailed Description

Use this class to rectify stereo images if the same distortion maps are reused over and over again.

The rectify maps are cached internally and only computed once for the camera parameters. The stereo camera calibration must be supplied in a mrpt::util::TStereoCamera structure (which provides method for loading from a plain text config file) or directly from the parameters of a mrpt::obs::CObservationStereoImages object.

Remember that the rectified images have a different set of intrinsic parameters than the original images, which can be retrieved with getRectifiedImageParams()

Works with grayscale or color images.

Refer to the program stereo-calib-gui for a tool that generates the required stereo camera parameters from a set of stereo images of a checkerboard.

Example of usage with mrpt::obs::CObservationStereoImages:

  CStereoRectifyMap   rectify_map;
  // Set options as desired:
  // rectify_map.setAlpha(...);
  // rectify_map.enableBothCentersCoincide(...);
  while (true) {
    mrpt::obs::CObservationStereoImages::Ptr obs_stereo = ... // Grab stereo
observation from wherever
    // Only once, construct the rectification maps:
    if (!rectify_map.isSet())
      rectify_map.setFromCamParams(*obs_stereo);
    // Rectify in place:
    unmap.rectify(*obs_stereo);
    // Rectified images are now in: obs_stereo->imageLeft &
obs_stereo->imageRight
  }

Read also the tutorial page online: https://www.mrpt.org/Rectifying_stereo_images

See also: CUndistortMap, mrpt::obs::CObservationStereoImages, mrpt::img::TCamera, the application camera-calib for calibrating a camera.

Note: This class provides a uniform wrap over different OpenCV versions. The "alpha" parameter is ignored if built against OpenCV 2.0.X

Definition at line 74 of file CStereoRectifyMap.h.

#include <mrpt/vision/CStereoRectifyMap.h>

Public Member Functions
	CStereoRectifyMap ()=default

Rectify map preparation and setting/getting of parameters
bool	isSet () const
	Returns true if setFromCamParams() has been already called, false otherwise. More...

void	setFromCamParams (const mrpt::img::TStereoCamera &params)
	Prepares the mapping from the intrinsic, distortion and relative pose parameters of a stereo camera. More...

void	setFromCamParams (const mrpt::obs::CObservationStereoImages &stereo_obs)
	A wrapper to setFromCamParams() which takes the parameters from an stereo observation object. More...

const mrpt::img::TStereoCamera &	getCameraParams () const
	Returns the camera parameters which were used to generate the distortion map, as passed by the user to setFromCamParams. More...

const mrpt::img::TStereoCamera &	getRectifiedImageParams () const
	After computing the rectification maps, this method retrieves the calibration parameters of the rectified images (which won't have any distortion). More...

const mrpt::img::TCamera &	getRectifiedLeftImageParams () const
	Just like getRectifiedImageParams() but for the left camera only. More...

const mrpt::img::TCamera &	getRectifiedRightImageParams () const
	Just like getRectifiedImageParams() but for the right camera only. More...

void	setAlpha (double alpha)
	Sets the alpha parameter which controls the zoom in/out of the rectified images, such that: More...

double	getAlpha () const
	Return the alpha parameter. More...

void	enableResizeOutput (bool enable, unsigned int target_width=0, unsigned int target_height=0)
	If enabled, the computed maps will rectify images to a size different than their original size. More...

bool	isEnabledResizeOutput () const
	Returns whether resizing is enabled (default=false) More...

mrpt::img::TImageSize	getResizeOutputSize () const
	Only when isEnabledResizeOutput() returns true, this gets the target size. More...

void	setInterpolationMethod (const mrpt::img::TInterpolationMethod interp)
	Change remap interpolation method (default=Lineal). More...

mrpt::img::TInterpolationMethod	getInterpolationMethod () const
	Get the currently selected interpolation method. More...

void	enableBothCentersCoincide (bool enable=true)
	If enabled (default=false), the principal points in both output images will coincide. More...

bool	isEnabledBothCentersCoincide () const

const mrpt::poses::CPose3DQuat &	getLeftCameraRot () const
	After computing the rectification maps, get the rotation applied to the left/right camera so their virtual image plane is the same after rectification. More...

const mrpt::poses::CPose3DQuat &	getRightCameraRot () const
	See getLeftCameraRot() More...

void	setRectifyMaps (const std::vector< int16_t > &left_x, const std::vector< uint16_t > &left_y, const std::vector< int16_t > &right_x, const std::vector< uint16_t > &right_y)
	Direct input access to rectify maps. More...

void	setRectifyMapsFast (std::vector< int16_t > &left_x, std::vector< uint16_t > &left_y, std::vector< int16_t > &right_x, std::vector< uint16_t > &right_y)
	Direct input access to rectify maps. More...

Rectify methods
void	rectify (const mrpt::img::CImage &in_left_image, const mrpt::img::CImage &in_right_image, mrpt::img::CImage &out_left_image, mrpt::img::CImage &out_right_image) const
	Rectify the input image pair and save the result in a different output images - setFromCamParams() must have been set prior to calling this. More...

void	rectify (mrpt::obs::CObservationStereoImages &stereo_image_observation, const bool use_internal_mem_cache=true) const
	Overloaded version for in-place rectification of image pairs stored in a mrpt::obs::CObservationStereoImages. More...

Private Member Functions
void	internal_invalidate ()

Private Attributes
double	m_alpha {-1}

bool	m_resize_output {false}

bool	m_enable_both_centers_coincide {false}

mrpt::img::TImageSize	m_resize_output_value {0, 0}

mrpt::img::TInterpolationMethod	m_interpolation_method

std::vector< int16_t >	m_dat_mapx_left

std::vector< int16_t >	m_dat_mapx_right

std::vector< uint16_t >	m_dat_mapy_left

std::vector< uint16_t >	m_dat_mapy_right

mrpt::img::TStereoCamera	m_camera_params
	A copy of the data provided by the user. More...

mrpt::img::TStereoCamera	m_rectified_image_params
	Resulting images params. More...

mrpt::poses::CPose3DQuat	m_rot_left
	The rotation applied to the left/right camera so their virtual image plane is the same after rectification. More...

mrpt::poses::CPose3DQuat	m_rot_right

Constructor & Destructor Documentation

◆ CStereoRectifyMap()

mrpt::vision::CStereoRectifyMap::CStereoRectifyMap ( )

default

Member Function Documentation

◆ enableBothCentersCoincide()

void CStereoRectifyMap::enableBothCentersCoincide ( bool enable = true )

If enabled (default=false), the principal points in both output images will coincide.

Note: Call this method before building the rectification maps, otherwise they'll be marked as invalid.

Definition at line 94 of file CStereoRectifyMap.cpp.

◆ enableResizeOutput()

void CStereoRectifyMap::enableResizeOutput	(	bool	enable,
		unsigned int	target_width = `0`,
		unsigned int	target_height = `0`
	)

If enabled, the computed maps will rectify images to a size different than their original size.

Note: Call this method before building the rectification maps, otherwise they'll be marked as invalid.

Definition at line 84 of file CStereoRectifyMap.cpp.

◆ getAlpha()

double mrpt::vision::CStereoRectifyMap::getAlpha ( ) const

inline

Return the alpha parameter.

See also: setAlpha

Definition at line 140 of file CStereoRectifyMap.h.

References m_alpha.

Referenced by main().

Here is the caller graph for this function:

◆ getCameraParams()

const mrpt::img::TStereoCamera& mrpt::vision::CStereoRectifyMap::getCameraParams ( ) const

inline

Returns the camera parameters which were used to generate the distortion map, as passed by the user to setFromCamParams.

Definition at line 108 of file CStereoRectifyMap.h.

References m_camera_params.

Referenced by do_rectify().

Here is the caller graph for this function:

◆ getInterpolationMethod()

mrpt::img::TInterpolationMethod mrpt::vision::CStereoRectifyMap::getInterpolationMethod ( ) const

inline

Get the currently selected interpolation method.

See also: setInterpolationMethod

Definition at line 169 of file CStereoRectifyMap.h.

References m_interpolation_method.

◆ getLeftCameraRot()

const mrpt::poses::CPose3DQuat& mrpt::vision::CStereoRectifyMap::getLeftCameraRot ( ) const

inline

After computing the rectification maps, get the rotation applied to the left/right camera so their virtual image plane is the same after rectification.

Definition at line 190 of file CStereoRectifyMap.h.

References m_rot_left.

◆ getRectifiedImageParams()

const mrpt::img::TStereoCamera & CStereoRectifyMap::getRectifiedImageParams ( ) const

After computing the rectification maps, this method retrieves the calibration parameters of the rectified images (which won't have any distortion).

Exceptions

std::exception If the rectification maps have not been computed.

Definition at line 339 of file CStereoRectifyMap.cpp.

References THROW_EXCEPTION.

◆ getRectifiedLeftImageParams()

const mrpt::img::TCamera & CStereoRectifyMap::getRectifiedLeftImageParams ( ) const

Just like getRectifiedImageParams() but for the left camera only.

Definition at line 347 of file CStereoRectifyMap.cpp.

References THROW_EXCEPTION.

◆ getRectifiedRightImageParams()

const mrpt::img::TCamera & CStereoRectifyMap::getRectifiedRightImageParams ( ) const

Just like getRectifiedImageParams() but for the right camera only.

Definition at line 354 of file CStereoRectifyMap.cpp.

References THROW_EXCEPTION.

◆ getResizeOutputSize()

mrpt::img::TImageSize mrpt::vision::CStereoRectifyMap::getResizeOutputSize ( ) const

inline

Only when isEnabledResizeOutput() returns true, this gets the target size.

See also: enableResizeOutput

Definition at line 155 of file CStereoRectifyMap.h.

References m_resize_output_value.

Referenced by do_rectify().

Here is the caller graph for this function:

◆ getRightCameraRot()

const mrpt::poses::CPose3DQuat& mrpt::vision::CStereoRectifyMap::getRightCameraRot ( ) const

inline

See getLeftCameraRot()

Definition at line 195 of file CStereoRectifyMap.h.

References m_rot_right.

◆ internal_invalidate()

void CStereoRectifyMap::internal_invalidate ( )

private

Definition at line 68 of file CStereoRectifyMap.cpp.

◆ isEnabledBothCentersCoincide()

bool mrpt::vision::CStereoRectifyMap::isEnabledBothCentersCoincide ( ) const

inline

See also: enableBothCentersCoincide

Definition at line 182 of file CStereoRectifyMap.h.

References m_enable_both_centers_coincide.

◆ isEnabledResizeOutput()

bool mrpt::vision::CStereoRectifyMap::isEnabledResizeOutput ( ) const

inline

Returns whether resizing is enabled (default=false)

See also: enableResizeOutput

Definition at line 152 of file CStereoRectifyMap.h.

References m_resize_output.

Referenced by do_rectify().

Here is the caller graph for this function:

◆ isSet()

bool mrpt::vision::CStereoRectifyMap::isSet ( ) const

inline

Returns true if setFromCamParams() has been already called, false otherwise.

Can be used within loops to determine the first usage of the object and when it needs to be initialized.

Definition at line 86 of file CStereoRectifyMap.h.

References m_dat_mapx_left.

Referenced by do_rectify().

Here is the caller graph for this function:

◆ rectify() [1/2]

void CStereoRectifyMap::rectify	(	const mrpt::img::CImage &	in_left_image,
		const mrpt::img::CImage &	in_right_image,
		mrpt::img::CImage &	out_left_image,
		mrpt::img::CImage &	out_right_image
	)		const

Rectify the input image pair and save the result in a different output images - setFromCamParams() must have been set prior to calling this.

The previous contents of the output images are completely ignored, but if they are already of the correct size and type, allocation time will be saved. Recall that getRectifiedImageParams() provides you the new intrinsic parameters of these images.

Exceptions

std::exception If the rectification maps have not been computed.

Note: The same image CANNOT be at the same time input and output, in which case an exception will be raised (but see the overloaded version for in-place rectification)

Definition at line 264 of file CStereoRectifyMap.cpp.

References mrpt::img::CImage::asCvMat(), mrpt::img::CImage::asCvMatRef(), do_rectify(), mrpt::img::CImage::getChannelCount(), MRPT_END, MRPT_START, mrpt::img::CImage::resize(), and mrpt::img::SHALLOW_COPY.

Referenced by mrpt::hwdrivers::CDUO3DCamera::getObservations().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ rectify() [2/2]

void CStereoRectifyMap::rectify	(	mrpt::obs::CObservationStereoImages &	stereo_image_observation,
		const bool	use_internal_mem_cache = `true`
	)		const

Overloaded version for in-place rectification of image pairs stored in a mrpt::obs::CObservationStereoImages.

Upon return, the new camera intrinsic parameters will be already stored in the observation object. If use_internal_mem_cache is set to true (recommended), will reuse over and over again the same auxiliary images (kept internally to this object) needed for in-place rectification. The only reason not to enable this cache is when multiple threads can invoke this method simultaneously.

Note: This method uses the left & right camera rotations computed by the rectification map to update mrpt::obs::CObservationStereoImages::cameraPose (left camera wrt the robot frame) and mrpt::obs::CObservationStereoImages::rightCameraPose (right wrt left camera).

Definition at line 303 of file CStereoRectifyMap.cpp.

References ASSERT_, mrpt::obs::CObservationStereoImages::cameraPose, mrpt::obs::CObservationStereoImages::hasImageRight, mrpt::obs::CObservationStereoImages::imageLeft, mrpt::obs::CObservationStereoImages::imageRight, mrpt::poses::CPose3DQuat::m_coords, MRPT_END, MRPT_START, mrpt::obs::CObservationStereoImages::rightCameraPose, and mrpt::obs::CObservationStereoImages::setStereoCameraParams().

Here is the call graph for this function:

◆ setAlpha()

void CStereoRectifyMap::setAlpha ( double alpha )

Sets the alpha parameter which controls the zoom in/out of the rectified images, such that:

alpha=0 => rectified images are zoom in so that only valid pixels are visible
alpha=1 => rectified images will contain large "black areas" but no pixel from the original image will be lost. Intermediary values leads to intermediary results. Its default value (-1) means auto guess by the OpenCV's algorithm.
Note
Call this method before building the rectification maps, otherwise they'll be marked as invalid.

Definition at line 77 of file CStereoRectifyMap.cpp.

◆ setFromCamParams() [1/2]

void CStereoRectifyMap::setFromCamParams ( const mrpt::img::TStereoCamera & params )

Prepares the mapping from the intrinsic, distortion and relative pose parameters of a stereo camera.

Prepares the mapping from the distortion parameters of a camera.

Must be called before invoking rectify(). The alpha parameter can be changed with setAlpha() before invoking this method; otherwise, the current rectification maps will be marked as invalid and should be prepared again.

See also: setAlpha()

Must be called before invoking undistort().

Definition at line 104 of file CStereoRectifyMap.cpp.

References ASSERT_, mrpt::img::TCamera::dist, mrpt::poses::CPoseOrPoint< DERIVEDCLASS, DIM >::getInverseHomogeneousMatrix(), mrpt::img::TCamera::intrinsicParams, MRPT_END, MRPT_START, mrpt::img::TCamera::ncols, mrpt::img::TCamera::nrows, params, R, mrpt::poses::CPose3D::setRotationMatrix(), and THROW_EXCEPTION.

Referenced by main(), mrpt::hwdrivers::CDUO3DCamera::open(), and setFromCamParams().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ setFromCamParams() [2/2]

void mrpt::vision::CStereoRectifyMap::setFromCamParams ( const mrpt::obs::CObservationStereoImages & stereo_obs )

inline

A wrapper to setFromCamParams() which takes the parameters from an stereo observation object.

Definition at line 99 of file CStereoRectifyMap.h.

References mrpt::obs::CObservationStereoImages::getStereoCameraParams(), params, and setFromCamParams().

Here is the call graph for this function:

◆ setInterpolationMethod()

void mrpt::vision::CStereoRectifyMap::setInterpolationMethod ( const mrpt::img::TInterpolationMethod interp )

inline

Change remap interpolation method (default=Lineal).

This parameter can be safely changed at any instant without consequences.

Definition at line 162 of file CStereoRectifyMap.h.

References m_interpolation_method.

◆ setRectifyMaps()

void CStereoRectifyMap::setRectifyMaps	(	const std::vector< int16_t > &	left_x,
		const std::vector< uint16_t > &	left_y,
		const std::vector< int16_t > &	right_x,
		const std::vector< uint16_t > &	right_y
	)

Direct input access to rectify maps.

Definition at line 362 of file CStereoRectifyMap.cpp.

◆ setRectifyMapsFast()

void CStereoRectifyMap::setRectifyMapsFast	(	std::vector< int16_t > &	left_x,
		std::vector< uint16_t > &	left_y,
		std::vector< int16_t > &	right_x,
		std::vector< uint16_t > &	right_y
	)

Direct input access to rectify maps.

This method swaps the vectors so the inputs are no longer available.

Definition at line 377 of file CStereoRectifyMap.cpp.

Member Data Documentation

◆ m_alpha

double mrpt::vision::CStereoRectifyMap::m_alpha {-1}

private

Definition at line 259 of file CStereoRectifyMap.h.

Referenced by getAlpha().

◆ m_camera_params

mrpt::img::TStereoCamera mrpt::vision::CStereoRectifyMap::m_camera_params

private

A copy of the data provided by the user.

Definition at line 270 of file CStereoRectifyMap.h.

Referenced by getCameraParams().

◆ m_dat_mapx_left

std::vector<int16_t> mrpt::vision::CStereoRectifyMap::m_dat_mapx_left

private

Definition at line 266 of file CStereoRectifyMap.h.

Referenced by isSet().

◆ m_dat_mapx_right

std::vector<int16_t> mrpt::vision::CStereoRectifyMap::m_dat_mapx_right

private

Definition at line 266 of file CStereoRectifyMap.h.

◆ m_dat_mapy_left

std::vector<uint16_t> mrpt::vision::CStereoRectifyMap::m_dat_mapy_left

private

Definition at line 267 of file CStereoRectifyMap.h.

◆ m_dat_mapy_right

std::vector<uint16_t> mrpt::vision::CStereoRectifyMap::m_dat_mapy_right

private

Definition at line 267 of file CStereoRectifyMap.h.

◆ m_enable_both_centers_coincide

bool mrpt::vision::CStereoRectifyMap::m_enable_both_centers_coincide {false}

private

Definition at line 261 of file CStereoRectifyMap.h.

Referenced by isEnabledBothCentersCoincide().

◆ m_interpolation_method

mrpt::img::TInterpolationMethod mrpt::vision::CStereoRectifyMap::m_interpolation_method

private

Initial value:

{

mrpt::img::IMG_INTERP_LINEAR}

Definition at line 263 of file CStereoRectifyMap.h.

Referenced by getInterpolationMethod(), and setInterpolationMethod().

◆ m_rectified_image_params

mrpt::img::TStereoCamera mrpt::vision::CStereoRectifyMap::m_rectified_image_params

private

Resulting images params.

Definition at line 272 of file CStereoRectifyMap.h.

◆ m_resize_output

bool mrpt::vision::CStereoRectifyMap::m_resize_output {false}

private

Definition at line 260 of file CStereoRectifyMap.h.

Referenced by isEnabledResizeOutput().

◆ m_resize_output_value

mrpt::img::TImageSize mrpt::vision::CStereoRectifyMap::m_resize_output_value {0, 0}

private

Definition at line 262 of file CStereoRectifyMap.h.

Referenced by getResizeOutputSize().

◆ m_rot_left

mrpt::poses::CPose3DQuat mrpt::vision::CStereoRectifyMap::m_rot_left

private

The rotation applied to the left/right camera so their virtual image plane is the same after rectification.

Definition at line 276 of file CStereoRectifyMap.h.

Referenced by getLeftCameraRot().

◆ m_rot_right

mrpt::poses::CPose3DQuat mrpt::vision::CStereoRectifyMap::m_rot_right

private

Definition at line 276 of file CStereoRectifyMap.h.

Referenced by getRightCameraRot().

Detailed Description

Public Member Functions

Private Member Functions

Private Attributes

Constructor & Destructor Documentation

◆ CStereoRectifyMap()

Member Function Documentation

◆ enableBothCentersCoincide()

◆ enableResizeOutput()

◆ getAlpha()

◆ getCameraParams()

◆ getInterpolationMethod()

◆ getLeftCameraRot()

◆ getRectifiedImageParams()

◆ getRectifiedLeftImageParams()

◆ getRectifiedRightImageParams()

◆ getResizeOutputSize()

◆ getRightCameraRot()

◆ internal_invalidate()

◆ isEnabledBothCentersCoincide()

◆ isEnabledResizeOutput()

◆ isSet()

◆ rectify() [1/2]

◆ rectify() [2/2]

◆ setAlpha()

◆ setFromCamParams() [1/2]

◆ setFromCamParams() [2/2]

◆ setInterpolationMethod()

◆ setRectifyMaps()

◆ setRectifyMapsFast()

Member Data Documentation

◆ m_alpha

◆ m_camera_params

◆ m_dat_mapx_left

◆ m_dat_mapx_right

◆ m_dat_mapy_left

◆ m_dat_mapy_right

◆ m_enable_both_centers_coincide

◆ m_interpolation_method

◆ m_rectified_image_params

◆ m_resize_output

◆ m_resize_output_value

◆ m_rot_left

◆ m_rot_right