Detailed Description

Back to list of all libraries | See all modules

Library `mrpt-vision`

This library includes some extensions to OpenCV functionality, plus some original classes:

The namespace mrpt::vision::pinhole contains several projection and Jacobian auxiliary functions for projective cameras.
Sparse Bundle Adjustment algorithms. Implemented versions are:
- mrpt::vision::bundle_adj_full : Levenberg-Marquart full optimization of camera frames (6D) + feature points (3D).
A versatile feature tracker. See mrpt::vision::CGenericFeatureTracker and the implementation mrpt::vision::CFeatureTracker_KL
mrpt::vision::CFeature: A generic representation of a visual feature, with or without patch, with or without a set of descriptors.
mrpt::vision::CFeatureExtraction: A hub for a number of detection algorithms and different descriptors.
mrpt::vision::TSIFTDescriptorsKDTreeIndex, mrpt::vision::find_descriptor_pairings() and others: KD-tree-based SIFT/SURF feature matching.
mrpt::vision::CVideoFileWriter: A class to write video files.
mrpt::vision::CUndistortMap: A cache of the map for undistorting image, very efficient for sequences of images all with the same distortion parameters.
This library defines a new type of observation (mrpt::slam::CObservationVisualLandmarks) and a new type of map (mrpt::slam::CLandmarksMap).
Notice that sets of parameters for monocular and stereo cameras are defined in [mrpt-base] for convenience, in the classes:
- mrpt::utils::TCamera
- mrpt::utils::TStereoCamera
mrpt::vision::CDifodo: A class which implements a Visual Odometry algorithm based on depth images and the "range flow constraint equation".

See all the classes in mrpt::vision

Collaboration diagram for [mrpt-vision]:

Classes
class	mrpt::slam::CLandmark
	The class for storing "landmarks" (visual or laser-scan-extracted features,...) More...

class	mrpt::slam::CLandmarksMap
	A class for storing a map of 3D probabilistic landmarks. More...

class	mrpt::slam::CObservationVisualLandmarks
	Declares a class derived from "CObservation" that stores a Landmarks Map as seen from a stereo camera at a given instant of time. More...

class	mrpt::vision::CCamModel
	This class represent a pinhole camera model for Monocular SLAM and implements some associated Jacobians. More...

class	mrpt::vision::CDifodo
	This abstract class implements a method called "Difodo" to perform Visual odometry with range cameras. More...

class	mrpt::vision::CImagePyramid
	Holds and builds a pyramid of images: starting with an image at full resolution (octave=1), it builds a number of half-resolution images: octave=2 at 1/2 , octave=3 at 1/2^2, octave=N at 1/2^(N-1). More...

class	mrpt::vision::CStereoRectifyMap
	Use this class to rectify stereo images if the same distortion maps are reused over and over again. More...

class	mrpt::vision::CUndistortMap
	Use this class to undistort monocular images if the same distortion map is used over and over again. More...

class	mrpt::vision::CVideoFileWriter
	An output stream which takes a sequence of images and writes a video file in any of a given of compatible formats. More...

struct	mrpt::vision::RobustKernel< TRobustKernelType, T >

struct	mrpt::vision::RobustKernel< static_cast< int >(rkLeastSquares), T >
	No robust kernel, use standard least squares: rho(r)= 1/2 * r^2. More...

struct	mrpt::vision::RobustKernel< static_cast< int >(rkPseudoHuber), T >
	Pseudo-huber robust kernel: rho(r) = b^2 * (-1+sqrt(1+(r)^2)) More...

struct	mrpt::vision::TFeatureObservation
	One feature observation entry, used within sequences with TSequenceFeatureObservations. More...

struct	mrpt::vision::TRelativeFeaturePos
	One relative feature observation entry, used with some relative bundle-adjustment functions. More...

struct	mrpt::vision::TSequenceFeatureObservations
	A complete sequence of observations of features from different camera frames (poses). More...

struct	mrpt::vision::TStereoSystemParams
	Parameters associated to a stereo system. More...

struct	mrpt::vision::TROI
	A structure for storing a 3D ROI. More...

struct	mrpt::vision::TImageROI
	A structure for defining a ROI within an image. More...

struct	mrpt::vision::TMatchingOptions
	A structure containing options for the matching. More...

struct	mrpt::vision::TMultiResMatchingOutput
	Struct containing the output after matching multi-resolution SIFT-like descriptors. More...

struct	mrpt::vision::TMultiResDescMatchOptions
	Struct containing the options when matching multi-resolution SIFT-like descriptors. More...

struct	mrpt::vision::TMultiResDescOptions
	Struct containing the options when computing the multi-resolution SIFT-like descriptors. More...

Namespaces
	mrpt::slam
	This namespace contains algorithms for SLAM, localization, map building, representation of robot's actions and observations, and representation of many kinds of metric maps.

	mrpt::vision::pinhole
	Functions related to pinhole camera models, point projections, etc.

	mrpt::vision
	Classes for computer vision, detectors, features, etc.

Modules
	Bundle-Adjustment methods

	Feature detection, descriptors and matching

	KD-Tree construction of visual descriptors

	Chessboard calibration

	Multiresolution SIFTs (experimental)

	Feature detection and tracking

Typedefs
typedef uint64_t	mrpt::vision::TFeatureID
	Definition of a feature ID. More...

typedef uint64_t	mrpt::vision::TLandmarkID
	Unique IDs for landmarks. More...

typedef uint64_t	mrpt::vision::TCameraPoseID
	Unique IDs for camera frames (poses) More...

typedef mrpt::aligned_containers< TCameraPoseID, CPose3D >::map_t	mrpt::vision::TFramePosesMap
	A list of camera frames (6D poses) indexed by unique IDs. More...

typedef mrpt::aligned_containers< CPose3D >::vector_t	mrpt::vision::TFramePosesVec
	A list of camera frames (6D poses), which assumes indexes are unique, consecutive IDs. More...

typedef std::map< TLandmarkID, TPoint3D >	mrpt::vision::TLandmarkLocationsMap
	A list of landmarks (3D points) indexed by unique IDs. More...

typedef std::vector< TPoint3D >	mrpt::vision::TLandmarkLocationsVec
	A list of landmarks (3D points), which assumes indexes are unique, consecutive IDs. More...

typedef std::map< mrpt::vision::TFeatureID, TRelativeFeaturePos >	mrpt::vision::TRelativeFeaturePosMap
	An index of feature IDs and their relative locations. More...

Enumerations
enum	mrpt::vision::TRobustKernelType { mrpt::vision::rkLeastSquares = 0, mrpt::vision::rkPseudoHuber }
	The different types of kernels for usage within a robustified least-squares estimator. More...

enum	mrpt::vision::TFeatureType { mrpt::vision::featNotDefined = -1, mrpt::vision::featKLT = 0, mrpt::vision::featHarris, mrpt::vision::featBCD, mrpt::vision::featSIFT, mrpt::vision::featSURF, mrpt::vision::featBeacon, mrpt::vision::featFAST, mrpt::vision::featFASTER9, mrpt::vision::featFASTER10, mrpt::vision::featFASTER12 }
	Types of features - This means that the point has been detected with this algorithm, which is independent of additional descriptors a feature may also have. More...

enum	mrpt::vision::TDescriptorType { mrpt::vision::descAny = 0, mrpt::vision::descSIFT = 1, mrpt::vision::descSURF = 2, mrpt::vision::descSpinImages = 4, mrpt::vision::descPolarImages = 8, mrpt::vision::descLogPolarImages = 16 }
	The bitwise OR combination of values of TDescriptorType are used in CFeatureExtraction::computeDescriptors to indicate which descriptors are to be computed for features. More...

enum	mrpt::vision::TFeatureTrackStatus { mrpt::vision::status_IDLE = 0, mrpt::vision::status_TRACKED = 5, mrpt::vision::status_OOB = 1, mrpt::vision::status_LOST = 10 }

Functions
void VISION_IMPEXP	mrpt::vision::pinhole::projectPoints_no_distortion (const std::vector< mrpt::poses::CPoint3D > &in_points_3D, const mrpt::poses::CPose3D &cameraPose, const mrpt::math::CMatrixDouble33 &intrinsicParams, std::vector< TPixelCoordf > &projectedPoints, bool accept_points_behind=false)
	Project a set of 3D points into a camera at an arbitrary 6D pose using its calibration matrix (undistorted projection model) More...

template<bool INVERSE_CAM_POSE>
TPixelCoordf	mrpt::vision::pinhole::projectPoint_no_distortion (const mrpt::utils::TCamera &cam_params, const mrpt::poses::CPose3D &F, const mrpt::math::TPoint3D &P)
	Project a single 3D point with global coordinates P into a camera at pose F, without distortion parameters. More...

template<typename POINT >
void	mrpt::vision::pinhole::projectPoint_no_distortion (const POINT &in_point_wrt_cam, const mrpt::utils::TCamera &cam_params, TPixelCoordf &out_projectedPoints)

void VISION_IMPEXP	mrpt::vision::pinhole::projectPoints_with_distortion (const std::vector< mrpt::poses::CPoint3D > &in_points_3D, const mrpt::poses::CPose3D &cameraPose, const mrpt::math::CMatrixDouble33 &intrinsicParams, const std::vector< double > &distortionParams, std::vector< TPixelCoordf > &projectedPoints, bool accept_points_behind=false)
	Project a set of 3D points into a camera at an arbitrary 6D pose using its calibration matrix and distortion parameters (radial and tangential distortions projection model) More...

void VISION_IMPEXP	mrpt::vision::pinhole::projectPoint_with_distortion (const mrpt::math::TPoint3D &in_point_wrt_cam, const mrpt::utils::TCamera &in_cam_params, TPixelCoordf &out_projectedPoints, bool accept_points_behind=false)
	Project one 3D point into a camera using its calibration matrix and distortion parameters (radial and tangential distortions projection model) More...

void VISION_IMPEXP	mrpt::vision::pinhole::projectPoints_with_distortion (const std::vector< mrpt::math::TPoint3D > &P, const mrpt::utils::TCamera &params, const CPose3DQuat &cameraPose, std::vector< TPixelCoordf > &pixels, bool accept_points_behind=false)

void VISION_IMPEXP	mrpt::vision::pinhole::undistort_points (const std::vector< TPixelCoordf > &srcDistortedPixels, std::vector< TPixelCoordf > &dstUndistortedPixels, const mrpt::math::CMatrixDouble33 &intrinsicParams, const std::vector< double > &distortionParams)
	Undistort a list of points given by their pixel coordinates, provided the camera matrix and distortion coefficients. More...

void VISION_IMPEXP	mrpt::vision::pinhole::undistort_points (const std::vector< TPixelCoordf > &srcDistortedPixels, std::vector< TPixelCoordf > &dstUndistortedPixels, const mrpt::utils::TCamera &cameraModel)
	Undistort a list of points given by their pixel coordinates, provided the camera matrix and distortion coefficients. More...

void VISION_IMPEXP	mrpt::vision::pinhole::undistort_point (const TPixelCoordf &inPt, TPixelCoordf &outPt, const mrpt::utils::TCamera &cameraModel)
	Undistort one point given by its pixel coordinates and the camera parameters. More...

void VISION_IMPEXP	mrpt::vision::openCV_cross_correlation (const CImage &img, const CImage &patch_img, size_t &x_max, size_t &y_max, double &max_val, int x_search_ini=-1, int y_search_ini=-1, int x_search_size=-1, int y_search_size=-1)
	Computes the correlation between this image and another one, encapsulating the openCV function cvMatchTemplate This implementation reduced computation time. More...

void VISION_IMPEXP	mrpt::vision::flip (CImage &img)
	Invert an image using OpenCV function. More...

TPoint3D VISION_IMPEXP	mrpt::vision::pixelTo3D (const TPixelCoordf &xy, const CMatrixDouble33 &A)
	Extract a UNITARY 3D vector in the direction of a 3D point, given from its (x,y) pixels coordinates, and the camera intrinsic coordinates. More...

CMatrixDouble33 VISION_IMPEXP	mrpt::vision::buildIntrinsicParamsMatrix (const double focalLengthX, const double focalLengthY, const double centerX, const double centerY)
	Builds the intrinsic parameters matrix A from parameters: More...

CMatrixDouble33 VISION_IMPEXP	mrpt::vision::defaultIntrinsicParamsMatrix (unsigned int camIndex=0, unsigned int resolutionX=320, unsigned int resolutionY=240)
	Returns the stored, default intrinsic params matrix for a given camera: More...

void VISION_IMPEXP	mrpt::vision::deleteRepeatedFeats (CFeatureList &list)
	Explore the feature list and removes features which are in the same coordinates. More...

void VISION_IMPEXP	mrpt::vision::rowChecking (CFeatureList &leftList, CFeatureList &rightList, float threshold=1.0)
	Search for correspondences which are not in the same row and deletes them. More...

void VISION_IMPEXP	mrpt::vision::getDispersion (const CFeatureList &list, vector_float &std, vector_float &mean)
	Computes the dispersion of the features in the image. More...

void	mrpt::vision::correctDistortion (const CImage &in_img, CImage &out_img, const CMatrixDouble33 &A, const vector_double &dist_coeffs)
	Returns a new image where distortion has been removed. More...

double VISION_IMPEXP	mrpt::vision::computeMsd (const TMatchingPairList &list, const poses::CPose3D &Rt)
	Computes the mean squared distance between a set of 3D correspondences ... More...

void VISION_IMPEXP	mrpt::vision::cloudsToMatchedList (const CObservationVisualLandmarks &cloud1, const CObservationVisualLandmarks &cloud2, TMatchingPairList &outList)
	Transform two clouds of 3D points into a matched list of points ... More...

float VISION_IMPEXP	mrpt::vision::computeMainOrientation (const CImage &image, unsigned int x, unsigned int y)
	Computes the main orientation of a set of points with an image (for using in SIFT-based algorithms) More...

void VISION_IMPEXP	mrpt::vision::normalizeImage (const CImage &image, CImage &nimage)
	Normalizes the brigthness and contrast of an image by setting its mean value to zero and its standard deviation to unit. More...

size_t VISION_IMPEXP	mrpt::vision::matchFeatures (const CFeatureList &list1, const CFeatureList &list2, CMatchedFeatureList &matches, const TMatchingOptions &options=TMatchingOptions(), const TStereoSystemParams &params=TStereoSystemParams())
	Find the matches between two lists of features which must be of the same type. More...

void VISION_IMPEXP	mrpt::vision::generateMask (const CMatchedFeatureList &mList, CMatrixBool &mask1, CMatrixBool &mask2, int wSize=10)
	Calculates the Sum of Absolutes Differences (range [0,1]) between two patches. More...

double VISION_IMPEXP	mrpt::vision::computeSAD (const CImage &patch1, const CImage &patch2)
	Calculates the Sum of Absolutes Differences (range [0,1]) between two patches. More...

void VISION_IMPEXP	mrpt::vision::addFeaturesToImage (const CImage &inImg, const CFeatureList &theList, CImage &outImg)
	Draw rectangles around each of the features on a copy of the input image. More...

void VISION_IMPEXP	mrpt::vision::projectMatchedFeatures (const CFeatureList &leftList, const CFeatureList &rightList, vector< TPoint3D > &vP3D, const TStereoSystemParams &params=TStereoSystemParams())
	Computes the 3D position of a set of matched features from their coordinates in the images. More...

void VISION_IMPEXP	mrpt::vision::projectMatchedFeature (const CFeaturePtr &leftFeat, const CFeaturePtr &rightFeat, TPoint3D &p3D, const TStereoSystemParams &params=TStereoSystemParams())
	Computes the 3D position of a particular matched feature. More...

void VISION_IMPEXP	mrpt::vision::projectMatchedFeatures (CMatchedFeatureList &mfList, const TStereoSystemParams &param, CLandmarksMap &landmarks)
	Project a list of matched features into the 3D space, using the provided parameters of the stereo system. More...

void VISION_IMPEXP	mrpt::vision::projectMatchedFeatures (CFeatureList &leftList, CFeatureList &rightList, const TStereoSystemParams &param, CLandmarksMap &landmarks)
	Project a pair of feature lists into the 3D space, using the provided options for the stereo system. More...

void VISION_IMPEXP	mrpt::vision::StereoObs2BRObs (const CObservationStereoImages &inObs, const vector< double > &sg, CObservationBearingRange &outObs)
	Converts a stereo images observation into a bearing and range observation. More...

void VISION_IMPEXP	mrpt::vision::StereoObs2BRObs (const CMatchedFeatureList &inMatches, const CMatrixDouble33 &intrinsicParams, const double &baseline, const CPose3D &sensorPose, const vector< double > &sg, CObservationBearingRange &outObs)
	Converts a matched feature list into a bearing and range observation (some of the stereo camera system must be provided). More...

void VISION_IMPEXP	mrpt::vision::StereoObs2BRObs (const CObservationVisualLandmarks &inObs, CObservationBearingRange &outObs)
	Converts a CObservationVisualLandmarks into a bearing and range observation (without any covariances). More...

void VISION_IMPEXP	mrpt::vision::computeStereoRectificationMaps (const TCamera &cam1, const TCamera &cam2, const poses::CPose3D &rightCameraPose, void outMap1x, void outMap1y, void outMap2x, void outMap2y)
	Computes a pair of x-and-y maps for stereo rectification from a pair of cameras and the relative pose of the second one wrt the first one. More...

Typedef Documentation

◆ TCameraPoseID

typedef uint64_t mrpt::vision::TCameraPoseID

Unique IDs for camera frames (poses)

Definition at line 35 of file vision/include/mrpt/vision/types.h.

◆ TFeatureID

typedef uint64_t mrpt::vision::TFeatureID

Definition of a feature ID.

Definition at line 32 of file vision/include/mrpt/vision/types.h.

◆ TFramePosesMap

typedef mrpt::aligned_containers<TCameraPoseID,CPose3D>::map_t mrpt::vision::TFramePosesMap

A list of camera frames (6D poses) indexed by unique IDs.

Definition at line 37 of file vision/include/mrpt/vision/types.h.

◆ TFramePosesVec

typedef mrpt::aligned_containers<CPose3D>::vector_t mrpt::vision::TFramePosesVec

A list of camera frames (6D poses), which assumes indexes are unique, consecutive IDs.

Definition at line 38 of file vision/include/mrpt/vision/types.h.

◆ TLandmarkID

typedef uint64_t mrpt::vision::TLandmarkID

Unique IDs for landmarks.

Definition at line 34 of file vision/include/mrpt/vision/types.h.

◆ TLandmarkLocationsMap

typedef std::map<TLandmarkID,TPoint3D> mrpt::vision::TLandmarkLocationsMap

A list of landmarks (3D points) indexed by unique IDs.

Definition at line 40 of file vision/include/mrpt/vision/types.h.

◆ TLandmarkLocationsVec

typedef std::vector<TPoint3D> mrpt::vision::TLandmarkLocationsVec

A list of landmarks (3D points), which assumes indexes are unique, consecutive IDs.

Definition at line 41 of file vision/include/mrpt/vision/types.h.

◆ TRelativeFeaturePosMap

typedef std::map<mrpt::vision::TFeatureID, TRelativeFeaturePos> mrpt::vision::TRelativeFeaturePosMap

An index of feature IDs and their relative locations.

Definition at line 110 of file vision/include/mrpt/vision/types.h.

Enumeration Type Documentation

◆ TDescriptorType

enum mrpt::vision::TDescriptorType

The bitwise OR combination of values of TDescriptorType are used in CFeatureExtraction::computeDescriptors to indicate which descriptors are to be computed for features.

Enumerator
descAny	Used in some methods to mean "any of the present descriptors".
descSIFT	SIFT descriptors.
descSURF	SURF descriptors.
descSpinImages	Intensity-domain spin image descriptors.
descPolarImages	Polar image descriptor.
descLogPolarImages	Log-Polar image descriptor.

Definition at line 63 of file vision/include/mrpt/vision/types.h.

◆ TFeatureTrackStatus

enum mrpt::vision::TFeatureTrackStatus

Enumerator
status_IDLE	Inactive (right after detection, and before being tried to track)
status_TRACKED	Feature correctly tracked.
status_OOB	Feature fell Out Of Bounds (out of the image limits, too close to image borders)
status_LOST	Unable to track this feature (mismatch is too high for the given tracking window: lack of texture? oclussion?)

Definition at line 73 of file vision/include/mrpt/vision/types.h.

◆ TFeatureType

enum mrpt::vision::TFeatureType

Types of features - This means that the point has been detected with this algorithm, which is independent of additional descriptors a feature may also have.

Enumerator
featNotDefined	Non-defined feature (also used for Occupancy features)
featKLT	Kanade-Lucas-Tomasi feature [SHI'94].
featHarris	Harris border and corner detector [HARRIS].
featBCD	Binary corder detector.
featSIFT	Scale Invariant Feature Transform [LOWE'04].
featSURF	Speeded Up Robust Feature [BAY'06].
featBeacon	A especial case: this is not an image feature, but a 2D/3D beacon (used for range-only SLAM from mrpt::slam::CLandmark)
featFAST	FAST feature detector, OpenCV's implementation ("Faster and better: A machine learning approach to corner detection", E. Rosten, R. Porter and T. Drummond, PAMI, 2009).
featFASTER9	FASTER-9 detector, Edward Rosten's libcvd implementation optimized for SSE2.
featFASTER10	FASTER-9 detector, Edward Rosten's libcvd implementation optimized for SSE2.
featFASTER12	FASTER-9 detector, Edward Rosten's libcvd implementation optimized for SSE2.

Definition at line 46 of file vision/include/mrpt/vision/types.h.

◆ TRobustKernelType

enum mrpt::vision::TRobustKernelType

The different types of kernels for usage within a robustified least-squares estimator.

See also: Use these types as arguments of the template RobustKernel<>

Enumerator
rkLeastSquares	No robust kernel, use standard least squares: rho(r)= 1/2 * r^2.
rkPseudoHuber	Pseudo-huber robust kernel.

Definition at line 29 of file robust_kernels.h.

Function Documentation

◆ addFeaturesToImage()

void VISION_IMPEXP mrpt::vision::addFeaturesToImage	(	const CImage &	inImg,
		const CFeatureList &	theList,
		CImage &	outImg
	)

Draw rectangles around each of the features on a copy of the input image.

Parameters

inImg	[IN] The input image where to draw the features.
theList	[IN] The list of features.
outImg	[OUT] The copy of the input image with the marked features.

◆ buildIntrinsicParamsMatrix()

CMatrixDouble33 VISION_IMPEXP mrpt::vision::buildIntrinsicParamsMatrix	(	const double	focalLengthX,
		const double	focalLengthY,
		const double	centerX,
		const double	centerY
	)

Builds the intrinsic parameters matrix A from parameters:

Parameters

focalLengthX	[IN] The focal length, in X (horizontal) pixels
focalLengthY	[IN] The focal length, in Y (vertical) pixels
centerX	[IN] The image center, horizontal, in pixels
centerY	[IN] The image center, vertical, in pixels

This method returns the matrix:

f_x	0	cX
0	f_y	cY
0	0	1

See also the tutorial discussing the camera model parameters.

See also: defaultIntrinsicParamsMatrix, pixelTo3D

◆ cloudsToMatchedList()

void VISION_IMPEXP mrpt::vision::cloudsToMatchedList	(	const CObservationVisualLandmarks &	cloud1,
		const CObservationVisualLandmarks &	cloud2,
		TMatchingPairList &	outList
	)

Transform two clouds of 3D points into a matched list of points ...

◆ computeMainOrientation()

float VISION_IMPEXP mrpt::vision::computeMainOrientation	(	const CImage &	image,
		unsigned int	x,
		unsigned int	y
	)

Computes the main orientation of a set of points with an image (for using in SIFT-based algorithms)

Parameters

image	[IN] The input image.
x	[IN] A vector containing the 'x' coordinates of the image points.
y	[IN] A vector containing the 'y' coordinates of the image points.

Returns: The main orientation of the image point.

◆ computeMsd()

double VISION_IMPEXP mrpt::vision::computeMsd	(	const TMatchingPairList &	list,
		const poses::CPose3D &	Rt
	)

Computes the mean squared distance between a set of 3D correspondences ...

◆ computeSAD()

double VISION_IMPEXP mrpt::vision::computeSAD	(	const CImage &	patch1,
		const CImage &	patch2
	)

Calculates the Sum of Absolutes Differences (range [0,1]) between two patches.

Both patches must have the same size.

Parameters

patch1	[IN] One patch.
patch2	[IN] The other patch.

Returns: The value of computed SAD normalized to [0,1]

◆ computeStereoRectificationMaps()

void VISION_IMPEXP mrpt::vision::computeStereoRectificationMaps	(	const TCamera &	cam1,
		const TCamera &	cam2,
		const poses::CPose3D &	rightCameraPose,
		void *	outMap1x,
		void *	outMap1y,
		void *	outMap2x,
		void *	outMap2y
	)

Computes a pair of x-and-y maps for stereo rectification from a pair of cameras and the relative pose of the second one wrt the first one.

Parameters

cam1,cam2	[IN] The pair of involved cameras
rightCameraPose	[IN] The change in pose of the second camera wrt the first one
outMap1x,outMap1y	[OUT] The x-and-y maps corresponding to cam1 (should be converted to *cv::Mat)
outMap2x,outMap2y	[OUT] The x-and-y maps corresponding to cam2 (should be converted to *cv::Mat)

See also: An easier to use class for stereo rectification mrpt::vision::CStereoRectifyMap

◆ correctDistortion()

void mrpt::vision::correctDistortion	(	const CImage &	in_img,
		CImage &	out_img,
		const CMatrixDouble33 &	A,
		const vector_double &	dist_coeffs
	)

inline

Returns a new image where distortion has been removed.

Parameters

A	The 3x3 intrinsic parameters matrix
dist_coeffs	The 1x4 (or 1x5) vector of distortion coefficients

Definition at line 190 of file vision/include/mrpt/vision/utils.h.

References mrpt::utils::CImage::rectifyImage().

◆ defaultIntrinsicParamsMatrix()

CMatrixDouble33 VISION_IMPEXP mrpt::vision::defaultIntrinsicParamsMatrix	(	unsigned int	camIndex = `0`,
		unsigned int	resolutionX = `320`,
		unsigned int	resolutionY = `240`
	)

Returns the stored, default intrinsic params matrix for a given camera:

Parameters

camIndex	[IN] Posible values are listed next.
resolutionX	[IN] The number of pixel columns
resolutionY	[IN] The number of pixel rows

The matrix is generated for the indicated camera resolution configuration. The following table summarizes the current supported cameras and the values as ratios of the corresponding horz. or vert. resolution:

camIndex	Manufacturer	Camera model	fx	fy	cx	cy
0	Point Grey Research	Bumblebee	0.79345	1.05793	0.55662	0.52692
1	Sony	???	0.95666094	1.3983423f	0.54626328f	0.4939191f

See also: buildIntrinsicParamsMatrix, pixelTo3D

◆ deleteRepeatedFeats()

void VISION_IMPEXP mrpt::vision::deleteRepeatedFeats ( CFeatureList & list )

Explore the feature list and removes features which are in the same coordinates.

Parameters

list	[IN] The list of features.

◆ flip()

void VISION_IMPEXP mrpt::vision::flip ( CImage & img )

Invert an image using OpenCV function.

◆ generateMask()

void VISION_IMPEXP mrpt::vision::generateMask	(	const CMatchedFeatureList &	mList,
		CMatrixBool &	mask1,
		CMatrixBool &	mask2,
		int	wSize = `10`
	)

Calculates the Sum of Absolutes Differences (range [0,1]) between two patches.

Both patches must have the same size.

Parameters

mList	[IN] The list of matched features.
mask1	[OUT] The output mask for left features.
mask2	[OUT] The output mask for right features.
wSize	[IN] The value of the masking window for each features.

Exceptions

if	mList.size() = 0

◆ getDispersion()

void VISION_IMPEXP mrpt::vision::getDispersion	(	const CFeatureList &	list,
		vector_float &	std,
		vector_float &	mean
	)

Computes the dispersion of the features in the image.

Parameters

list	[IN] Input list of features
std	[OUT] 2 element vector containing the standard deviations in the 'x' and 'y' coordinates.
mean	[OUT] 2 element vector containing the mean in the 'x' and 'y' coordinates.

◆ matchFeatures()

size_t VISION_IMPEXP mrpt::vision::matchFeatures	(	const CFeatureList &	list1,
		const CFeatureList &	list2,
		CMatchedFeatureList &	matches,
		const TMatchingOptions &	options = `TMatchingOptions()`,
		const TStereoSystemParams &	params = `TStereoSystemParams()`
	)

Find the matches between two lists of features which must be of the same type.

Parameters

list1	[IN] One list.
list2	[IN] Other list.
matches	[OUT] A vector of pairs of correspondences.
options	[IN] A struct containing matching options

Returns: Returns the number of matched pairs of features.

◆ normalizeImage()

void VISION_IMPEXP mrpt::vision::normalizeImage	(	const CImage &	image,
		CImage &	nimage
	)

Normalizes the brigthness and contrast of an image by setting its mean value to zero and its standard deviation to unit.

Parameters

image	[IN] The input image.
nimage	[OUTPUT] The new normalized image.

◆ openCV_cross_correlation()

void VISION_IMPEXP mrpt::vision::openCV_cross_correlation	(	const CImage &	img,
		const CImage &	patch_img,
		size_t &	x_max,
		size_t &	y_max,
		double &	max_val,
		int	x_search_ini = `-1`,
		int	y_search_ini = `-1`,
		int	x_search_size = `-1`,
		int	y_search_size = `-1`
	)

Computes the correlation between this image and another one, encapsulating the openCV function cvMatchTemplate This implementation reduced computation time.

Parameters

img	[IN] The imput image. This function supports gray-scale (1 channel only) images.
patch_img	[IN] The "patch" image, which must be equal, or smaller than "this" image. This function supports gray-scale (1 channel only) images.
x_max	[OUT] The x coordinate where it was found the maximun cross correlation value.
y_max	[OUT] The y coordinate where it was found the maximun cross correlation value.
max_val	[OUT] The maximun value of cross correlation which we can find
x_search_ini	[IN] The "x" coordinate of the search window.
y_search_ini	[IN] The "y" coordinate of the search window.
x_search_size	[IN] The width of the search window.
y_search_size	[IN] The height of the search window. Note: By default, the search area is the whole (this) image.

See also: cross_correlation

◆ pixelTo3D()

TPoint3D VISION_IMPEXP mrpt::vision::pixelTo3D	(	const TPixelCoordf &	xy,
		const CMatrixDouble33 &	A
	)

Extract a UNITARY 3D vector in the direction of a 3D point, given from its (x,y) pixels coordinates, and the camera intrinsic coordinates.

Parameters

xy	[IN] Pixels coordinates, from the top-left corner of the image.
A	[IN] The 3x3 intrinsic parameters matrix for the camera.

Returns: The TPoint3D containing the output unitary vector.

See also: buildIntrinsicParamsMatrix, defaultIntrinsicParamsMatrix, TPixelCoordf

◆ projectMatchedFeature()

void VISION_IMPEXP mrpt::vision::projectMatchedFeature	(	const CFeaturePtr &	leftFeat,
		const CFeaturePtr &	rightFeat,
		TPoint3D &	p3D,
		const TStereoSystemParams &	params = `TStereoSystemParams()`
	)

Computes the 3D position of a particular matched feature.

Parameters

leftList	[IN] The left feature.
rightList	[IN] The right feature.
vP3D	[OUT] The 3D position of the projected point.
params	[IN] The intrinsic and extrinsic parameters of the stereo pair.

◆ projectMatchedFeatures() [1/3]

void VISION_IMPEXP mrpt::vision::projectMatchedFeatures	(	const CFeatureList &	leftList,
		const CFeatureList &	rightList,
		vector< TPoint3D > &	vP3D,
		const TStereoSystemParams &	params = `TStereoSystemParams()`
	)

Computes the 3D position of a set of matched features from their coordinates in the images.

The list have to be matched in order, e.g. leftList[0]<->rightList[0]

Parameters

leftList	[IN] The left list of features.
rightList	[IN] The right list of features.
vP3D	[OUT] A vector of TPoint3D containing the 3D positions of the projected points.
params	[IN] The intrinsic and extrinsic parameters of the stereo pair.

◆ projectMatchedFeatures() [2/3]

void VISION_IMPEXP mrpt::vision::projectMatchedFeatures	(	CMatchedFeatureList &	mfList,
		const TStereoSystemParams &	param,
		CLandmarksMap &	landmarks
	)

Project a list of matched features into the 3D space, using the provided parameters of the stereo system.

Parameters

mfList	[IN/OUT] The list of matched features. Features which yields a 3D point outside the area defined in TStereoSystemParams are removed from the lists.
param	[IN] The parameters of the stereo system.
landmarks	[OUT] A map containing the projected landmarks.

See also: TStereoSystemParams, CLandmarksMap

◆ projectMatchedFeatures() [3/3]

void VISION_IMPEXP mrpt::vision::projectMatchedFeatures	(	CFeatureList &	leftList,
		CFeatureList &	rightList,
		const TStereoSystemParams &	param,
		CLandmarksMap &	landmarks
	)

Project a pair of feature lists into the 3D space, using the provided options for the stereo system.

The matches must be in order, i.e. leftList[0] corresponds to rightList[0] and so on. Features which yields a 3D point outside the area defined in TStereoSystemParams are removed from the lists.

Parameters

leftList	[IN/OUT] The left list of matched features.
rightList	[IN/OUT] The right list of matched features.
param	[IN] The options of the stereo system.
landmarks	(OUT] A map containing the projected landmarks.

See also: TStereoSystemParams, CLandmarksMap

◆ projectPoint_no_distortion() [1/2]

template<bool INVERSE_CAM_POSE>

TPixelCoordf mrpt::vision::pinhole::projectPoint_no_distortion	(	const mrpt::utils::TCamera &	cam_params,
		const mrpt::poses::CPose3D &	F,
		const mrpt::math::TPoint3D &	P
	)

inline

Project a single 3D point with global coordinates P into a camera at pose F, without distortion parameters.

The template argument INVERSE_CAM_POSE is related on how the camera pose "F" is stored:

INVERSE_CAM_POSE:false -> The local coordinates of the feature wrt the camera F are: $P \ominus F$
INVERSE_CAM_POSE:true -> The local coordinates of the feature wrt the camera F are: $F \oplus P$

Definition at line 53 of file pinhole.h.

References ASSERT_, mrpt::poses::CPose3D::composePoint(), mrpt::utils::TCamera::cx(), mrpt::utils::TCamera::cy(), mrpt::utils::TCamera::fx(), mrpt::utils::TCamera::fy(), mrpt::poses::CPose3D::inverseComposePoint(), mrpt::math::TPoint3D::x, mrpt::math::TPoint3D::y, and mrpt::math::TPoint3D::z.

◆ projectPoint_no_distortion() [2/2]

template<typename POINT >

void mrpt::vision::pinhole::projectPoint_no_distortion	(	const POINT &	in_point_wrt_cam,
		const mrpt::utils::TCamera &	cam_params,
		TPixelCoordf &	out_projectedPoints
	)

inline

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Definition at line 72 of file pinhole.h.

References ASSERT_, mrpt::utils::TCamera::cx(), mrpt::utils::TCamera::cy(), mrpt::utils::TCamera::fx(), mrpt::utils::TCamera::fy(), mrpt::utils::TPixelCoordf::x, and mrpt::utils::TPixelCoordf::y.

◆ projectPoint_with_distortion()

void VISION_IMPEXP mrpt::vision::pinhole::projectPoint_with_distortion	(	const mrpt::math::TPoint3D &	in_point_wrt_cam,
		const mrpt::utils::TCamera &	in_cam_params,
		TPixelCoordf &	out_projectedPoints,
		bool	accept_points_behind = `false`
	)

Project one 3D point into a camera using its calibration matrix and distortion parameters (radial and tangential distortions projection model)

Parameters

in_point_wrt_cam	[IN] The 3D point wrt the camera focus, with +Z=optical axis, +X=righthand in the image plane, +Y=downward in the image plane.
in_cam_params	[IN] The camera parameters. See http://www.mrpt.org/Camera_Parameters
out_projectedPoints	[OUT] The projected point, in pixel units.
accept_points_behind	[IN] See the note below.

Note: Points "behind" the camera (which couldn't be physically seen in the real world) are marked with pixel coordinates (-1,-1) to detect them as invalid, unless accept_points_behind is true. In that case they'll be projected normally.

See also: projectPoints_with_distortion

◆ projectPoints_no_distortion()

void VISION_IMPEXP mrpt::vision::pinhole::projectPoints_no_distortion	(	const std::vector< mrpt::poses::CPoint3D > &	in_points_3D,
		const mrpt::poses::CPose3D &	cameraPose,
		const mrpt::math::CMatrixDouble33 &	intrinsicParams,
		std::vector< TPixelCoordf > &	projectedPoints,
		bool	accept_points_behind = `false`
	)

Project a set of 3D points into a camera at an arbitrary 6D pose using its calibration matrix (undistorted projection model)

Parameters

in_points_3D	[IN] The list of 3D points in world coordinates (meters) to project.
cameraPose	[IN] The pose of the camera in the world.
intrinsicParams	[IN] The 3x3 calibration matrix. See http://www.mrpt.org/Camera_Parameters
projectedPoints	[OUT] The list of image coordinates (in pixels) for the projected points. At output this list is resized to the same number of input points.
accept_points_behind	[IN] See the note below.

Note: Points "behind" the camera (which couldn't be physically seen in the real world) are marked with pixel coordinates (-1,-1) to detect them as invalid, unless accept_points_behind is true. In that case they'll be projected normally.

See also: projectPoints_with_distortion, projectPoint_no_distortion

◆ projectPoints_with_distortion() [1/2]

void VISION_IMPEXP mrpt::vision::pinhole::projectPoints_with_distortion	(	const std::vector< mrpt::poses::CPoint3D > &	in_points_3D,
		const mrpt::poses::CPose3D &	cameraPose,
		const mrpt::math::CMatrixDouble33 &	intrinsicParams,
		const std::vector< double > &	distortionParams,
		std::vector< TPixelCoordf > &	projectedPoints,
		bool	accept_points_behind = `false`
	)

Project a set of 3D points into a camera at an arbitrary 6D pose using its calibration matrix and distortion parameters (radial and tangential distortions projection model)

Parameters

in_points_3D	[IN] The list of 3D points in world coordinates (meters) to project.
cameraPose	[IN] The pose of the camera in the world.
intrinsicParams	[IN] The 3x3 calibration matrix. See http://www.mrpt.org/Camera_Parameters
distortionParams	[IN] The 4-length vector with the distortion parameters [k1 k2 p1 p2]. See http://www.mrpt.org/Camera_Parameters
projectedPoints	[OUT] The list of image coordinates (in pixels) for the projected points. At output this list is resized to the same number of input points.
accept_points_behind	[IN] See the note below.

Note: Points "behind" the camera (which couldn't be physically seen in the real world) are marked with pixel coordinates (-1,-1) to detect them as invalid, unless accept_points_behind is true. In that case they'll be projected normally.

See also: projectPoint_with_distortion, projectPoints_no_distortion

◆ projectPoints_with_distortion() [2/2]

void VISION_IMPEXP mrpt::vision::pinhole::projectPoints_with_distortion	(	const std::vector< mrpt::math::TPoint3D > &	P,
		const mrpt::utils::TCamera &	params,
		const CPose3DQuat &	cameraPose,
		std::vector< TPixelCoordf > &	pixels,
		bool	accept_points_behind = `false`
	)

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

◆ rowChecking()

void VISION_IMPEXP mrpt::vision::rowChecking	(	CFeatureList &	leftList,
		CFeatureList &	rightList,
		float	threshold = `1.0`
	)

Search for correspondences which are not in the same row and deletes them.

Parameters

leftList	[IN/OUT] The left list of matched features.
rightList	[IN/OUT] The right list of matched features.
threshold	[IN] The tolerance value for the row checking: valid matched are within this threshold.

◆ StereoObs2BRObs() [1/3]

void VISION_IMPEXP mrpt::vision::StereoObs2BRObs	(	const CObservationStereoImages &	inObs,
		const vector< double > &	sg,
		CObservationBearingRange &	outObs
	)

Converts a stereo images observation into a bearing and range observation.

Converts a CObservationVisualLandmarks into a bearing and range observation (without any covariances).

Parameters

inObs	[IN] The input stereo images observation.
sg	[IN] The sigma of the row, col, and disparity variables involved in the feature detection.
outObs	[OUT] The output bearing and range observation (including covariances).

Fields of view are not computed.

Parameters

inObs	[IN] The input observation.
sg	[IN] The sigma of the row, col, and disparity variables involved in the feature detection.
outObs	[OUT] The output bearing and range observation.

◆ StereoObs2BRObs() [2/3]

void VISION_IMPEXP mrpt::vision::StereoObs2BRObs	(	const CMatchedFeatureList &	inMatches,
		const CMatrixDouble33 &	intrinsicParams,
		const double &	baseline,
		const CPose3D &	sensorPose,
		const vector< double > &	sg,
		CObservationBearingRange &	outObs
	)

Converts a matched feature list into a bearing and range observation (some of the stereo camera system must be provided).

Parameters

inMatches	[IN] The input list of matched features.
intrinsicParams	[IN] The intrisic params of the reference (left) camera of the stereo system.
baseline	[IN] The distance among the X axis of the right camera wrt the reference (left) camera.
sg	[IN] The sigma of the row, col, and disparity variables involved in the feature detection.
outObs	[OUT] The output bearing and range observation (including covariances).

◆ StereoObs2BRObs() [3/3]

void VISION_IMPEXP mrpt::vision::StereoObs2BRObs	(	const CObservationVisualLandmarks &	inObs,
		CObservationBearingRange &	outObs
	)

Converts a CObservationVisualLandmarks into a bearing and range observation (without any covariances).

Fields of view are not computed.

Parameters

inObs	[IN] The input observation.
outObs	[OUT] The output bearing and range observation.

◆ undistort_point()

void VISION_IMPEXP mrpt::vision::pinhole::undistort_point	(	const TPixelCoordf &	inPt,
		TPixelCoordf &	outPt,
		const mrpt::utils::TCamera &	cameraModel
	)

Undistort one point given by its pixel coordinates and the camera parameters.

See also: undistort_points

◆ undistort_points() [1/2]

void VISION_IMPEXP mrpt::vision::pinhole::undistort_points	(	const std::vector< TPixelCoordf > &	srcDistortedPixels,
		std::vector< TPixelCoordf > &	dstUndistortedPixels,
		const mrpt::math::CMatrixDouble33 &	intrinsicParams,
		const std::vector< double > &	distortionParams
	)

Undistort a list of points given by their pixel coordinates, provided the camera matrix and distortion coefficients.

Parameters

srcDistortedPixels	[IN] The pixel coordinates as in the distorted image.
dstUndistortedPixels	[OUT] The computed pixel coordinates without distortion.
intrinsicParams	[IN] The 3x3 calibration matrix. See http://www.mrpt.org/Camera_Parameters
distortionParams	[IN] The 4-length vector with the distortion parameters [k1 k2 p1 p2]. See http://www.mrpt.org/Camera_Parameters

See also: undistort_point

◆ undistort_points() [2/2]

void VISION_IMPEXP mrpt::vision::pinhole::undistort_points	(	const std::vector< TPixelCoordf > &	srcDistortedPixels,
		std::vector< TPixelCoordf > &	dstUndistortedPixels,
		const mrpt::utils::TCamera &	cameraModel
	)

Undistort a list of points given by their pixel coordinates, provided the camera matrix and distortion coefficients.

Parameters

srcDistortedPixels	[IN] The pixel coordinates as in the distorted image.
dstUndistortedPixels	[OUT] The computed pixel coordinates without distortion.
cameraModel	[IN] The camera parameters.

See also: undistort_point

Detailed Description

Library mrpt-vision

Classes

Namespaces

Modules

Typedefs

Enumerations

Functions

Typedef Documentation

◆ TCameraPoseID

◆ TFeatureID

◆ TFramePosesMap

◆ TFramePosesVec

◆ TLandmarkID

◆ TLandmarkLocationsMap

◆ TLandmarkLocationsVec

◆ TRelativeFeaturePosMap

Enumeration Type Documentation

◆ TDescriptorType

◆ TFeatureTrackStatus

◆ TFeatureType

◆ TRobustKernelType

Function Documentation

◆ addFeaturesToImage()

◆ buildIntrinsicParamsMatrix()

◆ cloudsToMatchedList()

◆ computeMainOrientation()

◆ computeMsd()

◆ computeSAD()

◆ computeStereoRectificationMaps()

◆ correctDistortion()

◆ defaultIntrinsicParamsMatrix()

◆ deleteRepeatedFeats()

◆ flip()

◆ generateMask()

◆ getDispersion()

◆ matchFeatures()

◆ normalizeImage()

◆ openCV_cross_correlation()

◆ pixelTo3D()

◆ projectMatchedFeature()

◆ projectMatchedFeatures() [1/3]

◆ projectMatchedFeatures() [2/3]

◆ projectMatchedFeatures() [3/3]

◆ projectPoint_no_distortion() [1/2]

◆ projectPoint_no_distortion() [2/2]

◆ projectPoint_with_distortion()

◆ projectPoints_no_distortion()

◆ projectPoints_with_distortion() [1/2]

◆ projectPoints_with_distortion() [2/2]

◆ rowChecking()

◆ StereoObs2BRObs() [1/3]

◆ StereoObs2BRObs() [2/3]

◆ StereoObs2BRObs() [3/3]

◆ undistort_point()

◆ undistort_points() [1/2]

◆ undistort_points() [2/2]

Library `mrpt-vision`