Main MRPT website > C++ reference for MRPT 1.9.9
COpenNI2_RGBD360.h
Go to the documentation of this file.
1 /* +------------------------------------------------------------------------+
2  | Mobile Robot Programming Toolkit (MRPT) |
3  | http://www.mrpt.org/ |
4  | |
5  | Copyright (c) 2005-2018, Individual contributors, see AUTHORS file |
6  | See: http://www.mrpt.org/Authors - All rights reserved. |
7  | Released under BSD License. See details in http://www.mrpt.org/License |
8  +------------------------------------------------------------------------+ */
9 #ifndef mrpt_OPENNI2_RGBD360_H
10 #define mrpt_OPENNI2_RGBD360_H
11 
17 
18 namespace mrpt
19 {
20 namespace hwdrivers
21 {
22 /** A class for grabing RGBD images from several OpenNI2 sensors. This is used
23  *to obtain larger fields of view using a radial configuration of the sensors.
24  * The same options (resolution, fps, etc.) are used for every sensor.
25  *
26  * <h2>Configuration and usage:</h2> <hr>
27  * Data is returned as observations of type mrpt::obs::CObservationRGBD360.
28  * See those classes for documentation on their fields.
29  *
30  * As with any other CGenericSensor class, the normal sequence of methods to be
31  *called is:
32  * - CGenericSensor::loadConfig() - Or calls to the individual setXXX() to
33  *configure the sensor parameters.
34  * - COpenNI2_RGBD360::initialize() - to start the communication with the
35  *sensor.
36  * - call COpenNI2_RGBD360::getNextObservation() for getting the data.
37  *
38  * <h2>Calibration parameters</h2><hr>
39  * The reference system for both depth and RGB images provided by each
40  *individual OpenNI2 sensors are referred to the
41  * RGB Camera.
42  * The extrinsic parameters of each RGBD sensor are provided from a
43  *configuration file. This calibration was obtained
44  * using the method reported in [].
45  *
46  * <h2>Coordinates convention</h2><hr>
47  * The origin of coordinates is the focal point of the RGB camera of the
48  *first indexed sensor, with the axes oriented
49  * as in the diagram shown in mrpt::obs::CObservation3DRangeScan. Notice in
50  *that picture that the RGB camera is
51  * assumed to have axes as usual in computer vision, which differ from those
52  *for the depth camera.
53  *
54  * The X,Y,Z axes used to report the data from accelerometers coincide with
55  *those of the depth camera
56  * (e.g. the camera standing on a table would have an ACC_Z=-9.8m/s2).
57  *
58  * Notice however that, for consistency with stereo cameras, when loading the
59  *calibration parameters from
60  * a configuration file, the left-to-right pose increment is expected as if
61  *both RGB & IR cameras had
62  * their +Z axes pointing forward, +X to the right, +Y downwards (just like
63  *it's the standard in stereo cameras
64  * and in computer vision literature). In other words: the pose stored in
65  *this class uses a different
66  * axes convention for the depth camera than in a stereo camera, so when a
67  *pose L2R is loaded from a calibration file
68  * it's actually converted like:
69  *
70  * L2R(this class convention) = CPose3D(0,0,0,-90deg,0deg,-90deg) (+)
71  *L2R(in the config file)
72  *
73  *
74  * <h2>Some general comments</h2><hr>
75  * - Depth is grabbed in 10bit depth, and a range N it's converted to
76  *meters
77  *as: range(m) = 0.1236 * tan(N/2842.5 + 1.1863)
78  * - This sensor can be also used from within rawlog-grabber to grab
79  *datasets
80  *within a robot with more sensors.
81  * - There is no built-in threading support, so if you use this class
82  *manually
83  *(not with-in rawlog-grabber),
84  * the ideal would be to create a thread and continuously request data
85  *from
86  *that thread (see mrpt::system::createThread ).
87  * - The intensity channel default to the RGB images, but it can be changed
88  *with setVideoChannel() to read the IR camera images (useful for calibrating).
89  * - There is a built-in support for an optional preview of the data on a
90  *window, so you don't need to even worry on creating a window to show them.
91  * - This class relies on an embedded version of libfreenect (you do NOT
92  *need
93  *to install it in your system). Thanks guys for the great job!
94  *
95  * <h2>Converting to 3D point cloud </h2><hr>
96  * You can convert the 3D observation into a 3D point cloud with this piece
97  *of code:
98  *
99  * \code
100  * mrpt::obs::CObservationRGBD360 obs3D;
101  * mrpt::maps::CColouredPointsMap pntsMap;
102  * pntsMap.colorScheme.scheme = CColouredPointsMap::cmFromIntensityImage;
103  * pntsMap.loadFromRangeScan(obs3D);
104  * \endcode
105  *
106  * Then the point cloud mrpt::maps::CColouredPointsMap can be converted into
107  *an OpenGL object for
108  * rendering with mrpt::maps::CMetricMap::getAs3DObject() or alternatively
109  *with:
110  *
111  * \code
112  * mrpt::opengl::CPointCloudColoured::Ptr gl_points =
113  *mrpt::make_aligned_shared<mrpt::opengl::CPointCloudColoured>();
114  * gl_points->loadFromPointsMap(&pntsMap);
115  * \endcode
116  *
117  *
118  * <h2>Platform-specific comments</h2><hr>
119  * For more details, refer to <a href="http://openkinect.org/wiki/Main_Page"
120  *>libfreenect</a> documentation:
121  * - Linux: You'll need root privileges to access Kinect. Or, install
122  *<code>
123  *MRPT/scripts/51-kinect.rules </code> in <code>/etc/udev/rules.d/</code> to
124  *allow access to all users.
125  * - Windows:
126  * - Since MRPT 0.9.4 you'll only need to install <a
127  *href="http://sourceforge.net/projects/libusb-win32/files/libusb-win32-releases/"
128  *>libusb-win32</a>: download and extract the latest
129  *libusb-win32-bin-x.x.x.x.zip
130  * - To install the drivers, read this:
131  *http://openkinect.org/wiki/Getting_Started#Windows
132  * - MacOS: (write me!)
133  *
134  *
135  * <h2>Format of parameters for loading from a .ini file</h2><hr>
136  *
137  * \code
138  * PARAMETERS IN THE ".INI"-LIKE CONFIGURATION STRINGS:
139  * -------------------------------------------------------
140  * [supplied_section_name]
141  * sensorLabel = RGBD360 // A text description
142  * preview_window = false // Show a window with a preview of the
143  *grabbed data in real-time
144  *
145  * device_number = 0 // Device index to open (0:first Kinect,
146  *1:second Kinect,...)
147  *
148  * grab_image = true // Grab the RGB image channel?
149  *(Default=true)
150  * grab_depth = true // Grab the depth channel? (Default=true)
151  * grab_3D_points = true // Grab the 3D point cloud? (Default=true)
152  *If disabled, points can be generated later on.
153  *
154  * video_channel = VIDEO_CHANNEL_RGB // Optional. Can be:
155  *VIDEO_CHANNEL_RGB (default) or VIDEO_CHANNEL_IR
156  *
157  * pose_x=0 // Camera position in the robot (meters)
158  * pose_y=0
159  * pose_z=0
160  * pose_yaw=0 // Angles in degrees
161  * pose_pitch=0
162  * pose_roll=0
163  *
164  *
165  * // Left/Depth camera
166  * [supplied_section_name_LEFT]
167  * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this
168  *section (it is not a separate device!)
169  *
170  * resolution = [640 488]
171  * cx = 314.649173
172  * cy = 240.160459
173  * fx = 572.882768
174  * fy = 542.739980
175  * dist = [-4.747169e-03 -4.357976e-03 0.000000e+00 0.000000e+00
176  *0.000000e+00] // The order is: [K1 K2 T1 T2 K3]
177  *
178  * // Right/RGB camera
179  * [supplied_section_name_RIGHT]
180  * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this
181  *section (it is not a separate device!)
182  *
183  * resolution = [640 480]
184  * cx = 322.515987
185  * cy = 259.055966
186  * fx = 521.179233
187  * fy = 493.033034
188  * dist = [5.858325e-02 3.856792e-02 0.000000e+00 0.000000e+00
189  *0.000000e+00] // The order is: [K1 K2 T1 T2 K3]
190  *
191  * // Relative pose of the right camera wrt to the left camera:
192  * // This assumes that both camera frames are such that +Z points
193  * // forwards, and +X and +Y to the right and downwards.
194  * // For the actual coordinates employed in 3D observations, see figure in
195  *mrpt::obs::CObservation3DRangeScan
196  * [supplied_section_name_LEFT2RIGHT_POSE]
197  * rawlog-grabber-ignore = true // Instructs rawlog-grabber to ignore this
198  *section (it is not a separate device!)
199  *
200  * pose_quaternion = [0.025575 -0.000609 -0.001462 0.999987 0.002038
201  *0.004335 -0.001693]
202  *
203  * \endcode
204  *
205  * More references to read:
206  * - http://RGBD360
207  * - http://http://www.openni.org/
208  * \ingroup mrpt_hwdrivers_grp
209  */
212 {
214 
215  public:
216  /** Default ctor
217  */
219  /** Default ctor
220  */
222 
223  /** Initializes the 3D camera - should be invoked after calling loadConfig()
224  * or setting the different parameters with the set*() methods.
225  * \exception This method must throw an exception with a descriptive
226  * message if some critical error is found.
227  */
228  virtual void initialize();
229 
230  /** To be called at a high rate (>XX Hz), this method populates the
231  * internal buffer of received observations.
232  * This method is mainly intended for usage within rawlog-grabber or
233  * similar programs.
234  * For an alternative, see getNextObservation()
235  * \exception This method must throw an exception with a descriptive
236  * message if some critical error is found.
237  * \sa getNextObservation
238  */
239  virtual void doProcess();
240 
241  /** The main data retrieving function, to be called after calling
242  * loadConfig() and initialize().
243  * \param out_obs The output retrieved observation (only if
244  * there_is_obs=true).
245  * \param there_is_obs If set to false, there was no new observation.
246  * \param hardware_error True on hardware/comms error.
247  *
248  * \sa doProcess
249  */
250  void getNextObservation(
251  mrpt::obs::CObservationRGBD360& out_obs, bool& there_is_obs,
252  bool& hardware_error);
253 
254  /** Set the path where to save off-rawlog image files (this class DOES take
255  * into account this path).
256  * An empty string (the default value at construction) means to save
257  * images embedded in the rawlog, instead of on separate files.
258  * \exception std::exception If the directory doesn't exists and cannot be
259  * created.
260  */
261  virtual void setPathForExternalImages(const std::string& directory);
262 
263  /** @name Sensor parameters (alternative to \a loadConfig ) and manual
264  control
265  @{ */
266 
267  /** Get the maximum range (meters) that can be read in the observation field
268  * "rangeImage" */
269  inline double getMaxRange() const { return m_maxRange; }
270  /** Enable/disable the grabbing of the RGB channel */
271  inline void enableGrabRGB(bool enable = true) { m_grab_rgb = enable; }
272  inline bool isGrabRGBEnabled() const { return m_grab_rgb; }
273  /** Enable/disable the grabbing of the depth channel */
274  inline void enableGrabDepth(bool enable = true) { m_grab_depth = enable; }
275  inline bool isGrabDepthEnabled() const { return m_grab_depth; }
276  /** Enable/disable the grabbing of the 3D point clouds */
277  inline void enableGrab3DPoints(bool enable = true)
278  {
279  m_grab_3D_points = enable;
280  }
281  inline bool isGrab3DPointsEnabled() const { return m_grab_3D_points; }
282  /** @} */
283 
284  protected:
285  virtual void loadConfig_sensorSpecific(
286  const mrpt::config::CConfigFileBase& configSource,
287  const std::string& section);
288 
290 
291  static const int NUM_SENSORS = 2;
292 
293  /** Show preview window while grabbing
294  */
296  /** If preview is enabled, only show 1 out of N images.
297  */
302 
303  /** Sensor max range (meters)
304  */
305  double m_maxRange;
306 
307  /** Default: all true
308  */
310 
311 }; // End of class
312 } // End of NS
313 
314 } // End of NS
315 
316 #endif
A generic interface for a wide-variety of sensors designed to be used in the application RawLogGrabbe...
bool m_grab_rgb
Default: all true.
Declares a class derived from "CObservation" that encapsules an omnidirectional RGBD measurement from...
void enableGrab3DPoints(bool enable=true)
Enable/disable the grabbing of the 3D point clouds.
void enableGrabRGB(bool enable=true)
Enable/disable the grabbing of the RGB channel.
mrpt::gui::CDisplayWindow::Ptr m_win_int[NUM_SENSORS]
size_t m_preview_window_decimation
If preview is enabled, only show 1 out of N images.
virtual void doProcess()
To be called at a high rate (>XX Hz), this method populates the internal buffer of received observati...
A class for grabing RGBD images from several OpenNI2 sensors.
void enableGrabDepth(bool enable=true)
Enable/disable the grabbing of the depth channel.
This class allows loading and storing values and vectors of different types from a configuration text...
virtual void setPathForExternalImages(const std::string &directory)
Set the path where to save off-rawlog image files (this class DOES take into account this path)...
An abstract class for accessing OpenNI2 compatible sensors.
virtual void loadConfig_sensorSpecific(const mrpt::config::CConfigFileBase &configSource, const std::string &section)
Loads specific configuration for the device from a given source of configuration parameters, for example, an ".ini" file, loading from the section "[iniSection]" (see config::CConfigFileBase and derived classes)
virtual void initialize()
Initializes the 3D camera - should be invoked after calling loadConfig() or setting the different par...
double getMaxRange() const
Get the maximum range (meters) that can be read in the observation field "rangeImage".
GLsizei const GLchar ** string
Definition: glext.h:4101
#define DEFINE_GENERIC_SENSOR(class_name)
This declaration must be inserted in all CGenericSensor classes definition, within the class declarat...
This is the global namespace for all Mobile Robot Programming Toolkit (MRPT) libraries.
A class used to store a 3D pose (a 3D translation + a rotation in 3D).
Definition: CPose3D.h:88
mrpt::poses::CPose3D m_sensorPoseOnRobot
bool m_preview_window
Show preview window while grabbing.
void getNextObservation(mrpt::obs::CObservationRGBD360 &out_obs, bool &there_is_obs, bool &hardware_error)
The main data retrieving function, to be called after calling loadConfig() and initialize().
mrpt::gui::CDisplayWindow::Ptr m_win_range[NUM_SENSORS]
double m_maxRange
Sensor max range (meters)



Page generated by Doxygen 1.8.14 for MRPT 1.9.9 Git: ad3a9d8ae Tue May 1 23:10:22 2018 -0700 at lun oct 28 00:14:14 CET 2019