Error message here!

Hide Error message here!

Error message here!

Hide Error message here!

Error message here!

Close

Orbslam note 1 -- several methods in tracking

Cc1924 2022-01-23 13:10:15 阅读数:6 评论数:0 点赞数:0 收藏数:0

1.ReferenceKF Reference keyframes

1.1. Who has RefKF

• Tracking class ：Tracking Class RefKF Basically the same as the current ordinary frame RefKF It's the same , From the practical point of view 2 Solutions in .
• Normal frame ： Most of the time it's The key frame with the greatest degree of CO viewing with this ordinary frame , This RefKF Basically and Tracking Class RefKF It's the same .
• Map points ： Map points RefKF It's usually fixed , Namely Triangulate the key frame that generates this map point .

1.2.RefKF What's the usage?

• Tracking Of RefKF： It is used in tracking , such as `TrackReferenceKeyFrame` Is to match the current frame through the word bag 2D Point and Tracking Of the reference key frame 3D Map points . But actually when you call this function Tracking Of RefKF yes On a frame RefKF, This is also reasonable , Because the relationship between the current frame and the previous frame is still relatively close , And there is no information of the current frame available at this time , On a frame RefKF Basically, it is also the frame most closely related to the current frame .
• Normal frame RefKF： Most of the time it's The key frame with the greatest degree of CO viewing with this ordinary frame , The pose of this ordinary frame can be recorded more accurately . In the use of `TrackWithMotionModel` Before tracking , The pose of the previous frame will be updated first , Which uses the previous frame RefKF. The reason is that LocalMapping Local maps will be optimized in , So the last frame RefKF Your pose may be updated , So use the last frame relative to its RefKF The pose relationship of the and its Ref With the updated pose , You can get a more accurate pose in the previous frame , Thus, a more accurate pose under the uniform velocity estimation of the current frame is obtained .

1.3.RefKF When to generate

1.3.1.TrackLocalMap Time generation

Use... In the previous step `TrackReferenceKeyFrame` perhaps `TrackWithMotionModel` After the initial pose of the current frame is obtained by tracking , It will go further `TrackLocalMap` Tracking local maps . At this time, the local key frames of the current frame will be filtered , There will be statistics and The keyframe with the largest common view of the current frame , Then take this key frame as Current frame and Tracking class Of RefKF.

1.3.2. Generated when the current frame generates a new key frame

After this tracking , Determines whether to generate a new keyframe . If the current frame is generated into a new key frame , Then update the current frame and Tracking Class RefKF by The newly generated key frame .

1.4.Tracking Of RefKF Select for TrackReferenceKeyFrame() The rationality of

Both of the above methods make sense ,1 The choice is to consider Maximum co visibility in space ,2 The choice is to consider The last frame in time .

For example, the following use of constant speed model tracking failed , For example, there may be a fast motion in this frame , Beyond the radius search range of constant speed model , So use 1 Of Ref It is the key frame with the largest degree of CO viewing in the space of the immediately adjacent previous frame , It must be more common with the current frame .

If you use 2 Of Ref, So there is only one case , The previous frame just generated a keyframe , Then, the current frame moves rapidly, resulting in the loss of the constant speed motion model , So use 2 Of Ref It is equivalent to taking the previous frame as the reference key frame , They are adjacent in time , So it's also reasonable . Then to the local map tracking , The program will find the key frame with the highest degree of CO viewing with the current frame as Ref, If 2 Of Ref Not the one with the highest degree of CO vision , Then it will still be replaced .

2. Tracking and relocation methods

2.1.TrackReferenceKeyFrame() Reference keyframe tracking

2.1.1. technological process

Since there is no a priori information about the pose of the current frame , Can't use `SearchByProjection` This more efficient matching method . So the word bag is used to match the word of the reference key frame with the word of the current frame , Use on matching g2o Optimize the pose of the current frame , The pose of the previous frame is taken as the initial value of the pose of the current frame to accelerate convergence .

2.1.2. summary

• Yes g2o The rationality of setting the initial value of the optimized pose is in doubt , Because for reference key tracking just after relocation , If you've lost it for a long time , The last frame was also a long time ago , Then it makes no sense to use the pose of the previous frame to initialize the pose of the current frame . So I think we should first use the result of word bag matching to solve a RANSAC Of PnP As the initial value of optimization .
• For the above question , If the system has just been initialized , The reference key frame is the previous frame , That's all right. .
• For the above question , If it is a constant velocity model, the tracking fails （ For example, because the camera suddenly moves ）, Then the reference key frame is the key frame with the largest co view with the previous frame , No problem at this time .

2.2.TrackWithMotionModel() Uniform motion model tracking

2.2.1. technological process

The pose of the current frame is estimated according to the motion speed of the camera and the pose of the previous frame , Then project the map points of the previous frame to the current frame , Use `SearchByProjection` Match efficiently , For matching 3D and 2D Use... Between points g2o Optimize the pose of the current frame .

2.2.2. summary

• On the whole, it is reasonable . And one of the details is handled well , It is because of LocalMapping The pose of keyframes may be optimized in the thread , So here we use the relative pose of the previous frame to its reference key frame before tracking （ Local relations , Think accurate ） And the world pose of the reference keyframe （ May be in LocalMapping Updated ） To get the updated pose of the previous frame , The estimated pose of the current frame is more accurate .

2.3.TrackLocalMap() Tracking local maps

2.3.1. technological process

• First `UpdateLocalMap()` Construct a local map ： Local map = Local keyframes + Map points corresponding to local keyframes , So the key is how to select local keys . The specific selection strategy is to select the common view key frame of the current frame 、 The front of the common view keyframe 10 Common view keyframes 、 Parent child keyframes for common view keyframes . Local map points are the map points corresponding to these key frames .
• then `SearchLocalPoints()` Find more matches with the current frame in the local map points . Because the current frame already has a preliminary pose , So you can use `SearchByProjection` Match efficiently .
• Finally, match the 3D and 2D Use... Between points g2o Optimize the pose of the current frame .

2.3.2. summary

• Selection of local key frames of the above local map , I don't particularly understand the reason for this choice , It is better to select the common view key frame of the current frame , But the rest of the options still don't understand .

2.4.Relocalization() relocation

2.4.1. technological process

• Filter matching candidate keyframes ： Use the inverted list of the dictionary , That is, it stores the key frames under the current word , Filter the key frames most similar to the current frame as candidate key frames （recognation）.
• Word bag matching for candidate key frames ： Similarly, there is no a priori pose information of the current frame , Matches can only be made through word bags .
• Preliminarily solve the pose of the current frame ： At this time, many candidate key frames similar to the current frame are filtered out , Use g2o Optimization is too slow , So here we use RANSAC and EPnP Solving pose . For these keyframes , If the number of interior points solved >50, Then it is directly considered that the relocation using this key frame is successful .
• Finally, after successful relocation , Get the pose of the current frame , The matching relationship between the current frame and the map point of the relocation key frame is also obtained .

2.4.2. summary

• After a key frame is found and relocated successfully , This key frame will not be used as the reference key frame of the current frame , But in `TrackLocalMap` The key frame with the maximum common view of the current frame will be searched again , The key frame with the maximum common view is taken as the key frame of the current frame RefKF.

3. Map points during tracking

3.1. Map point source analysis

• The latest frame after initialization , Use RefKF track ： At this time RefKF This is the second frame during initialization . At this time, the tracked map point is RefKF Map points of , Then the current frame will match some map points , Therefore, the map points of the current frame are a subset of the map points of the reference frame .

• Use Ref The next frame after tracking , There's a motion model , After that, the constant velocity model is used to track ： At this time, it matches the map points of the previous frame and the current frame 2D spot , The map point in the previous frame is RefKF A subset of map points in , Therefore, the map point tracked in the current frame is still the map point of the previous frame RefKF A subset of .

• Tracking local maps ： The map points of the common view key frame of the current frame will be projected into the current frame , Look for more 3D-2D matching , Therefore, the new map point of the current frame obtained here is still the map point in the original key frame .

To sum up, we can see that , During the tracking process , If you don't think about LocalMapping The newly generated map points in the thread , Always use the map points generated during initialization , So in the process of follow-up The map points of each frame are a subset of the initialized map points . If the camera angle gradually deviates from the scene at initialization during tracking , Then there will be fewer and fewer map points in the tracking process , Finally, it leads to tracking failure . therefore LocalMapping The new map point in is to give Tracking Provide enough map points , It's very important ！