Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glomap mapper results are significantly worse than clomap #128

Open
Jacky56 opened this issue Oct 23, 2024 · 12 comments
Open

glomap mapper results are significantly worse than clomap #128

Jacky56 opened this issue Oct 23, 2024 · 12 comments

Comments

@Jacky56
Copy link

Jacky56 commented Oct 23, 2024

I've read multiple related issues #86 #123 #76 where glomap is not populating the sparse ponitcloud correctly. I assume this is the case for my issue as well.

versions:

ffmpeg version 4.4.2-0ubuntu0.22.04.1
cmake version 3.31.0-rc2
COLMAP 3.9.1 -- Structure-from-Motion and Multi-View Stereo
glomap 1.0.0

sample:

commands:

mkdir /tmp/test
cd /tmp/test
mkdir images sparse

ffmpeg -y -i '/tmp/test/vid.mp4' -vf fps=2,scale=-1:480 '/tmp/test/images/%06d.bmp'
colmap feature_extractor --database_path '/tmp/test/database.db' --image_path '/tmp/test/images' --SiftExtraction.use_gpu 0
colmap exhaustive_matcher --database_path '/tmp/test/database.db' --SiftMatching.use_gpu 0

mapper command:

# colmap
colmap mapper --database_path '/tmp/test/database.db' --image_path '/tmp/test/images' --output_path '/tmp/test/sparse'
# glomap
glomap mapper --database_path '/tmp/test/database.db' --image_path '/tmp/test/images' --output_path '/tmp/test/sparse'

a gsplat example:

@ljjTYJR
Copy link

ljjTYJR commented Oct 24, 2024

But in fact, it can be imagined that the global bundle adjustment would not be as robust as the incremental one

@lpanaf
Copy link
Collaborator

lpanaf commented Oct 25, 2024

Hi @Jacky56, thanks for the questions. Regarding the result, would you mind sharing the extracted database.db file directly so we can reproduce the result? I noticed in your command line that you do not enforce the images to share cameras. In your case, the image does not have too many features, so it would be quite difficult for GLOMAP to get good results. You can do so by setting --ImageReader.single_camera=1 in colmap feature_extractor

@Jacky56
Copy link
Author

Jacky56 commented Oct 25, 2024

Hello @lpanaf,

colmap feature_extractor --database_path '/tmp/test/database.db' --image_path '/tmp/test/images' --SiftExtraction.use_gpu 0

result without single camera

colmap feature_extractor --database_path '/tmp/test/database.db' --image_path '/tmp/test/images' --SiftExtraction.use_gpu 0 --ImageReader.single_camera=1

result with single camera

@lpanaf
Copy link
Collaborator

lpanaf commented Oct 25, 2024

Hi @Jacky56, thanks for the data. I tried with exhaustive_matcher on the dataset with a single camera, and this is the reconstruction result I got.
GLOMAP
image
COLMAP
image

If you feed these reconstructions into your gsplat pipeline, you should see similar result :)

@Jacky56
Copy link
Author

Jacky56 commented Oct 25, 2024

@lpanaf definitely major improvements on the results, although glomap variant may seem to cause 3dgs pipeline to incorrectly layer ellipsoids

result

@lpanaf
Copy link
Collaborator

lpanaf commented Oct 25, 2024

Hi @Jacky56, can you maybe be more specific on this comment? I am not very familiar with gsplat terms so I don't quite understand what this means. I checked gsplat from both GLOMAP and COLMAP, and they look similar to me.

@pablovela5620
Copy link

pablovela5620 commented Nov 13, 2024

Hi @lpanaf, I am having similar difficulties with the quality of output from glomap compared to colmap. I have tried with varying size datasets with different levels of success. For sake of iteration I chose a small self collected dataset that I just could not get glomap to reconstruct where as colmap did just fine. Here is the dataset

https://huggingface.co/datasets/pablovela5620/example-colmap-glomap/tree/main

and I used colmap/glomap with superpoint and lightglue using the following mapper command

mapper_cmd = [
        f"{colmap_cmd} mapper",
        f"--database_path {database}",
        f"--image_path {image_dir}",
        f"--output_path {sparse_dir}",
        f"--TrackEstablishment.max_num_tracks {1000*num_images}",
    ]

I will also add that I made matching sequential instead of exhaustive or vocab tree since it was taken via a video sequence so I also used a shared camera paramter

@pablovela5620
Copy link

I wanted to add that upping the number of images from 100 to 300 allowed for a reasonable reconstruction, but with glomap one gets the crazy outliers

glomap 100
glomap100

colmap 100
colmap 100

glomap 300
glomap300

colmap 300
colmap300

@lpanaf
Copy link
Collaborator

lpanaf commented Nov 15, 2024

@pablovela5620 Hi, thanks for sharing the data. I noticed only the matches and descriptors are available but not the database.db. Do you mind sharing it as well so we can look into it and try to replicate the result?

@pablovela5620
Copy link

Happy to, here you go
https://huggingface.co/datasets/pablovela5620/example-colmap-glomap/tree/main/colmap/sparse/0

I ran and reuploaded (basically the same results, but this time on the 100 it got split into 2 different models). The database.db should be there for both

@lpanaf
Copy link
Collaborator

lpanaf commented Nov 18, 2024

Hi @pablovela5620, thanks for the database. I have tried to reconstruct with it, and it turns out that the reconstruction of GLOMAP is very similar to COLMAP
image
Here, I manually align the reconstruction to COLMAP (with scaling).

The reason that it looks very different is that COLMAP rescales the reconstruction to be approximate at range (0, 5), while GLOMAP does not perform this step, and in the case of this dataset, it is very small so that you cannot properly view it.

We would work on a pull request to include this reconstruction normalization step for better visualization.

@pablovela5620
Copy link

This is awesome thank you! I appreciate you taking the time to help me debug this. It would be very helpful to have the visualizations between glomap/colmap to have the same post processing. I'm currently working on doing a bunch more testing for glomap/colmap as the speed of reconstruction from glomap is just so much faster

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants