Vision and geomagnetic indoor positioning (MS thesis)

Overview

This is the experiment work behind my MS thesis in Computer Science at the University of Gujrat (2024–2026), "Fusion of Geomagnetic Field to Mitigate Architectural Similarities in Vision-Based Indoor Positioning Systems".

The system locates a phone photo inside two building blocks by matching it against a database of reference photos and estimating the camera pose, then fuses that position with one from geomagnetic fingerprinting. A learned ResNet18 pose-regression model serves as a comparison baseline.

Context and problem

Vision-based indoor positioning gets confused by places that look alike, such as matching corridors in different blocks. The magnetic field inside a building varies from place to place, so I fused magnetic fingerprints with the visual estimate to tell those places apart.

The research appears aimed at a thesis committee and indoor-positioning researchers rather than at end users, so the output is experiments, comparisons and figures, not a product.

My role

It was my own research, and I was the only developer: every commit in the experiment repository is mine, as are the earlier working snapshot and the data-collection app.

I collected the field data with my own Flutter app, calibrated several phone cameras, built the matching, pose and fusion pipeline, trained the pose-regression baseline, and later answered the reviewers' and external examiner's comments.

Research and planning

The dataset is a grid of reference photos: each file name encodes the block, floor, grid position and viewing angle, with four orientations per grid cell, and lookup tables plus stored transforms map grid positions onto the floor map.

Each reference photo has SIFT descriptors and a grid-cell label in the reference database, and the reference magnetic readings form a separate fingerprint database.

The main method is a geo-guided signature–FLANN pipeline, in six steps:

1. The magnetic reading gives a first position, and only reference photos within a configured 10 m radius of it are searched.
2. Mean SIFT descriptor signatures shortlist the top 300 references for the query photo.
3. FLANN nearest-neighbour matching with a 0.77 ratio test scores each shortlisted reference.
4. Up to 30 leading candidates are re-ranked on a mix of visual score and closeness to the magnetic position.
5. Relative camera pose against the ranked candidates refines the visual position, preferring a candidate consistent with the magnetic point.
6. The visual and magnetic positions are fused into the final location.

To see what each step contributes, I compared this pipeline with several other matchers (k-means shortlists with brute-force or FLANN matching, RANSAC verification, and the same pipeline without re-ranking), and with a learned pose-regression baseline on the same query set.

Key features

• Reference database: SIFT descriptors and grid-cell location labels for every reference photo.
• Matching variants: geo-guided signature–FLANN with and without re-ranking, k-means shortlists at three sizes with brute-force or FLANN matching, and RANSAC verification.
• Magnetic fingerprinting and fusion: a nearest-fingerprint magnetic position, blended with the visual position using a fixed weight.
• Fusion-weight search: particle swarm optimisation over the fusion weight.
• Pose-regression baseline: a ResNet18 model that predicts a 2D position and the block, evaluated and fused the same way as the classical pipeline.
• Figures: error CDFs, block-confusion maps and top-view maps of the floor plan.
• Data collection: my own Flutter camera app that saves timestamped reference and query photos into per-location folders, plus per-phone camera calibrations.

Architecture and stack

The work runs along three paths: the original pipeline (MySQL ingestion, matching, pose and fusion-weight experiments), classical matcher comparisons that replay saved magnetic positions, and the pose-regression path (train, infer, fuse, plot).

• Computer vision: Python and OpenCV for SIFT features, brute-force and FLANN matching with a ratio test, RANSAC, relative-pose recovery and checkerboard camera calibration.
• Shortlisting: k-means descriptor clusters narrow the reference set before full matching.
• Data: the original scripts ingest reference images, descriptors and sensor readings into MySQL.
• Deep learning: PyTorch for the ResNet18 pose-regression model; earlier TensorFlow/Keras VGG16 and image-classification experiments in Jupyter notebooks.
• Optimisation: particle swarm optimisation for the fusion weights.
• Plots: matplotlib for error CDFs and floor-map overlays.
• Field app: Flutter and Dart for the data-collection camera app.

Implementation details

Magnetic matching picks the reference fingerprint with the smallest summed absolute difference across the field magnitude and its three components.

Pose refinement estimates an essential matrix with RANSAC from SIFT correspondences, recovers the relative camera pose with OpenCV, and shifts the matched reference position by a scaled translation offset.

The pose-regression model starts from an ImageNet-pretrained ResNet18 with one head for coordinates and one for the block. It is trained with SmoothL1 and cross-entropy losses on a step learning-rate schedule.

Its validation split holds out whole grid cells, so different viewing angles of the same spot never end up in both training and validation, and the query photos are kept out of training.

Challenges

• Look-alike corridors: vision alone confuses similar places in different blocks, which is the problem the magnetic fusion addresses.
• Examiner review: I answered reviewer and external-examiner comments on the objectives, the accuracy definitions and the justification for PSO, and re-checked the fusion weights with a retrospective PSO cross-validation.
• Fusion weights: that re-check concluded the reproduced PSO output is not a validated optimum for the fixed weights.
• Reconstruction: a structure-from-motion experiment produced OpenCV-based outputs, not a finished COLMAP reconstruction.

Trade-offs

The classical pipeline with magnetic fusion is likely the more interpretable option and needs no training beyond the reference set, while the learned baseline needs training but probably answers each query in a single forward pass.

I documented the project's limits in the README rather than present the saved results as reproducible: the "vision" pipeline already uses magnetic guidance, some saved magnetic estimates and block labels depend on ground truth, some tuning used test truth, and several core modules survive only as bytecode caches, so a fresh clone cannot rerun everything.

Links

No public link or repository for this project.

Skills demonstrated

• Computer vision: SIFT features, descriptor matching, RANSAC and camera pose with OpenCV.
• Sensor fusion of vision and geomagnetic fingerprints.
• Deep learning: a PyTorch ResNet18 pose-regression model with a leakage-free validation split.
• Optimisation with particle swarm search.
• Research methodology: matcher comparisons, a learned baseline and a response to examiner review.
• Mobile data-collection tooling in Flutter.