Skip to content

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024

Booster-SHOT: Boosting Stacked Homography Transformations for Multiview Pedestrian Detection with Attention

A Homography Attention Module selects channels and spatial regions per homography plane, boosting end-to-end multiview pedestrian detection to state-of-the-art MODA on Wildtrack and MultiviewX.

Jinwoo Hwang1* Philipp Benz2 Pete Kim2

*Work performed while Jinwoo Hwang was at Deeping Source Inc.

1Seoul National University2Deeping Source Inc.

PDF Slides

Why this matters

Better multiview aggregation

Homography-based attention improves how features from calibrated camera views are fused into a bird’s-eye-view occupancy map.

Plug-in attention module

HAM’s channel gate and spatial gate drop into existing end-to-end detectors (MVDet, SHOT, MVDeTr) and improve their performance.

State-of-the-art performance

Booster-SHOT combines HAM with stacked homography transformations and view-coherent augmentation to reach state-of-the-art MODA.

Attention-guided homography fusion improves multiview pedestrian detection and bird’s-eye-view occupancy mapping.

Abstract

Improving multi-view aggregation is integral for multiview pedestrian detection, which aims to obtain a bird’s-eye-view pedestrian occupancy map from images captured through a set of calibrated cameras. Inspired by the success of attention modules for deep neural networks, we first propose a Homography Attention Module (HAM) which is shown to boost the performance of existing end-to-end multiview detection approaches by utilizing a novel channel gate and spatial gate. Additionally, we propose Booster-SHOT, an end-to-end convolutional approach to multiview pedestrian detection incorporating our proposed HAM as well as elements from previous approaches such as view-coherent augmentation or stacked homography transformations. Booster-SHOT achieves 92.9% and 94.2% for MODA on Wildtrack and MultiviewX respectively, outperforming the state-of-the-art by 1.4% on Wildtrack and 0.5% on MultiviewX, achieving state-of-the-art performance overall for standard evaluation metrics used in multi-view pedestrian detection.

Framework at a glance

Overview of multiview detection with the homography attention module
Multiview images are encoded into image features, passed through the Homography Attention Module (HAM), and projected onto stacked homography planes. A BEV heatmap generator then produces the pedestrian occupancy map. Inside HAM, a channel gate selects channels for each homography and a spatial gate re-weights each selected feature map before projection.

Homography Attention Module

Channel gate

Channel gate with channel selection and top-K selection modules
A shared MLP over max- and average-pooled features produces a distinct channel score for each homography; the top-K channels are selected per homography.

Spatial gate

Spatial gate with pooling, concatenation, and CNN attention map
Channel-wise max and average pooling are concatenated and passed through a CNN to produce a spatial attention map that re-weights the selected features.

Main results

92.9% MODA on Wildtrack
94.2% MODA on MultiviewX
+1.4 MODA points over prior state of the art (Wildtrack)
+0.5 MODA points over prior state of the art (MultiviewX)

Citation

BibTeX

@inproceedings{hwang2024boostershot,
  title={Booster-SHOT: Boosting Stacked Homography Transformations for Multiview Pedestrian Detection with Attention},
  author={Hwang, Jinwoo and Benz, Philipp and Kim, Pete},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2024},
  pages={362--371},
  doi={10.1109/WACV57701.2024.00043}
}