Please use this identifier to cite or link to this item:
https://hdl.handle.net/2440/138860
Citations | ||
Scopus | Web of Science® | Altmetric |
---|---|---|
?
|
?
|
Type: | Conference paper |
Title: | Proposal-free temporal moment localization of a natural-language query in video using guided attention |
Author: | Rodriguez Opazo, C. Marrese-Taylor, E. Saleh, F.S. Li, H. Gould, S. |
Citation: | Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV 2020), 2020, pp.2453-2462 |
Publisher: | IEEE |
Issue Date: | 2020 |
Series/Report no.: | IEEE Winter Conference on Applications of Computer Vision |
ISBN: | 9781728165530 |
ISSN: | 2642-9381 2472-6737 |
Conference Name: | IEEE Winter Conference on Applications of Computer Vision (WACV) (1 Mar 2020 - 5 Mar 2020 : Snowmass, CO, USA) |
Statement of Responsibility: | Cristian Rodriguez-Opazo, Edison Marrese-Taylor, Fatemeh Sadat Saleh, Hongdong Li, Stephen Gould |
Abstract: | This paper studies the problem of temporal moment localization in a long untrimmed video using natural language as the query. Given an untrimmed video and a query sentence, the goal is to determine the start and end of the relevant visual moment in the video that corresponds to the query sentence. While most previous works have tackled this by a propose-and-rank approach, we introduce a more efficient, end-to-end trainable, and proposal-free approach that is built upon three key components: a dynamic filter which adaptively transfers language information to visual domain attention map, a new loss function to guide the model to attend the most relevant part of the video, and soft labels to cope with annotation uncertainties. Our method is evaluated on three standard benchmark datasets, Charades-STA, TACoS and ActivityNet-Captions. Experimental results show our method outperforms state-of-theart methods on these datasets, confirming the effectiveness of the method. We believe the proposed dynamic filter-based guided attention mechanism will prove valuable for other vision and language tasks as well. |
Rights: | ©2020 IEEE |
DOI: | 10.1109/WACV45572.2020.9093328 |
Grant ID: | http://purl.org/au-research/grants/arc/CE140100016 |
Published version: | https://ieeexplore.ieee.org/xpl/conhome/9087828/proceeding |
Appears in Collections: | Australian Institute for Machine Learning publications |
Files in This Item:
There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.