User variance and its impact on video retrieval benchmarking

Peter  Wilkins; Raphael Troncy; Martin Halvey; Daragh Byrne; Alia Amin; P Punitha; Alan F.  Smeaton; Robert Villa

doi:10.1145/1646396.1646400

User variance and its impact on video retrieval benchmarking

Peter Wilkins, Raphael Troncy, Martin Halvey, Daragh Byrne, Alia Amin, P Punitha, Alan F. Smeaton, Robert Villa

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

In this paper, we describe one of the largest multi-site interactive video retrieval experiments conducted in a laboratory setting. Interactive video retrieval performance is difficult to cross-compare as variables exist across users, interfaces and the underlying retrieval engine. Conducted within the framework of TRECVID 2008, we completed a multi-site, multi-interface experiment. Three institutes participated involving 36 users, 12 each from Dublin City University (DCU, Ireland), University of Glasgow (GU, Scotland) and Centrum Wiskunde & Informatica (CWI, the Netherlands). Three user interfaces were developed which all used the same search service. Using a latin squares arrangement, each user completed 12 topics, leading to 6 TRECVID runs per site, 18 in total.

Original language	English
Title of host publication	Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR) 2009
Publisher	Association for Computing Machinery (ACM)
Number of pages	8
ISBN (Print)	9781605584805
DOIs	https://doi.org/10.1145/1646396.1646400
Publication status	Published - 2009

Keywords

TRECVID
user variance
video retrieval

Documents and Links

10.1145/1646396.1646400

Cite this

@inproceedings{7a182d86ae4c47f2b1826b266e7f6647,

title = "User variance and its impact on video retrieval benchmarking",

abstract = "In this paper, we describe one of the largest multi-site interactive video retrieval experiments conducted in a laboratory setting. Interactive video retrieval performance is difficult to cross-compare as variables exist across users, interfaces and the underlying retrieval engine. Conducted within the framework of TRECVID 2008, we completed a multi-site, multi-interface experiment. Three institutes participated involving 36 users, 12 each from Dublin City University (DCU, Ireland), University of Glasgow (GU, Scotland) and Centrum Wiskunde & Informatica (CWI, the Netherlands). Three user interfaces were developed which all used the same search service. Using a latin squares arrangement, each user completed 12 topics, leading to 6 TRECVID runs per site, 18 in total.",

keywords = "TRECVID, user variance, video retrieval ",

author = "Peter Wilkins and Raphael Troncy and Martin Halvey and Daragh Byrne and Alia Amin and P Punitha and Smeaton, {Alan F.} and Robert Villa",

note = "No page numbers cited, article no.2 js 09.05.13",

year = "2009",

doi = "10.1145/1646396.1646400",

language = "English",

isbn = "9781605584805",

booktitle = "Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR) 2009",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

}

TY - GEN

T1 - User variance and its impact on video retrieval benchmarking

AU - Wilkins, Peter

AU - Troncy, Raphael

AU - Halvey, Martin

AU - Byrne, Daragh

AU - Amin, Alia

AU - Punitha, P

AU - Smeaton, Alan F.

AU - Villa, Robert

N1 - No page numbers cited, article no.2 js 09.05.13

PY - 2009

Y1 - 2009

N2 - In this paper, we describe one of the largest multi-site interactive video retrieval experiments conducted in a laboratory setting. Interactive video retrieval performance is difficult to cross-compare as variables exist across users, interfaces and the underlying retrieval engine. Conducted within the framework of TRECVID 2008, we completed a multi-site, multi-interface experiment. Three institutes participated involving 36 users, 12 each from Dublin City University (DCU, Ireland), University of Glasgow (GU, Scotland) and Centrum Wiskunde & Informatica (CWI, the Netherlands). Three user interfaces were developed which all used the same search service. Using a latin squares arrangement, each user completed 12 topics, leading to 6 TRECVID runs per site, 18 in total.

AB - In this paper, we describe one of the largest multi-site interactive video retrieval experiments conducted in a laboratory setting. Interactive video retrieval performance is difficult to cross-compare as variables exist across users, interfaces and the underlying retrieval engine. Conducted within the framework of TRECVID 2008, we completed a multi-site, multi-interface experiment. Three institutes participated involving 36 users, 12 each from Dublin City University (DCU, Ireland), University of Glasgow (GU, Scotland) and Centrum Wiskunde & Informatica (CWI, the Netherlands). Three user interfaces were developed which all used the same search service. Using a latin squares arrangement, each user completed 12 topics, leading to 6 TRECVID runs per site, 18 in total.

KW - TRECVID

KW - user variance

KW - video retrieval

U2 - 10.1145/1646396.1646400

DO - 10.1145/1646396.1646400

M3 - Conference contribution

SN - 9781605584805

BT - Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR) 2009

PB - Association for Computing Machinery (ACM)

ER -

User variance and its impact on video retrieval benchmarking

Abstract

Keywords

Documents and Links

Fingerprint

Cite this