Research in video quality assessment relies on the exploitation of subjective scores collected from subjects looking at the test video sequences. The availability of databases for performance comparison, development and testing of new models is of crucial importance. Only few databases are currently available and more popular in quality assessment field. State of arts data bases are limited to packet loss rate, coding and compression artifacts only. Here our goal is to create the data base with considering key Quality of Service (QoS) transmission parameters: dealy, delay variation/jitter, Packet Loss Rate (PLR) and throughput limitations as a possible artifacts.
Video Quality database, ReTRiEVED, to be used in evaluating the performances of video quality metrics is presented. The database contains 184 test videos obtained from eight source videos of different content characterized by wide span of spatial and temporal information and different motions. Test video sequences was generated by considering practical transmission scenario by using Network Emulator (NETEM) and Video LAN. Packet loss rate, jitter, delay, and throughput have been considered as possible distortions resulting from video transmission and its value is considered based on ITU and ETSI recommendations. Experiment setup has been designed based on the ITU standards to produce reliable and reproducible results. Original video sequence also being used for the evaluation as well, so that we could use Mean Opinion Score (MOS) and Difference Mean Opinion Score (DMOS) as a quality assessment metrics. Most stable and widely used video quality assessment technique called discrete single stimulus Absolute Category Rating (ACR) has been used for the quality evaluation. Video sequences, collected subjective scores, and results of the performed analysis are made publicly available for the research community, so that it could be used to design, test and compare subjective and objective video quality assessment metrics. For the further information about the database and results refer this paper.
We are making the ReTRiEVED Video Quality Database available to the research community free of charge.
If you use this database in your research, we kindly ask that you reference our papers listed below:
P. Paudyal, F. Battisti, and M. Carli, "A Study On The Effects Of Quality Of Service Parameters On Perceived Video Quality", Procs. of 5th European Workshop on Visual Information Processing, France, December 2014. [.pdf][.bib]
F. Battisti, M. Carli, and P. Paudyal, "QoS to QoE Mapping Model for Wired/Wireless Video Communication", Procs. of Euro Med Telco Conference, 12-15 November 2014, Naples, Italy. [.pdf][.bib]
P. Paudyal, F. Battisti, and M. Carli, "Study of the Effects of Video Content on Quality of Experience", Procs. of Image Quality and System Performance XII, IS&T/SPIE Electronic Imaging, 8-12 February, 2015, San Francisco, California, United States. (Accepted) [.pdf][.bib]
- Test Video Sequences
- 8 reference videos (size: 68 KB);
- 40 videos affected by PLR (7 PLR values (%): 0.1, 4, 1, 3, 5, 8 and 10, zipped folder size: 338 MB);
- 40 videos affected by jitter (5 jitter values (ms): 1, 2, 3, 4 and 5, zipped folder size: 221 MB);
- 40 videos affected by delay (5 delay values (ms): 100, 300, 500, 800 and 1000, zipped folder size: 248 MB);
- 40 videos affected by throughput (5 throughput values (Mbps): 0.512, 1, 2, 3 and 5, zipped folder size: 217 MB).
- Collected Subjective Scores
The investigators in this research are:
Mr. Pradip Paudyal
Dr. Federica Battisti
Dr. Marco Carli