SHREC 2022 Track: Sketch-Based 3D Shape Retrieval in the Wild Call for Papers

SHREC 2022 Track: Sketch-Based 3D Shape Retrieval in the Wild

 
* Website:

https://sites.google.com/site/firmamentqj/sbsrw


* Registration Deadline: January 22


* Organizers:

- Jie Qin, Nanjing University of Aeronautics and Astronautics, Nanjing, China

- Shuaihang Yuan, New York University, New York, USA

- Jiaxin Chen, Beihang University, Beijing, China

- Boulbaba Ben Amor - IMT Nord Europe, France & Inception Institute of
Artificial Intelligence, UAE

- Yi Fang, NYU Abu Dhabi, UAE and NYU Tandon, USA

 

============================ Objective =================================

The objective of this track is to evaluate the performance of
different sketch-based 3D shape retrieval algorithms based on a 2D
free-hand sketch dataset and a 3D shape dataset in a more realistic
and challenging setting.

 

============================ Introduction ================================

Sketch-based 3D shape retrieval (SBSR) [1-3] has drawn a significant
amount of attention, owing to the succinctness of free-hand sketches
and the increasing demands from real applications. It is an intuitive
yet challenging task due to the large discrepancy between the 2D and
3D modalities.

To foster the research on this important problem, several tracks
focusing on related tasks have been held in the past SHREC challenges,
such as [4-7]. However, the datasets they adopted are not quite
realistic, and thus cannot well simulate real application
scenarios. To mimic the real-world scenario, the dataset is expected
to meet the following requirements. First, there should exist a large
domain gap between the two modalities, i.e., sketches and 3D
shapes. However, current datasets unintentionally narrow this gap by
using projection-based/multi-view representations for 3D shapes (i.e.,
a 3D shape is manually rendered into a set of 2D images). In this way,
the large 2D-3D domain discrepancy is unnecessarily reduced to the
2D-2D one. Second, the data themselves from both modalities should be
realistic, mimicking the real-world scenario. More specifically, we
need a full variety of sketches per category as real users possess
various drawing skills. As for 3D shapes, we need to frame 3D models
with real-world settings more than create them artificially. However,
human sketches on existing datasets tend to be semi-photorealistic
drawn by experts and the number of sketches per category is quite
limited; in the meantime, most current 3D datasets used in SBSR are
composed of CAD models, losing certain details compared to the models
scanned from real objects.


To circumvent the above limitations, this track proposes a more
realistic and challenging setting for SBSR. On the one hand, we adopt
highly abstract 2D sketches drawn by amateurs, and at the same time,
bypass the projection-based representations for 3D shapes by directly
adopting and representing 3D point cloud data. On the other hand, we
adopt a full variety of free-hand sketches with various samples per
category, as well as a collection of realistic point cloud data framed
from indoor objects. Therefore, we name this track "sketch-based 3D
shape retrieval in the wild" (SBSRW). As stated above, the term
"in the wild" is reflected in two perspectives: 1) The domain
gap between the two modalities is realistic as we adopt sketches of
high abstraction levels and 3D point cloud data. 2) The data
themselves mimic the real-world setting as we adopt a full variety of
sketches (3,000 per category) and 3D point clouds captured from real
objects.

 

======================= Tasks ===========================

We proposed two tasks to evaluate the performance of different SBSR
algorithms, i.e., sketch-based 3D CAD model (point cloud data)
retrieval and sketch-based realistic scanned model (point cloud data)
retrieval.


For the first task, we select around 2,500 3D CAD models from 47
classes on ModelNet40/ShapeNet and 3,000 sketches from each
corresponding category (141,000 sketch samples in total) on
QuickDraw. We randomly select 2,500 sketches from each class for
training, and the remaining 500 sketches per class are used for
testing/query. All the 3D point clouds as a whole are utilized as the
target/gallery dataset to evaluate the retrieval
performance. Participants are asked to submit the results on the test
datasets.


For the second task, we select 2,000 realistic 3D models from 11
classes on ScanObjectNN and 3,000 sketches per class (33,000 sketch
samples in total) from QuickDraw. Similar to the first task, we
randomly select 2,500 sketches from each class for training, and the
remaining 500 sketches per class are used for testing/query. All the
3D point clouds as a whole are utilized as the target/gallery dataset
to evaluate the retrieval performance. Participants are asked to
submit the results on the test datasets.

 

======================= Evaluation Method ===========================

For a comprehensive evaluation of different algorithms, we employ the
following widely-adopted performance metrics in SBSR, including
nearest neighbor (NN), first tier (FT), second tier (ST), E-measure
(E), discounted cumulated gain (DCG), mean average precision (mAP),
and precision-recall (PR) curve. We will provide the source code to
compute all the aforementioned metrics.
 

======================= Procedure ===========================

The following list is a step-by-step description of the activities:

    The participants register the track by sending an email to
    qinjiebuaa@gmail.com with 'SHREC 2022 - SBSRW Track Registration'
    as the title and indicating which task they are interested in.

    The organizers release the dataset via their website.

    The participants submit the distance matrices for the test sets,
    with one-page descriptions of their methods.

    Evaluation is automatically performed based on the submitted
    matrices, by computing all the performance metrics via the
    official source code.

    The organizers announce the results and the final rank list of all
    the participants.

    The track results are combined into a joint paper, which is
    subject to a two-stage peer review process. Accepted papers will
    be published in Computers & Graphics.

    The description of the track and the results will be presented at
    Eurographics 2022 Symposium on 3D Object Retrieval (1-2 September
    2022).

 

======================= Schedule ===========================

    January 1: Call for participation.
    January 15: Release a few sample sketches and 3D models.
    January 22: Registration deadline.
    January 29: Release the training set for the first task.
    February 5: Release the training set for the second task.
    February 28: Submission deadline for the first task.
    March 4: Submission deadline for the second task.
    March 8: Release the final results for both tasks; jointly write the track report.
    March 15: Submission deadline for the joint paper for C&G review.