DrawMon:
A Distributed System for Detection of Atypical Sketch Content in Concurrent Pictionary Games

Nikhil Bansal        Kartik Gupta        Kiruthika Kannan        Sivani Pentapati        Ravi Kiran Sarvadevabhatla       

ACMMM 2022

Paper        Dataset        Code       

What is Pictionary?

Pictionary is a popular casual social game played by people of all age groups. In Pictionary, one player of the team is chosen as the Drawer while the other player or players is the Guesser.The goal of the game is for the Drawer to convey a randomly chosen target word by sketching to the teammate Guesser within a time limit. The Drawer is provided with a canvas on which the Drawer can sketch the target word or hints related to the target word. The Guesser looks at the canvas and provides guesses of the target word. The game ends successfully when the Guesser correctly guesses the target word and the players earn points. The game ends in a failure if the time runs out without the Guesser guessing the target word.

 

An example game session from our data collection tool

Atypical sketch content

The rules of Pictionary forbid the Drawer from writing text on canvas. However, in anonymized web-based version of the game, the Drawer may cheat by writing text on the digital whiteboard. Apart from such rule violation, atypical sketch content can also exist in non-malicious, benign scenarios. For instance, the Drawer may choose to draw arrows and other such icons to attract the Guesser’s attention and provide indirect hints regarding the target word. Accurately localizing such activities can aid statistical learning approaches which associate sketch-based representations with corresponding target words.

AtyPict- the first ever dataset of atypical whiteboard content

 

Examples of atypical content text, numbers, question marks, arrows, circles and other icons (e.g. tick marks, addition symbol).

The categories of atypical content usually encountered in Pictionary sessions are:

  • Text: Drawer directly writes the target word or hints related to the target word on the canvas.
  • Numerical: Drawer writes numbers on canvas.
  • Circles: Drawers often circle a portion of the canvas to emphasize relevant or important content.
  • Iconic: Other items used for emphasizing content and abstract compositional structures include drawing a question mark, arrow and other miscellaneous structures (e.g. double-headed arrow, tick marks, addition symbol, cross) and striking out the sketch (which usually implies negation of thesketched item).

 

The distribution of atypical content categories show significant imbalance with category 'Individual letters' occurring more often than others.

CanvasDash: an intuitive dashboard UI for annotation and visualization

 

An illustration of annotation using our Canvas-Dash interface.

 

An example game session from our data collection tool

CanvasNet: a deep network for detecting atypical sketch content

CanvasNet processes the rendered image as input and outputs a list of atypical activities (if any) along with associated meta-information (atypical content category, 2-D spatial location).

 

Architecture of CanvasNet deep neural network.

 

Examples of atypical content detection. False negatives are shown as dashed rectangles and false positives as dotted rectangles. Color codes are: text, numbers, question marks, arrows, circles and other icons (e.g. tick marks, addition symbol).

DrawMon: a distributed system for sketchcontent-based alert generation

DrawMon - a distributed alert generation system (see figure below). Each game session is managed by a central Session Manager which assigns a unique session id.

  • For a given session, whenever a sketch stroke is drawn, the accumulated canvas content (i.e. strokes rendered so far) is tagged with session id and relayed to a shared Session Canvas Queue.
  • For efficiency, the canvas content is represented as a lightweight Scalable Vector Graphic (SVG) object. The contents of the Session Canvas Queue are dequeued and rendered into corresponding 512×512 binary images by Distributed Rendering Module in a distributed and parallel fashion.
  • The rendered binary images tagged with session id are placed in the Rendered Image Queue. The contents of Rendered Image Queue are dequeued and processed by Distributed Detection Module. Each Detection module consists of our custom-designed deep neural network CanvasNet.

 

A pictorial overview of DrawMon- our distributed atypical sketch content response system.

 

DrawMon in Action

 

Acknowledgements

We wish to acknowledge grant from KCIS - TCS foundation.

 

Please consider citing the following works if you make use of our work:

@InProceedings{DrawMonACMMM2022,
author="Bansal, Nikhil
and Gupta, Kartik
and Kannan, Kiruthika
and Pentapati, Sivani
and Sarvadevabhatla, Ravi Kiran",
title="DrawMon: A Distributed System for Detection of Atypical Sketch Content in Concurrent Pictionary Games",
booktitle = "ACM conference on Multimedia (ACMMM)",
year="2022"
}