#include <CUGestureRecognizer.h>

Public Types
enum class	Algorithm { ONEDOLLAR , PROTRACTOR }

Public Member Functions
	GestureRecognizer ()

	~GestureRecognizer ()

bool	init ()

bool	init (size_t samples, Size bounds)

bool	initWithJson (const std::shared_ptr< JsonValue > &json)

void	dispose ()

float	getSampleSize () const

Size	getNormalizedBounds () const

Algorithm	getAlgorithm () const

void	setAlgorithm (Algorithm algorithm)

float	getSimilarityThreshold () const

void	setSimilarityThreshold (float accuracy)

float	getOrientationTolerance () const

void	setOrientationTolerance (float tolerance)

const std::string	match (const std::vector< Vec2 > points)

const std::string	match (const Vec2 *points, size_t psize)

const std::string	match (const Path2 &path)

const std::string	match (const std::vector< Vec2 > points, float &similarity)

const std::string	match (const Vec2 *points, size_t psize, float &similarity)

const std::string	match (const Path2 &path, float &similarity)

float	similarity (const std::string name, const std::vector< Vec2 > &points, bool invariant=true)

float	similarity (const std::string name, const Vec2 *points, size_t psize, bool invariant=true)

float	similarity (const std::string name, const Path2 &path, bool invariant=true)

bool	addGesture (const std::string name, const std::vector< Vec2 > &points, bool unique=false)

bool	addGesture (const std::string name, const Vec2 *points, size_t psize, bool unique=false)

bool	addGesture (const std::string name, const Path2 &path, bool unique=false)

void	addGestures (const std::shared_ptr< JsonValue > &json, bool unique=false)

void	removeGesture (const std::string name)

void	clearGestures ()

bool	hasGesture (const std::string name) const

const UnistrokeGesture	getGesture (const std::string name) const

std::vector< std::string >	getGestureNames () const

std::vector< UnistrokeGesture >	getGestures () const

Static Public Member Functions
static std::shared_ptr< GestureRecognizer >	alloc ()

static std::shared_ptr< GestureRecognizer >	alloc (size_t samples, Size bounds)

static std::shared_ptr< GestureRecognizer >	allocWithJson (const std::shared_ptr< JsonValue > &json)

Detailed Description

A class representing the gesture recognition engine.

This class contains all the required funtionality needed for recognizing user defined gestures. It's major responsibilities are to store a collection of template gestures (for comparison), and normalize a gesture to a grid of fixed size. Gesture matches are determined by computing a similarity score between normalized instances.

Normalization involves resampling the gesture to a fixed number of points, as defined by getSampleSize. In addition, the gesture is rescaled to a box defined by getNormalizedBounds.

The recognition algorithm can use either the traditional $1 algorithm algorithm or the PROTACTOR method, depending on what suits you and what gives you the best results for your application. For more information on these algorithms, see

https ://dl.acm.org/citation.cfm?id=1294238 https ://dl.acm.org/citation.cfm?id=1753654/

Both of these algorithms are rotationally oblivious, meaning that they can recognize the gestures at any orientation. Typically this is not what we want in game development, however. Therfore, this recognizer includes the option to reject any matches whose angles of rotation exceed a certain threshold. See getOrientationTolerance for more information.

Similarity is determined on a scale of 0 to 1 where 1 is a complete match and 0 is no match at all. A pure 0 is difficult to achieve. By default, we consider any gesture a possible match if it has a similarity of at least 0.8.

Member Enumeration Documentation

◆ Algorithm

enum class cugl::GestureRecognizer::Algorithm

strong

An enumeration listing the type of matching algorithms

Enumerator

ONEDOLLAR

The classic $1 algorithm (default)

This is a slower, but more accurate method. It has to to iterate through several angles when comparing two gestures to each other

PROTRACTOR

The PROTRACTOR algorithm

This is a faster, less accurate method. It converts the gesture into a high dimensional feature vector so that it only needs one step to compare two gestures.

Constructor & Destructor Documentation

◆ GestureRecognizer()

cugl::GestureRecognizer::GestureRecognizer ( )

Creates an uninitialized Gesture Recognizer.

NEVER USE A CONSTRUCTOR WITH NEW. If you want to allocate an object on the heap, use one of the static constructors instead.

◆ ~GestureRecognizer()

cugl::GestureRecognizer::~GestureRecognizer ( )

inline

Deletes this recognizer and all of its resources.

Member Function Documentation

◆ addGesture() [1/3]

bool cugl::GestureRecognizer::addGesture	(	const std::string	name,
		const Path2 &	path,
		bool	unique = `false`
	)

inline

Adds the given gesture to this recognizer using the given name.

The gesture will be normalized before storing it. If the gesture has the same name as an existing one, the previous gesture will be replaced.

If the optional parameter unique is set to true, this method will first check that the gesture is unique (e.g. it does not exceed the similarity threshold when compared to any existing gestures) before adding it. If the gesture is too close to an existing one, this method will return false.

Note that uniqueness is determined according the current algorithm, similarity threshold, and orientation tolerance. If any of these values change, then uniqueness is no longer guaranteed.

When converting to a gesture, the path will be treated as a linear sequence of points. Corner classifications and whether the path is closed will be ignored.

Parameters

name	the name of the gesture to compare
path	a path representing a candidate gesture
unique	whether to enforce gesture uniqueness

Returns: true if the gesture was added to this recognizer

◆ addGesture() [2/3]

bool cugl::GestureRecognizer::addGesture	(	const std::string	name,
		const std::vector< Vec2 > &	points,
		bool	unique = `false`
	)

inline

Adds the given gesture to this recognizer using the given name.

The gesture will be normalized before storing it. If the gesture has the same name as an existing one, the previous gesture will be replaced.

If the optional parameter unique is set to true, this method will first check that the gesture is unique (e.g. it does not exceed the similarity threshold when compared to any existing gestures) before adding it. If the gesture is too close to an existing one, this method will return false.

Note that uniqueness is determined according the current algorithm, similarity threshold, and orientation tolerance. If any of these values change, then uniqueness is no longer guaranteed.

Parameters

name	the gesture name
points	a vector of points representing a gesture
unique	whether to enforce gesture uniqueness

Returns: true if the gesture was added to this recognizer

◆ addGesture() [3/3]

bool cugl::GestureRecognizer::addGesture	(	const std::string	name,
		const Vec2 *	points,
		size_t	psize,
		bool	unique = `false`
	)

Adds the given gesture to this recognizer using the given name.

The gesture will be normalized before storing it. If the gesture has the same name as an existing one, the previous gesture will be replaced.

If the optional parameter unique is set to true, this method will first check that the gesture is unique (e.g. it does not exceed the similarity threshold when compared to any existing gestures) before adding it. If the gesture is too close to an existing one, this method will return false.

Note that uniqueness is determined according the current algorithm, similarity threshold, and orientation tolerance. If any of these values change, then uniqueness is no longer guaranteed.

Parameters

name	the gesture name
points	an array of points representing a gesture
psize	the number of points in the gesture
unique	whether to enforce gesture uniqueness

Returns: true if the gesture was added to this recognizer

◆ addGestures()

void cugl::GestureRecognizer::addGestures	(	const std::shared_ptr< JsonValue > &	json,
		bool	unique = `false`
	)

Adds all of the gestures in the given JSON specification

The JSON value should be a list of entries that all follow the format used by the Path2 class.

If the optional parameter unique is set to true, this method will first check that each gesture is unique (e.g. it does not exceed the similarity threshold when compared to any existing gestures) before adding it. Any gesture that matches one that came previously in the list will be dropped.

Note that uniqueness is determined according the current algorithm, similarity threshold, and orientation tolerance. If any of these values change, then uniqueness is no longer guaranteed.

Parameters

json	The JSON object specifying the recognizer
unique	whether to enforce gesture uniqueness

◆ alloc() [1/2]

static std::shared_ptr< GestureRecognizer > cugl::GestureRecognizer::alloc ( )

inlinestatic

Returns a newly allocated gesture recognizer with the default settings.

The sample rate and normalization bounds will be set the default values. The recognier will be empty, and therefore will not match any gestures until some are added via addGesture.

Returns: a newly allocated gesture recognizer with the default settings.

◆ alloc() [2/2]

static std::shared_ptr< GestureRecognizer > cugl::GestureRecognizer::alloc	(	size_t	samples,
		Size	bounds
	)

inlinestatic

Returns a newly allocated gesture recognizer with the given accuracy.

The recognier will be empty, and therefore will not match any gestures until some are added via addGesture.

Parameters

samples	The number of samples to use in a normalized gesture
bounds	The bounding box for normalized gestures

Returns: a newly allocated gesture recognizer with the default accuracy.

◆ allocWithJson()

static std::shared_ptr< GestureRecognizer > cugl::GestureRecognizer::allocWithJson ( const std::shared_ptr< JsonValue > & json )

inlinestatic

Returns a newly allocated gesture recognizer with the given JSON entry

The JSON entry supports the following attribute values:

 "algorithm": One of "onedollar" or "protractor"
 "accuracy":  A float with the similarity threshold
 "tolerance": A float with the rotational tolerance
 "samples":   An int for the number of sample points
 "bounds":    A two-element list of floats representing the
              bounding box for normalized gestures
 "gestures":  A list of path entries

The path entries should all follow the format used by the Path2 class.

Parameters

json	The JSON object specifying the recognizer

Returns: a newly allocated gesture recognizer with the given JSON entry

◆ clearGestures()

void cugl::GestureRecognizer::clearGestures ( )

inline

Removes all gestures from this recognizer

◆ dispose()

void cugl::GestureRecognizer::dispose ( )

Empties the recongizer of all gestures and resets all attributes.

This will set the sample size to 0, meaning no future matches are possible. You must reinitialize the object to use it.

◆ getAlgorithm()

Algorithm cugl::GestureRecognizer::getAlgorithm ( ) const

inline

Returns the current matching algorithm

This gesture recognizer can use either the classic $1 algorithm or the PROTRACTOR algorithm. The PROTRACTOR is faster with less accuracy. However the need for speed really depends on the number of gestures stored in this recognizer. Determining which algorithm is best is typically a matter of experiementation.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the active algorithm. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Returns: the current matching algorithm

◆ getGesture()

const UnistrokeGesture cugl::GestureRecognizer::getGesture ( const std::string name ) const

inline

Returns (a copy of) gesture of the given name

If there is no gesture of that name, it will return an empty gesture.

Parameters

name	the name of the gesture to query

Returns: (a copy of) gesture of the given name

◆ getGestureNames()

std::vector< std::string > cugl::GestureRecognizer::getGestureNames ( ) const

Returns a vector of all the gesture names

Returns: a vector of all the gesture names

◆ getGestures()

std::vector< UnistrokeGesture > cugl::GestureRecognizer::getGestures ( ) const

Returns all the (normalized) gestures stored in this recognizer

Returns: all the (normalized) gestures stored in this recognizer

◆ getNormalizedBounds()

Size cugl::GestureRecognizer::getNormalizedBounds ( ) const

inline

Returns the bounding box of the normalization space

When a gesture is normalized, it is resized to so that its bounding box matches that of all other gestures. This value is the size of the box. By default it is 250x250.

This value is set when the gesture recognizer is allocated and cannot be changed. Changing this value would require the reintialization of all gestures.

Returns: the bounding box of the normalization space

◆ getOrientationTolerance()

float cugl::GestureRecognizer::getOrientationTolerance ( ) const

inline

Returns the rotational tolerance for gesture matching.

The matching algorithms are rotationally oblivious, meaning that they can recognize the gestures at any orientation. Typically this is not what we want in game development, however. If a gesture is rotated too far, we want to reject it.

The tolerance is the maximum allowable angle (in radians) of rotation for a gesture to be recognized. The angle of a gesture is measured using the vector from its first point to the (normalized) centroid. If θ is the angle between the gesture and potential match, it will be rejected if it is greater than +/- the tolerance.

By default this value is 20°. Setting this to a negative value will allow free rotation of gestures.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the current rotational tolerance. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Returns: the rotational tolerance for gesture matching.

◆ getSampleSize()

float cugl::GestureRecognizer::getSampleSize ( ) const

inline

Returns the sample rate of a normalized gesture

When a gesture is either added to the recognizer or posed as a candidate for matching, it will first be normalized. All normalized gestures will have this many points (the the PROCTRACTOR algorithm will only use 1/4 this many points).

Increasing this number will hurt performance, while decreasing this value will hurt accuracy. Ideally it should be between 30-128. By default this value is 64 (and hence 16 for the PROTRACTOR algorithm).

This value is set when the gesture recognizer is allocated and cannot be changed. Changing this value would require the reintialization of all gestures.

Returns: the sample rate of a normalized gesture

◆ getSimilarityThreshold()

float cugl::GestureRecognizer::getSimilarityThreshold ( ) const

inline

Returns the similarity threshold

When matching a candidate gesture against the stored values, we only consider matches whose similarity value is above this threshold. If this value is non-zero, it is possible that a candidate gesture will have no matches. By default this value is 0.8.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the current similarity threshold. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Returns: the similarity threshold

◆ hasGesture()

bool cugl::GestureRecognizer::hasGesture ( const std::string name ) const

inline

Returns true if this recognizer has a gesture of the given name

Parameters

name	the name of the gesture to query

Returns: true if this recognizer has a gesture of the given name

◆ init() [1/2]

bool cugl::GestureRecognizer::init ( )

Initializes an empty gesture recognizer with the default settings.

Calling this method initializes the sample rate and the bounding box for normalization. These values will be set to the defaults. The recognizer will be empty, and therefore will not match any gestures until some are added via addGesture.

Returns: true if the recognizer is initialized properly, false otherwise.

◆ init() [2/2]

bool cugl::GestureRecognizer::init	(	size_t	samples,
		Size	bounds
	)

Initializes an empty gesture recognizer with the given settings.

The recognier will be empty, and therefore will not match any gestures until some are added via addGesture.

Parameters

samples	The number of samples to use in a normalized gesture
bounds	The bounding box for normalized gestures

Returns: true if the recognizer is initialized properly, false otherwise.

◆ initWithJson()

bool cugl::GestureRecognizer::initWithJson ( const std::shared_ptr< JsonValue > & json )

Initializes the gesture recognizer with the given JSON entry

The JSON entry supports the following attribute values:

 "algorithm": One of "onedollar" or "protractor"
 "accuracy":  A float with the similarity threshold
 "tolerance": A float with the rotational tolerance
 "samples":   An int for the number of sample points
 "bounds":    A two-element list of floats representing the
              bounding box for normalized gestures
 "gestures":  A list of path entries

The path entries should all follow the format used by the Path2 class.

Parameters

json	The JSON object specifying the recognizer

Returns: true if the recognizer is initialized properly, false otherwise.

◆ match() [1/6]

const std::string cugl::GestureRecognizer::match ( const Path2 & path )

inline

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm. If there is no match within the similarity threshold or orientation tolerance, this method will return the empty string. A gesture must consist of at least two points.

When matching as a gesture, the path will be treated as a linear sequence of points. Corner classifications and whether the path is closed will be ignored.

Parameters

path	a path representing a candidate gesture.

Returns: the name of the gesture with the closest match to the given one.

◆ match() [2/6]

const std::string cugl::GestureRecognizer::match	(	const Path2 &	path,
		float &	similarity
	)

inline

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm, with the (rotationally invariant) similarity score stored in the reference parameter. If there is no match within the similarity threshold, or orientation tolerance, this method will return the empty string (though it will still report the rotationally invariant similarity. A gesture must consist of at least two points.

When matching as a gesture, the path will be treated as a linear sequence of points. Corner classifications and whether the path is closed will be ignored.

Parameters

path	a path representing a candidate gesture.
similarity	parameter to store the similarity value

Returns: the name of the gesture with the closest match to the given one.

◆ match() [3/6]

const std::string cugl::GestureRecognizer::match ( const std::vector< Vec2 > points )

inline

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm. If there is no match within the similarity threshold or orientation tolerance, this method will return the empty string. A gesture must consist of at least two points.

Parameters

points a vector of points representing a candidate gesture.

Returns: the name of the gesture with the closest match to the given one.

◆ match() [4/6]

const std::string cugl::GestureRecognizer::match	(	const std::vector< Vec2 >	points,
		float &	similarity
	)

inline

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm, with the (rotationally invariant) similarity score stored in the reference parameter. If there is no match within the similarity threshold, or orientation tolerance, this method will return the empty string (though it will still report the rotationally invariant similarity. A gesture must consist of at least two points.

Parameters

points	a vector of points representing a candidate gesture.
similarity	parameter to store the similarity score

Returns: the name of the gesture with the closest match to the given one.

◆ match() [5/6]

const std::string cugl::GestureRecognizer::match	(	const Vec2 *	points,
		size_t	psize
	)

inline

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm. If there is no match within the similarity threshold or orientation tolerance, this method will return the empty string. A gesture must consist of at least two points.

Parameters

points	an array of points representing a candidate gesture.
psize	the number of points in the candidate gesture.

Returns: the name of the gesture with the closest match to the given one.

◆ match() [6/6]

const std::string cugl::GestureRecognizer::match	(	const Vec2 *	points,
		size_t	psize,
		float &	similarity
	)

Returns the name of the gesture with the closest match to the given one.

The match will be performed using the current active algorithm, with the (rotationally invariant) similarity score stored in the reference parameter. If there is no match within the similarity threshold, or orientation tolerance, this method will return the empty string (though it will still report the rotationally invariant similarity. A gesture must consist of at least two points.

Parameters

points	an array of points representing a candidate gesture.
psize	the number of points in the candidate gesture.
similarity	parameter to store the similarity value

Returns: the name of the gesture with the closest match to the given one.

◆ removeGesture()

void cugl::GestureRecognizer::removeGesture ( const std::string name )

Removes the gesture with the given name from the recongizer

Parameters

name	The name of the gesture to remove

◆ setAlgorithm()

void cugl::GestureRecognizer::setAlgorithm ( Algorithm algorithm )

inline

Sets the current matching algorithm

This gesture recognizer can use either the classic $1 algorithm or the PROTRACTOR algorithm. The PROTRACTOR is faster with less accuracy. However the need for speed really depends on the number of gestures stored in this recognizer. Determining which algorithm is best is typically a matter of experiementation.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the active algorithm. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Parameters

algorithm the matching algorithm

◆ setOrientationTolerance()

void cugl::GestureRecognizer::setOrientationTolerance ( float tolerance )

inline

Sets the rotational tolerance for gesture matching.

The matching algorithms are rotationally oblivious, meaning that they can recognize the gestures at any orientation. Typically this is not what we want in game development, however. If a gesture is rotated too far, we want to reject it.

The tolerance is the maximum allowable angle (in radians) of rotation for a gesture to be recognized. The angle of a gesture is measured using the vector from its first point to the (normalized) centroid. If θ is the angle between the gesture and potential match, it will be rejected if it is greater than +/- the tolerance.

By default this value is 20°. Setting this to a negative value will allow free rotation of gestures.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the current rotational tolerance. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Parameters

tolerance the rotational tolerance for gesture matching.

◆ setSimilarityThreshold()

void cugl::GestureRecognizer::setSimilarityThreshold ( float accuracy )

inline

Sets the similarity threshold

When matching a candidate gesture against the stored values, we only consider matches whose similarity value is above this threshold. If this value is non-zero, it is possible that a candidate gesture will have no matches. By default this value is 0.3.

Note that changing this value can possible invalidate any uniqueness constraints previously applied to the gestures. That is because uniqueness is determined using the current similarity threshold. If uniqueness is important, we recommend that you clear the existing gestures and add them back.

Parameters

accuracy the similarity threshold

◆ similarity() [1/3]

float cugl::GestureRecognizer::similarity	(	const std::string	name,
		const Path2 &	path,
		bool	invariant = `true`
	)

inline

Returns the similarity measure of the named gesture to this one.

The similarity measure will be computed using the active algorithm. As those algorithms are rotationally invariant, it will ignore the rotation when computing that value. However, if the parameter invariant is set to false, this method will return 0 for gestures not within the orientation tolerance.

If there is no gesture of the given name, this method will return 0. A gesture must consist of at least two points.

When matching as a gesture, the path will be treated as a linear sequence of points. Corner classifications and whether the path is closed will be ignored.

Parameters

name	the name of the gesture to compare
path	a path representing a candidate gesture
invariant	whether to ignore rotation when computing similarity

Returns: the similarity measure of the named gesture to this one.

◆ similarity() [2/3]

float cugl::GestureRecognizer::similarity	(	const std::string	name,
		const std::vector< Vec2 > &	points,
		bool	invariant = `true`
	)

inline

Returns the similarity measure of the named gesture to this one.

The similarity measure will be computed using the active algorithm. As those algorithms are rotationally invariant, it will ignore the rotation when computing that value. However, if the parameter invariant is set to false, this method will return 0 for gestures not within the orientation tolerance.

If there is no gesture of the given name, this method will return 0. A gesture must consist of at least two points.

Parameters

name	the name of the gesture to compare
points	a vector of points representing a candidate gesture
invariant	whether to ignore rotation when computing similarity

Returns: the similarity measure of the named gesture to this one.

◆ similarity() [3/3]

float cugl::GestureRecognizer::similarity	(	const std::string	name,
		const Vec2 *	points,
		size_t	psize,
		bool	invariant = `true`
	)

Returns the similarity measure of the named gesture to this one.

The similarity measure will be computed using the active algorithm. As those algorithms are rotationally invariant, it will ignore the rotation when computing that value. However, if the parameter invariant is set to false, this method will return 0 for gestures not within the orientation tolerance.

If there is no gesture of the given name, this method will return 0. A gesture must consist of at least two points.

Parameters

name	the name of the gesture to compare
points	an array of points representing a candidate gesture
psize	the number of points in the candidate gesture
invariant	whether to ignore rotation when computing similarity

Returns: the similarity measure of the named gesture to this one.

The documentation for this class was generated from the following file:

/Users/wmwhite/Developer/CUGL/include/cugl/input/gestures/CUGestureRecognizer.h

Public Types

Public Member Functions

Static Public Member Functions

Detailed Description

Member Enumeration Documentation

◆ Algorithm

Constructor & Destructor Documentation

◆ GestureRecognizer()

◆ ~GestureRecognizer()

Member Function Documentation

◆ addGesture() [1/3]

◆ addGesture() [2/3]

◆ addGesture() [3/3]

◆ addGestures()

◆ alloc() [1/2]

◆ alloc() [2/2]

◆ allocWithJson()

◆ clearGestures()

◆ dispose()

◆ getAlgorithm()

◆ getGesture()

◆ getGestureNames()

◆ getGestures()

◆ getNormalizedBounds()

◆ getOrientationTolerance()

◆ getSampleSize()

◆ getSimilarityThreshold()

◆ hasGesture()

◆ init() [1/2]

◆ init() [2/2]

◆ initWithJson()

◆ match() [1/6]

◆ match() [2/6]

◆ match() [3/6]

◆ match() [4/6]

◆ match() [5/6]

◆ match() [6/6]

◆ removeGesture()

◆ setAlgorithm()

◆ setOrientationTolerance()

◆ setSimilarityThreshold()

◆ similarity() [1/3]

◆ similarity() [2/3]

◆ similarity() [3/3]