make

How to Use

SVM^multiclass consists of a learning module (svm_multiclass_learn) and a classification module (svm_multiclass_classify). The classification module can be used to apply the learned model to new examples. See also the examples below for how to use svm_multiclass_learn and svm_multiclass_classify.

      svm_multiclass_learn -c 1.0 example_file model_file

General options:
         -?          -> this help
         -v [0..3]   -> verbosity level (default 1)
         -y [0..3]   -> verbosity level for svm_light (default 0)
Learning options:
         -c float    -> C: trade-off between training error
                        and margin (default 0.01)
         -p [1,2]    -> L-norm to use for slack variables. Use 1 for L1-norm,
                        use 2 for squared slacks. (default 1)
         -o [1,2]    -> Rescaling method to use for loss.
                        1: slack rescaling
                        2: margin rescaling
                        (default 2)
         -l [0..]    -> Loss function to use.
                        0: zero/one loss
                        (default 0)
Kernel options:
         -t int      -> type of kernel function:
                        0: linear (default)
                        1: polynomial (s a*b+c)^d
                        2: radial basis function exp(-gamma ||a-b||^2)
                        3: sigmoid tanh(s a*b + c)
                        4: user defined kernel from kernel.h
         -d int      -> parameter d in polynomial kernel
         -g float    -> parameter gamma in rbf kernel
         -s float    -> parameter s in sigmoid/poly kernel
         -r float    -> parameter c in sigmoid/poly kernel
         -u string   -> parameter of user defined kernel
Optimization options (see [2][3]):
         -w [1,2,3,4]-> choice of structural learning algorithm (default 3):
                        1: algorithm described in [2]
                        2: joint constraint algorithm (primal) [to be published]
                        3: joint constraint algorithm (dual) [to be published]
                        4: joint constraint algorithm (dual) with constr. cache
         -q [2..]    -> maximum size of QP-subproblems (default 10)
         -n [2..q]   -> number of new variables entering the working set
                        in each iteration (default n = q). Set n < q to prevent
                        zig-zagging.
         -m [5..]    -> size of cache for kernel evaluations in MB (default 40)
                        (used only for -w 1 with kernels)
         -f [5..]    -> number of constraints to cache for each example
                        (default 5) (used with -w 4)
         -e float    -> eps: Allow that error for termination criterion
                        (default 0.100000)
         -h [5..]    -> number of iterations a variable needs to be
                        optimal before considered for shrinking (default 100)
         -k [1..]    -> number of new constraints to accumulate before
                        recomputing the QP solution (default 100) (-w 1 only)
         -# int      -> terminate QP subproblem optimization, if no progress
                        after this number of iterations. (default 100000)
Output options:
         -a string   -> write all alphas to this file after learning
                        (in the same order as in the training set)
Structure learning options:
          none

The input file example_file contains the training examples. The file format is the same as for SVM^light, just that the target value is now a positive integer that indicates the class. The first lines may contain comments and are ignored if they start with #. Each of the following lines represents one training example and is of the following format:

The target value and each of the feature/value pairs are separated by a space character. Feature/value pairs MUST be ordered by increasing feature number. Features with value zero can be skipped. The target value denotes the class of the example via a positive integer. So, for example, the line

3 1:0.43 3:0.12 9284:0.2 # abcdef

specifies an example of class 3 for which feature number 1 has the value 0.43, feature number 3 has the value 0.12, feature number 9284 has the value 0.2, and all the other features have value 0. In addition, the string abcdef is stored with the vector, which can serve as a way of providing additional information when adding user defined kernels.

The result of svm_multiclass_learn is the model which is learned from the training data in example_file. The model is written to model_file. To make predictions on test examples, svm_multiclass_classify reads this file. svm_multiclass_classify is called as follows:

For all test examples in test_example_file the predicted classes are written to output_file. There is one line per test example in output_file in the same order as in test_example_file.

Getting started: an Example Problem

You will find an example problem with 7 classes at

Download this file into your svm_multiclass directory and unpack it with

This will create a subdirectory example4. There are 300 examples in the file train.dat and 2000 in the file test.dat. To run the example, execute the commands:

The accuracy on the test set is printed to stdout.

Disclaimer

This software is free only for non-commercial use. It must not be distributed without prior permission of the author. The author is not responsible for implications from the use of this software.

Known Problems

• On 64-bit AMD and Intel machines, there can be a precision problem when using the w3 or w4 algorithms on problems with many features. To fix this problem, change "#define CFLOAT" to "double" in svm_common.h. Thank you to Chih-Jen Lin for pointing this out.
• The RBF kernel is broken.

History

V1.01 - V2.12

• New training algorithm based on equivalent 1-slack reformulation of the training problem. This makes training on the linear kernel several orders of magnitude faster than in V1.01. See changes introduced by SVM^struct V3.00 for a general properties of the new structural SVM training algorithm.
• New IO routines that are faster for reading large data and model files. SVM^multiclass V1.011.

V1.00 - V1.01

svm_multiclass_classify that occurred when true class labels are not given in test file (thanks to Jouni Paulus).

References

[1] K. Crammer and Y. Singer. On the Algorithmic Implementation of Multi-class SVMs, JMLR, 2001.

[2] I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun. Support Vector Learning for Interdependent and Structured Output Spaces, ICML, 2004. [Postscript]] [PDF]

[3] T. Joachims, Making Large-Scale SVM Learning Practical. Advances in Kernel Methods - Support Vector Learning, B. Schölkopf and C. Burges and A. Smola (ed.), MIT Press, 1999. [Postscript (gz)] [PDF]