<< Chapter < Page Chapter >> Page >

But from Equation  [link] , the last term must be zero, so we obtain

L ( w , b , α ) = i = 1 m α i - 1 2 i , j = 1 m y ( i ) y ( j ) α i α j ( x ( i ) ) T x ( j ) .

Recall that we got to the equation above by minimizing L with respect to w and b . Putting this together with the constraints α i 0 (that we always had) and the constraint [link] , we obtain the following dual optimization problem:

max α W ( α ) = i = 1 m α i - 1 2 i , j = 1 m y ( i ) y ( j ) α i α j x ( i ) , x ( j ) . s.t. α i 0 , i = 1 , ... , m i = 1 m α i y ( i ) = 0 ,

You should also be able to verify that the conditions required for p * = d * and the KKT conditions ( [link] ) to hold are indeed satisfied in our optimization problem. Hence, we can solve the dual in lieu of solving the primal problem. Specifically, in the dual problemabove, we have a maximization problem in which the parameters are the α i 's. We'll talk later about the specific algorithm that we're going to use to solve the dual problem, but if we areindeed able to solve it (i.e., find the α 's that maximize W ( α ) subject to the constraints), then we can use Equation  [link] to go back and find the optimal w 's as a function of the α 's. Having found w * , by considering the primal problem, it is also straightforward to find the optimal value forthe intercept term b as

b * = - max i : y ( i ) = - 1 w * T x ( i ) + min i : y ( i ) = 1 w * T x ( i ) 2 .

(Check for yourself that this is correct.)

Before moving on, let's also take a more careful look at Equation  [link] , which gives the optimal value of w in terms of (the optimal value of) α . Suppose we've fit our model's parametersto a training set, and now wish to make a prediction at a new point input x . We would then calculate w T x + b , and predict y = 1 if and only if this quantity is bigger than zero. But using  [link] , this quantity can also be written:

w T x + b = i = 1 m α i y ( i ) x ( i ) T x + b = i = 1 m α i y ( i ) x ( i ) , x + b .

Hence, if we've found the α i 's, in order to make a prediction, we have to calculate a quantity that depends only on the inner product between x and the points in the training set. Moreover, we saw earlier that the α i 's will all be zero except for the support vectors. Thus, many of the terms in the sum above will be zero, and wereally need to find only the inner products between x and the support vectors (of which there is often only a small number) in order calculate  [link] and make our prediction.

By examining the dual form of the optimization problem, we gained significant insight into the structure of the problem,and were also able to write the entire algorithm in terms of only inner products between input feature vectors. In the next section, we will exploit this property toapply the kernels to our classification problem. The resulting algorithm, support vector machines , will be able to efficiently learn in very high dimensional spaces.

Kernels

Back in our discussion of linear regression, we had a problem in which the input x was the living area of a house, and we considered performing regression using the features x , x 2 and x 3 (say) to obtain a cubic function. To distinguish between these two sets of variables, we'llcall the “original” input value the input attributes of a problem (in this case, x , the living area). When that is mapped to some new set of quantities that are then passed to the learning algorithm, we'll call thosenew quantities the input features . (Unfortunately, different authors use different terms to describe these two things, but we'll try to use thisterminology consistently in these notes.) We will also let Φ denote the feature mapping , which maps from the attributes to the features. For instance, in our example, we had

Questions & Answers

what is mutation
Janga Reply
what is a cell
Sifune Reply
how is urine form
Sifune
what is antagonism?
mahase Reply
classification of plants, gymnosperm features.
Linsy Reply
what is the features of gymnosperm
Linsy
how many types of solid did we have
Samuel Reply
what is an ionic bond
Samuel
What is Atoms
Daprince Reply
what is fallopian tube
Merolyn
what is bladder
Merolyn
what's bulbourethral gland
Eduek Reply
urine is formed in the nephron of the renal medulla in the kidney. It starts from filtration, then selective reabsorption and finally secretion
onuoha Reply
State the evolution relation and relevance between endoplasmic reticulum and cytoskeleton as it relates to cell.
Jeremiah
what is heart
Konadu Reply
how is urine formed in human
Konadu
how is urine formed in human
Rahma
what is the diference between a cavity and a canal
Pelagie Reply
what is the causative agent of malaria
Diamond
malaria is caused by an insect called mosquito.
Naomi
Malaria is cause by female anopheles mosquito
Isaac
Malaria is caused by plasmodium Female anopheles mosquitoe is d carrier
Olalekan
a canal is more needed in a root but a cavity is a bad effect
Commander
what are pathogens
Don Reply
In biology, a pathogen (Greek: πάθος pathos "suffering", "passion" and -γενής -genēs "producer of") in the oldest and broadest sense, is anything that can produce disease. A pathogen may also be referred to as an infectious agent, or simply a germ. The term pathogen came into use in the 1880s.[1][2
Zainab
A virus
Commander
Definition of respiration
Muhsin Reply
respiration is the process in which we breath in oxygen and breath out carbon dioxide
Achor
how are lungs work
Commander
where does digestion begins
Achiri Reply
in the mouth
EZEKIEL
what are the functions of follicle stimulating harmones?
Rashima Reply
stimulates the follicle to release the mature ovum into the oviduct
Davonte
what are the functions of Endocrine and pituitary gland
Chinaza
endocrine secrete hormone and regulate body process
Achor
while pituitary gland is an example of endocrine system and it's found in the Brain
Achor
what's biology?
Egbodo Reply
Biology is the study of living organisms, divided into many specialized field that cover their morphology, physiology,anatomy, behaviour,origin and distribution.
Lisah
biology is the study of life.
Alfreda
Biology is the study of how living organisms live and survive in a specific environment
Sifune
Got questions? Join the online conversation and get instant answers!
Jobilize.com Reply

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Machine learning. OpenStax CNX. Oct 14, 2013 Download for free at http://cnx.org/content/col11500/1.4
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Machine learning' conversation and receive update notifications?

Ask