An Information Theoretic Approach toAdaptive SystemTrainingUsing UnlabeledData

Traditionally, supervised learning isperformed with pairwise input-output labeled data. After thetraining procedure, theadaptive systemweights arefixed andthesystem istested withunlabeled data.Recently, exploiting theunlabeled data toimprove classification performance hasbeenproposed in themachinelearning community. Inthispaper, we present aninformation theoretic approach basedondensity divergence minimization toobtain anextended training algorithm using unlabeled dataduring testing. Thesimulations forclassification problems suggest thatourmethodcanimprove theperformance ofadaptive system intheapplication phase. I.INTRODUCTION Supervised learning, including systemidentification and regression, isperformed withinput-output labeled datausing linear andnonlinear systemtopologies andoptimal criteria based onstatistics oftheerror between thedesired samples andsystem output. Thepurpose oflearning istoextract as muchinformation aspossible fromthelabeled training data toobtain optimal systemweights sothattheygeneralize in unlabeled data(typically bysplitting thedatasetinto training andtesting sets). Oncethesystem istrained, there isnofurther optimization carried outovertheunlabeled dataduring the actual application (testing) phase(1), (2). Thisapproach is considered natural toallsince wedonothavethelabels for thetesting datatofurther train theadaptive system.