Machine Learning: Abduction and Transduction

Deduction & Induction

IN HIGH SCHOOL MATH CLASS WE LEARNED ABOUT THE TWO MOST COMMON MODES OF LOGICAL ARGUMENTATION ; DEDUCTION AND INDUCTION.  USING DEDUCTION WE REASON FROM THE GENERAL TO THE SPECIFIC. IF "A" IS A SPECIFIC ELEMENT OF A GENERAL CLASS "C", THEN "A" SHARES PROPERTIES (CHARACTERISTICS) WITH OTHER ELEMENTS OF "C".  USING INDUCTION WE REASON FROM THE SPECIFIC TO THE GENERAL. GIVEN ENOUGH SPECIFIC ELEMENTS A1, A2, A3...AN WE CAN INFER THE EXISTENCE OF A GENERAL CLASS C DEFINING ONE OR MORE COMMON PROPERTIES AMONG THOSE ELEMENTS.

Abduction & Transduction

FOR MACHINE LEARNING, WE NEED TO AUGMENT DEDUCTION AND INDUCTION WITH TWO ADDITIONAL MODES OF REASONING -- ABDUCTION AND TRANSDUCTION.  ABDUCTION PROVIDES THE JUSTIFICATION OF USING STATISTICAL METHODS ("MOSTLY TRUE") TO LOOK FOR PATTERNS IN DATA. TRANSDUCTION IMPLIES A SPECIFIC-TO-SPECIFIC MAPPING BY WAY OF A GENERAL CLASS.  

VLADIMIR VAPNIK IN HIS BOOK "THE NATURE OF STATISTICAL LEARNING" (SPRINGER Y2000) POINTS OUT THAT "LEARNING IS A PROBLEM OF FUNCTION ESTIMATION ON THE BASIS OF EMPIRICAL DATA."  HE DESCRIBES THE INDUCTIVE PRINCIPLE OF EMPIRICAL RISK MINIMIZATION AS CONSISTING OF TWO STAGES:

  1. LEAST-SQUARES METHOD REGRESSION FOR THE LOSS FUNCTION

  2. MAXIMUM-LIKELIHOOD METHOD DENSITY ESTIMATION OF THE LOSS FUNCTION

STATISTICAL LEARNING INVOLVES ESTIMATING THE VALUES OF AN UNKNOWN FUNCTION AT GIVEN POINTS OF INTEREST.  THIS PROBLEM IS SOLVED BY USING A TWO-STEP SCHEME:

  1. USING THE TRAINING DATA EXAMPLES TO ESTIMATE AN APPROXIMATING FUNCTION BY INDUCTIVE INFERENCE (PATTERN RECOGNITION)

  2. USING THE APPROXIMATING FUNCTION TO DEDUCTIVELY INFER (ESTIMATE) VALUES OF THE FUNCTION AT TEST DATA POINTS OF INTEREST.

THE COMBINATION OF THESE TWO STEPS DESCRIBES A NEW FORM OF INFERENCE, TRANSDUCTION, WHICH MOVES FROM SPECIFIC TO SPECIFIC (VIA GENERAL) TO PROVIDE THE BEST RESULT FROM LIMITED INFORMATION.

BY BOHDAN SHMORHAY

Bohdan Shmorhay