Machine Learning
An artificial Intelligence Approach
Ryszard S. Michalski
Kao,eG/CArbpme;;
Tom M. Mitchell
Scope of Machine learning
there are two basic forms of learning: Knowledge acquisition and skill refinement.
Knowledge acquisition is defined as learning new symbolic information and coupled with the ability to apply that information in an effective manner. When we say someone learned physics, understood their meaning, and understood their relationship to each other and to the physical world. the essence of learning in this case is the acquisition of new knowledge, including descriptions and models of physical system and behaviors. A person is said to learn more if his knowledge explains a broader scope of situations, is more accurate, and is better able to predict the behavior of the physical world.
A second kind of learning is the gradual improvement of motor and cognitive skills through practice, such as learning to ride a bicycle or to play the piano.
The machine learning focuses on the knowledge acquisition aspect of learning.
Rote Learning and direct implanting of new knowledge:
No inference or other transformation of the knowledge is required on the part of the learner.
Learning by being programmed, constructed or modified by an external entity, requiring no effort on the part of the learner.
Learning by memorization of given facts and data with no inferences drawn form the incoming information.
Learning from instruction
Acquiring knowledge form a teacher or other organized source, such as textbook, requiring that the learning transform the knowledge form the input language to an internally-usable representation, and that the new information be integrated with prior knowledge for effective use. Hence, the learner is required to perform some inference, but a large fraction of the burden remains with the teacher, who must present and organize knowledge in a way that incrementally augments the student's existing knowledge. Learning from instruction parallels most formal education methods. Therefore, machine learning task is one of building a system that can accept instruction or advice and can store and apply this learned knowledge effectively. This form of learning is discussed in Chapters 12, 13, and 14.
Learning by analogy
Acquiring new facts or skills by transforming and augmenting existing knowledge that bears strong similarity to the desired new concept or skill into a form effectively useful in the new situation. for instance, a person who has never driven a small truck, but who drives automobiles, may well transform his existing skill to (perhaps imperfectly) the new task. This form of learning requires more inference on the part of the learner than does rote learning or learning form instruction. A fact or skill analogous in relevant parameters must be retrieved from memory; then the retrieved knowledge must be transformed, applied to the new situation, and store for future use. This kind of learning is discussed in Chapters 5 and 7.
Learning From examples
( a special case of inductive learning) - given a set of examples and counterexamples of concept, the learner induces a general concept description that describes all of the positive examples and none of the counterexamples. Learning from examples is a method that has been heavily investigated in artificial intelligence. the amount of inference performed by the learner is much greater than in learning from instruction, as no general concepts are provided by a teacher, and is some-what greater than in learning by analogy, as no similar concepts are provided as "seeds" around which the new concept may be grown. Learning from examples can be subcategorized according to the source of the examples:
the source is the teacher who knows the concept and generates sequences of examples that are meant to be as helpful as possible.
the source is the learner itself. The learner typically knows its own knowledge state, but clearly does not know the concept to be acquired. there for, the learner can generate instances ( and have an external entity such as the environment or teacher classify them as positive or negative examples) on basis of the information it believes necessary to discriminate among contending concept descriptions.
The source is the external environment. In this case the example generation process is operationally random, as the learner must rely on relatively uncontrolled observations. For example, an astronomer attempting to infer precursors to supernovas must rely mainly upon unstructured data presentation.
One also can classify learning from examples by the type of examples available to the learner.
Only positive examples available: whereas positive example provide instances of the concept to be acquired, they do not provide information for preventing overgeneralization of the inferred concept. In this kind of learning situation. overgeneralization might be avoided by considering only the minimal generalization necessary, or by relying upon a priori domain knowledge to constrain the concept to be inferred.
positive and negative examples available. In this kind of situation, positive examples force generalization whereas negative examples prevent overgeneralization (the induced concept should never be so general as to include any of the negative examples). This is the most typical form of learning form examples.
learning form examples may be one-trial or incremental. In the former case, all examples are presented at once. In the latter case, the system must form one or more hypotheses of the concept ( or range of concepts) consistent with the available data, and subsequently refine the hypotheses after considering additional examples. The incremental approach more closely parallels human learning, allows the learner to use partially learned concepts (for performance, or to guide the example generation process), and enables a teacher to focus on the basis aspects of a new concept before attempting to impart less central details. Various aspects of learning from examples are discussed in Chapters 3, 4, 5, 6, 7, 8, 15 and 16.
Learning from observation and discovery (also called unsupervised learning)
this is the very general form of inductive learning that includes discovery systems, theory-formation tasks, the creation of classification criteria to form taxonomic hierarchies, and similar tasks without benefit of an external teacher. This form of unsupervised learning requires the learner to perform more inference than any approach thus far discussed.
One may subclassify learning from observation to the degree of interaction with an external environment. the extreme points in this dimension are:
passive observation, where the learner classifies and taxonomizes observations of multiple aspects of the environment.
Active experimentation, where the learner perturbs the environment to observe the results of its perturbations. Experimentation may be random, dynamically focused according to general criteria of interestingness, or strongly guided by theoretical constraints. As a system acquires knowledge, and hypothesizes theories it may be driven to confirm or disconfirm its theories, and hence explore its environment applying different observation and experimentation strategies as the need arises. Often this form of learning involves the generation of examples to test hypothesized or partially acquired concepts.
Classification According to the type of knowledge acquired
A learning system may acquire rules of behaviors, descriptions of physical objects, problem-solving heuristics, classification taxonomies over a sample space, and many other types of knowledge useful in the performance of a wide variety of tasks. The list of spans types of knowledge acquired, primarily as a function of the representation of that knowledge.
parameters in algebraic expression - learning in this context consists of adjusting numerical parameters or coefficients in algebraic expressions of fixed functional form so as to obtain desired performance. For instance, perceptrons adjust weighting coefficients for threshold logic elements when learning to recognize two-dimensional patterns.
Decision trees - some systems acquire decision trees to discriminate among classes of objects. The nodes in a decision tree correspond to selected object attributes, and the edges correspond to predetermined alternative values for these attributes. Leaves of the tree correspond to sets of objects with an identical classification.
formal grammars - In learning to recognize a particular (usually artificial) language, formal grammars are induced from sequences of expressions in the language. These grammars are typically represented as regular expressions, finite-state automata, context-free grammar rules, or transformation rules.
Production rules - a production rule is a condition-action pair {C => A), where C is a set of conditions, and A is a sequence of actions. If all the conditions in a production rule are satisfied, then the sequence of action is executed. Due to their simplicity and ease of interpretation, production rules are widely used knowledge representation in learning systems. The four basic operations whereby production rules may be acquired and refined are:
Creation : a new rule is constructed by the system or acquired from an external entity.
Generalization: Conditions are dropped or made less restrictive, so that the rule applies in a larger number of situations.
Specialization: Additional conditions are added to the condition set, or existing conditions
Formal logic-based expressions and related formalisms
Graphs and networks
Frames and schemas
Computer programs and other procedural encodings
Taxonomies
Multiple representations
Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same task or tasks drawn from the same population more efficiently and more effectively the next time.
Chapter 3 A comparative review of selected methods for learning from examples