Fuzzy neural network
| Rudolf Kruse (2008), Scholarpedia, 3(11):6043. | doi:10.4249/scholarpedia.6043 | revision #91290 [link to/cite this article] |
A fuzzy neural network or neuro-fuzzy system is a learning machine that finds the parameters of a fuzzy system (i.e., fuzzy sets, fuzzy rules) by exploiting approximation techniques from neural networks.
Contents |
Combining fuzzy systems with neural networks
Both neural networks and fuzzy systems have some things in common. They can be used for solving a problem (e.g. pattern recognition, regression or density estimation) if there does not exist any mathematical model of the given problem. They solely do have certain disadvantages and advantages which almost completely disappear by combining both concepts.
Neural networks can only come into play if the problem is expressed by a sufficient amount of observed examples. These observations are used to train the black box. On the one hand no prior knowledge about the problem needs to be given. On the other hand, however, it is not straightforward to extract comprehensible rules from the neural network's structure.
On the contrary, a fuzzy system demands linguistic rules instead of learning examples as prior knowledge. Furthermore the input and output variables have to be described linguistically. If the knowledge is incomplete, wrong or contradictory, then the fuzzy system must be tuned. Since there is not any formal approach for it, the tuning is performed in a heuristic way. This is usually very time consuming and error-prone.
| Neural Networks | Fuzzy Systems |
|---|---|
| no mathematical model necessary | no mathematical model necessary |
| learning from scratch | apriori knowledge essential |
| several learning algorithms | not capable to learn |
| black-box behavior | simple interpretation and implementation |
It is desirable for fuzzy systems to have an automatic adaption procedure which is comparable to neural networks. As it can be seen in Table 1, combining both approaches should unite advantages and exclude disadvantages.
Characteristics
Compared to a common neural network, connection weights and propagation and activation functions of fuzzy neural networks differ a lot. Although there are many different approaches to model a fuzzy neural network (Buckley and Hayashi, 1994, 1995; Nauck and Kruse, 1996), most of them agree on certain characteristics such as the following:
- A neuro-fuzzy system based on an underlying fuzzy system is trained by means of a data-driven learning method derived from neural network theory. This heuristic only takes into account local information to cause local changes in the fundamental fuzzy system.
- It can be represented as a set of fuzzy rules at any time of the learning process, i.e., before, during and after.
- Thus the system might be initialized with or without prior knowledge in terms of fuzzy rules.
- The learning procedure is constrained to ensure the semantic properties of the underlying fuzzy system.
- A neuro-fuzzy system approximates a n-dimensional unknown function which is partly represented by training examples.
- Fuzzy rules can thus be interpreted as vague prototypes of the training data.
- A neuro-fuzzy system is represented as special three-layer feedforward neural network as it is shown in Figure <ref>F1</ref>.
- The first layer corresponds to the input variables.
- The second layer symbolizes the fuzzy rules.
- The third layer represents the output variables.
- The fuzzy sets are converted as (fuzzy) connection weights.
- Some approaches also use five layers where the fuzzy sets are encoded in the units of the second and fourth layer, respectively. However, these models can be transformed into a three-layer architecture.
One can basically distinguish between two different kinds of fuzzy neural networks, i.e., cooperative and hybrid FNNs (Nauck et al., 1997).
Cooperative Fuzzy Neural Network
In the case of cooperative neural fuzzy systems, both artificial neural network and fuzzy system work independently from each other. The ANN tries to learn the parameters from the fuzzy system. This can be either performed offline or online while the fuzzy system is applied. Figure <>
Hybrid Fuzzy Neural Network
References
- Berenji, H.R. (1992). A Reinforcement Learning Based Architecture for Fuzzy Logic Control. International Journal of Approximate Reasoning 6, 267-292.
- Berenji, H. R. and Khedkar, P. (1992). Learning and Tuning Fuzzy Logic Controllers Through Reinforcements, IEEE Trans. Neural Networks, 3, pp. 724-740.
- Bezdek, J. C., Tsao, E. C.-K. and Pal, N. R. (1992). Fuzzy Kohonen Clustering Networks, in Proc. IEEE Int. Conf. on Fuzzy Systems 1992 (San Diego), pp. 1035-1043.
- Buckley, J. J. and Hayashi, Y. (1994). Fuzzy neural networks: A survey, Fuzzy Sets and Systems 66, pp. 1-13.
- Buckley, J. J. and Hayashi, Y. (1995). Neural networks for fuzzy systems, Fuzzy Sets and Systems 71, pp. 265-276.
- Hayashi, I., Nomura, H., Yamasaki, H. and Wakami, N. (1992). Construction of Fuzzy Inference Rules by NFD and NDFL. International Journal of Approximate Reasoning, 6, pp. 241-266.
- Jang, J.-S. R. (1993). ANFIS: Adaptive-Network-Based Fuzzy Inference Systems, IEEE Transaction on Systems, Man, and Cybernetics 23, pp. 665-685.
- Kosko, B. (1992). Neural Networks and Fuzzy Systems. A Dynamical Systems Approach to Machine Intelligence (Prentice-Hall, Englewood Cliffs).
- Nauck, D. (1994). A Fuzzy Perceptron as a Generic Model for Neuro-Fuzzy Approaches, in Proc. Fuzzy-Systeme 94 (Munich).
- Nauck, D. and Kruse, R. (1996). Neuro-Fuzzy Classification with NEFCLASS, in P. Kleinschmidt, A. Bachem, U. Derigs, D. Fischer, U. Leopold-Wildburger and R. Möhring (eds.), Operations Research Proceedings 1995, (Berlin), pp. 294-299.
- Nauck, D. and Kruse, R. (1997). Function Approximation by NEFPROX, in Proc. Second European Workshop on Fuzzy Decision Analysis and Neural Networks for Management, Planning, and Optimization (EFDAN'97), (Dortmund), pp. 160-169.
- Nomura, H., Hayashi, I. and Wakami, N. (1992). A Learning Method of Fuzzy Inference Rules by Descent Method, in Proc. IEEE Int. Conf. on Fuzzy Systems 1992 (San Diego), pp. 203-210.
- Vuorimaa, P. (1994). Fuzzy Self-Organizing Map, Fuzzy Sets and Systems 66, pp. 223-231.
- Wang, L.-X. and Mendel, J. M. (1992). Generating Fuzzy Rules by Learning from Examples, IEEE Transaction on Systems, Man, and Cybernetics 22, pp. 1414-1427.
Recommended reading
- Klawonn, F. Nauck D. and Kruse, R. (1997). Foundations of Neuro-Fuzzy Systems (Wiley, Chichester, United Kingdom).
- Klawonn, F., Kruse R., Nauck, D. and Borgelt, C. (2003). Neuro-Fuzzy-Systeme (Vieweg, Wiesbaden).
