- ▪E.D. Sontag, "VC dimension of neural networks", In Neural Networks and Machine Learning, pp. 69-95, 1998. pdf
Abstract
The Vapnik-Chervonenkis (VC) dimension is an integer which helps to characterize distribution-independent learning of binary concepts from positive and negative samples. This paper, based on lectures delivered at the Isaac Newton Institute in August of 1997, presents a brief introduction, establishes various elementary results, and discusses how to estimate the VC dimension in several examples of interest in neural network theory. (It does not address the learning and estimation-theoretic applications of VC dimension, and the applications to uniform convergence theorems for empirical probabilities, for which many suitable references are available.)
- ▪E.D. Sontag, "Recurrent neural networks: Some systems-theoretic aspects", In Dealing with Complexity: a Neural Network Approach, pp. 1–12, 1997. pdfmachine learning · artificial intelligence · neural networks · recurrent neural networks · learning · VC dimension
Abstract
This paper provides an exposition of some recent results regarding system-theoretic aspects of continuous-time recurrent (dynamic) neural networks with sigmoidal activation functions. The class of systems is introduced and discussed, and a result is cited regarding their universal approximation properties. Known characterizations of controllability, observability, and parameter identifiability are reviewed, as well as a result on minimality. Facts regarding the computational power of recurrent nets are also mentioned.