Version 18 of Neural Networks
Updated 2015-03-11 14:40:26 by SEHA Tcl extension for neural networks processing using the FANN library can be found at tcl-fann.
(RS would call this A little feed-forward back-propagation learning neural network.)
There came a time when I needed to do a series of simulations of a feed-forward back-propagation (FFBP) neural network . Rather then spend just an hour doing the homework, I instead spent over 10 hours writing a full-fledged network builder and simulator. As you can see from the screenshot above the result is a little tool that allows the user to graphically draw a network and then run simulations upon it.
Features include:
- hidden and bias nodes
- choice of the sigmoid (or logistic function ) or bipolar squashing function
- configurable eta value (learning rate)
- learning by way of steepest descent and delta rule
The bipolar function is g(x)=2f(x)-1, where f(x) is the sigmoid function. Conveniently, its derivative is g'(x)=0.5(1+g(x))(1-g(x)). A more general-purpose steepest descent algorithm may be found at Differentiation and steepest-descent, though my FFBP does not use his code.
Usage
Shift-click the canvas to place a node. Shift-drag between nodes to add a weight link. Reposition nodes by dragging them around. Double click a node or weight to change its properties.
The real power of the program is its ability to run a number of trials and to learn after each test datum. To do this requires modifying the code a bit. See the last three functions in ffbp.tcl. The current functions demonstrate how to learn the XOR function; the corresponding network is saved in ffbp.net.
While you are at it, take a look at another kind of neural network, Hopfield Networks.
To do list:
- add momentum value
- better interface for running trials
Downloads
Version 0.2 - http://tcl.jtang.org/ffbp/ffbp-0.2.tar.gz
- This version breaks save file compatibility with previous version!
- Remove nodes and weights by double-clicking them, then clicking on 'Remove' button.
- Show current weights directly on the canvas.
- Adjust weight format using format codes; set string to empty to see raw value.
Version 0.1 - http://tcl.jtang.org/ffbp/ffbp-0.1.tar.gz
- Initial hack-ish release.
See also ANN
Lens is an efficient, yet flexible, neural network simulator that runs on a variety of platforms, is able to handle large, complex simulations, but is also reasonably easy for novices to operate. Lens is written primarily in C. The graphical interfaces and script interpreter were implemented using the Tcl/Tk toolkit.
Delve (Data for Evaluating Learning in Valid Experiments): Neural network library in C with Tcl binding.
Xerion is made up of a set of C libraries that can be used to build complex or experimental neural networks, and pre-built simulators written with these libraries. It uses the Tcl command language as its scripting language. Xerion extends the Tcl language by providing commands for: building and modifying neural networks; optimizing functions using many different methods, including conjugate gradient optimization; and examining and modifying C data structures.
Xerion has a graphical interface written with Tk. The graphical interface allows you to easily load and examine networks, train them using any of the training methods, and test them on different data.
Xerion contains pre-built modules that implement the Back Propagation algorithm and a Mixture of Gaussians that can be trained using EM.
Siemens offers a commercial forecasting product called SENN (Simulation Environment for Neural Networking), which is a C++ library with Tcl interface bindings [L1 ] [L2 ].