Neural Networks
See Also
- Lens, by Douglas Rohde
- A neural network simulator that runs on a variety of platforms, is able to handle large, complex simulations, but is also reasonably easy for novices to operate. Written primarily in C, with a Tcl/Tk GUI and an embedded Tcl interpreter.
- ANN
- A Tcl extension for building, training, testing, and running neural network designs.
- Delve
- Data for Evaluating Learning in Valid Experiments. Designed to evalute the performane of methods that learn relationships based primarily on empirical data. Written in C and Tcl.
Description
(RS would call this A little feed-forward back-propagation learning neural network.)
There came a time when I needed to do a series of simulations of a feed-forward back-propagation (FFBP) neural network . Rather then spend just an hour doing the homework, I instead spent over 10 hours writing a full-fledged network builder and simulator. As you can see from the screenshot above the result is a little tool that allows the user to graphically draw a network and then run simulations upon it.
Features include:
- hidden and bias nodes
- choice of the sigmoid (or logistic function ) or bipolar squashing function
- configurable eta value (learning rate)
- learning by way of steepest descent and delta rule
The bipolar function is g(x)=2f(x)-1, where f(x) is the sigmoid function. Conveniently, its derivative is g'(x)=0.5(1+g(x))(1-g(x)). A more general-purpose steepest descent algorithm may be found at Differentiation and steepest-descent, though my FFBP does not use his code.
Usage
Shift-click the canvas to place a node. Shift-drag between nodes to add a weight link. Reposition nodes by dragging them around. Double click a node or weight to change its properties.
The real power of the program is its ability to run a number of trials and to learn after each test datum. To do this requires modifying the code a bit. See the last three functions in ffbp.tcl. The current functions demonstrate how to learn the XOR function; the corresponding network is saved in ffbp.net.
While you are at it, take a look at another kind of neural network, Hopfield Networks.
To do list:
- add momentum value
- better interface for running trials
Downloads
Version 0.2 - http://tcl.jtang.org/ffbp/ffbp-0.2.tar.gz
- This version breaks save file compatibility with previous version!
- Remove nodes and weights by double-clicking them, then clicking on 'Remove' button.
- Show current weights directly on the canvas.
- Adjust weight format using format codes; set string to empty to see raw value.
Version 0.1 - http://tcl.jtang.org/ffbp/ffbp-0.1.tar.gz
- Initial hack-ish release.
Xerion is made up of a set of C libraries that can be used to build complex or experimental neural networks, and pre-built simulators written with these libraries. It uses the Tcl command language as its scripting language. Xerion extends the Tcl language by providing commands for: building and modifying neural networks; optimizing functions using many different methods, including conjugate gradient optimization; and examining and modifying C data structures.
Xerion has a graphical interface written with Tk. The graphical interface allows you to easily load and examine networks, train them using any of the training methods, and test them on different data.
Xerion contains pre-built modules that implement the Back Propagation algorithm and a Mixture of Gaussians that can be trained using EM.
A rewritten version of Xerion called Uts (University of Toronto Simulator) makes extended use of Tcl as a scripting language. Source code: [L1 ]
Siemens offers a commercial forecasting product called SENN (Simulation Environment for Neural Networking), which is a C++ library with Tcl interface bindings [L2 ] [L3 ].
NSL (Neural Simulation Language) is a simulation system for large-scale general neural networks. NSL provides a simulation environment simplifying the task of modeling neural networks. It is delivered as a set of C++ libraries with a Tcl/Tk interface.
