Calling Maxima from Tcl

See also Maxima through a pipe for a rudimentary programmatic interface to the command line Maxima.


TV:

I've thus far tested Maxima on windows (XP), and it proved to be useful and useable, not in the least because of its integration with tcl, though maybe that isn't perfect from a strict programmer's point of view.

The mechanism which is used in maxima to interface between the Tcl and Tk supplied user interface (at least on windows) is that a maxima process communicates with the Tcl/Tk interface process over a socket, over which mainly text is being sent back and forth containing mathematica statements/commands/formulas and the answers from the maxima 'engine', a technique succesfully used in professional software of various kinds. The possible disadvantage is that for each communication, process change is needed in the process time slicer, which probably works at at most a granularity of a millisecond or so. Most maxima commands aren't executed blazing fast, and usually don't really need to, so that in practice is probably not a granularity problem.

As stated on the maxima page, the way normally the tcl/tk + maxima 'program' maxima.exe communicates the user input to the maxima lisp based core primarily by reading from and outputting to a text widget, which from UI perspective is not a bad idea. In fact it appears the maxima.exe is a tclkit (tcl+tk+incrtcl+metakit) appearently calling some scripts, and in the same dir with as.exe, gcc.exe, and some dlls. Not very handy in my opinion, I'd prefer a few script files which simply normally start up tcl/tk of which I control the installation, which works fine for me, and would seem to require less files, and be more overseeable. Maybe I'll try, and in the process see if that way a more recent tcl/tk can be used.

The maxima tcl procedure to make user interactions work is that somehow pressing return in the .maxima.text widget invokes the routine CMeval with (normally) the window name .maxima.text as only argument. This procedure does some regexpr stuff and basically pushes the maxima input line where the cursor is through the maxima command socket stream to the engine.

The socket of this non-blocking connection has a fileevent readable routine scheduled to activate when data arrives back from the engine: maximaFilter .maxima.text sock156 (second argument depending on session of course). There seem to be various return kinds, and it could be data can be multi line and maybe even intra-line split up in separate return packets.

The below is a first very prototypical hack to make a proc that executes maxima commands and returns 'clean' results, which I made by starting maxima, loading bwise034.tcl in the opened console (not needed to run the below), somehow redefining the below proc (source in console or as I did edit the bwise procedure window), and of course loading the domax proc.

I've come up with a first simple (read not too tested, subjected to the above considerations of possible failure) way to have a procedure to give a expression to maxima, and have the return value return to the tcl caller. I've changed the following proc:

proc maximaFilter { {win} {sock} } {
    linkLocal $win  plotPending
    global pdata
#TV added:
    global passres
###
    if { [eof $sock] } {
        # puts "at end"
        close $sock
        return ""
    }
    set it [read $sock]
    # puts "read=<$it>"
    if { [string first "\032\032" $it] >= 0 &&
        [regexp  -indices "\032\032(\[^:]+):(\[0-9]+):\[^\n]*\n" $it junk file line] } {
        
        dblDisplayFrame [getMatch $it $file] [getMatch $it $line]
        append res [string range $it 0 [expr { [lindex $junk 0] -1 } ]]
        append res [string range $it [expr { 1+[lindex $junk 1]}] end]
        set it $res
    }
    if { [string first "\032\031tcl:" $it] >= 0 &&  [regexp  -indices "\032\031tcl:(\\[^\n]*)\n" $it junk com]} {
        eval $com
        append res [string range $it 0 [expr { [lindex $junk 0] -1 } ]]
        append res [string range $it [expr { 1+[lindex $junk 1]}] end]
        set it $res
    }
    # puts it=<$it>
    if { [regexp -indices "\{plot\[d23]\[fd]" $it inds] } {
        set plotPending [string range $it [lindex $inds 0] end]
        set it ""
        if { [regexp {\(C[0-9]+\) $} $it ff] } {
            regexp "\{plot\[d23]\[df].*\}" $ff it
            #        set it $ff
        }
    }
    if { [info exists plotPending] } {
        #puts "plotPending=<$plotPending>,it=<$it>"
        append plotPending $it
        set it ""
        if { [regexp -indices "\n\\(D\[0-9\]+\\)" $plotPending  inds] } {
            set it [string range $plotPending [lindex $inds 0] end]
            set plotPending [string range $plotPending 0 [lindex $inds 0]]
            set data $plotPending
            unset plotPending
            #puts itplot=<$it>,$inds
            #puts plotdata=<$data>
            doShowPlot $win $data
 
        }
    }
 
    $win insert end $it "output"
    $win mark set  lastStart "end -1char"
#TV added:
    if [string compare -length 2 $it "(D"] {
################## here you want to get the first characters of the next prompt, for Maxima 5.9.1 on Windowa,
################## that has become "\n(%i" (otherwise your output contains the next prompt).
        set i [string first "\n(C" $it];
        if {$i <= 0} {set i end-1} {incr i -1}
        set passres [string range $it [expr [string first )\  $it]+2] $i]
    }
###
    if { [regexp {\(C[0-9]+\) $|\(dbm:[0-9]+\) $|([A-Z]+>[>]*)$} $it junk lisp]  } {
        #puts "junk=$junk, lisp=$lisp,[expr { 0 == [string compare $lisp {}] }]"
        #puts "it=<$it>,pdata={[array get pdata *]},[$win index end],[$win index insert]"
 
        if { [info exists pdata($sock,wait) ] && $pdata($sock,wait) > 0 } {
            #puts "it=<$it>,begin=$pdata($sock,begin),end=[$win index {end linestart}]"
            #puts dump=[$win dump -all "insert -3 lines" end]
            setAct pdata($sock,result) [$win get $pdata($sock,begin) "end -1char linestart" ]
            #puts result=$pdata($sock,result)
            set pdata($sock,wait) 0
        }
        $win mark set lastStart "end -1char"
        $win tag add  input "end -1char" end
        oset $win atMaximaPrompt [expr { 0 == [string compare $lisp ""] }]
        
    }
    $win see end
    return
} 

Now the routine to do the maxima calling from tcl:

proc domax e {
    global passres;
    .maxima.text insert insert "$e;\n" ;
    CMeval .maxima.text;
    set passres {};
    while {$passres eq {}}  {vwait passres};
    return $passres
}

And it's needed to set maxima's reply mode to 1 dimensional, so that formatting is simple formula return which can be fed back even to a new evaluation by maxima, and so that my output line parse finds the number of the result (DXX) at the beginning, and not in the middle line of the output:

domax display2d:false

Now we can use:

  domax 1+5
6
  domax "solve(x^2+x+1=0,x)"
[x = -(SQRT(3)*%I+1)/2,x = (SQRT(3)*%I-1)/2]
  • Note that at this moment, there is no real error handling, and
  • the domax proc can hang (leaving the maxima console running, type a ; on a new prompt to mostly make domax return...),
  • I've had a few wrong responses (missing first part of reply) though I'm not sure in the latest version, but the principle appears to work.
  • Important note is that I didn't contentwise think at all what might happen when calling domax recursively...

Meanwhile I've tried using domax in recursive calls, which appears to work right, but the above remark still holds....

Possible/needed improvements:

  • maybe join the return lines into one line for long return values
  • as the 'solve' example indicated, return values can be lists
  • certain integrate() and probably other commands require the user to add input, which could be done in the text widget, but that makes domax at least no normal proc anymore (possibly there is an settable options to prevent this)
  • for certain cases maybe it is desirable to be able to change back and forth to tcl expr syntax
  • for higher speed, possible it would be better to batch commands
  • the stream setup should be able to work fine, but as it is, it may be not to good how domax works, but that would require alternatives like stream based processing to be used instead of a proc.
  • Multiple line responses are not really dealt with by me, so when the socket starts buffering, I might not get the whole response line and mess up the interaction.

Anyone from the maxima scene follow the wiki? Or: maybe someone wants to make a touch more decent 'domax'? And maybe a Tk based pretty printer ? Just askin' ...

DieMongo 2006-09-08: I'm working on a project where I reuse the output from Maxima, modify it and send it back. Look at the example and result here below running from the Tcl Console in Maxima version 5.9.3.99rc1:

(%o3) false
(%i4) 34*x + 17 +4 = 48*x - 30+4;

(%o4) 34*x+21 = 48*x-26
(%i5) 34*x+21 -33 = 48*x-26
(%i5)-33;
Incorrect syntax: Syntax error
x+21 -33 = 48*x-26\n(
                   ^

- The 'false' is just the result from the call domax "display2d:false" - To the output (%o4) I add -33 on both sides and resend this to maxima as input (%i5) - As you can see the output (%o4) contained a newline which prevent me from reusing the output from Maxima "as is"

The solution is to modify the final line in the Domax routine:

proc domax e {
    global passres;
    .maxima.text insert insert "$e;\n";
    CMeval .maxima.text;
    set passres {};
    while {$passres eq {}}  {vwait passres};
    regsub -all \n $passres {} passres
    return $passres
}

And to handle multiple lines, change the following line in maximaFilter:

################## that has become "\n(%i" (otherwise your output contains the next prompt).
     set i [string first "\n(C" $it];

to this

     set i [string first "\n(\%i" $it];