Error processing request

Parameters

Body

Error

Unknow state transition: CODE -> END

-code

1

-level

0

-errorstack

INNER {returnImm {Unknow state transition: CODE -> END} {}} CALL {my render_wikit {Tcl IO performance} \[MJ\]\ -\ 20071105,\ Sparked\ by\ an\ inquiry\ by\ \[Larry\ McVoy\]\ on\ the\ \[Tcl\ chatroom\],\ the\ IO\ performance\ of\ Tcl\ and\ specifically\ compared\ to\ \[Perl\]\ has\ been\ benchmarked\ quite\ extensively\ and\ was\ shown\ to\ be\ slower\ than\ \[Perl\].\ In\ order\ to\ get\ closer\ to\ the\ speed\ of\ Perl\ the\ IO\ implementation\ in\ Tcl\ is\ currently\ undergoing\ scrutiny\ to\ see\ where\ performance\ gains\ can\ be\ made.\n\nThe\ goal\ of\ the\ page\ is\ to\ summarize\ the\ discussion\ as\ it\ has\ been\ held\ up\ to\ this\ point,\ solutions\ that\ have\ been\ suggested\ and\ patches\ that\ have\ been\ tried.\ This\ should\ prevent\ circular\ discussions\ on\ the\ chat,\ because\ people\ can\ be\ directed\ to\ this\ page\ to\ get\ an\ update\ of\ the\ current\ state\ of\ affairs.\n\nThe\ benchmarks\ used\ to\ time\ the\ effect\ of\ changes\ can\ be\ downloaded\ from\ \[http://www.bitmover.com/lm/langbench.shar\]\n\n\[JE\]\ -\ 20071105.\ \ In\ particular,\ it's\ important\ to\ note\ that\ the\ point\ of\ the\ exercise\ is\ not\ to\ make\ a\ \"fast\ cat\"\ in\ Tcl\nor\ to\ make\ a\ \"fast\ grep\"\ in\ Tcl:\ these\ are\ both\ easily\ doable.\ \ The\ point\ of\ the\ exercise\ is\ to\ find\ where\ the\ real\ bottlenecks\ are\nin\ real\ text\ processing\ filters,\ and\ to\ make\ those\ hotspots\ cooler.\n\n----\n\[KBK\]\ -\ 20071105:\n\n'''Hypothesis:'''\ The\ variable\ access\ in\ the\ three-argument\ form\ of\ \[\[gets\]\]\ is\ a\ singificant\ contributor\ to\ the\ observed\ poor\ performance.\n\n'''Experiment:'''\ A\ disc\ file\ was\ created\ containing\ 1\ million\ lines,\ each\ comprising\ the\ single\ letter\ 'x'.\nA\ modification\ was\ made\ to\ Tcl_GetsObjCmd\ as\ shown:\n======\n===================================================================\nRCS\ file:\ /cvsroot/tcl/tcl/generic/tclIOCmd.c,v\nretrieving\ revision\ 1.44\ndiff\ -u\ -r1.44\ tclIOCmd.c\n---\ generic/tclIOCmd.c\ \ 15\ Oct\ 2007\ 21:27:48\ -0000\ \ \ \ \ \ 1.44\n+++\ generic/tclIOCmd.c\ \ 5\ Nov\ 2007\ 21:02:36\ -0000\n@@\ -287,9\ +287,13\ @@\n\ \ \ \ \ \ \ \ lineLen\ =\ -1\;\n\ \ \ \ \ \}\n\ \ \ \ \ if\ (objc\ ==\ 3)\ \{\n-\ \ \ \ \ \ \ if\ (Tcl_ObjSetVar2(interp,\ objv\[2\],\ NULL,\ linePtr,\n-\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ TCL_LEAVE_ERR_MSG)\ ==\ NULL)\ \{\n-\ \ \ \ \ \ \ \ \ \ \ return\ TCL_ERROR\;\n+\ \ \ \ \ \ \ if\ (*Tcl_GetString(objv\[2\])\ !=\ '\\0')\ \{\n+\ \ \ \ \ \ \ \ \ \ \ if\ (Tcl_ObjSetVar2(interp,\ objv\[2\],\ NULL,\ linePtr,\n+\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ TCL_LEAVE_ERR_MSG)\ ==\ NULL)\ \{\n+\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ return\ TCL_ERROR\;\n+\ \ \ \ \ \ \ \ \ \ \ \}\n+\ \ \ \ \ \ \ \}\ else\ \{\n+\ \ \ \ \ \ \ \ \ \ \ Tcl_DecrRefCount(linePtr)\;\n\ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \ \ \ Tcl_SetObjResult(interp,\ Tcl_NewIntObj(lineLen))\;\n\ \ \ \ \ \ \ \ return\ TCL_OK\;\n======\n\nThe\ following\ script,\ roughly\ equivalent\ to\ 'wc\ -l'\ was\ used\ as\ benchmark.\n\n======\nproc\ read1\ \{\}\ \{\n\ \ \ \ set\ lc\ 0\n\ \ \ \ set\ f\ \[open\ ~/tmp/1mlines\ r\]\n\ \ \ \ while\ \{\[gets\ \$f\ line\]\ >=\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n\nproc\ read2\ \{\}\ \{\n\ \ \ \ set\ lc\ 0\n\ \ \ \ set\ f\ \[open\ ~/tmp/1mlines\ r\]\n\ \ \ \ while\ \{\[gets\ \$f\ \{\}\]\ >=\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n\nset\ methods\ \{read1\ read2\}\nforeach\ method\ \$methods\ \{\n\ \ \ \ set\ min(\$method)\ Inf\ \ \ \ \n\}\nfor\ \{set\ i\ 0\}\ \{\$i\ <\ 5\}\ \{incr\ i\}\ \{\n\ \ \ \ foreach\ method\ \$methods\ \{\n\ \ \ \ \ \ \ \ set\ us\ \[lindex\ \[time\ \$method\]\ 0\]\n\tputs\ \[list\ \$method:\ \$us\]\n\tif\ \{\$us\ <\ \$min(\$method)\}\ \{\n\t\ \ \ \ set\ min(\$method)\ \$us\n\t\}\n\ \ \ \ \}\n\}\nforeach\ method\ \$methods\ \{\n\ \ \ \ puts\ \[list\ \$method\ \$min(\$method)\]\n\}\n======\n\n'''Results:'''\n\nThe\ minimum\ observed\ run\ time\ for\ the\ script\ where\ the\ line\ was\ returned\ into\ a\ variable\ was\ 2.46\ seconds\;\ the\ minimum\ observed\ time\ where\ the\ line\ was\ discarded\ was\ 2.08\ seconds.\ \ (The\ maxima\ were\ 2.52\ and\ 2.17\ respectively,\ so\ system\ load\ was\ presumed\ not\ to\ be\ an\ issue).\n\n'''Conclusion:'''\ \n\nStoring\ the\ return\ string\ in\ a\ variable\ appears\ to\ consume\ roughly\ 0.28\ microseconds\ per\ line,\ or\ about\ 12%\ of\ the\ total\ time\ spent\ in\ this\ micro-benchmark.\n----\n\nFurther\ experiments\ by\ \[kbk\]\ as\ reported\ on\ the\ chat:\n\n\ \ \ *\ insuring\ that\ ''buf''\ is\ a\ compiled\ local\ variable\ (read1a)\ gives\ the\ times:\ read1:\ 2.44\ s,\ read1a\ 2.20\ s,\ read2\ 2.09\ s\n\ \ \ *\ the\ same\ as\ above,\ on\ larry's\ larger\ data\ file:\ read1:\ 2.85\ read1a\ 2.52\ read2\ 2.57\n\ \ \ *\ with\ '''fakegets'''\ and\ '''readm'''\ as\ below:\ 1.91\ s\ ''This\ means\ that\ the\ call/setVar/return\ is\ no\ more\ than\ 1.91s''\n\n=======\nproc\ fakegets\ \{chan\ v\}\ \{\n\ \ \ \ global\ ct\n\ \ \ \ if\ \{!\[incr\ ct\ -1\]\}\ \{return\ -1\}\n\ \ \ \ \ \ \ \ return\ 1\n\ \ \ \ \}\n\}\nproc\ readm\ \{\}\ \{\n\ \ \ \ set\ ::ct\ 1000000\n\ \ \ \ set\ f\ \[open\ ~/tmp/DATA\ r\]\n\ \ \ \ while\ \{\[fakegets\ \$f\ \{\}\]\ >=\ 0\}\ \{\n\ \ \ \ \ \ \ \ incr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\n======\n\n\ \ \ *\ finally,\ the\ following\ experiment\ puts\ an\ upper\ bound\ of\ 0.69s\ on\ call/setVar/return\n\n======\nproc\ readm2\ \{\}\ \{\n\ \ \ \ set\ x\ incr\n\ \ \ \ set\ ct\ 1000000\n\ \ \ \ set\ f\ \[open\ ~/tmp/DATA\ r\]\n\ \ \ \ while\ \{\[\$x\ ct\ -1\]\ >\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n====== regexp2} CALL {my render {Tcl IO performance} \[MJ\]\ -\ 20071105,\ Sparked\ by\ an\ inquiry\ by\ \[Larry\ McVoy\]\ on\ the\ \[Tcl\ chatroom\],\ the\ IO\ performance\ of\ Tcl\ and\ specifically\ compared\ to\ \[Perl\]\ has\ been\ benchmarked\ quite\ extensively\ and\ was\ shown\ to\ be\ slower\ than\ \[Perl\].\ In\ order\ to\ get\ closer\ to\ the\ speed\ of\ Perl\ the\ IO\ implementation\ in\ Tcl\ is\ currently\ undergoing\ scrutiny\ to\ see\ where\ performance\ gains\ can\ be\ made.\n\nThe\ goal\ of\ the\ page\ is\ to\ summarize\ the\ discussion\ as\ it\ has\ been\ held\ up\ to\ this\ point,\ solutions\ that\ have\ been\ suggested\ and\ patches\ that\ have\ been\ tried.\ This\ should\ prevent\ circular\ discussions\ on\ the\ chat,\ because\ people\ can\ be\ directed\ to\ this\ page\ to\ get\ an\ update\ of\ the\ current\ state\ of\ affairs.\n\nThe\ benchmarks\ used\ to\ time\ the\ effect\ of\ changes\ can\ be\ downloaded\ from\ \[http://www.bitmover.com/lm/langbench.shar\]\n\n\[JE\]\ -\ 20071105.\ \ In\ particular,\ it's\ important\ to\ note\ that\ the\ point\ of\ the\ exercise\ is\ not\ to\ make\ a\ \"fast\ cat\"\ in\ Tcl\nor\ to\ make\ a\ \"fast\ grep\"\ in\ Tcl:\ these\ are\ both\ easily\ doable.\ \ The\ point\ of\ the\ exercise\ is\ to\ find\ where\ the\ real\ bottlenecks\ are\nin\ real\ text\ processing\ filters,\ and\ to\ make\ those\ hotspots\ cooler.\n\n----\n\[KBK\]\ -\ 20071105:\n\n'''Hypothesis:'''\ The\ variable\ access\ in\ the\ three-argument\ form\ of\ \[\[gets\]\]\ is\ a\ singificant\ contributor\ to\ the\ observed\ poor\ performance.\n\n'''Experiment:'''\ A\ disc\ file\ was\ created\ containing\ 1\ million\ lines,\ each\ comprising\ the\ single\ letter\ 'x'.\nA\ modification\ was\ made\ to\ Tcl_GetsObjCmd\ as\ shown:\n======\n===================================================================\nRCS\ file:\ /cvsroot/tcl/tcl/generic/tclIOCmd.c,v\nretrieving\ revision\ 1.44\ndiff\ -u\ -r1.44\ tclIOCmd.c\n---\ generic/tclIOCmd.c\ \ 15\ Oct\ 2007\ 21:27:48\ -0000\ \ \ \ \ \ 1.44\n+++\ generic/tclIOCmd.c\ \ 5\ Nov\ 2007\ 21:02:36\ -0000\n@@\ -287,9\ +287,13\ @@\n\ \ \ \ \ \ \ \ lineLen\ =\ -1\;\n\ \ \ \ \ \}\n\ \ \ \ \ if\ (objc\ ==\ 3)\ \{\n-\ \ \ \ \ \ \ if\ (Tcl_ObjSetVar2(interp,\ objv\[2\],\ NULL,\ linePtr,\n-\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ TCL_LEAVE_ERR_MSG)\ ==\ NULL)\ \{\n-\ \ \ \ \ \ \ \ \ \ \ return\ TCL_ERROR\;\n+\ \ \ \ \ \ \ if\ (*Tcl_GetString(objv\[2\])\ !=\ '\\0')\ \{\n+\ \ \ \ \ \ \ \ \ \ \ if\ (Tcl_ObjSetVar2(interp,\ objv\[2\],\ NULL,\ linePtr,\n+\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ TCL_LEAVE_ERR_MSG)\ ==\ NULL)\ \{\n+\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ return\ TCL_ERROR\;\n+\ \ \ \ \ \ \ \ \ \ \ \}\n+\ \ \ \ \ \ \ \}\ else\ \{\n+\ \ \ \ \ \ \ \ \ \ \ Tcl_DecrRefCount(linePtr)\;\n\ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \ \ \ Tcl_SetObjResult(interp,\ Tcl_NewIntObj(lineLen))\;\n\ \ \ \ \ \ \ \ return\ TCL_OK\;\n======\n\nThe\ following\ script,\ roughly\ equivalent\ to\ 'wc\ -l'\ was\ used\ as\ benchmark.\n\n======\nproc\ read1\ \{\}\ \{\n\ \ \ \ set\ lc\ 0\n\ \ \ \ set\ f\ \[open\ ~/tmp/1mlines\ r\]\n\ \ \ \ while\ \{\[gets\ \$f\ line\]\ >=\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n\nproc\ read2\ \{\}\ \{\n\ \ \ \ set\ lc\ 0\n\ \ \ \ set\ f\ \[open\ ~/tmp/1mlines\ r\]\n\ \ \ \ while\ \{\[gets\ \$f\ \{\}\]\ >=\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n\nset\ methods\ \{read1\ read2\}\nforeach\ method\ \$methods\ \{\n\ \ \ \ set\ min(\$method)\ Inf\ \ \ \ \n\}\nfor\ \{set\ i\ 0\}\ \{\$i\ <\ 5\}\ \{incr\ i\}\ \{\n\ \ \ \ foreach\ method\ \$methods\ \{\n\ \ \ \ \ \ \ \ set\ us\ \[lindex\ \[time\ \$method\]\ 0\]\n\tputs\ \[list\ \$method:\ \$us\]\n\tif\ \{\$us\ <\ \$min(\$method)\}\ \{\n\t\ \ \ \ set\ min(\$method)\ \$us\n\t\}\n\ \ \ \ \}\n\}\nforeach\ method\ \$methods\ \{\n\ \ \ \ puts\ \[list\ \$method\ \$min(\$method)\]\n\}\n======\n\n'''Results:'''\n\nThe\ minimum\ observed\ run\ time\ for\ the\ script\ where\ the\ line\ was\ returned\ into\ a\ variable\ was\ 2.46\ seconds\;\ the\ minimum\ observed\ time\ where\ the\ line\ was\ discarded\ was\ 2.08\ seconds.\ \ (The\ maxima\ were\ 2.52\ and\ 2.17\ respectively,\ so\ system\ load\ was\ presumed\ not\ to\ be\ an\ issue).\n\n'''Conclusion:'''\ \n\nStoring\ the\ return\ string\ in\ a\ variable\ appears\ to\ consume\ roughly\ 0.28\ microseconds\ per\ line,\ or\ about\ 12%\ of\ the\ total\ time\ spent\ in\ this\ micro-benchmark.\n----\n\nFurther\ experiments\ by\ \[kbk\]\ as\ reported\ on\ the\ chat:\n\n\ \ \ *\ insuring\ that\ ''buf''\ is\ a\ compiled\ local\ variable\ (read1a)\ gives\ the\ times:\ read1:\ 2.44\ s,\ read1a\ 2.20\ s,\ read2\ 2.09\ s\n\ \ \ *\ the\ same\ as\ above,\ on\ larry's\ larger\ data\ file:\ read1:\ 2.85\ read1a\ 2.52\ read2\ 2.57\n\ \ \ *\ with\ '''fakegets'''\ and\ '''readm'''\ as\ below:\ 1.91\ s\ ''This\ means\ that\ the\ call/setVar/return\ is\ no\ more\ than\ 1.91s''\n\n=======\nproc\ fakegets\ \{chan\ v\}\ \{\n\ \ \ \ global\ ct\n\ \ \ \ if\ \{!\[incr\ ct\ -1\]\}\ \{return\ -1\}\n\ \ \ \ \ \ \ \ return\ 1\n\ \ \ \ \}\n\}\nproc\ readm\ \{\}\ \{\n\ \ \ \ set\ ::ct\ 1000000\n\ \ \ \ set\ f\ \[open\ ~/tmp/DATA\ r\]\n\ \ \ \ while\ \{\[fakegets\ \$f\ \{\}\]\ >=\ 0\}\ \{\n\ \ \ \ \ \ \ \ incr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\n======\n\n\ \ \ *\ finally,\ the\ following\ experiment\ puts\ an\ upper\ bound\ of\ 0.69s\ on\ call/setVar/return\n\n======\nproc\ readm2\ \{\}\ \{\n\ \ \ \ set\ x\ incr\n\ \ \ \ set\ ct\ 1000000\n\ \ \ \ set\ f\ \[open\ ~/tmp/DATA\ r\]\n\ \ \ \ while\ \{\[\$x\ ct\ -1\]\ >\ 0\}\ \{\n\tincr\ lc\n\ \ \ \ \}\n\ \ \ \ close\ \$f\n\ \ \ \ return\ \$lc\n\}\ \n======} CALL {my revision {Tcl IO performance}} CALL {::oo::Obj4892174 process revision/Tcl+IO+performance} CALL {::oo::Obj4892172 process}

-errorcode

NONE

-errorinfo

Unknow state transition: CODE -> END
    while executing
"error $msg"
    (class "::Wiki" method "render_wikit" line 6)
    invoked from within
"my render_$default_markup $N $C $mkup_rendering_engine"
    (class "::Wiki" method "render" line 8)
    invoked from within
"my render $name $C"
    (class "::Wiki" method "revision" line 31)
    invoked from within
"my revision $page"
    (class "::Wiki" method "process" line 56)
    invoked from within
"$server process [string trim $uri /]"

-errorline

4