Error processing request

Parameters

Body

Error

Unknow state transition: LINE -> END

-code

1

-level

0

-errorstack

INNER {returnImm {Unknow state transition: LINE -> END} {}} CALL {my render_wikit Cloverfield **\ Announcement\ **\nThe\ official\ announcement\ for\ the\ Cloverfield\ project\ can\ be\ found\ there:\ \[Cloverfield\ -\ Announcement\]\n\n----\n**\ Goals\ **\nAlong\ with\ the\ general\ goals\ listed\ in\ the\ above\ announcement,\ here\ are\ a\ few\ more\ specific\ technical\ goals:\n\n***\ Language\ ***\nImprove\ the\ Tcl\ language\ syntax\ on\ several\ points\ to\ address\ common\ criticisms\ as\ well\ as\ implement\ missing\ features.\ For\ example\ :\n\n\ \ \ *\ 'Fix'\ the\ comments\;\n\ \ \ *\ Auto-expand\ the\ first\ word\ of\ a\ command\ recursively.\ This\ will\ simplify\ currying\ and\ can\ give\ great\ results\ if\ namespaces\ become\ regular\ commands\ (spaces\ would\ thus\ become\ a\ valid\ namespace\ separator)\;\n\ \ \ *\ Improve\ variable\ access\ :\ allow\ e.g.\ `\$\$var`,\ and\ subscript\ access\ such\ as\ `var\[\[1\]\]`\ or\ `var(a)`\ along\ with\ interfaces\ (see\ Data\ structures\ below)\;\n\ \ \ *\ Allow\ variable\ references\ using\ the\ syntax\ `\$&var`.\ This\ can\ fill\ the\ gap\ between\ current\ value/reference\ access\ semantics,\ e.g.\ `lindex`\ vs\ `lappend`,\ and\ solve\ many\ mutability\ vs.\ immutability\ problems\;\n\ \ \ *\ Add\ a\ new\ quoting\ rule\ using\ parentheses,\ and\ drop\ the\ `list`\ command\ as\ we\ know\ it.\ For\ example,\ `(a\ b\ \$c)`\ should\ be\ equivalent\ to\ `\[\[list\ a\ b\ \$c\]\]`.\ The\ semantics\ of\ quotes\ and\ braces\ is\ preserved\ (minus\ changes\ needed\ for\ e.g.\ comments).\ Incidentally,\ this\ is\ the\ same\ syntax\ as\ LISP.\n\ \ \ *\ Extend\ the\ metasyntax\ pioneered\ by\ the\ argument\ expansion\ operator.\ This\ is\ the\ most\ controversial\ syntax\ change,\ but\ is\ unfortunately\ needed\ by\ the\ nature\ of\ some\ changes,\ like\ references\ or\ LISP-like\ delayed\ evaluation.\n\ \ \ *\ Define\ a\ syntax\ for\ specifying\ references.\ This\ can\ be\ used\ for\ example\ to\ serialize\ circular\ references,\ or\ keep\ references\ to\ variables\ that\ go\ out\ of\ scope\;\ for\ example,\ `\{ref\ self\}(a\ \{ref\ self\}\{\})`\ specifies\ a\ list\ whose\ second\ element\ points\ to\ its\ parent.\n\nFor\ more\ detailed\ information,\ see\ \[Cloverfield\ -\ Tridekalogue\]\n\n***\ Data\ structures\ ***\nUse\ \[rope\]s\ as\ the\ internal\ string\ representation.\ Ropes\ will\ use\ B-trees\ of\ immutable\ strings.\ This\ will\ give\ fast\ concatenation,\ slicing,\ insertion,\ and\ should\ dramatically\ reduce\ the\ memory\ usage\ and\ data\ copying.\n\nUse\ interfaces\ (à\ la\ \[Feather\])\ instead\ of\ Tcl_Obj.\ This\ should\ eliminate\ most\ cases\ of\ shimmering.\n\n***\ Runtime\ ***\nImplement\ the\ runtime\ on\ existing\ virtual\ machines.\ Primary\ target\ is\ \[LLVM\].\ Secondary\ target\ could\ be\ Java,\ .NET,\ Parrot.\ LLVM\ is\ the\ most\ interesting\ solution\ since\ it\ gives\ access\ to\ JIT\ compiling,\ platform\ independence,\ native\ performances,\ and\ allow\ total\ control\ over\ the\ internal\ model\ (contrary\ to\ JVM).\ Moreover,\ other\ languages\ such\ as\ C\ or\ C++\ are\ already\ supported,\ which\ means\ that\ we\ could\ get\ cross-platform\ \[Critcl\]-like\ features\ for\ free.\n\nTo\ achieve\ the\ goal\ of\ VM\ independence,\ internal\ data\ structures\ should\ be\ sufficiently\ high\ level.\n\nProvide\ a\ VM-less,\ purely\ interpreted\ reference\ platform\ for\ embedded\ and\ small\ footprint\ solutions.\n\nThe\ runtime\ should\ provide\ advanced\ execution\ modes\ such\ as\ coroutines,\ stackless,\ lightweight\ threads,\ etc.\ See\ \[Radical\ reform\ of\ the\ execution\ engine\]\ for\ some\ ideas.\n----\n**\ Related\ information\ **\n\nSee\ \[Cloverfield\ -\ The\ Gathering\]\ for\ all\ other\ pages\ related\ to\ language\ improvement.\n\n----\n**\ General\ Discussion\ **\n\n\[George\ Peter\ Staplin\]:\ Hi\ FB!\ \ I\ think\ you\ have\ some\ good\ ideas.\ \ I've\ read\ some\ of\ your\ code\ for\ \[TkGS\].\ \ \ I'm\ hoping\ that\ you\ can\ get\ developers\ behind\ this\ project,\ and\ it\ doesn't\ become\ moribund.\ \ I\ am\ interested.\ \ Cloverfield\ is\ a\ good\ name,\ and\ I\ think\ it\ gets\ away\ from\ many\ old\ misconceptions\ about\ Tcl.\n\n\[FB\]:\ Thank\ you!\ Yes,\ I\ hope\ Cloverfield\ will\ get\ more\ attention.\ TkGS\ scope\ was\ a\ bit\ too\ narrow\ to\ really\ get\ developers\ on\ the\ project.\ But\ I've\ learned\ a\ lot\ working\ on\ it,\ even\ if\ the\ project\ never\ completed\ due\ to\ lack\ of\ free\ time\ (building\ a\ family\ needs\ a\ lot\ of\ commitment).\ Anyway\ I\ think\ it\ is\ a\ bit\ obsolete\ now,\ since\ most\ of\ the\ work\ involved\ the\ creation\ a\ new\ graphic\ layer,\ and\ I\ feel\ that\ Cairo\ would\ do\ the\ job\ perfectly.\ I\ even\ had\ the\ project\ to\ port\ Tk\ to\ Cairo\ a\ few\ months\ ago,\ but\ given\ the\ success\ of\ \[Tile\]\ I\ came\ to\ the\ conclusion\ that\ Tk\ no\ longer\ needed\ significant\ improvements\ (at\ least\ for\ now),\ whereas\ Tcl\ was\ losing\ ground,\ so\ I\ moved\ on\ to\ what\ became\ Cloverfield.\n\nAbout\ the\ name:\ I\ chose\ Cloverfield\ only\ a\ couple\ of\ days\ ago\ after\ realizing\ that\ the\ date\ for\ the\ announcement\ was\ 18-1-08.\ But\ prior\ to\ that\ I\ made\ a\ list\ of\ possible\ names,\ see:\ \[Cloverfield\ -\ Alternate\ names\]\n\n----\n\[Lars\ H\]:\ Wow,\ I\ think\ there\ isn't\ a\ single\ thing\ on\ that\ list\ that\ doesn't\ strike\ me\ as\ completely\ wrong\ for\ Tcl.\ Fascinating!\ Well,\ as\ long\ as\ you're\ just\ starting\ it\ up\ as\ a\ separate\ project\ I\ suppose\ I\ can\ happily\ ignore\ all\ of\ it…\n\n''\[FB\]:\ I\ don't\ want\ to\ sound\ too\ harsh,\ but\ after\ reading\ your\ contributions\ to\ all\ the\ discussions\ I\ found\ on\ this\ wiki\ regarding\ language\ improvements,\ you\ seem\ to\ be\ very\ conservative\ when\ it\ comes\ to\ anything\ that\ might\ impact\ the\ \[Dodekalogue\].\ Frankly,\ I\ don't\ think\ that\ auto-expansion\ of\ leading\ words\ is\ totally\ un-Tclish,\ given\ that\ the\ same\ suggestion\ have\ been\ made\ by\ several\ reputable\ Tclers\ such\ as\ \[DKF\]\ of\ \[NEM\],\ and\ that\ it\ currently\ requires\ `unknown`\ hacks\ to\ work\ (see\ \[Let\ unknown\ know\]\ and\ \[An\ implicit\ \{*\}\$\ prefix\]).\ And\ the\ use\ of\ parentheses\ has\ been\ debated\ in\ \[Tcl\ 9.0\ WishList\],\ see\ #67.\ I\ understand\ you\ were\ against\ this\ change,\ but\ you\ should\ also\ concede\ that\ this\ change\ would\ greatly\ improve\ readability.''\ \n\n\[Lars\ H\]:\ No,\ it\ would\ not\ noticably\ improve\ readability.\n\n''\[FB\]:\ Let's\ have\ a\ look\ at\ the\ following\ code:''\n\n\ #\ Tcl:\n\ set\ l\ \[list\ \\\n\ \ \ \ \ \[list\ 1\ 2\ \$somevar\]\ \\\n\ \ \ \ \ \[list\ 3\ 4\ \[someproc\]\]\ \\\n\ \]\n\n\ #\ Cloverfield:\n\ set\ l\ (\n\ \ \ \ \ (1\ 2\ \$somevar)\n\ \ \ \ \ (3\ 4\ \[someproc\])\n\ )\n\n''Don't\ you\ agree\ that\ the\ latter\ form\ is\ more\ readable?\ Moreover\ the\ former\ is\ more\ error-prone\ (you\ can\ easily\ forget\ a\ backslash),\ and\ this\ is\ a\ simplistic\ case.\ Removing\ the\ noise\ created\ by\ `list`\ and\ backslashes\ leaves\ only\ meaningful\ data.\ Add\ new-style\ comments\ and\ you\ get\ a\ more\ declarative\ way\ of\ defining\ data\ vs.\ the\ old\ procedural\ style.\ I\ also\ feel\ that\ the\ new\ style\ would\ be\ faster\ to\ parse\ and\ interpret,\ because\ the\ whole\ tree\ is\ now\ a\ single\ word,\ versus\ a\ collection\ of\ subwords\ in\ the\ former\ case\ (and\ this\ is\ a\ very\ important\ condition\ for\ the\ proposed\ reference\ declaration\ syntax).''\n----\n\[LES\]\ Dude,\ if\ this\ entire\ business\ is\ about\ making\ Tcl\ more\ popular\ (and\ it\ looks\ like\ it\ is),\ a\ little\ more\ effort\ spent\ on\ Tk/Tile\ and\ a\ handful\ of\ useful\ and\ good-looking\ desktop\ applications\ would\ probably\ be\ a\ lot\ more\ effective\ than\ any\ sort\ of\ fragmentation.\ Fragmentation\ is\ usually\ a\ very\ good\ strategy\ to\ '''stifle'''\ an\ endeavor.\n\n\[KJN\]\ Better\ desktop\ applications\ would\ clearly\ be\ an\ asset.\ \ The\ \[OLPC\]\ adopted\ \[GTK\]\ as\ its\ main\ graphics\ toolkit,\ even\ though\ Tk\ is\ a\ much\ better\ fit\ for\ a\ resource-constrained\ system\ such\ as\ OLPC\;\ but\ it\ had\ no\ choice,\ because\ GTK\ has\ the\ applications\ (Abiword,\ Gnumeric,\ Firefox...),\ and\ Tk\ does\ not.\n\nHowever,\ it\ is\ always\ worth\ thinking\ about\ what\ we\ would\ like\ Tcl\ to\ become.\ \ Most\ suggestions\ will\ be\ explored\ and\ eventually\ rejected\ (see\ pages\ on\ this\ Wiki\ for\ many\ examples)\;\ a\ few\ will\ be\ adopted,\ after\ lengthy\ debate.\ \ Fragmentation\ has\ not\ occurred\ in\ the\ past\ (except\ for\ a\ few\ brave\ souls\ who\ still\ use\ Tcl\ 7\ or\ even\ 6\ because\ of\ their\ smaller\ footprint)\ -\ there\ aren't\ enough\ of\ us\ to\ maintain\ two\ major\ codebases.\n\n----\n'''[email protected]''':\ When\ casting\ about\ for\ a\ language\ that\ isn't\ object\ orientated,\ I\ happened\ upon\ Tcl.\ \ Apart\ from\ perl,\ which\ also\ seems\ to\ have\ become\ frozen\ in\ time,\ this\ seems\ to\ be\ the\ only\ one.\ \ But\ it's\ like\ stepping\ back\ in\ time.\ \ As\ far\ as\ I\ can\ see,\ there's\ one\ book\ that's\ modern\ (2004)\ and\ all\ the\ rest\ are\ from\ about\ 1997.\ \ I\ presume\ when\ they\ say\ they\ work\ with\ windows\ they\ mean\ 98?\ \ There\ doesn't\ seem\ to\ be\ an\ equivalent\ of\ CPAN,\ eggs,\ gems\ either,\ or\ if\ there\ is,\ it's\ not\ obvious.\ \ But\ that\ goes\ for\ the\ whole\ language,\ not\ obvious.\ \ It's\ like\ its\ pitched\ a\ C\ programmers\ and\ that's\ it.\ \ The\ only\ thing\ that\ will\ work\ is\ a\ compelling\ application\ that\ takes\ a\ share\ of\ mind.\ \ Other\ than\ that\ it\ will\ remain\ an\ overlooked\ language.\ \ It's\ problems\ are,\ no\ books,\ no\ tutorials,\ no\ obvious\ way\ to\ do\ things.\ Perhaps\ it's\ had\ its\ day,\ legacy\ only,\ isn't\ that\ the\ way\ of\ it?\n\n----\n**\ Specific\ subtopics\ **\n\n***\ Data\ structures\ ***\n****\ Ropes\ ****\n\n\[DKF\]:\ Experience\ with\ strings-implemented-as-trees\ in\ the\ past\ makes\ me\ point\ out\ that\ you'd\ better\ make\ sure\ that\ you\ take\ care\ to\ keep\ the\ trees\ balanced.\ Otherwise\ you'll\ have\ terrible\ performance.\ And\ using\ C\ arrays\ of\ characters\ seems\ to\ actually\ work\ quite\ well\ in\ practice...\n\n\[FB\]:\ flat\ strings\ (ie\ C\ arrays\ of\ chars)\ give\ good\ performances\ in\ Tcl's\ current\ context.\ Object\ sharing,\ COW,\ the\ lack\ of\ references,\ and\ the\ impossibility\ to\ build\ circular\ structures,\ all\ these\ factors\ suit\ flat\ strings\ perfectly.\ However,\ when\ you\ introduce\ references\ and\ mutability,\ you\ cannot\ use\ COW\ semantics\ anymore,\ because\ changing\ an\ object's\ value\ implies\ invalidating\ the\ string\ rep\ of\ all\ objects\ that\ reference\ it.\ This\ can\ represent\ a\ huge\ performance\ hit,\ as\ data\ sharing\ are\ obviously\ more\ likely\ with\ languages\ that\ allow\ references.\ With\ \[rope\]\ structures,\ you\ only\ have\ to\ invalidate\ the\ substring\ that\ has\ changed,\ by\ rebuilding\ the\ tree\ (or\ one\ of\ its\ leaves\ in\ the\ simplest\ cases).\n\nMoreover\ some\ platforms\ like\ Java\ only\ provide\ immutable\ strings,\ and\ allowing\ string\ mutability\ implies\ a\ huge\ performance\ hit.\ This\ can\ be\ a\ serious\ problem\ if\ we\ want\ to\ implement\ a\ Tcl\ interpreter\ over\ the\ JVM.\ In\ this\ case,\ a\ \[rope\]\ structure\ can\ be\ modified\ but\ the\ underlying\ data\ is\ stored\ in\ immutable\ string.\n\nYou\ can\ read\ the\ following\ paper\ for\ more\ info\ on\ a\ real\ world\ rope\ implementation,\ especially\ section\ 'C\ Cords':\ http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol25/issue12/spe986.pdf\n----\n****\ See\ also\ ****\n\ \ \ *\ \[Tcl9\ and\ annotated\ strings\].\n\n----\n***\ Comments\ ***\n\[FB\]:\ There\ is\ a\ slight\ misunderstanding\ about\ comments\ in\ Cloverfield.\ The\ new\ ''word\ comment''\ `\{#\}`\ syntax\ is\ not\ at\ all\ meant\ to\ replace\ the\ existing\ syntax,\ but\ to\ complement\ it.\ Besides,\ \"fixing\"\ the\ comment\ only\ involves\ changing\ the\ way\ braces\ are\ matched\ and\ allowing\ them\ at\ the\ beginning\ of\ words,\ so\ that\ comments\ work\ in\ a\ less\ surprising\ way.\ For\ example:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ #\ The\ following\ works\ as\ expected:\ the\ close\ brace\ after\ true\ doesn't\ close\ the\ proc.\n\ \ \ \ \ #if\ \{true\}\ \{\n\ \ \ \ \ if\ \{false\}\ \{\n\ \ \ \ \ \ \ \ \ switch\ \$v\ \{\n\ \ \ \ \ \ \ \ \ \ \ \ \ #\ This\ is\ a\ comment.\n\ \ \ \ \ \ \ \ \ \ \ \ \ default\ \{\n\ \ \ \ \ \ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \}\n\n\ \ \ \ \ \{#\}\{This\ comments\ out\ a\ whole\ block\ of\ code\n\ \ \ \ \ someproc\ \$somevar\n\ \ \ \ \ return\ \[foo\]\ #\ Will\ loop\ forever!\ (notice\ the\ lack\ of\ semicolon)\n\ \ \ \ \ \}\n\ \ \ \ \ return\n\ \}\n\n\[AMG\]:\ Regarding\ the\ #if\ comment:\ I\ see\ rule\ \[\[5\]\]\ causes\ Cloverfield\ to\ treat\ all\ braces\ on\ that\ line\ the\ same\ way\ Tcl\ treats\ braces\ preceded\ by\ backslashes.\ \ In\ a\ compiler\ I\ wrote\ that\ accepts\ a\ Tcl-like\ language,\ in\ order\ to\ find\ the\ end\ of\ a\ brace-quoted\ word,\ I\ count\ opening\ and\ closing\ braces\;\ backslashed\ braces\ do\ not\ contribute\ to\ the\ brace\ count.\n\nI\ see\ that\ you\ also\ updated\ the\ brace-counting\ rule\ to\ skip\ braces\ contained\ inside\ double\ quotes.\ \ You\ also\ skip\ braces\ quoted\ using\ the\ raw\ data\ word\ modifier,\ which\ is\ pretty\ much\ a\ new\ thing\ to\ this\ Wiki\;\ I'm\ not\ prepared\ to\ discuss\ it\ yet!\n\n''\[FB\]:\ In\ short,\ `\{data\}`\ is\ roughly\ equivalent\ to\ XML's\ CDATA.\ The\ goal\ is\ to\ ease\ the\ inclusion\ of\ foreign\ data\ into\ Tcl\ to\ improve\ its\ status\ as\ a\ glue\ language.\ Mixed\ with\ \[Critcl\]\ and\ \[tcc\]-like\ features\ it\ would\ simplify\ its\ use\ as\ a\ \[Scripted\ compiler\].\ Currently,\ when\ defining\ data\ in\ a\ foreign\ format,\ you\ have\ to\ properly\ quote\ the\ significant\ characters,\ which\ can\ be\ tedious\ and\ lead\ to\ \[Quoting\ hell\].''\n\nIn\ Cloverfield,\ the\ following\ code\ works\ fine,\ but\ in\ Tcl\ it\ has\ a\ mismatched\ brace:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ puts\ \"\}\"\n\ \}\n\nIn\ Tcl,\ it's\ weird\ (but\ understandable)\ that\ the\ following\ code:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ puts\ -nonewline\ \"digits\ =\ \{\"\n\ \ \ \ \ for\ \{set\ x\ 0\}\ \{\$x\ <\ 10\}\ \{incr\ x\}\ \{\n\ \ \ \ \ \ \ \ \ puts\ -nonewline\ \"\ \$x\"\n\ \ \ \ \ \}\n\ \ \ \ \ puts\ \"\ \}\"\n\ \}\n\nworks\ great\ unless\ I\ delete\ either\ the\ first\ or\ the\ last\ puts.\ \ But\ in\ Cloverfield,\ it's\ fine.\ \ Am\ I\ understanding\ that\ right?\n\n''\[FB\]:\ Yes.\ I've\ tried\ to\ remove\ the\ localities\ of\ the\ brace\ and\ comment\ syntaxes\ so\ that\ they\ work\ in\ the\ least\ astonishing\ way.''\n----\n\[Lars\ H\]:\ I'm\ a\ bit\ surprised,\ though,\ that\ you\ choose\ to\ label\ \[\{expand\}\]-style\ syntax\ as\ the\ most\ controversial\ part\ —\ I\ quite\ agree\ it\ would\ be\ a\ technical\ necessity\ after\ dropping\ \[Everything\ Is\ A\ String\]\ —\ but\ perhaps\ this\ just\ means\ that\ it\ is\ the\ thing\ that\ isn't\ directly\ borrowed\ from\ some\ other\ language.\n\n''\[FB\]:\ Well,\ I\ felt\ this\ was\ controversial\ wrt.\ the\ debate\ that\ preceded\ the\ adoption\ of\ this\ syntax\ change.\ Many\ Tclers\ were\ concerned\ that\ this\ would\ open\ a\ pandora\ box.\ Regarding\ my\ suggestions,\ I\ tried\ to\ limit\ them\ to\ cases\ where\ no\ alternative\ was\ viable,\ i.e.\ when\ the\ changes\ interfered\ with\ the\ way\ Tcl\ interprets\ the\ data.\ I\ think\ I\ managed\ to\ get\ fair\ compromises.\ However\ dropping\ \[EIAS\]\ is\ totally\ out\ of\ question,\ because\ it\ is\ at\ the\ heart\ of\ the\ Tcl\ way.\ On\ the\ contrary,\ if\ you\ re-read\ my\ suggestions\ carefully,''\n—\ \[Lars\ H\]:\ I\ can't\ say\ I\ care\ enough\ to\ bother.\ —\n''you'll\ see\ that\ I\ took\ great\ care\ to\ enforce\ this\ principle\ to\ solve\ some\ hairy\ problems\ (e.g.\ the\ representation\ of\ circular\ structures\ and\ references).''\n—\ \[Lars\ H\]:\ IMNSHO\ references\ are\ the\ data\ structure\ equivalent\ of\ assembly\ language,\ and\ I'm\ glad\ Tcl\ frees\ me\ of\ the\ dangers\ inherent\ in\ these.\ —\n\nAlso,\ I'm\ a\ bit\ surprised\ about\ the\ way\ in\ which\ you\ propose\ to\ \"fix\"\ comments\;\ I\ can't\ recall\ ever\ seeing\ any\ requests\ for\ ''comment\ words''\ in\ commands,\ and\ it\ is\ already\ perfectly\ possible\ to\ put\ comments\ even\ ''inside''\ words,\ using\ command\ substitution:\n\ \ proc\ \\#\ \{args\}\ \{\}\ \;\ #\ No-op\ command,\ for\ comments.\n\ \ \$w\[\\#\ \{That's\ the\ window\ name\}\].toolbar\[\\#\ \{This\ is\ a\ frame\}\].fire\[\\#\ \{The\ actual\ button\ widget\}\]\ configure\ -repeatinterval\[\\#\ \{Delays\ in\ auto\ mode\}\]\ 10\[\\#\ \{A\ veritable\ machine\ gun,\ this\ button\;\ 6000\ shots\ per\ minute\}\]\n\n''(\[MS\]\ hopes\ you\ don't\ have\ a\ comment\ like\ '\[\[exec\ rm\ -rf\ ~\]\]')''\n\n''\[FB\]:\ have\ you\ ever\ had\ to\ explain\ to\ a\ newcomer\ why\ the\ following\ code:''\n\n\ #if\ \{true\}\ \{\n\ if\ \{false\}\ \{\n\ \ \ \ \ puts\ something\ \n\ \}\n\n''works\ in\ an\ interactive\ shell\ but\ not\ from\ source\ or\ inside\ a\ proc?\ And\ why\ comments\ inside\ `switch`\ blocks\ sometimes\ don't\ work,\ or\ give\ unexpected\ results?''\n—\ \[Lars\ H\]:\ This\ is\ really\ a\ \[RTFM\].\ Being\ a\ teacher,\ I\ would\ suggest\ the\ beginner\ to\ (under\ supervision)\ apply\ the\ \[dodekalogue\]\ to\ the\ scripts\ in\ question.\ It's\ not\ a\ difficult\ exercise,\ but\ quite\ instructive.\ —\n''To\ be\ successful\ Tcl\ needs\ to\ follow\ the\ principle\ of\ least\ astonishment\ whenever\ possible.''\n—\ \[Lars\ H\]:\ On\ the\ contrary,\ most\ of\ Tcl's\ perceived\ defects\ are\ the\ result\ of\ literates\ in\ other\ languages\ not\ being\ astonished\ enough\ by\ the\ specific\ nature\ of\ Tcl\ that\ they\ learn\ the\ language\ properly,\ and\ instead\ struggle\ with\ analogies\ to\ other\ languages\;\ in\ this\ case\ that\ braces\ work\ as\ in\ \[C\]\ ''et\ al.''\ —\n\n''To\ be\ more\ specific,\ the\ problem\ is\ not\ really\ to\ \"fix\"\ comments\ but\ to\ \"fix\"\ the\ way\ braces\ are\ matched.\ Hence\ the\ changes\ I\ proposed\ to\ brace\ matching.\ The\ ''comment\ word''\ thing\ is\ needed\ for\ `switch`-like\ cases.\ But\ my\ proposal\ takes\ great\ care\ not\ to\ break\ the\ \[EIAS\]\ principle.\ OTOH,\ I\ don't\ find\ the\ way\ you\ choose\ to\ implement\ comments\ in\ words\ to\ be\ neither\ Tclish\ nor\ readable\ (let\ alone\ dangerous).''\n—\ \[Lars\ H\]:\ Not\ according\ to\ your\ understanding\ of\ Tcl,\ you\ mean?\ (My\ experience\ only\ goes\ back\ to\ Tcl\ 7,\ but\ I'm\ pretty\ sure\ this\ has\ been\ possible\ much\ earlier\ than\ that,\ so\ it's\ implicit\ quite\ deeply\ in\ the\ core\ of\ the\ language.)\ I'd\ be\ the\ first\ to\ admit\ that\ it\ is\ '''near-unreadable''',\ but\ `\[\[\\#\ \{...\}\]\]`\ isn't\ much\ worse\ than\ `\{#\}\{...\}`\ in\ that\ department,\ and\ it\ was\ rather\ this\ that\ was\ the\ point:\ how\ did\ this\ \"fix\"\ anything?\ —\n\n(If\ I\ recall\ correctly,\ \[dkf\]\ has\ mentioned\ in\ discussions\ of\ the\ \[K\]\ combinator\ that\ \$somevar\[\[command-returning-empty-string\]\]\ does\ not\ even\ cause\ shimmering\ in\ Tcl\ 8.5,\ hence\ no\ quality\ degradation\ of\ code.)\ \nAlternatively,\ one\ can\ do\ that\ with\ ordinary\ comments,\ provided\ one\ inserts\ the\ necessary\ newlines:\n\ \ \$w\[#\ That's\ the\ window\ name\n\ \ \].toolbar\[#\ This\ is\ a\ frame\n\ \ \].fire\[#\ The\ actual\ button\ widget\n\ \ \]\ configure\ -repeatinterval\[#\ Delays\ in\ auto\ mode\n\ \ \]\ 10\[#\ A\ veritable\ machine\ gun,\ this\ button\;\ 6000\ shots\ per\ minute\n\ \ \]\nBut\ perhaps\ the\ point\ is\ rather\ to\ allow\ comments\ in\ lists\ written\ as\ strings?\ ''That''\ I\ would\ often\ have\ found\ useful,\ but\ Cloverfield\ rather\ seems\ to\ turn\ away\ from\ this\ practice.\n\n''\[FB\]:\ on\ the\ contrary,\ that's\ exactly\ what\ Cloverfield\ proposes.\ The\ idea\ is\ to\ modify\ the\ way\ comments\ are\ parsed\ in\ braced\ strings.\ For\ example\ :''\n\n\ #\ Tcl:\n\ set\ v\ \{#\ The\ next\ brace\ close\ the\ string\ \}\n\ set\ v\ \{\"\ The\ next\ brace\ close\ the\ string\ \}\n\ set\ v\ \{#\ This\ isn't\ a\ comment\}\ \ #\ and\ this\ is\ just\ excess\ arguments\n\ set\ v\ \{#\ This\ isn't\ a\ comment\}\;\ #\ but\ this\ is\n\n\ #\ Cloverfield:\n\ set\ v\ \{#\ The\ next\ brace\ doesn't\ close\ the\ string\ \}\n\ \ \ \ \ \ \ \ \"neither\ does\ this\ one\ \}\"\n\ \ \ \ \ \ \ \ but\ this\ one\ does\ \}\n\ set\ v\ \{\"#\ This\ is\ not\ a\ comment\"\}\n\ set\ v\ \{\\#\ Neither\ is\ this\}\ \ \ #\ but\ this\ is!\n\n''This\ introduces\ an\ incompatibility\ but\ at\ the\ benefit\ of\ a\ less\ surprising\ behavior.\ But\ EIAS\ is\ preserved,\ as\ the\ comment\ is\ not\ stripped\ off\ the\ string,\ but\ only\ alter\ the\ way\ braces\ are\ matched.''\n\n\[AMG\]:\ It\ is\ my\ understanding\ from\ \[Cloverfield\ -\ Tridekalogue\]\ that\ Cloverfield\ comments\ are\ words\ preceded\ by\ the\ \{#\}\ modifier,\ which\ isn't\ represented\ in\ the\ above\ example.\ \ Oh\ wait,\ I\ see\ what\ you're\ getting\ at.\ \ You\ (\[FB\])\ also\ changed\ the\ way\ \"line\ comments\"\ work.\ \ Whereas\ Tcl\ only\ recognizes\ a\ comment\ when\ #\ appears\ as\ the\ first\ character\ of\ the\ first\ word\ of\ a\ command,\ Cloverfield\ recognizes\ a\ comment\ wherever\ #\ is\ the\ first\ character\ of\ any\ word.\ \ Watch\ out\ for\ \[uplevel\]!\ :^)\n\n''—\ \[Lars\ H\]:\ Ah,\ I\ missed\ that\;\ saw\ only\ the\ \{#\}\ style\ of\ comment.\ OK,\ I\ concede\ that's\ more\ a\ kind\ of\ \"fixing\"\ that\ I\ can\ imagine\ having\ been\ requested.\ Still\ doesn't\ lead\ to\ the\ behaviour\ shown\ in\ the\ first\ example\ above,\ but\ there\ could\ be\ other\ rules\ yet\ on\ commenting\ lurking\ in\ that\ tridekalogue.\ However,\ I\ don't\ care\ enough\ to\ look.\ —''\ \[FB\]:\ Exactly.\ If\ you\ re-read\ the\ Tridekalogue,\ you'll\ see\ that\ \"fixing\"\ the\ comments\ only\ need\ slight\ amendments\ to\ the\ existing\ Dodekalogue:\ rule\ \[\[6\]\]\ becomes\ \[\[5\]\]\ and\ changes\ the\ way\ braces\ are\ matched,\ and\ rule\ \[\[10\]\]\ allows\ the\ hash\ character\ wherever\ the\ beginning\ of\ a\ word\ is\ expected.\ And\ your\ code\ above\ would\ still\ work\ as\ expected.\ —\n\n''\[FB\]:\ Yes.\ Changes\ to\ line\ comments\ are\ necessary\ to\ make\ them\ work\ in\ the\ least\ surprising\ way.\ To\ do\ so\ #\ must\ be\ allowed\ as\ the\ first\ char\ of\ a\ word,\ to\ allow\ for\ in-list\ comments\ (case\ in\ point:\ `switch`).\ Unfortunately\ `uplevel`\ and\ friends\ are\ collateral\ victims\ of\ this\ choice\ (but\ URL\ fragments\ are\ not),\ as\ they\ will\ now\ need\ proper\ quoting\ of\ the\ #,\ for\ example\ with\ backslash.\ Which\ many\ editors\ do\ automatically\ anyway\ (because\ their\ Tcl\ parser\ usually\ fail\ to\ recognize\ comments\ properly,\ QED),\ and\ which\ is\ pretty\ harmless\ compared\ to\ the\ potential\ gain.\ But\ this\ single\ change\ is\ sufficient\ to\ turn\ Cloverfield\ into\ a\ distinct\ language\ because\ it\ impacts\ one\ of\ the\ fundamental\ rules\ of\ the\ \[Dodekalogue\].''\n\n''The\ new\ line\ comment\ rules\ allow\ the\ following\ code:''\n\n\ dict\ create\ \{\n\ \ \ \ \ FirstName\ John\n\ \ \ \ \ LastName\ Smith\n\ \ \ \ \ DateOfBirth\ 1/18/08\ #\ In\ mm/dd/yy\ format\n\ \}\n\n''As\ for\ the\ so-called\ \"word\ comment\",\ I\ think\ it\ is\ a\ bit\ of\ a\ misnomer,\ but\ chosen\ for\ overall\ consistency\ of\ the\ meta-syntax\ (''word\ modifiers'').\ The\ goal\ is\ not\ to\ allow\ the\ commenting\ of\ individual\ words\ (which\ seem\ pretty\ useless),\ but\ rather\ to\ make\ comments\ be\ recognized\ as\ individual\ words\ which\ are\ subsequently\ ignored.\ The\ typical\ use\ case\ for\ word\ comments\ would\ actual\ be\ block\ commenting.\ See\ below\ for\ examples.''\n\nBut\ for\ the\ sake\ of\ completeness,\ here's\ (what\ I\ think\ is)\ a\ Cloverfield\ \{#\}\ comment:\n\n\ set\ v\ \{#\}\{This\ is\ a\ comment\}\ value\n\nI\ guess\ the\ following\ is\ the\ Tcl\ 8.5\ analog\ for\ \"in-line\"\ comments.\ \ It's\ quite\ similar\ to\ something\ proposed\ above,\ except\ it\ doesn't\ work\ by\ appending\ empty\ string\ to\ existing\ words.\ \ It\ uses\ \[\{*\}\]\ to\ produce\ zero\ words,\ which\ makes\ it\ very\ much\ like\ Cloverfield's\ \{#\}.\n\n\ proc\ #\ \{comment\}\ \{\}\n\ set\ v\ \{*\}\[\\#\ \{This\ is\ a\ comment\}\]\ value\n\nIn\ an\ attempt\ to\ defang\ \[\[exec\ rm\ -rf\ ~\]\],\ \ I\ encourage\ the\ caller\ to\ brace\ the\ comment\ text.\ \ I\ do\ this\ by\ making\ \[\[#\]\]\ only\ accept\ one\ argument.\n\n''\[FB\]:\ exactly,\ the\ only\ difference\ being\ the\ substitution\ rules,\ as\ `\{#\}`\ would\ skip\ all\ the\ substitution\ phase.''\n\nI\ also\ added\ to\ your\ above\ comment\ example.\ \ I\ hope\ you\ don't\ mind.\n\n''\[FB\]:\ I've\ added\ what\ I\ think\ was\ a\ missing\ semicolon\ before\ the\ hash\ in\ the\ last\ Tcl\ example.\ Back\ to\ word\ comments,\ here\ is\ an\ example\ of\ block\ commenting.\ Note\ that\ braces\ must\ be\ properly\ balanced\ inside\ comment\ blocks,\ as\ the\ commented\ words\ must\ follow\ all\ formatting\ rules.''\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ \{#\}\{This\ is\ a\ comment\n\ \ \ \ \ that\ spans\ multiple\ lines.\n\ \ \ \ \ Regular\ parsing\ rules\ apply,\ e.g.\ #\}\ this\ brace\ doesn't\ close\ the\ block\n\ \ \ \ \ but\ this\ one\ does\}\n\n\ \ \ \ \ #\ In\ the\ following\ code\ the\ call\ to\ bar\ was\ commented\ out\ and\ replaced\ by\ baz.\n\ \ \ \ \ #\ Typical\ use\ case:\ debugging\ sessions.\n\ \ \ \ \ return\ \{#\}\[bar\]\ \[baz\]\n\ \}\n\n----\n\[KJN\]\ In\ the\ example\ above,\ I\ can\ see\ what\ Cloverfield\ is\ trying\ to\ do\ with\n===\n\ set\ v\ \{#\ The\ next\ brace\ doesn't\ close\ the\ string\ \}\n\ \ \ \ \ \ \ \ \"neither\ does\ this\ one\ \}\"\n\ \ \ \ \ \ \ \ but\ this\ one\ does\ \}\n===\nand\ it\ is\ more\ appealing\ than\ Tcl\ if\ the\ braced\ quantity\ is\ code\;\ but\ if\ the\ braced\ quantity\ is\ data,\ this\ is\ like\ allowing\ a\ comment\ inside\ a\ quoted\ string,\ which\ is\ a\ bit\ painful.\n\nTcl\ has\ an\ intrinsic\ problem,\ which\ is:\n\ \ \ 1.\ We\ uses\ braces\ to\ delimit\ a\ data\ \"word\"\n\ \ \ 2.\ Often\ that\ data\ is\ to\ be\ interpreted\ as\ code\n\ \ \ 3.\ The\ flexibility\ of\ Tcl\ means\ that\ when\ a\ braced\ word\ is\ first\ parsed\ we\ cannot\ in\ general\ know\ whether\ the\ contents\ are\ intended\ to\ be\ executed\ as\ code\n\nI\ don't\ think\ the\ comments/bracing\ problem\ is\ fixable:\ unless\ you\ throw\ away\ the\ power\ of\ Tcl,\ or\ add\ complexity,\ all\ you\ can\ do\ is\ swap\ one\ kind\ of\ pain\ for\ another.\ \ In\ this\ case,\ the\ suggested\ parsing\ rules\ for\ Cloverfield\ remove\ the\ pain\ from\ braced\ code,\ but\ they\ transfer\ it\ to\ braced\ data:\ sometimes\ the\ programmer\ will\ want\ to\ brace\ an\ arbitrary\ data\ string\ which\ is\ to\ be\ interpreted\ verbatim,\ without\ \"comments\".\n\n''\[FB\]:\ In\ this\ case\ you\ may\ want\ to\ use\ Cloverfield's\ raw\ data\ word\ modifier\ `\{data\}`,\ which\ is\ designed\ to\ allow\ the\ inclusion\ of\ arbitrary\ data\ (see\ also\ my\ comments\ higher\ on\ this\ page).\ That\ way\ you\ can\ have\ your\ cake\ (sensible\ brace\ parsing)\ and\ eat\ it\ (eliminate\ quoting\ hell)\ :''\n\n\ set\ s\ \{data\}SomeArbitraryTag\ the\ rest\ is\ ignored\ \$\}\[\{\"\\\n\ #include\ <stdio.h>\n\ int\ main()\ \{\n\ \ \ \ const\ char\ string\[\]\ =\ \"\$\\\[\}\ \{data\}SomeOtherTagWontMatchTheAboveOne\"\;\n\ \ \ \ return\ 0\;\n\ \}\n\ this\ is\ also\ ignored\ \$\}\[\{\"\\\ until\ the\ tag\ SomeArbitraryTag\ #\ here\ we're\ back\ to\ the\ interpreter.\n\n''Basically\ you\ can\ use\ the\ `\{data\}`\ modifier\ to\ enclose\ arbitrary\ data\ between\ a\ user-defined\ tag,\ in\ a\ similar\ way\ to\ MIME\ multi-part\ messages.\ The\ exact\ syntax\ is\ not\ final,\ but\ the\ concept\ is\ powerful.\ The\ goal\ of\ Cloverfield\ is\ obviously\ not\ to\ add,\ but\ remove\ complexity,\ moreover\ only\ a\ global\ proposal\ could\ address\ all\ these\ issues\ altogether\;\ each\ proposal\ taken\ individually\ would\ only\ provide\ marginal\ gain,\ if\ any.''\n\n''This\ concept\ is\ known\ as\ \[here\ document\]\ or\ \[heredoc\]\ in\ other\ languages\ such\ as\ \[Perl\]\ or\ \[Python\].''\n\n\n\[KJN\]:\ Here's\ a\ suggestion\ that\ adds\ complexity,\ and\ probably\ has\ more\ negatives\ than\ positives:\ \ interpret\ braces\ with\ the\ Cloverfield\ rules\;\ but\ also\ allow\ «guillemets/chevrons»\ to\ delimit\ words\ using\ the\ existing\ Tcl\ rules\ for\ braces.\n\nIn\ all\ cases,\ the\ Tcl\ idea\ of\ words\ is\ preserved\;\ but\ programmers\ will\ be\ discouraged\ from\ thinking\ of\ braced\ text\ as\ a\ \{quoted\ string\},\ and\ will\ still\ have\ a\ mechanism\ for\ quoting\ literal\ strings\ without\ substitution.\n\nNaturally\ «chevrons»\ are\ not\ as\ pleasant\ as\ ASCII\ delimiters,\ but\ I\ think\ we\ have\ run\ out\ of\ suitable\ ASCII\ codes,\ except\ possibly\ `these'\n\n''\[FB\]:\ I'd\ rather\ suggest\ that\ chevrons\ (or\ whatever)\ remove\ all\ Tcl-sensitive\ syntax\ and\ take\ verbatim\ data.\ In\ this\ case\ their\ behavior\ is\ close\ to\ my\ proposed\ `\{data\}`.\ The\ problem\ is\ that\ there\ must\ be\ a\ way\ to\ properly\ escape\ existing\ chevrons\ in\ the\ included\ data\ (=>\ quoting\ hell),\ notwithstanding\ the\ fact\ that\ they\ are\ hard\ to\ input\ on\ regular\ keyboards.\ But\ conceptually\ both\ ideas\ are\ similar.''\n\n\[KJN\]:\ I\ don't\ like\ the\ word\ modifiers\ very\ much\ (''see\ Word\ modifiers\ section\ below'').\ \ How\ many\ rules\ could\ you\ ''remove''\ from\ Cloverfield,\ and\ still\ fix\ the\ problems\ that\ you\ want\ to?\n\n''\[FB\]:\ regarding\ comments\ and\ literal\ strings,\ rewriting\ the\ brace\ matching\ rules\ and\ allowing\ `\{data\}`-like\ syntax\ is\ all\ that's\ needed.\ The\ other\ rules\ are\ orthogonal\ and\ target\ other\ issues.\ Anyway\ the\ two\ previous\ enhancements\ are\ incompatible\ changes\ and\ need\ a\ major\ version\ bump.\ But\ the\ merit\ of\ Cloverfield\ is\ to\ introduce\ the\ concept\ of\ word\ modifier\ to\ solve\ a\ range\ of\ problems\ where\ the\ typical\ solution\ would\ use\ ad-hoc\ techniques\ like\ opaque\ tokens\ (=>\ no\ more\ EIAS)\ or\ string\ manipulations\ (=>\ shimmering).''\n\n''\[FB\]:\ Moved\ word-modifier-related\ discussion\ to\ Word\ modifiers\ section\ below''\n\n\[KJN\]:\ If\ rules\ for\ chevrons\ correspond\ to\ those\ for\ Tcl\ braces,\ then\ chevrons\ would\ enclose\ verbatim\ data,\ and\ unmatched\ chevrons\ would\ have\ to\ be\ escaped.\n\nOne\ of\ the\ places\ where\ my\ chevron\ suggestion\ unwinds\ is\ the\ treatment\ of\ lists:\ if\ a\ list\ is\ represented\ as\ a\ string,\ should\ it\ use\ chevrons\ or\ braces\ as\ delimiters?\n\nI'm\ not\ sure\ there\ are\ any\ easy\ answers\ -\ the\ more\ I\ use\ Tcl,\ the\ more\ I\ appreciate\ how\ the\ different\ rules\ fit\ together\ (but\ at\ the\ cost\ of\ the\ comment/bracing\ hell).\ \ It\ is\ difficult\ to\ fix\ the\ comment/bracing\ hell\ without\ introducing\ a\ lot\ of\ extra\ complexity.\ \ But\ it\ is\ well\ worth\ trying!\n----\n\[KJN\]:\ Problems\ with\ \[Cloverfield\ -\ Tridekalogue\]\ re\ comment/bracing\ hell.\n\nRule\ 5\ for\ finding\ the\ matching\ brace,\ specifically,\ the\ rule\ to\ skip\ any\ braces\ in\ comments.\ \ The\ rule\ is\ to\ follow\ rule\ 10\ when\ identifying\ a\ comment.\ \ This\ seems\ to\ require\ that\ the\ braced\ text\ must\ be\ parsed\ into\ commands\ and\ words,\ in\ order\ to\ identify\ when\ '#'\ occurs\ at\ a\ place\ where\ the\ first\ character\ of\ a\ word\ is\ expected.\ \ The\ parsing\ of\ embedded\ braced\ text\ must\ be\ recursive,\ because\ embedded\ braces\ are\ required\ to\ match,\ and\ a\ comment\ string\ may\ occur\ at\ any\ depth\ in\ the\ nested\ braces.\ \ However\ the\ Tridekalogue\ does\ not\ require\ that\ braced\ text\ must\ be\ parsable\ into\ commands\ and\ words,\ only\ that\ braces\ match.\nA\ problem\ therefore\ arises\ when\ the\ recursive\ parsing\ encounters\ text\ that\ does\ not\ have\ Tcl/Cloverfield\ word\ form,\ e.g.\n======\n\ \ someCmd\ \{\"xx\ data\ not\ commands\ \{\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ #\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''\[FB\]:\ This\ example\ is\ simple\ actually.\ The\ first\ double\ quote\ just\ after\ the\ opening\ brace\ suspends\ brace\ matching\ until\ the\ next\ double\ quote,\ i.e.\ the\ one\ just\ before\ the\ closing\ brace.\ The\ \"comment\"\ isn't\ one\ because\ it's\ between\ double\ quotes.\ So\ Tcl\ and\ Cloverfield\ give\ the\ same\ result.\ However\ the\ following\ behaves\ differently\ in\ Tcl\ and\ Cloverfield\ (the\ leading\ double\ quote\ was\ moved\ after\ the\ second\ brace):''\n======\n\ \ someCmd\ \{xx\ data\ not\ commands\ \{\"\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ #\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''Here\ Cloverfield\ needs\ a\ second\ closing\ brace\ because\ the\ second\ opening\ brace\ is\ outside\ the\ couple\ of\ double\ quotes.\ The\ same\ for\ the\ following\ (the\ double\ quote\ was\ moved\ on\ the\ last\ line):''\n======\n\ \ someCmd\ \{xx\ data\ not\ commands\ \{\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ \"#\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''But\ this\ time\ because\ the\ first\ closing\ brace\ that\ matches\ the\ second\ opening\ brace\ in\ Tcl\ is\ commented\ out\ in\ Cloverfield.''\n\n''BTW,\ \[EIAS\]\ is\ preserved,\ meaning\ that\ whatever\ the\ commented\ chars\ may\ be,\ they\ are\ still\ part\ of\ the\ expression:''\n\n======\n%\ set\ l\ \{\n\ \ \ \ 1\ 2\n\ \ \ \ #\ 3\ 4\n\ \ \ \ 5\ 6\n\}\n%\ llength\ \$l\;\ #\ Gives\ 7\ in\ Tcl\ and\ 4\ in\ Cloverfield,\ as\ the\ list\ parser\ recognizes\ the\ comments.\n4\n%\ lindex\ \[split\ \$l\ \\n\]\ 2\;\ #\ Returns\ the\ \"commented\"\ line\ because\ it\ is\ part\ of\ the\ string\ rep.\n\ \ \ \ #\ 3\ 4\n======\n\n\[KJN\]:\ If\ a\ nested\ braced\ data\ string\ does\ not\ have\ code\ form,\ it\ may\ not\ be\ possible\ for\ the\ parser\ to\ decide\ how\ to\ match\ the\ braces.\ \ To\ avoid\ this\ problem,\ I\ think\ Cloverfield\ is\ forced\ to\ require\ that\ a\ braced\ string\ must\ be\ parsable\ into\ commands\ and\ words\ (i.e.\ into\ Cloverfield\ code,\ except\ that\ the\ commands\ need\ not\ be\ defined).\n\n''\[FB\]:\ That's\ a\ good\ point.\ However\ this\ does\ not\ necessarily\ imply\ that\ braced\ expressions\ be\ parsed\ into\ commands\ and\ words,\ only\ that\ beginning\ of\ words\ be\ recognized.\ For\ example,\ a\ double-quote\ character\ does\ not\ start\ a\ string\ when\ in\ the\ middle\ of\ a\ word,\ and\ thus\ does\ not\ suspend\ brace\ matching.\ However\ I'm\ conscious\ that\ rule\ \[\[5\]\]\ may\ sound\ a\ bit\ ambiguous\ and\ needs\ to\ be\ ironed\ out.\ A\ reference\ parser\ implementation\ should\ help\ sorting\ things\ out\ along\ with\ a\ good\ test\ suite.\ ''(\[KJN\]:\ that's\ a\ good\ idea\ -\ why\ not\ write\ the\ reference\ parser\ in\ Tcl,\ it\ should\ not\ be\ too\ difficult.\ ''\[FB\]:\ that\ was\ the\ plan\ \;-)'')''\ Anyway\ the\ goal\ of\ these\ new\ rules\ is\ to\ turn\ the\ language\ rules\ from\ parser-friendly\ into\ human-friendly:\ the\ rules\ should\ follow\ the\ principle\ of\ least\ astonishment\ in\ reading\ order.''\n\n\[KJN\]:\ Also,\ to\ eliminate\ comment\ hell,\ Cloverfield\ cannot\ escape\ parsing\ the\ contents\ of\ a\ quoted\ string\ in\ the\ same\ way,\ because\ code\ such\ as\ the\ following\ may\ occur:\n\n\ \ if\ \{\$a\ eq\ \$b\}\ \{\n\ \ \ \ proc\ foo\ \{bar\}\ \"\n\ \ \ \ \ \ #\ a\ comment\ \}\n\ \ \ \ \ \ puts\ \\\"Error\ foo\ in\ \\\$bar\\\"\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ The\ above\ doesn't\ work\ in\ Tcl\ but\ works\ as\ expected\ (if\ we\ trust\ the\ indentation)\ in\ Cloverfield:\ the\ `if`\ block\ contains\ the\ whole\ proc\ body,\ and\ the\ latter\ also\ behaves\ as\ expected\ (it\ puts\ the\ right\ message,\ skipping\ the\ commented\ line).''\n\n\[KJN\]:\ try\ instead\n\ \ if\ \{\$a\ eq\ \$b\}\ \{\n\ \ \ \ proc\ foo\ \{bar\}\ \"\n\ \ \ \ \ \ #\ a\ \"comment\"\n\ \ \ \ \ \ puts\ \\\"Error\ foo\ in\ \\\$bar\\\"\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ The\ braced\ expression\ is\ valid\ in\ both\ Tcl\ and\ Cloverfield,\ because\ it\ follows\ both\ sets\ of\ rules:''\n\n\ \ \ *\ ''In\ Tcl,\ the\ first\ brace\ matches\ the\ last\ one,\ because\ the\ inner\ braces\ around\ \"bar\"\ are\ well-balanced.''\n\ \ \ *\ ''Ditto\ for\ Cloverfield,\ which\ also\ requires\ that\ the\ double\ quotes\ be\ well-balanced.''\n\n''However,\ both\ Tcl\ and\ Cloverfield\ fail\ to\ evaluate\ the\ expression,\ because\ there\ are\ extra\ characters\ after\ the\ first\ close-quote\ (here,\ \"comment\").\ Tcl\ and\ Cloverfield\ rules\ aren't\ ''that''\ different,\ really\ \;-)\ \ They\ should\ only\ diverge\ in\ fringe\ cases\ or\ in\ very\ predictable\ ways\ (e.g.\ when\ using\ quotes\ and\ hashes\ in\ braced\ expressions).''\n\n\[KJN\]:\ This\ is\ my\ point\ -\ if\ Cloverfield\ aims\ to\ \"fix\"\ comments,\ then\ it\ should\ not\ fail\ with\ that\ code,\ or\ with\n\ \ foreach\ name\ \$list\ \{\n\ \ \ \ proc\ \$name\ \{\}\ \"\n\ \ \ \ \ \ #\ Using\ \"quotes\"\ for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\n\ \ \ \ \ \ return\ \$name\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ Cloverfields\ aims\ to\ \"fix\"\ comments\ within\ braces\ (notice\ how\ I\ always\ put\ \"fix\"\ between\ quotes,\ are\ comments\ are\ not\ really\ broken\ but\ rather\ parser-friendly\ vs.\ human-friendly).\ However\ the\ comments\ are\ not\ recognized\ within\ quoted\ strings\ (and\ I\ don't\ know\ of\ any\ mainstream\ language\ that\ allows\ that).\ Rule\ \[\[5\]\]\ specifies\ that\ characters\ (not\ just\ braces)\ lose\ their\ significance\ between\ quotes.\ So\ the\ above\ code\ will\ fail\ anyway\ because\ of\ the\ extra\ chars\ after\ the\ quotes,\ however\ let's\ consider\ the\ following\ code:''\n\n\ \ foreach\ name\ \$list\ \{\n\ \ \ \ proc\ \$name\ \{\}\ \"\n\ \ \ \ \ \ #\ Using\ \"\ quotes\ \"\ for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\n\ \ \ \ \ \ return\ \$name\n\ \ \ \ \"\n\ \ \}\n\n''Here\ the\ only\ differences\ are\ the\ extra\ spaces\ around\ the\ `quotes`\ string\ on\ the\ \"commented\ line\"\ (which\ isn't\ one).\ But\ now\ `proc`\ complains\ of\ extra\ arguments.\ Indeed,\ it\ is\ given\ 5\ args\ instead\ of\ the\ expected\ 3:''\n\n\ \ \ *\ ''3rd\ one\ (the\ expected\ body)\ is\ `\"\\n\ \ \ \ #\ Using\ \"`\ (notice\ the\ \"comment\")''\n\ \ \ *\ ''4th\ one\ is\ `\"quotes\"`''\n\ \ \ *\ ''5th\ one\ is\ `\"for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\\n\ \ \ \ return\ \$name\\n\"`''\n\n\n\[KJN\]:\ If\ we\ modify\ Cloverfield\ by\ adding\ rules\ for\ quoted\ strings,\ this\ seems\ to\ solve\ both\ these\ problems,\ but\ adds\ problems\ of\ its\ own:\n\n\ \ \ *\ a\ string\ such\ as\ \"\{\"\ is\ now\ illegal,\ but\ may\ be\ rendered\ \"\\\{\"\n\n''\[FB\]:\ On\ the\ contrary.\ In\ Tcl\ the\ string\ \"\{\"\ is\ legal\ at\ top\ level\ but\ not\ in\ braced\ code\ (e.g.\ in\ a\ proc),\ however\ it\ is\ always\ legal\ in\ Cloverfield.\ That's\ an\ area\ where\ the\ latter\ improves\ the\ overall\ consistency\ of\ the\ language.''\n\n\[KJN\]:\ No,\ the\ string\ is\ legal\ in\ Tcl\ at\ any\ level,\ but\ if\ it\ is\ inside\ braced\ code\ it\ may\ have\ an\ unexpected\ interpretation\ when\ that\ code\ is\ parsed.\ \ It\ is\ legal\ in\ Cloverfield\ too,\ but\ not\ in\ a\ Cloverfield\ that\ is\ modified\ as\ suggested\ above.\n\n''\[FB\]:\ I\ missed\ the\ \"modified\ Cloverfield\"\ part,\ sorry\ for\ the\ confusion.\ Anyway,\ let's\ consider\ the\ following\ code:''\n\n======\n#\ Legal\ in\ both\ Tcl\ and\ Cloverfield.\nset\ v\ \"\{\"\n\n#\ Legal\ in\ Cloverfield,\ but\ illegal\ in\ Tcl\ (requires\ an\ extra\ closing\ brace,\ and\ fails\ when\ evaluated\ anyway).\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ \"\{\"\n\}\n======\n\[KJN\]:\ bracing\ hell\ is\ that\ the\ code\ above\ is\ not\ illegal\ in\ Tcl,\ but\ incomplete:\ as\ you\ said,\ Tcl\ expects\ another\ close\ brace.\ \ If\ Cloverfield\ is\ modified\ as\ I\ have\ suggested\ to\ 'fix'\ comments\ inside\ quoted\ strings,\ then\ the\ code\ above\ becomes\ a\ problem\ -\ I\ think\ the\ modified-Cloverfield\ parser\ should\ interpret\ both\ the\ quotes\ as\ 'open\ quotes'\ and\ fail\ at\ the\ last\ '\}'\ because\ it\ is\ a\ close-brace\ with\ no\ open-brace\ inside\ a\ quoted\ string.\ \ This\ is\ an\ improvement\ over\ Tcl,\ because\ at\ least\ the\ parser\ fails\ close\ to\ the\ problematic\ code,\ and\ not\ at\ the\ end\ of\ the\ file\ when\ it\ cannot\ find\ the\ 'missing'\ close-brace.\ \ Hunting\ for\ missing\ braces\ has\ sometimes\ driven\ me\ to\ drink.\n''\[FB\]:\ But\ why\ and\ how\ exactly\ would\ you\ want\ to\ modify\ the\ Cloverfield\ rules?\ I\ fail\ to\ see\ the\ problem\ it\ tries\ to\ solve.\ Do\ you\ mean\ braces\ should\ also\ be\ balanced\ in\ quoted\ strings?''\n======\n#\ Legal\ in\ both.\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ \"\\\{\"\n\}\n======\n\n''So\ the\ string\ \"\{\"\ cannot\ be\ used\ verbatim\ in\ braced\ expressions\ with\ Tcl,\ it\ needs\ to\ be\ escaped\ as\ \"\\\{\".\ However\ both\ forms\ are\ valid\ in\ Cloverfield\ and\ give\ the\ same\ value\ whatever\ the\ context.''\n\n\ \ \ *\ braced\ text\ must\ have\ matching\ quotes\ as\ well\ as\ matching\ braces,\ making\ the\ string\ \{\"\}\ illegal,\ but\ expressible\ as\ \"\\\"\"\n\n''\[FB\]:\ True.\ And\ the\ same\ for\ all\ words\ that\ start\ with\ `#`,\ such\ as\ `uplevel`\ arguments,\ which\ will\ need\ to\ be\ escaped\ or\ put\ between\ quotes.\ As\ a\ side\ note,\ this\ will\ make\ most\ editors\ happier,\ since\ they\ often\ require\ hashes\ to\ be\ escaped\ in\ order\ to\ avoid\ rendering\ the\ line\ remainder\ as\ comment,\ but\ of\ course\ this\ is\ NOT\ the\ rationale\ for\ this\ change,\ just\ a\ side\ effect.''\n\n''\[KJN\]:\ the\ Cloverfield\ rules\ do\ not\ clearly\ state\ that\ \{\"\}\ is\ illegal,\ but\ this\ follows\ from\ analysis\ of\ the\ rules.\ \ Your\ idea\ of\ writing\ a\ reference\ parser\ is\ a\ good\ one,\ because\ cases\ like\ this\ would\ become\ obvious\ either\ when\ defining\ the\ parser\ or\ when\ running\ the\ test\ cases.''\n\n''\[FB\]:\ Indeed.\ The\ test\ case\ will\ be\ a\ nice\ combination\ of\ test-oriented\ development\ along\ with\ a\ tutorial\ for\ the\ new\ rules.\ Ultimately,\ all\ the\ test\ cases\ will\ be\ fine-tuned\ to\ give\ the\ least\ surprising\ result\ from\ a\ human\ POV,\ and\ could\ provide\ side-by-side\ comparison\ with\ Tcl\ as\ a\ porting\ guide.\ Anyway,\ I\ believe\ that\ existing\ code\ can\ be\ rewritten\ in\ such\ a\ way\ that\ it\ complies\ with\ both\ sets\ of\ rules\ in\ most\ cases\ (as\ with\ the\ above\ example).''\n\n\ \ \ *\ if\ we\ need\ to\ define\ a\ non-code-like\ data\ string,\ we\ will\ have\ to\ use\ either\ quotes\ and\ escapes,\ or\ string\ processing,\ or\ the\ \{data\}\ word\ modifier\ (rule\ 11).\n\nIt\ seems\ to\ me\ that\ Cloverfield\ removes\ the\ pain\ for\ braced\ code,\ but\ increases\ the\ pain\ for\ braced\ data.\n\n''\[FB\]:\ When\ handling\ \"problematic\"\ data,\ the\ use\ of\ \[heredoc\]\ features\ is\ strongly\ encouraged\ in\ most\ languages.\ Problem\ is,\ Tcl\ doesn't\ have\ an\ \[heredoc\]\ feature,\ but\ Cloverfield\ does\ provide\ such\ a\ feature\ with\ `\{data\}`.\ If\ we\ make\ the\ (reasonable\ IMHO)\ assumption\ that\ most\ braced\ expressions\ in\ Tcl\ are\ not\ arbitrary\ data\ but\ code,\ then\ this\ problem\ disappears\ anyway.\ Moreover\ Cloverfield\ provides\ a\ third\ quoting\ rule\ \[\[6\]\]\ using\ parentheses\ that\ combines\ some\ of\ the\ effects\ of\ quotes\ and\ braces\ (and\ paves\ the\ way\ for\ the\ mythical\ `list`\ refactoring).''\n\n----\n***\ Word\ modifiers\ ***\n\[KJN\]\ I'm\ not\ keen\ on\ the\ word\ modifiers\ -\ except\ \{*\}.\ \{*\}\ was\ accepted\ into\ 8.5\ because,\ despite\ introducing\ syntax,\ it\ simplified\ a\ lot\ of\ code.\ \ I\ see\ now\ the\ purpose\ served\ by\ \{data\}\ (providing\ a\ way\ to\ define\ a\ literal\ string,\ since\ Cloverfield\ braces\ no\ longer\ do\ that\ -\ ''\[FB\]:\ Yes\ they\ still\ do,\ only\ brace\ matching\ rules\ change,\ and\ \"comments\"\ are\ not\ stripped\ off\ the\ string\ but\ remain\ part\ of\ it'')\ ,\ but\ the\ other\ word\ modifiers\ in\ Cloverfield\ introduce\ a\ lot\ of\ syntax.\ \ Also,\ in\ some\ cases\ (like\ \{*\})\ the\ modifier\ determines\ what\ is\ done\ with\ the\ word\ after\ substitution,\ while\ in\ other\ cases\ it\ modifies\ the\ substitution\ itself.\ \ I\ think\ this\ adds\ too\ much\ complexity.\n\n\[FB\]:\ The\ merit\ of\ \[Cloverfield\]\ is\ to\ introduce\ the\ concept\ of\ word\ modifier\ to\ solve\ a\ range\ of\ problems\ where\ the\ typical\ solution\ would\ use\ ad-hoc\ techniques\ like\ opaque\ tokens\ (=>\ no\ more\ EIAS)\ or\ string\ manipulations\ (=>\ shimmering).\ For\ example,\ see\ \[Jim\ References\].\ The\ strings\ returned\ by\ `\[\[ref\]\]`\ are\ similar\ to\ Cloverfield's\ word\ modifier,\ with\ the\ difference\ that\ Jim's\ are\ opaque\ tokens\ whereas\ Cloverfield's\ are\ just\ metasyntax\ that\ preserve\ \[EIAS\]\ and\ prevent\ the\ loss\ of\ internal\ rep\ due\ to\ shimmering.\ Another\ example\ is\ `\[null\]`\ and\ TIP\ #185\ \[http://tip.tcl.tk/185\],\ which\ propose\ a\ very\ similar\ syntax.\ Word\ modifiers\ are\ just\ a\ consistent\ syntax\ to\ tag\ words\ with\ special\ meaning.\ Sometimes\ to\ modify\ the\ behavior\ of\ the\ parser\ like\ `\{data`\},\ word\ substitution\ like\ `\{*\}`\ or\ `\{#\}`,\ or\ evaluation\ like\ `\{delay\}`.\ I'm\ afraid\ there\ aren't\ many\ ways\ of\ enhancing\ the\ language\ to\ solve\ these\ problems\ without\ introducing\ a\ lot\ of\ syntax\ or\ breaking\ the\ semantics,\ and\ word\ modifiers\ don't\ introduce\ new\ syntactic\ rules\ but\ rather\ capitalize\ on\ the\ existing\ `\{*\}`.\n\nConcerning\ the\ `\{data\}`\ modifier,\ I've\ just\ come\ across\ an\ example\ that\ illustrates\ a\ strikingly\ similar\ syntax\ enhancement\ to\ \[Jim\]\ (near\ the\ end\ of\ the\ page):\n\n\ set\ v\ \{<<EOT\}\{\n\ blah\ blah...\n\ any\ old\ stuff:\ \$\$\$\ \{\{\{\ \[\[\[\ \\\\\\\ \]\]\]\ etc.\n\ EOT\}\n\nThis\ quoting\ style\ is\ known\ as\ \[heredoc\],\ or\ \[here\ document\],\ so\ it\ seems\ that\ my\ intuition\ was\ right\ (see\ also\ \[Quoting\ Heaven!\]).\ Given\ the\ properties\ of\ the\ Tcl\ parser,\ I'm\ afraid\ there\ isn't\ much\ alternative\ to\ a\ word-modifier-like\ syntax\ enhancement.\ The\ Jim\ patch\ follows\ the\ same\ path\ as\ Cloverfield\ by\ modifying\ the\ brace\ matching\ rules.\n----\n\[KJN\]:\ Is\ Cloverfield\ compatible\ with\ \[EIAS\]?\ ''-\ \[FB\]\ Yes,\ EIAS\ is\ the\ most\ important\ Tcl\ principle\ -''\ If\ so,\ what\ is\ the\ string\ representation\ of\ the\ following\ values?\n\ \ \ 1.\ \{null\}\{any\ word\}\ #\ Rule\ 11\ states\ that\ this\ is\ \"a\ special\ null\ value\ which\ is\ distinct\ from\ any\ other\ value,\ including\ the\ empty\ string\"\n\n''\[FB\]:\ null\ is\ the\ only\ value\ that\ has\ no\ string\ representation,\ which\ means\ that\ it\ can't\ be\ `\[puts\]`,\ or\ escape\ from\ the\ reach\ of\ the\ interpreter.\ On\ this\ point\ it\ resembles\ Smalltalk's\ nil.\ The\ only\ way\ to\ create\ such\ a\ value\ is\ when\ evaluating\ an\ expression\ prefixed\ by\ the\ `\{null\}`\ modifier.\ For\ example,\ the\ string\ `\"1\ 2\ \{null\}3\ 4\"`,\ when\ fed\ to\ `\[list\]`,\ gives\ a\ list\ whose\ 3rd\ element\ is\ null.\ The\ only\ gray\ area\ in\ the\ Tridekalogue\ is\ how\ to\ handle\ side\ effects.\ Does\ `\{null\}`\ totally\ discard\ all\ that\ follows,\ or\ does\ the\ word\ get\ evaluated\ as\ usual,\ but\ its\ value\ replaced\ by\ null?\ I.e.,\ does\ `\{null\}\[\[puts\ foo\]\]`\ outputs\ `foo`?\ Does\ `\{null\}\[\[puts\ \[\[incr\ foo\]\]\]\]`\ increments\ variable\ `foo`?\ Word\ modifiers\ such\ as\ `\{null\}`\ need\ more\ complete\ specifications.''\n\n\ \ \ 2.\ \{meta\ foo\}bar\ #\ this\ value\ has\ data\ \{bar\}\ and\ metadata\ \{foo\}\;\ are\ both\ expressed\ by\ its\ string\ representation?\n\n''\[FB\]:\ the\ metadata\ is\ not\ part\ of\ a\ word's\ string\ representation\;\ `puts\ \{meta\ foo\}bar`\ prints\ `bar`.\ The\ only\ way\ to\ get\ a\ word's\ metadata\ is\ to\ prefix\ it\ with\ `\{meta\}`.\ Metadata\ are\ transient,\ arbitrary\ client\ data\ that\ can\ be\ associated\ with\ words,\ notably\ for\ debugging\ purpose.\ Library\ code\ should\ avoid\ using\ metadata\ and\ leave\ it\ to\ the\ application.''\n\n\ \ \ 3.\ \{delay\}\$bar\ #\ what\ is\ the\ string\ representation\ of\ this\ value\ that\ forces\ substitution\ when\ the\ value\ is\ read?\n\n''\[FB\]:\ Think\ of\ `\{delay\}`\ as\ a\ \"safer\"\ \[future\].\ The\ latter\ are\ objects\ created\ with\ no\ string\ rep,\ whose\ expressions\ are\ evaluated\ when\ the\ string\ rep\ is\ required.\ The\ problem\ is\ that\ string\ rep\ generation\ can\ occur\ in\ unpredictable\ ways\ (e.g\ when\ shimmering).\ Whereas\ `\{delay\}`\ expressions\ are\ only\ evaluated\ when\ the\ word\ is\ explicitly\ accessed.\ So\ the\ string\ rep\ is\ the\ original\ word\ until\ its\ value\ is\ queried.\ For\ example,\ the\ string\ `\"foo\ \{delay\}\[\[bar\]\]\"`,\ when\ fed\ to\ `\[list\]`,\ gives\ a\ list\ where\ the\ second\ element\ is\ a\ delayed\ call\ to\ `bar`.\ It\ can\ be\ turned\ into\ a\ `\[dict\]`\ where\ the\ delayed\ expression\ is\ the\ value\ associated\ with\ `foo`,\ causing\ shimmering,\ but\ without\ evaluating\ the\ delayed\ expression.\ However\ a\ list\ or\ dict\ operation\ on\ this\ element\ will\ do.\ So\ here\ we\ have\ a\ string\ rep\ that\ may\ change\ dynamically\ within\ a\ larger\ string,\ and\ that's\ the\ main\ reason\ for\ using\ \[rope\]s\ as\ the\ primary\ internal\ string\ representation.''\n\n''However,\ like\ with\ `\{null\}`,\ there\ are\ still\ gray\ areas\ in\ the\ spec.\ For\ example,\ does\ `\{delay\}`\ delay\ the\ evaluation\ of\ the\ topmost\ word,\ or\ all\ the\ subwords?\ E.g.\ does\ `\{delay\}\[\[foo\ \[\[bar\]\]\]\]`\ also\ delays\ `\[\[bar\]\]`\ or\ is\ it\ substituted\ in\ the\ delayed\ expression?\ I'd\ favor\ the\ latter.\ Also,\ how\ does\ it\ behave\ with\ variables:\ does\ it\ delay\ the\ variable\ substitutions,\ or\ do\ variables\ have\ to\ contain\ an\ expression\ to\ be\ evaluated\ (à\ la\ `\[subst\]`)?\ And\ the\ same\ for\ other\ substitution\ rules.''\n\n======\nset\ v\ 1\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ 2\n\ \ \ \ return\ \[list\ \{delay\}\[set\ v\]\ \{delay\}\"\$v\"\ \{delay\}\$v\]\n\}\nfoo\ \;\ #\ Should\ it\ return\ \{1\ 1\ 1\},\ \{1\ 2\ 1\}\ or\ \{1\ 2\ 2\}?\nproc\ bar\ \{\}\ \{\n\ \ \ \ set\ e\ \{\[set\ v\]\}\n\ \ \ \ return\ \{delay\}\$e\ \n\}\nbar\ \;\ #\ Should\ it\ return\ \"1\"\ or\ \"\[set\ v\]\"\ ?\n\nset\ w\ 2\nset\ x\ v\nproc\ baz\ \{\}\ \{\n\ \ \ \ set\ x\ w\n\ \ \ \ return\ \{delay\}\[set\ \$x\]\n\}\nbaz\ \;\ #\ Should\ it\ return\ 1\ or\ 2?\n\n\n\n----\n****\ See\ also\ ****\n\ \ \ *\ \[Cloverfield\ -\ Tridekalogue\]\n\ \ \ *\ The\ Reference\ section\ below.\n\n\n----\n***\ References\ ***\n\n\[Cloverfield\]\ references\ using\ the\ `\{ref\}`\ word\ modifier\ is\ a\ solution\ to\ the\ problem\ of\ defining\ and\ serializing\ structures\ with\ circular\ references\ (see\ \[Cloverfield\ -\ Tridekalogue\]).\ Let's\ consider\ a\ tree\ whose\ nodes\ have\ to\ reference\ their\ root\ or\ parent\ node\ in\ some\ way\ (for\ instance,\ a\ DOM\ tree\ from\ XML\ data).\ E.g:\n\n======\nroot\n|\n+-node1\n|\n+-node2\n|\ |\n|\ +-node21\n|\ |\n|\ +-node22\n|\ |\n|\ +-node23\n|\n+-node3\n======\n\nEach\ node\ (including\ the\ root)\ is\ a\ structure\ that\ holds\ a\ value\ (here\ `root`,\ `node1`...),\ a\ list\ of\ children,\ and\ a\ reference\ to\ its\ root\ parent.\ Expressing\ this\ structure\ in\ plain\ Tcl\ gives:\n\n======\nset\ tree\ \{root\ \{\n\ \ \ \ \{node1\ \{\}\}\n\ \ \ \ \{node2\ \{\n\ \ \ \ \ \ \ \ \{node21\ \{\}\}\n\ \ \ \ \ \ \ \ \{node22\ \{\}\}\n\ \ \ \ \ \ \ \ \{node23\ \{\}\}\n\ \ \ \ \}\n\ \ \ \ \{node3\ \{\}\}\n\}\}\n======\n\nThis\ is\ a\ tree\ expressed\ as\ a\ recursive\ list.\ Each\ node\ holds\ 2\ elements:\ a\ name\ and\ a\ list\ of\ children.\ Nodes\ \"know\"\ their\ root\ and\ parent\ implicitly\ because\ they\ are\ part\ of\ the\ same\ Tcl\ variable.\ If\ we\ want\ to\ extract\ a\ given\ node,\ we\ lose\ the\ reference\ information\ so\ we\ have\ to\ retrieve\ this\ metadata\ somewhere.\ Typically\ one\ would\ have\ to\ maintain\ this\ information\ in\ variables,\ with\ some\ in-band\ signaling\ convention,\ or\ with\ specific\ naming\ conventions.\ For\ example,\ we\ could\ store\ the\ names\ of\ the\ root\ and\ parent\ nodes\ in\ the\ node\ structure\ (in-band\ signaling).\ \n\n======\nset\ tree\ \{root\ root\ root\ \{\n\ \ \ \ \{node1\ root\ root\ \{\}\}\n\ \ \ \ \{node2\ root\ root\ \{\n\ \ \ \ \ \ \ \ \{node21\ root\ node2\ \{\}\}\n\ \ \ \ \ \ \ \ \{node22\ root\ node2\ \{\}\}\n\ \ \ \ \ \ \ \ \{node23\ root\ node2\ \{\}\}\n\ \ \ \ \}\n\ \ \ \ \{node3\ root\ root\ \{\}\}\n\}\}\n======\n\nExtracting\ `node22`\ gives:\n\n\{node22\ root\ node2\ \{\}\}\n\nKnowing\ that\ the\ data\ is\ stored\ in\ variable\ `tree`,\ one\ could\ find\ the\ root\ and\ parent\ nodes\ by\ searching\ the\ tree.\ But\ this\ is\ a\ potentially\ expensive\ operation\ and\ needs\ explicit\ coding.\n\nNow\ consider\ the\ same\ structure\ with\ Cloverfield's\ `\{ref\}`\ word\ modifier:\n\n======\nset\ tree\ \{ref\ root\}(root\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \{ref\ node21\}(node21\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \{ref\ node22\}(node22\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ ))\n\ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n)\n======\n\n(Note\ that\ this\ is\ the\ string\ representation\ of\ the\ tree.\ Behind\ the\ curtains,\ the\ structure\ maintains\ pointers\ to\ the\ referenced\ data.)\n\nHere\ we\ chose\ to\ use\ the\ node\ name\ as\ the\ reference\ ID,\ but\ IDs\ are\ arbitrary.\ The\ tree\ is\ now\ expressed\ as\ a\ recursive\ list\ with\ all\ nodes\ tagged\ by\ a\ unique\ reference\ ID.\ Each\ node\ holds\ 4\ elements:\ a\ name,\ a\ reference\ to\ the\ root\ node,\ a\ reference\ to\ the\ parent\ node,\ and\ a\ list\ of\ children.\ This\ string\ representation\ is\ similar\ to\ the\ above\ Tcl\ example\ with\ in-band\ signaling.\ The\ difference\ is\ that\ elements\ 1\ and\ 2\ respectively\ hold\ references\ to\ the\ root\ and\ parent\ nodes,\ whereas\ the\ Tcl\ version\ required\ an\ explicit\ naming\ convention.\n\nSome\ example\ use:\n\n======\n%\ set\ node\ \[lindex\ \$tree\ 3\ 2\ 3\ 2\]\;\ #\ 3's\ are\ for\ each\ node's\ children\ list.\n#\ Note:\ indentation\ and\ inline\ comments\ added\ to\ the\ below\ string\ representation\ for\ clarity.\n\{ref\ node22\}(\n\ \ \ \ #\ Value:\n\ \ \ \ node22\n\ \ \ \ \n\ \ \ \ #\ Root:\ first\ reference\ to\ root\ is\ serialized.\ \n\ \ \ \ \{ref\ root\}(\n\ \ \ \ \ \ \ \ root\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ #\ Subsequent\ references\ to\ root\ aren't\ serialized.\n\ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \n\ \ \ \ \ \ \ \ #\ Children:\n\ \ \ \ \ \ \ \ (\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ Siblings\ of\ node22\ get\ serialized...\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node21\}(node21\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ ...\ but\ not\ node21.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node22\}\{\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ ))\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ )\n\ \ \ \ )\n\ \ \ \ \n\ \ \ \ #\ Parent:\ reference\ was\ already\ serialized\ above.\n\ \ \ \ \{ref\ node2\}\{\}\n\ \ \ \ \n\ \ \ \ #\ Childen:\n\ \ \ \ \{\}\n)\n\n#\ Get\ node\ name.\n%\ lindex\ \$node\ 0\nnode22\n\n#\ Get\ node\ parent\ name.\n%\ lindex\ \$node\ 2\ 0\ \;\ #\ 2\ is\ for\ parent.\nnode2\n\n#\ Get\ siblings.\n%\ lindex\ \$node\ 2\ 3\ \;\ #\ 2\ for\ parent,\ 3\ for\ children.\n#\ Note:\ indentation\ and\ inline\ comments\ added\ to\ the\ below\ string\ representation\ for\ clarity.\n(\n\ \ \ \ #\ First\ sibling\ is\ the\ first\ to\ reference\ root.\n\ \ \ \ \{ref\ node21\}(\n\ \ \ \ \ \ \ \ #\ Value:\n\ \ \ \ \ \ \ \ node21\ \n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ #\ Root:\ first\ reference\ to\ root\ is\ serialized.\ \n\ \ \ \ \ \ \ \ \{ref\ root\}(\n\ \ \ \ \ \ \ \ \ \ \ \ root\n\ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ #\ Subsequent\ references\ to\ root\ aren't\ serialized.\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ #\ Children:\n\ \ \ \ \ \ \ \ \ \ \ \ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ First\ sibling\ is\ currently\ being\ serialized.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node21\}\{\}\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ Subsequent\ siblings\ need\ serialization.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node22\}(node22\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ))\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ )\n\ \ \ \ \ \ \ \ )\n\ \ \ \ )\n\ \ \ \ \n\ \ \ \ #\ Subsequent\ siblings\ were\ already\ serialized.\n\ \ \ \ \{ref\ node22\}\{\}\n\ \ \ \ \{ref\ node23\}\{\}\n)\n\n======\n\nReferenced\ structures\ are\ never\ serialized\ more\ than\ once\ (the\ referenced\ value\ is\ serialized\ at\ the\ first\ occurrence).\ That\ way,\ structures\ with\ circular\ references\ can\ be\ serialized\ in\ finite\ space,\ expressed\ as\ strings,\ and\ compared\ for\ equality.\ For\ a\ simpler\ example,\ consider\ this:\n\n======\nset\ l\ \{ref\ a\}(a\ \{ref\ a\}\{\})\n======\n\nThe\ above\ structure\ is\ a\ list\ whose\ 2nd\ element\ is\ a\ reference\ to\ itself.\ Hence:\n\n======\n%\ lindex\ \$l\ 1\;\ #\ Or\ \$l\[1\]\ using\ Cloverfield\ conventions.\n\{ref\ a\}(a\ \{ref\ a\}\{\})\n%\ expr\ \{\[lindex\ \$l\ 1\]\ ==\ \$l\}\n1\n======\n\n----\nReferences\ can\ be\ combined\ with\ argument\ expansion.\ For\ example:\n\n======\n\{ref\ root\}(a\ \{*\}\{ref\ root\}\{\})\n======\n\ngives\ an\ infinite\ (flat)\ list\ of\ \"a\".\ The\ following\ code\ loops\ indefinitely:\n\n======\nforeach\ v\ \{ref\ root\}(a\ \{*\}\{ref\ root\}\{\})\ \{\n\ \ \ \ puts\ \$v\n\}\n======\n\nThe\ argument\ expansion\ word\ modifier\ must\ be\ placed\ before\ the\ reference\ so\ that\ the\ referenced\ value\ be\ expanded\ in-place\ in\ the\ containing\ list.\ Placing\ `\{*\}`\ after\ `\{ref\}`\ has\ no\ effect\ since\ the\ reference\ word\ modifier\ tags\ a\ single\ word.\n\n----\nNotice\ also\ the\ use\ of\ parentheses\ in\ place\ of\ braces,\ in\ order\ for\ the\ reference\ IDs\ to\ share\ the\ same\ scope.\ For\ example:\n\n======\n\{ref\ a\}(a\ \{ref\ a\}\{\})\ \n\{ref\ a\}\{a\ \{ref\ a\}\{\}\}\n======\n\nIn\ the\ first\ case,\ both\ references\ `a`\ designate\ the\ same\ reference,\ contrary\ to\ the\ second\ case\ because\ words\ are\ not\ evaluated\ at\ the\ same\ time.\ The\ first\ case\ is\ a\ self-referencing\ structure,\ whereas\ the\ second\ is\ equal\ to\ the\ list\ `\{a\ \{\}\}`\ value-wise.\n\nThe\ consequence\ is\ that\ the\ following\ works\ as\ expected:\n\n======\n%\ set\ l1\ (\{ref\ a\}1\ 2\ \{ref\ a\}\{\})\n%\ set\ l2\ (\$l1\ \{ref\ a\}3\ \{ref\ a\}\{\})\n\{\{ref\ a\}1\ 2\ \{ref\ a\}\{\}\}\ \{ref\ a\}3\ \{ref\ a\}\{\}\n#\ The\ above\ equals\ \{\{1\ 2\ 1\}\ 3\ 3\}\ value-wise.\n======\n`l1`'s\ references\ don't\ pollute\ `l2`'s\ unexpectedly.\n\nIn\ summary,\ references\ don't\ cross\ brace\ boundaries,\ because\ braces\ delay\ the\ evaluation\ of\ their\ content.\ That\ is\ the\ main\ reason\ why\ parentheses\ are\ necessary\ for\ references\ to\ work.\n----\n****\ See\ also\ ****\n\n\ \ \ *\ \[Reference\]\n\ \ \ *\ \[TclRef\]\n\ \ \ *\ \[Tcl\ references\ in\ Tcl\]\n\ \ \ *\ \[Jim\ References\]\n\ \ \ *\ \[Abstract\ Data\ Types\]\n----\n\n\[CMcC\]\ I've\ been\ persuaded\ to\ lift\ the\ RecentChanges\ suppression\ of\ Cloverfield,\ which\ I\ have\ done.\ \ I\ still\ think\ it's\ a\ walled\ garden,\ but\ ...\ let\ a\ thousand\ flowers\ bloom.\ \ Please\ don't\ feel\ that\ it\ can't\ be\ refactored,\ should\ people\ want\ to\ do\ that,\ and\ be\ aware\ that\ I'm\ considering\ some\ kind\ of\ repetition\ filtering\ for\ RecentChanges\ which\ would\ preclude\ all\ the\ Cloverfield\ changes\ appearing\ on\ the\ same\ day,\ just\ a\ signal-to-noise\ issue.\ \ But\ above\ all,\ I\ behaved\ inhospitably,\ and\ that's\ not\ good\ for\ a\ wiki.\n\n\[FB\]:\ Thank\ you.\ I\ understand\ your\ concerns\ about\ the\ amount\ of\ noise\ generated\ on\ RecentChanges.\ It\ is\ true\ that\ there\ have\ been\ a\ lot\ of\ Cloverfield-related\ changes\ on\ the\ feed,\ but\ the\ reason\ is\ simply\ that\ the\ project\ has\ just\ started\ and\ that\ I'm\ indexing\ and\ cross-linking\ a\ lot\ of\ information\ on\ the\ Wiki.\ Anyway\ I'll\ going\ to\ start\ moving\ some\ of\ the\ above\ pages\ back\ here\ so\ as\ to\ limit\ the\ impact.\ Rest\ assured\ that\ my\ goal\ is\ not\ to\ hijack\ or\ disrupt\ anything\ but\ to\ generate\ discussion\ and\ push\ Tcl\ forward.\n\n\[CMcC\]\ I\ absolutely\ believe\ that's\ your\ intention,\ and\ I\ regret\ imposing\ my\ will\ on\ the\ way\ you\ were\ articulating\ it.\ \ Don't\ rearrange\ it\ to\ suit\ my\ particular\ aesthetic,\ please.\ \ Carry\ on\ as\ you\ were\ and\ we'll\ let\ the\ wiki\ community\ work\ its\ magic\ on\ the\ content.\ \ I\ overreacted,\ and\ was\ wrong\ to\ do\ so.\n\n\[FB\]:\ Oops,\ too\ late!\ Ah,\ never\ mind.\ Maybe\ it's\ better\ that\ way!\ \ I'll\ recreate\ these\ pages\ only\ if\ this\ one\ really\ gets\ too\ large.\ If\ I\ had\ to\ make\ a\ suggestion\ for\ a\ Wikit\ improvement,\ that\ would\ be\ page\ anchors\ (e.g.\ \[\[Cloverfield#References\]\]\ or\ something\ like\ that).\n----\n!!!!!!\n%|\ \[Category\ Language\]\ |\ \[Category\ Cloverfield\]\ |%\n!!!!!! regexp2} CALL {my render Cloverfield **\ Announcement\ **\nThe\ official\ announcement\ for\ the\ Cloverfield\ project\ can\ be\ found\ there:\ \[Cloverfield\ -\ Announcement\]\n\n----\n**\ Goals\ **\nAlong\ with\ the\ general\ goals\ listed\ in\ the\ above\ announcement,\ here\ are\ a\ few\ more\ specific\ technical\ goals:\n\n***\ Language\ ***\nImprove\ the\ Tcl\ language\ syntax\ on\ several\ points\ to\ address\ common\ criticisms\ as\ well\ as\ implement\ missing\ features.\ For\ example\ :\n\n\ \ \ *\ 'Fix'\ the\ comments\;\n\ \ \ *\ Auto-expand\ the\ first\ word\ of\ a\ command\ recursively.\ This\ will\ simplify\ currying\ and\ can\ give\ great\ results\ if\ namespaces\ become\ regular\ commands\ (spaces\ would\ thus\ become\ a\ valid\ namespace\ separator)\;\n\ \ \ *\ Improve\ variable\ access\ :\ allow\ e.g.\ `\$\$var`,\ and\ subscript\ access\ such\ as\ `var\[\[1\]\]`\ or\ `var(a)`\ along\ with\ interfaces\ (see\ Data\ structures\ below)\;\n\ \ \ *\ Allow\ variable\ references\ using\ the\ syntax\ `\$&var`.\ This\ can\ fill\ the\ gap\ between\ current\ value/reference\ access\ semantics,\ e.g.\ `lindex`\ vs\ `lappend`,\ and\ solve\ many\ mutability\ vs.\ immutability\ problems\;\n\ \ \ *\ Add\ a\ new\ quoting\ rule\ using\ parentheses,\ and\ drop\ the\ `list`\ command\ as\ we\ know\ it.\ For\ example,\ `(a\ b\ \$c)`\ should\ be\ equivalent\ to\ `\[\[list\ a\ b\ \$c\]\]`.\ The\ semantics\ of\ quotes\ and\ braces\ is\ preserved\ (minus\ changes\ needed\ for\ e.g.\ comments).\ Incidentally,\ this\ is\ the\ same\ syntax\ as\ LISP.\n\ \ \ *\ Extend\ the\ metasyntax\ pioneered\ by\ the\ argument\ expansion\ operator.\ This\ is\ the\ most\ controversial\ syntax\ change,\ but\ is\ unfortunately\ needed\ by\ the\ nature\ of\ some\ changes,\ like\ references\ or\ LISP-like\ delayed\ evaluation.\n\ \ \ *\ Define\ a\ syntax\ for\ specifying\ references.\ This\ can\ be\ used\ for\ example\ to\ serialize\ circular\ references,\ or\ keep\ references\ to\ variables\ that\ go\ out\ of\ scope\;\ for\ example,\ `\{ref\ self\}(a\ \{ref\ self\}\{\})`\ specifies\ a\ list\ whose\ second\ element\ points\ to\ its\ parent.\n\nFor\ more\ detailed\ information,\ see\ \[Cloverfield\ -\ Tridekalogue\]\n\n***\ Data\ structures\ ***\nUse\ \[rope\]s\ as\ the\ internal\ string\ representation.\ Ropes\ will\ use\ B-trees\ of\ immutable\ strings.\ This\ will\ give\ fast\ concatenation,\ slicing,\ insertion,\ and\ should\ dramatically\ reduce\ the\ memory\ usage\ and\ data\ copying.\n\nUse\ interfaces\ (à\ la\ \[Feather\])\ instead\ of\ Tcl_Obj.\ This\ should\ eliminate\ most\ cases\ of\ shimmering.\n\n***\ Runtime\ ***\nImplement\ the\ runtime\ on\ existing\ virtual\ machines.\ Primary\ target\ is\ \[LLVM\].\ Secondary\ target\ could\ be\ Java,\ .NET,\ Parrot.\ LLVM\ is\ the\ most\ interesting\ solution\ since\ it\ gives\ access\ to\ JIT\ compiling,\ platform\ independence,\ native\ performances,\ and\ allow\ total\ control\ over\ the\ internal\ model\ (contrary\ to\ JVM).\ Moreover,\ other\ languages\ such\ as\ C\ or\ C++\ are\ already\ supported,\ which\ means\ that\ we\ could\ get\ cross-platform\ \[Critcl\]-like\ features\ for\ free.\n\nTo\ achieve\ the\ goal\ of\ VM\ independence,\ internal\ data\ structures\ should\ be\ sufficiently\ high\ level.\n\nProvide\ a\ VM-less,\ purely\ interpreted\ reference\ platform\ for\ embedded\ and\ small\ footprint\ solutions.\n\nThe\ runtime\ should\ provide\ advanced\ execution\ modes\ such\ as\ coroutines,\ stackless,\ lightweight\ threads,\ etc.\ See\ \[Radical\ reform\ of\ the\ execution\ engine\]\ for\ some\ ideas.\n----\n**\ Related\ information\ **\n\nSee\ \[Cloverfield\ -\ The\ Gathering\]\ for\ all\ other\ pages\ related\ to\ language\ improvement.\n\n----\n**\ General\ Discussion\ **\n\n\[George\ Peter\ Staplin\]:\ Hi\ FB!\ \ I\ think\ you\ have\ some\ good\ ideas.\ \ I've\ read\ some\ of\ your\ code\ for\ \[TkGS\].\ \ \ I'm\ hoping\ that\ you\ can\ get\ developers\ behind\ this\ project,\ and\ it\ doesn't\ become\ moribund.\ \ I\ am\ interested.\ \ Cloverfield\ is\ a\ good\ name,\ and\ I\ think\ it\ gets\ away\ from\ many\ old\ misconceptions\ about\ Tcl.\n\n\[FB\]:\ Thank\ you!\ Yes,\ I\ hope\ Cloverfield\ will\ get\ more\ attention.\ TkGS\ scope\ was\ a\ bit\ too\ narrow\ to\ really\ get\ developers\ on\ the\ project.\ But\ I've\ learned\ a\ lot\ working\ on\ it,\ even\ if\ the\ project\ never\ completed\ due\ to\ lack\ of\ free\ time\ (building\ a\ family\ needs\ a\ lot\ of\ commitment).\ Anyway\ I\ think\ it\ is\ a\ bit\ obsolete\ now,\ since\ most\ of\ the\ work\ involved\ the\ creation\ a\ new\ graphic\ layer,\ and\ I\ feel\ that\ Cairo\ would\ do\ the\ job\ perfectly.\ I\ even\ had\ the\ project\ to\ port\ Tk\ to\ Cairo\ a\ few\ months\ ago,\ but\ given\ the\ success\ of\ \[Tile\]\ I\ came\ to\ the\ conclusion\ that\ Tk\ no\ longer\ needed\ significant\ improvements\ (at\ least\ for\ now),\ whereas\ Tcl\ was\ losing\ ground,\ so\ I\ moved\ on\ to\ what\ became\ Cloverfield.\n\nAbout\ the\ name:\ I\ chose\ Cloverfield\ only\ a\ couple\ of\ days\ ago\ after\ realizing\ that\ the\ date\ for\ the\ announcement\ was\ 18-1-08.\ But\ prior\ to\ that\ I\ made\ a\ list\ of\ possible\ names,\ see:\ \[Cloverfield\ -\ Alternate\ names\]\n\n----\n\[Lars\ H\]:\ Wow,\ I\ think\ there\ isn't\ a\ single\ thing\ on\ that\ list\ that\ doesn't\ strike\ me\ as\ completely\ wrong\ for\ Tcl.\ Fascinating!\ Well,\ as\ long\ as\ you're\ just\ starting\ it\ up\ as\ a\ separate\ project\ I\ suppose\ I\ can\ happily\ ignore\ all\ of\ it…\n\n''\[FB\]:\ I\ don't\ want\ to\ sound\ too\ harsh,\ but\ after\ reading\ your\ contributions\ to\ all\ the\ discussions\ I\ found\ on\ this\ wiki\ regarding\ language\ improvements,\ you\ seem\ to\ be\ very\ conservative\ when\ it\ comes\ to\ anything\ that\ might\ impact\ the\ \[Dodekalogue\].\ Frankly,\ I\ don't\ think\ that\ auto-expansion\ of\ leading\ words\ is\ totally\ un-Tclish,\ given\ that\ the\ same\ suggestion\ have\ been\ made\ by\ several\ reputable\ Tclers\ such\ as\ \[DKF\]\ of\ \[NEM\],\ and\ that\ it\ currently\ requires\ `unknown`\ hacks\ to\ work\ (see\ \[Let\ unknown\ know\]\ and\ \[An\ implicit\ \{*\}\$\ prefix\]).\ And\ the\ use\ of\ parentheses\ has\ been\ debated\ in\ \[Tcl\ 9.0\ WishList\],\ see\ #67.\ I\ understand\ you\ were\ against\ this\ change,\ but\ you\ should\ also\ concede\ that\ this\ change\ would\ greatly\ improve\ readability.''\ \n\n\[Lars\ H\]:\ No,\ it\ would\ not\ noticably\ improve\ readability.\n\n''\[FB\]:\ Let's\ have\ a\ look\ at\ the\ following\ code:''\n\n\ #\ Tcl:\n\ set\ l\ \[list\ \\\n\ \ \ \ \ \[list\ 1\ 2\ \$somevar\]\ \\\n\ \ \ \ \ \[list\ 3\ 4\ \[someproc\]\]\ \\\n\ \]\n\n\ #\ Cloverfield:\n\ set\ l\ (\n\ \ \ \ \ (1\ 2\ \$somevar)\n\ \ \ \ \ (3\ 4\ \[someproc\])\n\ )\n\n''Don't\ you\ agree\ that\ the\ latter\ form\ is\ more\ readable?\ Moreover\ the\ former\ is\ more\ error-prone\ (you\ can\ easily\ forget\ a\ backslash),\ and\ this\ is\ a\ simplistic\ case.\ Removing\ the\ noise\ created\ by\ `list`\ and\ backslashes\ leaves\ only\ meaningful\ data.\ Add\ new-style\ comments\ and\ you\ get\ a\ more\ declarative\ way\ of\ defining\ data\ vs.\ the\ old\ procedural\ style.\ I\ also\ feel\ that\ the\ new\ style\ would\ be\ faster\ to\ parse\ and\ interpret,\ because\ the\ whole\ tree\ is\ now\ a\ single\ word,\ versus\ a\ collection\ of\ subwords\ in\ the\ former\ case\ (and\ this\ is\ a\ very\ important\ condition\ for\ the\ proposed\ reference\ declaration\ syntax).''\n----\n\[LES\]\ Dude,\ if\ this\ entire\ business\ is\ about\ making\ Tcl\ more\ popular\ (and\ it\ looks\ like\ it\ is),\ a\ little\ more\ effort\ spent\ on\ Tk/Tile\ and\ a\ handful\ of\ useful\ and\ good-looking\ desktop\ applications\ would\ probably\ be\ a\ lot\ more\ effective\ than\ any\ sort\ of\ fragmentation.\ Fragmentation\ is\ usually\ a\ very\ good\ strategy\ to\ '''stifle'''\ an\ endeavor.\n\n\[KJN\]\ Better\ desktop\ applications\ would\ clearly\ be\ an\ asset.\ \ The\ \[OLPC\]\ adopted\ \[GTK\]\ as\ its\ main\ graphics\ toolkit,\ even\ though\ Tk\ is\ a\ much\ better\ fit\ for\ a\ resource-constrained\ system\ such\ as\ OLPC\;\ but\ it\ had\ no\ choice,\ because\ GTK\ has\ the\ applications\ (Abiword,\ Gnumeric,\ Firefox...),\ and\ Tk\ does\ not.\n\nHowever,\ it\ is\ always\ worth\ thinking\ about\ what\ we\ would\ like\ Tcl\ to\ become.\ \ Most\ suggestions\ will\ be\ explored\ and\ eventually\ rejected\ (see\ pages\ on\ this\ Wiki\ for\ many\ examples)\;\ a\ few\ will\ be\ adopted,\ after\ lengthy\ debate.\ \ Fragmentation\ has\ not\ occurred\ in\ the\ past\ (except\ for\ a\ few\ brave\ souls\ who\ still\ use\ Tcl\ 7\ or\ even\ 6\ because\ of\ their\ smaller\ footprint)\ -\ there\ aren't\ enough\ of\ us\ to\ maintain\ two\ major\ codebases.\n\n----\n'''[email protected]''':\ When\ casting\ about\ for\ a\ language\ that\ isn't\ object\ orientated,\ I\ happened\ upon\ Tcl.\ \ Apart\ from\ perl,\ which\ also\ seems\ to\ have\ become\ frozen\ in\ time,\ this\ seems\ to\ be\ the\ only\ one.\ \ But\ it's\ like\ stepping\ back\ in\ time.\ \ As\ far\ as\ I\ can\ see,\ there's\ one\ book\ that's\ modern\ (2004)\ and\ all\ the\ rest\ are\ from\ about\ 1997.\ \ I\ presume\ when\ they\ say\ they\ work\ with\ windows\ they\ mean\ 98?\ \ There\ doesn't\ seem\ to\ be\ an\ equivalent\ of\ CPAN,\ eggs,\ gems\ either,\ or\ if\ there\ is,\ it's\ not\ obvious.\ \ But\ that\ goes\ for\ the\ whole\ language,\ not\ obvious.\ \ It's\ like\ its\ pitched\ a\ C\ programmers\ and\ that's\ it.\ \ The\ only\ thing\ that\ will\ work\ is\ a\ compelling\ application\ that\ takes\ a\ share\ of\ mind.\ \ Other\ than\ that\ it\ will\ remain\ an\ overlooked\ language.\ \ It's\ problems\ are,\ no\ books,\ no\ tutorials,\ no\ obvious\ way\ to\ do\ things.\ Perhaps\ it's\ had\ its\ day,\ legacy\ only,\ isn't\ that\ the\ way\ of\ it?\n\n----\n**\ Specific\ subtopics\ **\n\n***\ Data\ structures\ ***\n****\ Ropes\ ****\n\n\[DKF\]:\ Experience\ with\ strings-implemented-as-trees\ in\ the\ past\ makes\ me\ point\ out\ that\ you'd\ better\ make\ sure\ that\ you\ take\ care\ to\ keep\ the\ trees\ balanced.\ Otherwise\ you'll\ have\ terrible\ performance.\ And\ using\ C\ arrays\ of\ characters\ seems\ to\ actually\ work\ quite\ well\ in\ practice...\n\n\[FB\]:\ flat\ strings\ (ie\ C\ arrays\ of\ chars)\ give\ good\ performances\ in\ Tcl's\ current\ context.\ Object\ sharing,\ COW,\ the\ lack\ of\ references,\ and\ the\ impossibility\ to\ build\ circular\ structures,\ all\ these\ factors\ suit\ flat\ strings\ perfectly.\ However,\ when\ you\ introduce\ references\ and\ mutability,\ you\ cannot\ use\ COW\ semantics\ anymore,\ because\ changing\ an\ object's\ value\ implies\ invalidating\ the\ string\ rep\ of\ all\ objects\ that\ reference\ it.\ This\ can\ represent\ a\ huge\ performance\ hit,\ as\ data\ sharing\ are\ obviously\ more\ likely\ with\ languages\ that\ allow\ references.\ With\ \[rope\]\ structures,\ you\ only\ have\ to\ invalidate\ the\ substring\ that\ has\ changed,\ by\ rebuilding\ the\ tree\ (or\ one\ of\ its\ leaves\ in\ the\ simplest\ cases).\n\nMoreover\ some\ platforms\ like\ Java\ only\ provide\ immutable\ strings,\ and\ allowing\ string\ mutability\ implies\ a\ huge\ performance\ hit.\ This\ can\ be\ a\ serious\ problem\ if\ we\ want\ to\ implement\ a\ Tcl\ interpreter\ over\ the\ JVM.\ In\ this\ case,\ a\ \[rope\]\ structure\ can\ be\ modified\ but\ the\ underlying\ data\ is\ stored\ in\ immutable\ string.\n\nYou\ can\ read\ the\ following\ paper\ for\ more\ info\ on\ a\ real\ world\ rope\ implementation,\ especially\ section\ 'C\ Cords':\ http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol25/issue12/spe986.pdf\n----\n****\ See\ also\ ****\n\ \ \ *\ \[Tcl9\ and\ annotated\ strings\].\n\n----\n***\ Comments\ ***\n\[FB\]:\ There\ is\ a\ slight\ misunderstanding\ about\ comments\ in\ Cloverfield.\ The\ new\ ''word\ comment''\ `\{#\}`\ syntax\ is\ not\ at\ all\ meant\ to\ replace\ the\ existing\ syntax,\ but\ to\ complement\ it.\ Besides,\ \"fixing\"\ the\ comment\ only\ involves\ changing\ the\ way\ braces\ are\ matched\ and\ allowing\ them\ at\ the\ beginning\ of\ words,\ so\ that\ comments\ work\ in\ a\ less\ surprising\ way.\ For\ example:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ #\ The\ following\ works\ as\ expected:\ the\ close\ brace\ after\ true\ doesn't\ close\ the\ proc.\n\ \ \ \ \ #if\ \{true\}\ \{\n\ \ \ \ \ if\ \{false\}\ \{\n\ \ \ \ \ \ \ \ \ switch\ \$v\ \{\n\ \ \ \ \ \ \ \ \ \ \ \ \ #\ This\ is\ a\ comment.\n\ \ \ \ \ \ \ \ \ \ \ \ \ default\ \{\n\ \ \ \ \ \ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \ \ \ \ \}\n\ \ \ \ \ \}\n\n\ \ \ \ \ \{#\}\{This\ comments\ out\ a\ whole\ block\ of\ code\n\ \ \ \ \ someproc\ \$somevar\n\ \ \ \ \ return\ \[foo\]\ #\ Will\ loop\ forever!\ (notice\ the\ lack\ of\ semicolon)\n\ \ \ \ \ \}\n\ \ \ \ \ return\n\ \}\n\n\[AMG\]:\ Regarding\ the\ #if\ comment:\ I\ see\ rule\ \[\[5\]\]\ causes\ Cloverfield\ to\ treat\ all\ braces\ on\ that\ line\ the\ same\ way\ Tcl\ treats\ braces\ preceded\ by\ backslashes.\ \ In\ a\ compiler\ I\ wrote\ that\ accepts\ a\ Tcl-like\ language,\ in\ order\ to\ find\ the\ end\ of\ a\ brace-quoted\ word,\ I\ count\ opening\ and\ closing\ braces\;\ backslashed\ braces\ do\ not\ contribute\ to\ the\ brace\ count.\n\nI\ see\ that\ you\ also\ updated\ the\ brace-counting\ rule\ to\ skip\ braces\ contained\ inside\ double\ quotes.\ \ You\ also\ skip\ braces\ quoted\ using\ the\ raw\ data\ word\ modifier,\ which\ is\ pretty\ much\ a\ new\ thing\ to\ this\ Wiki\;\ I'm\ not\ prepared\ to\ discuss\ it\ yet!\n\n''\[FB\]:\ In\ short,\ `\{data\}`\ is\ roughly\ equivalent\ to\ XML's\ CDATA.\ The\ goal\ is\ to\ ease\ the\ inclusion\ of\ foreign\ data\ into\ Tcl\ to\ improve\ its\ status\ as\ a\ glue\ language.\ Mixed\ with\ \[Critcl\]\ and\ \[tcc\]-like\ features\ it\ would\ simplify\ its\ use\ as\ a\ \[Scripted\ compiler\].\ Currently,\ when\ defining\ data\ in\ a\ foreign\ format,\ you\ have\ to\ properly\ quote\ the\ significant\ characters,\ which\ can\ be\ tedious\ and\ lead\ to\ \[Quoting\ hell\].''\n\nIn\ Cloverfield,\ the\ following\ code\ works\ fine,\ but\ in\ Tcl\ it\ has\ a\ mismatched\ brace:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ puts\ \"\}\"\n\ \}\n\nIn\ Tcl,\ it's\ weird\ (but\ understandable)\ that\ the\ following\ code:\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ puts\ -nonewline\ \"digits\ =\ \{\"\n\ \ \ \ \ for\ \{set\ x\ 0\}\ \{\$x\ <\ 10\}\ \{incr\ x\}\ \{\n\ \ \ \ \ \ \ \ \ puts\ -nonewline\ \"\ \$x\"\n\ \ \ \ \ \}\n\ \ \ \ \ puts\ \"\ \}\"\n\ \}\n\nworks\ great\ unless\ I\ delete\ either\ the\ first\ or\ the\ last\ puts.\ \ But\ in\ Cloverfield,\ it's\ fine.\ \ Am\ I\ understanding\ that\ right?\n\n''\[FB\]:\ Yes.\ I've\ tried\ to\ remove\ the\ localities\ of\ the\ brace\ and\ comment\ syntaxes\ so\ that\ they\ work\ in\ the\ least\ astonishing\ way.''\n----\n\[Lars\ H\]:\ I'm\ a\ bit\ surprised,\ though,\ that\ you\ choose\ to\ label\ \[\{expand\}\]-style\ syntax\ as\ the\ most\ controversial\ part\ —\ I\ quite\ agree\ it\ would\ be\ a\ technical\ necessity\ after\ dropping\ \[Everything\ Is\ A\ String\]\ —\ but\ perhaps\ this\ just\ means\ that\ it\ is\ the\ thing\ that\ isn't\ directly\ borrowed\ from\ some\ other\ language.\n\n''\[FB\]:\ Well,\ I\ felt\ this\ was\ controversial\ wrt.\ the\ debate\ that\ preceded\ the\ adoption\ of\ this\ syntax\ change.\ Many\ Tclers\ were\ concerned\ that\ this\ would\ open\ a\ pandora\ box.\ Regarding\ my\ suggestions,\ I\ tried\ to\ limit\ them\ to\ cases\ where\ no\ alternative\ was\ viable,\ i.e.\ when\ the\ changes\ interfered\ with\ the\ way\ Tcl\ interprets\ the\ data.\ I\ think\ I\ managed\ to\ get\ fair\ compromises.\ However\ dropping\ \[EIAS\]\ is\ totally\ out\ of\ question,\ because\ it\ is\ at\ the\ heart\ of\ the\ Tcl\ way.\ On\ the\ contrary,\ if\ you\ re-read\ my\ suggestions\ carefully,''\n—\ \[Lars\ H\]:\ I\ can't\ say\ I\ care\ enough\ to\ bother.\ —\n''you'll\ see\ that\ I\ took\ great\ care\ to\ enforce\ this\ principle\ to\ solve\ some\ hairy\ problems\ (e.g.\ the\ representation\ of\ circular\ structures\ and\ references).''\n—\ \[Lars\ H\]:\ IMNSHO\ references\ are\ the\ data\ structure\ equivalent\ of\ assembly\ language,\ and\ I'm\ glad\ Tcl\ frees\ me\ of\ the\ dangers\ inherent\ in\ these.\ —\n\nAlso,\ I'm\ a\ bit\ surprised\ about\ the\ way\ in\ which\ you\ propose\ to\ \"fix\"\ comments\;\ I\ can't\ recall\ ever\ seeing\ any\ requests\ for\ ''comment\ words''\ in\ commands,\ and\ it\ is\ already\ perfectly\ possible\ to\ put\ comments\ even\ ''inside''\ words,\ using\ command\ substitution:\n\ \ proc\ \\#\ \{args\}\ \{\}\ \;\ #\ No-op\ command,\ for\ comments.\n\ \ \$w\[\\#\ \{That's\ the\ window\ name\}\].toolbar\[\\#\ \{This\ is\ a\ frame\}\].fire\[\\#\ \{The\ actual\ button\ widget\}\]\ configure\ -repeatinterval\[\\#\ \{Delays\ in\ auto\ mode\}\]\ 10\[\\#\ \{A\ veritable\ machine\ gun,\ this\ button\;\ 6000\ shots\ per\ minute\}\]\n\n''(\[MS\]\ hopes\ you\ don't\ have\ a\ comment\ like\ '\[\[exec\ rm\ -rf\ ~\]\]')''\n\n''\[FB\]:\ have\ you\ ever\ had\ to\ explain\ to\ a\ newcomer\ why\ the\ following\ code:''\n\n\ #if\ \{true\}\ \{\n\ if\ \{false\}\ \{\n\ \ \ \ \ puts\ something\ \n\ \}\n\n''works\ in\ an\ interactive\ shell\ but\ not\ from\ source\ or\ inside\ a\ proc?\ And\ why\ comments\ inside\ `switch`\ blocks\ sometimes\ don't\ work,\ or\ give\ unexpected\ results?''\n—\ \[Lars\ H\]:\ This\ is\ really\ a\ \[RTFM\].\ Being\ a\ teacher,\ I\ would\ suggest\ the\ beginner\ to\ (under\ supervision)\ apply\ the\ \[dodekalogue\]\ to\ the\ scripts\ in\ question.\ It's\ not\ a\ difficult\ exercise,\ but\ quite\ instructive.\ —\n''To\ be\ successful\ Tcl\ needs\ to\ follow\ the\ principle\ of\ least\ astonishment\ whenever\ possible.''\n—\ \[Lars\ H\]:\ On\ the\ contrary,\ most\ of\ Tcl's\ perceived\ defects\ are\ the\ result\ of\ literates\ in\ other\ languages\ not\ being\ astonished\ enough\ by\ the\ specific\ nature\ of\ Tcl\ that\ they\ learn\ the\ language\ properly,\ and\ instead\ struggle\ with\ analogies\ to\ other\ languages\;\ in\ this\ case\ that\ braces\ work\ as\ in\ \[C\]\ ''et\ al.''\ —\n\n''To\ be\ more\ specific,\ the\ problem\ is\ not\ really\ to\ \"fix\"\ comments\ but\ to\ \"fix\"\ the\ way\ braces\ are\ matched.\ Hence\ the\ changes\ I\ proposed\ to\ brace\ matching.\ The\ ''comment\ word''\ thing\ is\ needed\ for\ `switch`-like\ cases.\ But\ my\ proposal\ takes\ great\ care\ not\ to\ break\ the\ \[EIAS\]\ principle.\ OTOH,\ I\ don't\ find\ the\ way\ you\ choose\ to\ implement\ comments\ in\ words\ to\ be\ neither\ Tclish\ nor\ readable\ (let\ alone\ dangerous).''\n—\ \[Lars\ H\]:\ Not\ according\ to\ your\ understanding\ of\ Tcl,\ you\ mean?\ (My\ experience\ only\ goes\ back\ to\ Tcl\ 7,\ but\ I'm\ pretty\ sure\ this\ has\ been\ possible\ much\ earlier\ than\ that,\ so\ it's\ implicit\ quite\ deeply\ in\ the\ core\ of\ the\ language.)\ I'd\ be\ the\ first\ to\ admit\ that\ it\ is\ '''near-unreadable''',\ but\ `\[\[\\#\ \{...\}\]\]`\ isn't\ much\ worse\ than\ `\{#\}\{...\}`\ in\ that\ department,\ and\ it\ was\ rather\ this\ that\ was\ the\ point:\ how\ did\ this\ \"fix\"\ anything?\ —\n\n(If\ I\ recall\ correctly,\ \[dkf\]\ has\ mentioned\ in\ discussions\ of\ the\ \[K\]\ combinator\ that\ \$somevar\[\[command-returning-empty-string\]\]\ does\ not\ even\ cause\ shimmering\ in\ Tcl\ 8.5,\ hence\ no\ quality\ degradation\ of\ code.)\ \nAlternatively,\ one\ can\ do\ that\ with\ ordinary\ comments,\ provided\ one\ inserts\ the\ necessary\ newlines:\n\ \ \$w\[#\ That's\ the\ window\ name\n\ \ \].toolbar\[#\ This\ is\ a\ frame\n\ \ \].fire\[#\ The\ actual\ button\ widget\n\ \ \]\ configure\ -repeatinterval\[#\ Delays\ in\ auto\ mode\n\ \ \]\ 10\[#\ A\ veritable\ machine\ gun,\ this\ button\;\ 6000\ shots\ per\ minute\n\ \ \]\nBut\ perhaps\ the\ point\ is\ rather\ to\ allow\ comments\ in\ lists\ written\ as\ strings?\ ''That''\ I\ would\ often\ have\ found\ useful,\ but\ Cloverfield\ rather\ seems\ to\ turn\ away\ from\ this\ practice.\n\n''\[FB\]:\ on\ the\ contrary,\ that's\ exactly\ what\ Cloverfield\ proposes.\ The\ idea\ is\ to\ modify\ the\ way\ comments\ are\ parsed\ in\ braced\ strings.\ For\ example\ :''\n\n\ #\ Tcl:\n\ set\ v\ \{#\ The\ next\ brace\ close\ the\ string\ \}\n\ set\ v\ \{\"\ The\ next\ brace\ close\ the\ string\ \}\n\ set\ v\ \{#\ This\ isn't\ a\ comment\}\ \ #\ and\ this\ is\ just\ excess\ arguments\n\ set\ v\ \{#\ This\ isn't\ a\ comment\}\;\ #\ but\ this\ is\n\n\ #\ Cloverfield:\n\ set\ v\ \{#\ The\ next\ brace\ doesn't\ close\ the\ string\ \}\n\ \ \ \ \ \ \ \ \"neither\ does\ this\ one\ \}\"\n\ \ \ \ \ \ \ \ but\ this\ one\ does\ \}\n\ set\ v\ \{\"#\ This\ is\ not\ a\ comment\"\}\n\ set\ v\ \{\\#\ Neither\ is\ this\}\ \ \ #\ but\ this\ is!\n\n''This\ introduces\ an\ incompatibility\ but\ at\ the\ benefit\ of\ a\ less\ surprising\ behavior.\ But\ EIAS\ is\ preserved,\ as\ the\ comment\ is\ not\ stripped\ off\ the\ string,\ but\ only\ alter\ the\ way\ braces\ are\ matched.''\n\n\[AMG\]:\ It\ is\ my\ understanding\ from\ \[Cloverfield\ -\ Tridekalogue\]\ that\ Cloverfield\ comments\ are\ words\ preceded\ by\ the\ \{#\}\ modifier,\ which\ isn't\ represented\ in\ the\ above\ example.\ \ Oh\ wait,\ I\ see\ what\ you're\ getting\ at.\ \ You\ (\[FB\])\ also\ changed\ the\ way\ \"line\ comments\"\ work.\ \ Whereas\ Tcl\ only\ recognizes\ a\ comment\ when\ #\ appears\ as\ the\ first\ character\ of\ the\ first\ word\ of\ a\ command,\ Cloverfield\ recognizes\ a\ comment\ wherever\ #\ is\ the\ first\ character\ of\ any\ word.\ \ Watch\ out\ for\ \[uplevel\]!\ :^)\n\n''—\ \[Lars\ H\]:\ Ah,\ I\ missed\ that\;\ saw\ only\ the\ \{#\}\ style\ of\ comment.\ OK,\ I\ concede\ that's\ more\ a\ kind\ of\ \"fixing\"\ that\ I\ can\ imagine\ having\ been\ requested.\ Still\ doesn't\ lead\ to\ the\ behaviour\ shown\ in\ the\ first\ example\ above,\ but\ there\ could\ be\ other\ rules\ yet\ on\ commenting\ lurking\ in\ that\ tridekalogue.\ However,\ I\ don't\ care\ enough\ to\ look.\ —''\ \[FB\]:\ Exactly.\ If\ you\ re-read\ the\ Tridekalogue,\ you'll\ see\ that\ \"fixing\"\ the\ comments\ only\ need\ slight\ amendments\ to\ the\ existing\ Dodekalogue:\ rule\ \[\[6\]\]\ becomes\ \[\[5\]\]\ and\ changes\ the\ way\ braces\ are\ matched,\ and\ rule\ \[\[10\]\]\ allows\ the\ hash\ character\ wherever\ the\ beginning\ of\ a\ word\ is\ expected.\ And\ your\ code\ above\ would\ still\ work\ as\ expected.\ —\n\n''\[FB\]:\ Yes.\ Changes\ to\ line\ comments\ are\ necessary\ to\ make\ them\ work\ in\ the\ least\ surprising\ way.\ To\ do\ so\ #\ must\ be\ allowed\ as\ the\ first\ char\ of\ a\ word,\ to\ allow\ for\ in-list\ comments\ (case\ in\ point:\ `switch`).\ Unfortunately\ `uplevel`\ and\ friends\ are\ collateral\ victims\ of\ this\ choice\ (but\ URL\ fragments\ are\ not),\ as\ they\ will\ now\ need\ proper\ quoting\ of\ the\ #,\ for\ example\ with\ backslash.\ Which\ many\ editors\ do\ automatically\ anyway\ (because\ their\ Tcl\ parser\ usually\ fail\ to\ recognize\ comments\ properly,\ QED),\ and\ which\ is\ pretty\ harmless\ compared\ to\ the\ potential\ gain.\ But\ this\ single\ change\ is\ sufficient\ to\ turn\ Cloverfield\ into\ a\ distinct\ language\ because\ it\ impacts\ one\ of\ the\ fundamental\ rules\ of\ the\ \[Dodekalogue\].''\n\n''The\ new\ line\ comment\ rules\ allow\ the\ following\ code:''\n\n\ dict\ create\ \{\n\ \ \ \ \ FirstName\ John\n\ \ \ \ \ LastName\ Smith\n\ \ \ \ \ DateOfBirth\ 1/18/08\ #\ In\ mm/dd/yy\ format\n\ \}\n\n''As\ for\ the\ so-called\ \"word\ comment\",\ I\ think\ it\ is\ a\ bit\ of\ a\ misnomer,\ but\ chosen\ for\ overall\ consistency\ of\ the\ meta-syntax\ (''word\ modifiers'').\ The\ goal\ is\ not\ to\ allow\ the\ commenting\ of\ individual\ words\ (which\ seem\ pretty\ useless),\ but\ rather\ to\ make\ comments\ be\ recognized\ as\ individual\ words\ which\ are\ subsequently\ ignored.\ The\ typical\ use\ case\ for\ word\ comments\ would\ actual\ be\ block\ commenting.\ See\ below\ for\ examples.''\n\nBut\ for\ the\ sake\ of\ completeness,\ here's\ (what\ I\ think\ is)\ a\ Cloverfield\ \{#\}\ comment:\n\n\ set\ v\ \{#\}\{This\ is\ a\ comment\}\ value\n\nI\ guess\ the\ following\ is\ the\ Tcl\ 8.5\ analog\ for\ \"in-line\"\ comments.\ \ It's\ quite\ similar\ to\ something\ proposed\ above,\ except\ it\ doesn't\ work\ by\ appending\ empty\ string\ to\ existing\ words.\ \ It\ uses\ \[\{*\}\]\ to\ produce\ zero\ words,\ which\ makes\ it\ very\ much\ like\ Cloverfield's\ \{#\}.\n\n\ proc\ #\ \{comment\}\ \{\}\n\ set\ v\ \{*\}\[\\#\ \{This\ is\ a\ comment\}\]\ value\n\nIn\ an\ attempt\ to\ defang\ \[\[exec\ rm\ -rf\ ~\]\],\ \ I\ encourage\ the\ caller\ to\ brace\ the\ comment\ text.\ \ I\ do\ this\ by\ making\ \[\[#\]\]\ only\ accept\ one\ argument.\n\n''\[FB\]:\ exactly,\ the\ only\ difference\ being\ the\ substitution\ rules,\ as\ `\{#\}`\ would\ skip\ all\ the\ substitution\ phase.''\n\nI\ also\ added\ to\ your\ above\ comment\ example.\ \ I\ hope\ you\ don't\ mind.\n\n''\[FB\]:\ I've\ added\ what\ I\ think\ was\ a\ missing\ semicolon\ before\ the\ hash\ in\ the\ last\ Tcl\ example.\ Back\ to\ word\ comments,\ here\ is\ an\ example\ of\ block\ commenting.\ Note\ that\ braces\ must\ be\ properly\ balanced\ inside\ comment\ blocks,\ as\ the\ commented\ words\ must\ follow\ all\ formatting\ rules.''\n\n\ proc\ foo\ \{\}\ \{\n\ \ \ \ \ \{#\}\{This\ is\ a\ comment\n\ \ \ \ \ that\ spans\ multiple\ lines.\n\ \ \ \ \ Regular\ parsing\ rules\ apply,\ e.g.\ #\}\ this\ brace\ doesn't\ close\ the\ block\n\ \ \ \ \ but\ this\ one\ does\}\n\n\ \ \ \ \ #\ In\ the\ following\ code\ the\ call\ to\ bar\ was\ commented\ out\ and\ replaced\ by\ baz.\n\ \ \ \ \ #\ Typical\ use\ case:\ debugging\ sessions.\n\ \ \ \ \ return\ \{#\}\[bar\]\ \[baz\]\n\ \}\n\n----\n\[KJN\]\ In\ the\ example\ above,\ I\ can\ see\ what\ Cloverfield\ is\ trying\ to\ do\ with\n===\n\ set\ v\ \{#\ The\ next\ brace\ doesn't\ close\ the\ string\ \}\n\ \ \ \ \ \ \ \ \"neither\ does\ this\ one\ \}\"\n\ \ \ \ \ \ \ \ but\ this\ one\ does\ \}\n===\nand\ it\ is\ more\ appealing\ than\ Tcl\ if\ the\ braced\ quantity\ is\ code\;\ but\ if\ the\ braced\ quantity\ is\ data,\ this\ is\ like\ allowing\ a\ comment\ inside\ a\ quoted\ string,\ which\ is\ a\ bit\ painful.\n\nTcl\ has\ an\ intrinsic\ problem,\ which\ is:\n\ \ \ 1.\ We\ uses\ braces\ to\ delimit\ a\ data\ \"word\"\n\ \ \ 2.\ Often\ that\ data\ is\ to\ be\ interpreted\ as\ code\n\ \ \ 3.\ The\ flexibility\ of\ Tcl\ means\ that\ when\ a\ braced\ word\ is\ first\ parsed\ we\ cannot\ in\ general\ know\ whether\ the\ contents\ are\ intended\ to\ be\ executed\ as\ code\n\nI\ don't\ think\ the\ comments/bracing\ problem\ is\ fixable:\ unless\ you\ throw\ away\ the\ power\ of\ Tcl,\ or\ add\ complexity,\ all\ you\ can\ do\ is\ swap\ one\ kind\ of\ pain\ for\ another.\ \ In\ this\ case,\ the\ suggested\ parsing\ rules\ for\ Cloverfield\ remove\ the\ pain\ from\ braced\ code,\ but\ they\ transfer\ it\ to\ braced\ data:\ sometimes\ the\ programmer\ will\ want\ to\ brace\ an\ arbitrary\ data\ string\ which\ is\ to\ be\ interpreted\ verbatim,\ without\ \"comments\".\n\n''\[FB\]:\ In\ this\ case\ you\ may\ want\ to\ use\ Cloverfield's\ raw\ data\ word\ modifier\ `\{data\}`,\ which\ is\ designed\ to\ allow\ the\ inclusion\ of\ arbitrary\ data\ (see\ also\ my\ comments\ higher\ on\ this\ page).\ That\ way\ you\ can\ have\ your\ cake\ (sensible\ brace\ parsing)\ and\ eat\ it\ (eliminate\ quoting\ hell)\ :''\n\n\ set\ s\ \{data\}SomeArbitraryTag\ the\ rest\ is\ ignored\ \$\}\[\{\"\\\n\ #include\ <stdio.h>\n\ int\ main()\ \{\n\ \ \ \ const\ char\ string\[\]\ =\ \"\$\\\[\}\ \{data\}SomeOtherTagWontMatchTheAboveOne\"\;\n\ \ \ \ return\ 0\;\n\ \}\n\ this\ is\ also\ ignored\ \$\}\[\{\"\\\ until\ the\ tag\ SomeArbitraryTag\ #\ here\ we're\ back\ to\ the\ interpreter.\n\n''Basically\ you\ can\ use\ the\ `\{data\}`\ modifier\ to\ enclose\ arbitrary\ data\ between\ a\ user-defined\ tag,\ in\ a\ similar\ way\ to\ MIME\ multi-part\ messages.\ The\ exact\ syntax\ is\ not\ final,\ but\ the\ concept\ is\ powerful.\ The\ goal\ of\ Cloverfield\ is\ obviously\ not\ to\ add,\ but\ remove\ complexity,\ moreover\ only\ a\ global\ proposal\ could\ address\ all\ these\ issues\ altogether\;\ each\ proposal\ taken\ individually\ would\ only\ provide\ marginal\ gain,\ if\ any.''\n\n''This\ concept\ is\ known\ as\ \[here\ document\]\ or\ \[heredoc\]\ in\ other\ languages\ such\ as\ \[Perl\]\ or\ \[Python\].''\n\n\n\[KJN\]:\ Here's\ a\ suggestion\ that\ adds\ complexity,\ and\ probably\ has\ more\ negatives\ than\ positives:\ \ interpret\ braces\ with\ the\ Cloverfield\ rules\;\ but\ also\ allow\ «guillemets/chevrons»\ to\ delimit\ words\ using\ the\ existing\ Tcl\ rules\ for\ braces.\n\nIn\ all\ cases,\ the\ Tcl\ idea\ of\ words\ is\ preserved\;\ but\ programmers\ will\ be\ discouraged\ from\ thinking\ of\ braced\ text\ as\ a\ \{quoted\ string\},\ and\ will\ still\ have\ a\ mechanism\ for\ quoting\ literal\ strings\ without\ substitution.\n\nNaturally\ «chevrons»\ are\ not\ as\ pleasant\ as\ ASCII\ delimiters,\ but\ I\ think\ we\ have\ run\ out\ of\ suitable\ ASCII\ codes,\ except\ possibly\ `these'\n\n''\[FB\]:\ I'd\ rather\ suggest\ that\ chevrons\ (or\ whatever)\ remove\ all\ Tcl-sensitive\ syntax\ and\ take\ verbatim\ data.\ In\ this\ case\ their\ behavior\ is\ close\ to\ my\ proposed\ `\{data\}`.\ The\ problem\ is\ that\ there\ must\ be\ a\ way\ to\ properly\ escape\ existing\ chevrons\ in\ the\ included\ data\ (=>\ quoting\ hell),\ notwithstanding\ the\ fact\ that\ they\ are\ hard\ to\ input\ on\ regular\ keyboards.\ But\ conceptually\ both\ ideas\ are\ similar.''\n\n\[KJN\]:\ I\ don't\ like\ the\ word\ modifiers\ very\ much\ (''see\ Word\ modifiers\ section\ below'').\ \ How\ many\ rules\ could\ you\ ''remove''\ from\ Cloverfield,\ and\ still\ fix\ the\ problems\ that\ you\ want\ to?\n\n''\[FB\]:\ regarding\ comments\ and\ literal\ strings,\ rewriting\ the\ brace\ matching\ rules\ and\ allowing\ `\{data\}`-like\ syntax\ is\ all\ that's\ needed.\ The\ other\ rules\ are\ orthogonal\ and\ target\ other\ issues.\ Anyway\ the\ two\ previous\ enhancements\ are\ incompatible\ changes\ and\ need\ a\ major\ version\ bump.\ But\ the\ merit\ of\ Cloverfield\ is\ to\ introduce\ the\ concept\ of\ word\ modifier\ to\ solve\ a\ range\ of\ problems\ where\ the\ typical\ solution\ would\ use\ ad-hoc\ techniques\ like\ opaque\ tokens\ (=>\ no\ more\ EIAS)\ or\ string\ manipulations\ (=>\ shimmering).''\n\n''\[FB\]:\ Moved\ word-modifier-related\ discussion\ to\ Word\ modifiers\ section\ below''\n\n\[KJN\]:\ If\ rules\ for\ chevrons\ correspond\ to\ those\ for\ Tcl\ braces,\ then\ chevrons\ would\ enclose\ verbatim\ data,\ and\ unmatched\ chevrons\ would\ have\ to\ be\ escaped.\n\nOne\ of\ the\ places\ where\ my\ chevron\ suggestion\ unwinds\ is\ the\ treatment\ of\ lists:\ if\ a\ list\ is\ represented\ as\ a\ string,\ should\ it\ use\ chevrons\ or\ braces\ as\ delimiters?\n\nI'm\ not\ sure\ there\ are\ any\ easy\ answers\ -\ the\ more\ I\ use\ Tcl,\ the\ more\ I\ appreciate\ how\ the\ different\ rules\ fit\ together\ (but\ at\ the\ cost\ of\ the\ comment/bracing\ hell).\ \ It\ is\ difficult\ to\ fix\ the\ comment/bracing\ hell\ without\ introducing\ a\ lot\ of\ extra\ complexity.\ \ But\ it\ is\ well\ worth\ trying!\n----\n\[KJN\]:\ Problems\ with\ \[Cloverfield\ -\ Tridekalogue\]\ re\ comment/bracing\ hell.\n\nRule\ 5\ for\ finding\ the\ matching\ brace,\ specifically,\ the\ rule\ to\ skip\ any\ braces\ in\ comments.\ \ The\ rule\ is\ to\ follow\ rule\ 10\ when\ identifying\ a\ comment.\ \ This\ seems\ to\ require\ that\ the\ braced\ text\ must\ be\ parsed\ into\ commands\ and\ words,\ in\ order\ to\ identify\ when\ '#'\ occurs\ at\ a\ place\ where\ the\ first\ character\ of\ a\ word\ is\ expected.\ \ The\ parsing\ of\ embedded\ braced\ text\ must\ be\ recursive,\ because\ embedded\ braces\ are\ required\ to\ match,\ and\ a\ comment\ string\ may\ occur\ at\ any\ depth\ in\ the\ nested\ braces.\ \ However\ the\ Tridekalogue\ does\ not\ require\ that\ braced\ text\ must\ be\ parsable\ into\ commands\ and\ words,\ only\ that\ braces\ match.\nA\ problem\ therefore\ arises\ when\ the\ recursive\ parsing\ encounters\ text\ that\ does\ not\ have\ Tcl/Cloverfield\ word\ form,\ e.g.\n======\n\ \ someCmd\ \{\"xx\ data\ not\ commands\ \{\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ #\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''\[FB\]:\ This\ example\ is\ simple\ actually.\ The\ first\ double\ quote\ just\ after\ the\ opening\ brace\ suspends\ brace\ matching\ until\ the\ next\ double\ quote,\ i.e.\ the\ one\ just\ before\ the\ closing\ brace.\ The\ \"comment\"\ isn't\ one\ because\ it's\ between\ double\ quotes.\ So\ Tcl\ and\ Cloverfield\ give\ the\ same\ result.\ However\ the\ following\ behaves\ differently\ in\ Tcl\ and\ Cloverfield\ (the\ leading\ double\ quote\ was\ moved\ after\ the\ second\ brace):''\n======\n\ \ someCmd\ \{xx\ data\ not\ commands\ \{\"\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ #\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''Here\ Cloverfield\ needs\ a\ second\ closing\ brace\ because\ the\ second\ opening\ brace\ is\ outside\ the\ couple\ of\ double\ quotes.\ The\ same\ for\ the\ following\ (the\ double\ quote\ was\ moved\ on\ the\ last\ line):''\n======\n\ \ someCmd\ \{xx\ data\ not\ commands\ \{\n\ \ \ \ \ \ \ #\ but\ these\ are\ commands\ \}\n\ \ pwd\n\ \ set\ a\ foo\n\ \ \"#\ where\ is\ matching\ bracket?\ \ Is\ the\ next\ bracket\ surplus?\ \ Is\ there\ a\ missing\ \"\n\ \ \}\n======\n''But\ this\ time\ because\ the\ first\ closing\ brace\ that\ matches\ the\ second\ opening\ brace\ in\ Tcl\ is\ commented\ out\ in\ Cloverfield.''\n\n''BTW,\ \[EIAS\]\ is\ preserved,\ meaning\ that\ whatever\ the\ commented\ chars\ may\ be,\ they\ are\ still\ part\ of\ the\ expression:''\n\n======\n%\ set\ l\ \{\n\ \ \ \ 1\ 2\n\ \ \ \ #\ 3\ 4\n\ \ \ \ 5\ 6\n\}\n%\ llength\ \$l\;\ #\ Gives\ 7\ in\ Tcl\ and\ 4\ in\ Cloverfield,\ as\ the\ list\ parser\ recognizes\ the\ comments.\n4\n%\ lindex\ \[split\ \$l\ \\n\]\ 2\;\ #\ Returns\ the\ \"commented\"\ line\ because\ it\ is\ part\ of\ the\ string\ rep.\n\ \ \ \ #\ 3\ 4\n======\n\n\[KJN\]:\ If\ a\ nested\ braced\ data\ string\ does\ not\ have\ code\ form,\ it\ may\ not\ be\ possible\ for\ the\ parser\ to\ decide\ how\ to\ match\ the\ braces.\ \ To\ avoid\ this\ problem,\ I\ think\ Cloverfield\ is\ forced\ to\ require\ that\ a\ braced\ string\ must\ be\ parsable\ into\ commands\ and\ words\ (i.e.\ into\ Cloverfield\ code,\ except\ that\ the\ commands\ need\ not\ be\ defined).\n\n''\[FB\]:\ That's\ a\ good\ point.\ However\ this\ does\ not\ necessarily\ imply\ that\ braced\ expressions\ be\ parsed\ into\ commands\ and\ words,\ only\ that\ beginning\ of\ words\ be\ recognized.\ For\ example,\ a\ double-quote\ character\ does\ not\ start\ a\ string\ when\ in\ the\ middle\ of\ a\ word,\ and\ thus\ does\ not\ suspend\ brace\ matching.\ However\ I'm\ conscious\ that\ rule\ \[\[5\]\]\ may\ sound\ a\ bit\ ambiguous\ and\ needs\ to\ be\ ironed\ out.\ A\ reference\ parser\ implementation\ should\ help\ sorting\ things\ out\ along\ with\ a\ good\ test\ suite.\ ''(\[KJN\]:\ that's\ a\ good\ idea\ -\ why\ not\ write\ the\ reference\ parser\ in\ Tcl,\ it\ should\ not\ be\ too\ difficult.\ ''\[FB\]:\ that\ was\ the\ plan\ \;-)'')''\ Anyway\ the\ goal\ of\ these\ new\ rules\ is\ to\ turn\ the\ language\ rules\ from\ parser-friendly\ into\ human-friendly:\ the\ rules\ should\ follow\ the\ principle\ of\ least\ astonishment\ in\ reading\ order.''\n\n\[KJN\]:\ Also,\ to\ eliminate\ comment\ hell,\ Cloverfield\ cannot\ escape\ parsing\ the\ contents\ of\ a\ quoted\ string\ in\ the\ same\ way,\ because\ code\ such\ as\ the\ following\ may\ occur:\n\n\ \ if\ \{\$a\ eq\ \$b\}\ \{\n\ \ \ \ proc\ foo\ \{bar\}\ \"\n\ \ \ \ \ \ #\ a\ comment\ \}\n\ \ \ \ \ \ puts\ \\\"Error\ foo\ in\ \\\$bar\\\"\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ The\ above\ doesn't\ work\ in\ Tcl\ but\ works\ as\ expected\ (if\ we\ trust\ the\ indentation)\ in\ Cloverfield:\ the\ `if`\ block\ contains\ the\ whole\ proc\ body,\ and\ the\ latter\ also\ behaves\ as\ expected\ (it\ puts\ the\ right\ message,\ skipping\ the\ commented\ line).''\n\n\[KJN\]:\ try\ instead\n\ \ if\ \{\$a\ eq\ \$b\}\ \{\n\ \ \ \ proc\ foo\ \{bar\}\ \"\n\ \ \ \ \ \ #\ a\ \"comment\"\n\ \ \ \ \ \ puts\ \\\"Error\ foo\ in\ \\\$bar\\\"\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ The\ braced\ expression\ is\ valid\ in\ both\ Tcl\ and\ Cloverfield,\ because\ it\ follows\ both\ sets\ of\ rules:''\n\n\ \ \ *\ ''In\ Tcl,\ the\ first\ brace\ matches\ the\ last\ one,\ because\ the\ inner\ braces\ around\ \"bar\"\ are\ well-balanced.''\n\ \ \ *\ ''Ditto\ for\ Cloverfield,\ which\ also\ requires\ that\ the\ double\ quotes\ be\ well-balanced.''\n\n''However,\ both\ Tcl\ and\ Cloverfield\ fail\ to\ evaluate\ the\ expression,\ because\ there\ are\ extra\ characters\ after\ the\ first\ close-quote\ (here,\ \"comment\").\ Tcl\ and\ Cloverfield\ rules\ aren't\ ''that''\ different,\ really\ \;-)\ \ They\ should\ only\ diverge\ in\ fringe\ cases\ or\ in\ very\ predictable\ ways\ (e.g.\ when\ using\ quotes\ and\ hashes\ in\ braced\ expressions).''\n\n\[KJN\]:\ This\ is\ my\ point\ -\ if\ Cloverfield\ aims\ to\ \"fix\"\ comments,\ then\ it\ should\ not\ fail\ with\ that\ code,\ or\ with\n\ \ foreach\ name\ \$list\ \{\n\ \ \ \ proc\ \$name\ \{\}\ \"\n\ \ \ \ \ \ #\ Using\ \"quotes\"\ for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\n\ \ \ \ \ \ return\ \$name\n\ \ \ \ \"\n\ \ \}\n\n''\[FB\]:\ Cloverfields\ aims\ to\ \"fix\"\ comments\ within\ braces\ (notice\ how\ I\ always\ put\ \"fix\"\ between\ quotes,\ are\ comments\ are\ not\ really\ broken\ but\ rather\ parser-friendly\ vs.\ human-friendly).\ However\ the\ comments\ are\ not\ recognized\ within\ quoted\ strings\ (and\ I\ don't\ know\ of\ any\ mainstream\ language\ that\ allows\ that).\ Rule\ \[\[5\]\]\ specifies\ that\ characters\ (not\ just\ braces)\ lose\ their\ significance\ between\ quotes.\ So\ the\ above\ code\ will\ fail\ anyway\ because\ of\ the\ extra\ chars\ after\ the\ quotes,\ however\ let's\ consider\ the\ following\ code:''\n\n\ \ foreach\ name\ \$list\ \{\n\ \ \ \ proc\ \$name\ \{\}\ \"\n\ \ \ \ \ \ #\ Using\ \"\ quotes\ \"\ for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\n\ \ \ \ \ \ return\ \$name\n\ \ \ \ \"\n\ \ \}\n\n''Here\ the\ only\ differences\ are\ the\ extra\ spaces\ around\ the\ `quotes`\ string\ on\ the\ \"commented\ line\"\ (which\ isn't\ one).\ But\ now\ `proc`\ complains\ of\ extra\ arguments.\ Indeed,\ it\ is\ given\ 5\ args\ instead\ of\ the\ expected\ 3:''\n\n\ \ \ *\ ''3rd\ one\ (the\ expected\ body)\ is\ `\"\\n\ \ \ \ #\ Using\ \"`\ (notice\ the\ \"comment\")''\n\ \ \ *\ ''4th\ one\ is\ `\"quotes\"`''\n\ \ \ *\ ''5th\ one\ is\ `\"for\ the\ proc\ body\ sets\ the\ return\ value\ when\ the\ proc\ is\ defined\\n\ \ \ \ return\ \$name\\n\"`''\n\n\n\[KJN\]:\ If\ we\ modify\ Cloverfield\ by\ adding\ rules\ for\ quoted\ strings,\ this\ seems\ to\ solve\ both\ these\ problems,\ but\ adds\ problems\ of\ its\ own:\n\n\ \ \ *\ a\ string\ such\ as\ \"\{\"\ is\ now\ illegal,\ but\ may\ be\ rendered\ \"\\\{\"\n\n''\[FB\]:\ On\ the\ contrary.\ In\ Tcl\ the\ string\ \"\{\"\ is\ legal\ at\ top\ level\ but\ not\ in\ braced\ code\ (e.g.\ in\ a\ proc),\ however\ it\ is\ always\ legal\ in\ Cloverfield.\ That's\ an\ area\ where\ the\ latter\ improves\ the\ overall\ consistency\ of\ the\ language.''\n\n\[KJN\]:\ No,\ the\ string\ is\ legal\ in\ Tcl\ at\ any\ level,\ but\ if\ it\ is\ inside\ braced\ code\ it\ may\ have\ an\ unexpected\ interpretation\ when\ that\ code\ is\ parsed.\ \ It\ is\ legal\ in\ Cloverfield\ too,\ but\ not\ in\ a\ Cloverfield\ that\ is\ modified\ as\ suggested\ above.\n\n''\[FB\]:\ I\ missed\ the\ \"modified\ Cloverfield\"\ part,\ sorry\ for\ the\ confusion.\ Anyway,\ let's\ consider\ the\ following\ code:''\n\n======\n#\ Legal\ in\ both\ Tcl\ and\ Cloverfield.\nset\ v\ \"\{\"\n\n#\ Legal\ in\ Cloverfield,\ but\ illegal\ in\ Tcl\ (requires\ an\ extra\ closing\ brace,\ and\ fails\ when\ evaluated\ anyway).\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ \"\{\"\n\}\n======\n\[KJN\]:\ bracing\ hell\ is\ that\ the\ code\ above\ is\ not\ illegal\ in\ Tcl,\ but\ incomplete:\ as\ you\ said,\ Tcl\ expects\ another\ close\ brace.\ \ If\ Cloverfield\ is\ modified\ as\ I\ have\ suggested\ to\ 'fix'\ comments\ inside\ quoted\ strings,\ then\ the\ code\ above\ becomes\ a\ problem\ -\ I\ think\ the\ modified-Cloverfield\ parser\ should\ interpret\ both\ the\ quotes\ as\ 'open\ quotes'\ and\ fail\ at\ the\ last\ '\}'\ because\ it\ is\ a\ close-brace\ with\ no\ open-brace\ inside\ a\ quoted\ string.\ \ This\ is\ an\ improvement\ over\ Tcl,\ because\ at\ least\ the\ parser\ fails\ close\ to\ the\ problematic\ code,\ and\ not\ at\ the\ end\ of\ the\ file\ when\ it\ cannot\ find\ the\ 'missing'\ close-brace.\ \ Hunting\ for\ missing\ braces\ has\ sometimes\ driven\ me\ to\ drink.\n''\[FB\]:\ But\ why\ and\ how\ exactly\ would\ you\ want\ to\ modify\ the\ Cloverfield\ rules?\ I\ fail\ to\ see\ the\ problem\ it\ tries\ to\ solve.\ Do\ you\ mean\ braces\ should\ also\ be\ balanced\ in\ quoted\ strings?''\n======\n#\ Legal\ in\ both.\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ \"\\\{\"\n\}\n======\n\n''So\ the\ string\ \"\{\"\ cannot\ be\ used\ verbatim\ in\ braced\ expressions\ with\ Tcl,\ it\ needs\ to\ be\ escaped\ as\ \"\\\{\".\ However\ both\ forms\ are\ valid\ in\ Cloverfield\ and\ give\ the\ same\ value\ whatever\ the\ context.''\n\n\ \ \ *\ braced\ text\ must\ have\ matching\ quotes\ as\ well\ as\ matching\ braces,\ making\ the\ string\ \{\"\}\ illegal,\ but\ expressible\ as\ \"\\\"\"\n\n''\[FB\]:\ True.\ And\ the\ same\ for\ all\ words\ that\ start\ with\ `#`,\ such\ as\ `uplevel`\ arguments,\ which\ will\ need\ to\ be\ escaped\ or\ put\ between\ quotes.\ As\ a\ side\ note,\ this\ will\ make\ most\ editors\ happier,\ since\ they\ often\ require\ hashes\ to\ be\ escaped\ in\ order\ to\ avoid\ rendering\ the\ line\ remainder\ as\ comment,\ but\ of\ course\ this\ is\ NOT\ the\ rationale\ for\ this\ change,\ just\ a\ side\ effect.''\n\n''\[KJN\]:\ the\ Cloverfield\ rules\ do\ not\ clearly\ state\ that\ \{\"\}\ is\ illegal,\ but\ this\ follows\ from\ analysis\ of\ the\ rules.\ \ Your\ idea\ of\ writing\ a\ reference\ parser\ is\ a\ good\ one,\ because\ cases\ like\ this\ would\ become\ obvious\ either\ when\ defining\ the\ parser\ or\ when\ running\ the\ test\ cases.''\n\n''\[FB\]:\ Indeed.\ The\ test\ case\ will\ be\ a\ nice\ combination\ of\ test-oriented\ development\ along\ with\ a\ tutorial\ for\ the\ new\ rules.\ Ultimately,\ all\ the\ test\ cases\ will\ be\ fine-tuned\ to\ give\ the\ least\ surprising\ result\ from\ a\ human\ POV,\ and\ could\ provide\ side-by-side\ comparison\ with\ Tcl\ as\ a\ porting\ guide.\ Anyway,\ I\ believe\ that\ existing\ code\ can\ be\ rewritten\ in\ such\ a\ way\ that\ it\ complies\ with\ both\ sets\ of\ rules\ in\ most\ cases\ (as\ with\ the\ above\ example).''\n\n\ \ \ *\ if\ we\ need\ to\ define\ a\ non-code-like\ data\ string,\ we\ will\ have\ to\ use\ either\ quotes\ and\ escapes,\ or\ string\ processing,\ or\ the\ \{data\}\ word\ modifier\ (rule\ 11).\n\nIt\ seems\ to\ me\ that\ Cloverfield\ removes\ the\ pain\ for\ braced\ code,\ but\ increases\ the\ pain\ for\ braced\ data.\n\n''\[FB\]:\ When\ handling\ \"problematic\"\ data,\ the\ use\ of\ \[heredoc\]\ features\ is\ strongly\ encouraged\ in\ most\ languages.\ Problem\ is,\ Tcl\ doesn't\ have\ an\ \[heredoc\]\ feature,\ but\ Cloverfield\ does\ provide\ such\ a\ feature\ with\ `\{data\}`.\ If\ we\ make\ the\ (reasonable\ IMHO)\ assumption\ that\ most\ braced\ expressions\ in\ Tcl\ are\ not\ arbitrary\ data\ but\ code,\ then\ this\ problem\ disappears\ anyway.\ Moreover\ Cloverfield\ provides\ a\ third\ quoting\ rule\ \[\[6\]\]\ using\ parentheses\ that\ combines\ some\ of\ the\ effects\ of\ quotes\ and\ braces\ (and\ paves\ the\ way\ for\ the\ mythical\ `list`\ refactoring).''\n\n----\n***\ Word\ modifiers\ ***\n\[KJN\]\ I'm\ not\ keen\ on\ the\ word\ modifiers\ -\ except\ \{*\}.\ \{*\}\ was\ accepted\ into\ 8.5\ because,\ despite\ introducing\ syntax,\ it\ simplified\ a\ lot\ of\ code.\ \ I\ see\ now\ the\ purpose\ served\ by\ \{data\}\ (providing\ a\ way\ to\ define\ a\ literal\ string,\ since\ Cloverfield\ braces\ no\ longer\ do\ that\ -\ ''\[FB\]:\ Yes\ they\ still\ do,\ only\ brace\ matching\ rules\ change,\ and\ \"comments\"\ are\ not\ stripped\ off\ the\ string\ but\ remain\ part\ of\ it'')\ ,\ but\ the\ other\ word\ modifiers\ in\ Cloverfield\ introduce\ a\ lot\ of\ syntax.\ \ Also,\ in\ some\ cases\ (like\ \{*\})\ the\ modifier\ determines\ what\ is\ done\ with\ the\ word\ after\ substitution,\ while\ in\ other\ cases\ it\ modifies\ the\ substitution\ itself.\ \ I\ think\ this\ adds\ too\ much\ complexity.\n\n\[FB\]:\ The\ merit\ of\ \[Cloverfield\]\ is\ to\ introduce\ the\ concept\ of\ word\ modifier\ to\ solve\ a\ range\ of\ problems\ where\ the\ typical\ solution\ would\ use\ ad-hoc\ techniques\ like\ opaque\ tokens\ (=>\ no\ more\ EIAS)\ or\ string\ manipulations\ (=>\ shimmering).\ For\ example,\ see\ \[Jim\ References\].\ The\ strings\ returned\ by\ `\[\[ref\]\]`\ are\ similar\ to\ Cloverfield's\ word\ modifier,\ with\ the\ difference\ that\ Jim's\ are\ opaque\ tokens\ whereas\ Cloverfield's\ are\ just\ metasyntax\ that\ preserve\ \[EIAS\]\ and\ prevent\ the\ loss\ of\ internal\ rep\ due\ to\ shimmering.\ Another\ example\ is\ `\[null\]`\ and\ TIP\ #185\ \[http://tip.tcl.tk/185\],\ which\ propose\ a\ very\ similar\ syntax.\ Word\ modifiers\ are\ just\ a\ consistent\ syntax\ to\ tag\ words\ with\ special\ meaning.\ Sometimes\ to\ modify\ the\ behavior\ of\ the\ parser\ like\ `\{data`\},\ word\ substitution\ like\ `\{*\}`\ or\ `\{#\}`,\ or\ evaluation\ like\ `\{delay\}`.\ I'm\ afraid\ there\ aren't\ many\ ways\ of\ enhancing\ the\ language\ to\ solve\ these\ problems\ without\ introducing\ a\ lot\ of\ syntax\ or\ breaking\ the\ semantics,\ and\ word\ modifiers\ don't\ introduce\ new\ syntactic\ rules\ but\ rather\ capitalize\ on\ the\ existing\ `\{*\}`.\n\nConcerning\ the\ `\{data\}`\ modifier,\ I've\ just\ come\ across\ an\ example\ that\ illustrates\ a\ strikingly\ similar\ syntax\ enhancement\ to\ \[Jim\]\ (near\ the\ end\ of\ the\ page):\n\n\ set\ v\ \{<<EOT\}\{\n\ blah\ blah...\n\ any\ old\ stuff:\ \$\$\$\ \{\{\{\ \[\[\[\ \\\\\\\ \]\]\]\ etc.\n\ EOT\}\n\nThis\ quoting\ style\ is\ known\ as\ \[heredoc\],\ or\ \[here\ document\],\ so\ it\ seems\ that\ my\ intuition\ was\ right\ (see\ also\ \[Quoting\ Heaven!\]).\ Given\ the\ properties\ of\ the\ Tcl\ parser,\ I'm\ afraid\ there\ isn't\ much\ alternative\ to\ a\ word-modifier-like\ syntax\ enhancement.\ The\ Jim\ patch\ follows\ the\ same\ path\ as\ Cloverfield\ by\ modifying\ the\ brace\ matching\ rules.\n----\n\[KJN\]:\ Is\ Cloverfield\ compatible\ with\ \[EIAS\]?\ ''-\ \[FB\]\ Yes,\ EIAS\ is\ the\ most\ important\ Tcl\ principle\ -''\ If\ so,\ what\ is\ the\ string\ representation\ of\ the\ following\ values?\n\ \ \ 1.\ \{null\}\{any\ word\}\ #\ Rule\ 11\ states\ that\ this\ is\ \"a\ special\ null\ value\ which\ is\ distinct\ from\ any\ other\ value,\ including\ the\ empty\ string\"\n\n''\[FB\]:\ null\ is\ the\ only\ value\ that\ has\ no\ string\ representation,\ which\ means\ that\ it\ can't\ be\ `\[puts\]`,\ or\ escape\ from\ the\ reach\ of\ the\ interpreter.\ On\ this\ point\ it\ resembles\ Smalltalk's\ nil.\ The\ only\ way\ to\ create\ such\ a\ value\ is\ when\ evaluating\ an\ expression\ prefixed\ by\ the\ `\{null\}`\ modifier.\ For\ example,\ the\ string\ `\"1\ 2\ \{null\}3\ 4\"`,\ when\ fed\ to\ `\[list\]`,\ gives\ a\ list\ whose\ 3rd\ element\ is\ null.\ The\ only\ gray\ area\ in\ the\ Tridekalogue\ is\ how\ to\ handle\ side\ effects.\ Does\ `\{null\}`\ totally\ discard\ all\ that\ follows,\ or\ does\ the\ word\ get\ evaluated\ as\ usual,\ but\ its\ value\ replaced\ by\ null?\ I.e.,\ does\ `\{null\}\[\[puts\ foo\]\]`\ outputs\ `foo`?\ Does\ `\{null\}\[\[puts\ \[\[incr\ foo\]\]\]\]`\ increments\ variable\ `foo`?\ Word\ modifiers\ such\ as\ `\{null\}`\ need\ more\ complete\ specifications.''\n\n\ \ \ 2.\ \{meta\ foo\}bar\ #\ this\ value\ has\ data\ \{bar\}\ and\ metadata\ \{foo\}\;\ are\ both\ expressed\ by\ its\ string\ representation?\n\n''\[FB\]:\ the\ metadata\ is\ not\ part\ of\ a\ word's\ string\ representation\;\ `puts\ \{meta\ foo\}bar`\ prints\ `bar`.\ The\ only\ way\ to\ get\ a\ word's\ metadata\ is\ to\ prefix\ it\ with\ `\{meta\}`.\ Metadata\ are\ transient,\ arbitrary\ client\ data\ that\ can\ be\ associated\ with\ words,\ notably\ for\ debugging\ purpose.\ Library\ code\ should\ avoid\ using\ metadata\ and\ leave\ it\ to\ the\ application.''\n\n\ \ \ 3.\ \{delay\}\$bar\ #\ what\ is\ the\ string\ representation\ of\ this\ value\ that\ forces\ substitution\ when\ the\ value\ is\ read?\n\n''\[FB\]:\ Think\ of\ `\{delay\}`\ as\ a\ \"safer\"\ \[future\].\ The\ latter\ are\ objects\ created\ with\ no\ string\ rep,\ whose\ expressions\ are\ evaluated\ when\ the\ string\ rep\ is\ required.\ The\ problem\ is\ that\ string\ rep\ generation\ can\ occur\ in\ unpredictable\ ways\ (e.g\ when\ shimmering).\ Whereas\ `\{delay\}`\ expressions\ are\ only\ evaluated\ when\ the\ word\ is\ explicitly\ accessed.\ So\ the\ string\ rep\ is\ the\ original\ word\ until\ its\ value\ is\ queried.\ For\ example,\ the\ string\ `\"foo\ \{delay\}\[\[bar\]\]\"`,\ when\ fed\ to\ `\[list\]`,\ gives\ a\ list\ where\ the\ second\ element\ is\ a\ delayed\ call\ to\ `bar`.\ It\ can\ be\ turned\ into\ a\ `\[dict\]`\ where\ the\ delayed\ expression\ is\ the\ value\ associated\ with\ `foo`,\ causing\ shimmering,\ but\ without\ evaluating\ the\ delayed\ expression.\ However\ a\ list\ or\ dict\ operation\ on\ this\ element\ will\ do.\ So\ here\ we\ have\ a\ string\ rep\ that\ may\ change\ dynamically\ within\ a\ larger\ string,\ and\ that's\ the\ main\ reason\ for\ using\ \[rope\]s\ as\ the\ primary\ internal\ string\ representation.''\n\n''However,\ like\ with\ `\{null\}`,\ there\ are\ still\ gray\ areas\ in\ the\ spec.\ For\ example,\ does\ `\{delay\}`\ delay\ the\ evaluation\ of\ the\ topmost\ word,\ or\ all\ the\ subwords?\ E.g.\ does\ `\{delay\}\[\[foo\ \[\[bar\]\]\]\]`\ also\ delays\ `\[\[bar\]\]`\ or\ is\ it\ substituted\ in\ the\ delayed\ expression?\ I'd\ favor\ the\ latter.\ Also,\ how\ does\ it\ behave\ with\ variables:\ does\ it\ delay\ the\ variable\ substitutions,\ or\ do\ variables\ have\ to\ contain\ an\ expression\ to\ be\ evaluated\ (à\ la\ `\[subst\]`)?\ And\ the\ same\ for\ other\ substitution\ rules.''\n\n======\nset\ v\ 1\nproc\ foo\ \{\}\ \{\n\ \ \ \ set\ v\ 2\n\ \ \ \ return\ \[list\ \{delay\}\[set\ v\]\ \{delay\}\"\$v\"\ \{delay\}\$v\]\n\}\nfoo\ \;\ #\ Should\ it\ return\ \{1\ 1\ 1\},\ \{1\ 2\ 1\}\ or\ \{1\ 2\ 2\}?\nproc\ bar\ \{\}\ \{\n\ \ \ \ set\ e\ \{\[set\ v\]\}\n\ \ \ \ return\ \{delay\}\$e\ \n\}\nbar\ \;\ #\ Should\ it\ return\ \"1\"\ or\ \"\[set\ v\]\"\ ?\n\nset\ w\ 2\nset\ x\ v\nproc\ baz\ \{\}\ \{\n\ \ \ \ set\ x\ w\n\ \ \ \ return\ \{delay\}\[set\ \$x\]\n\}\nbaz\ \;\ #\ Should\ it\ return\ 1\ or\ 2?\n\n\n\n----\n****\ See\ also\ ****\n\ \ \ *\ \[Cloverfield\ -\ Tridekalogue\]\n\ \ \ *\ The\ Reference\ section\ below.\n\n\n----\n***\ References\ ***\n\n\[Cloverfield\]\ references\ using\ the\ `\{ref\}`\ word\ modifier\ is\ a\ solution\ to\ the\ problem\ of\ defining\ and\ serializing\ structures\ with\ circular\ references\ (see\ \[Cloverfield\ -\ Tridekalogue\]).\ Let's\ consider\ a\ tree\ whose\ nodes\ have\ to\ reference\ their\ root\ or\ parent\ node\ in\ some\ way\ (for\ instance,\ a\ DOM\ tree\ from\ XML\ data).\ E.g:\n\n======\nroot\n|\n+-node1\n|\n+-node2\n|\ |\n|\ +-node21\n|\ |\n|\ +-node22\n|\ |\n|\ +-node23\n|\n+-node3\n======\n\nEach\ node\ (including\ the\ root)\ is\ a\ structure\ that\ holds\ a\ value\ (here\ `root`,\ `node1`...),\ a\ list\ of\ children,\ and\ a\ reference\ to\ its\ root\ parent.\ Expressing\ this\ structure\ in\ plain\ Tcl\ gives:\n\n======\nset\ tree\ \{root\ \{\n\ \ \ \ \{node1\ \{\}\}\n\ \ \ \ \{node2\ \{\n\ \ \ \ \ \ \ \ \{node21\ \{\}\}\n\ \ \ \ \ \ \ \ \{node22\ \{\}\}\n\ \ \ \ \ \ \ \ \{node23\ \{\}\}\n\ \ \ \ \}\n\ \ \ \ \{node3\ \{\}\}\n\}\}\n======\n\nThis\ is\ a\ tree\ expressed\ as\ a\ recursive\ list.\ Each\ node\ holds\ 2\ elements:\ a\ name\ and\ a\ list\ of\ children.\ Nodes\ \"know\"\ their\ root\ and\ parent\ implicitly\ because\ they\ are\ part\ of\ the\ same\ Tcl\ variable.\ If\ we\ want\ to\ extract\ a\ given\ node,\ we\ lose\ the\ reference\ information\ so\ we\ have\ to\ retrieve\ this\ metadata\ somewhere.\ Typically\ one\ would\ have\ to\ maintain\ this\ information\ in\ variables,\ with\ some\ in-band\ signaling\ convention,\ or\ with\ specific\ naming\ conventions.\ For\ example,\ we\ could\ store\ the\ names\ of\ the\ root\ and\ parent\ nodes\ in\ the\ node\ structure\ (in-band\ signaling).\ \n\n======\nset\ tree\ \{root\ root\ root\ \{\n\ \ \ \ \{node1\ root\ root\ \{\}\}\n\ \ \ \ \{node2\ root\ root\ \{\n\ \ \ \ \ \ \ \ \{node21\ root\ node2\ \{\}\}\n\ \ \ \ \ \ \ \ \{node22\ root\ node2\ \{\}\}\n\ \ \ \ \ \ \ \ \{node23\ root\ node2\ \{\}\}\n\ \ \ \ \}\n\ \ \ \ \{node3\ root\ root\ \{\}\}\n\}\}\n======\n\nExtracting\ `node22`\ gives:\n\n\{node22\ root\ node2\ \{\}\}\n\nKnowing\ that\ the\ data\ is\ stored\ in\ variable\ `tree`,\ one\ could\ find\ the\ root\ and\ parent\ nodes\ by\ searching\ the\ tree.\ But\ this\ is\ a\ potentially\ expensive\ operation\ and\ needs\ explicit\ coding.\n\nNow\ consider\ the\ same\ structure\ with\ Cloverfield's\ `\{ref\}`\ word\ modifier:\n\n======\nset\ tree\ \{ref\ root\}(root\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \{ref\ node21\}(node21\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \{ref\ node22\}(node22\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ ))\n\ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n)\n======\n\n(Note\ that\ this\ is\ the\ string\ representation\ of\ the\ tree.\ Behind\ the\ curtains,\ the\ structure\ maintains\ pointers\ to\ the\ referenced\ data.)\n\nHere\ we\ chose\ to\ use\ the\ node\ name\ as\ the\ reference\ ID,\ but\ IDs\ are\ arbitrary.\ The\ tree\ is\ now\ expressed\ as\ a\ recursive\ list\ with\ all\ nodes\ tagged\ by\ a\ unique\ reference\ ID.\ Each\ node\ holds\ 4\ elements:\ a\ name,\ a\ reference\ to\ the\ root\ node,\ a\ reference\ to\ the\ parent\ node,\ and\ a\ list\ of\ children.\ This\ string\ representation\ is\ similar\ to\ the\ above\ Tcl\ example\ with\ in-band\ signaling.\ The\ difference\ is\ that\ elements\ 1\ and\ 2\ respectively\ hold\ references\ to\ the\ root\ and\ parent\ nodes,\ whereas\ the\ Tcl\ version\ required\ an\ explicit\ naming\ convention.\n\nSome\ example\ use:\n\n======\n%\ set\ node\ \[lindex\ \$tree\ 3\ 2\ 3\ 2\]\;\ #\ 3's\ are\ for\ each\ node's\ children\ list.\n#\ Note:\ indentation\ and\ inline\ comments\ added\ to\ the\ below\ string\ representation\ for\ clarity.\n\{ref\ node22\}(\n\ \ \ \ #\ Value:\n\ \ \ \ node22\n\ \ \ \ \n\ \ \ \ #\ Root:\ first\ reference\ to\ root\ is\ serialized.\ \n\ \ \ \ \{ref\ root\}(\n\ \ \ \ \ \ \ \ root\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ #\ Subsequent\ references\ to\ root\ aren't\ serialized.\n\ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \n\ \ \ \ \ \ \ \ #\ Children:\n\ \ \ \ \ \ \ \ (\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ Siblings\ of\ node22\ get\ serialized...\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node21\}(node21\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ ...\ but\ not\ node21.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node22\}\{\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ ))\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ )\n\ \ \ \ )\n\ \ \ \ \n\ \ \ \ #\ Parent:\ reference\ was\ already\ serialized\ above.\n\ \ \ \ \{ref\ node2\}\{\}\n\ \ \ \ \n\ \ \ \ #\ Childen:\n\ \ \ \ \{\}\n)\n\n#\ Get\ node\ name.\n%\ lindex\ \$node\ 0\nnode22\n\n#\ Get\ node\ parent\ name.\n%\ lindex\ \$node\ 2\ 0\ \;\ #\ 2\ is\ for\ parent.\nnode2\n\n#\ Get\ siblings.\n%\ lindex\ \$node\ 2\ 3\ \;\ #\ 2\ for\ parent,\ 3\ for\ children.\n#\ Note:\ indentation\ and\ inline\ comments\ added\ to\ the\ below\ string\ representation\ for\ clarity.\n(\n\ \ \ \ #\ First\ sibling\ is\ the\ first\ to\ reference\ root.\n\ \ \ \ \{ref\ node21\}(\n\ \ \ \ \ \ \ \ #\ Value:\n\ \ \ \ \ \ \ \ node21\ \n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ #\ Root:\ first\ reference\ to\ root\ is\ serialized.\ \n\ \ \ \ \ \ \ \ \{ref\ root\}(\n\ \ \ \ \ \ \ \ \ \ \ \ root\n\ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ #\ Subsequent\ references\ to\ root\ aren't\ serialized.\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \ \ \ \ \{ref\ root\}\{\}\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ #\ Children:\n\ \ \ \ \ \ \ \ \ \ \ \ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node1\}(node1\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node2\}(node2\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ (\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ First\ sibling\ is\ currently\ being\ serialized.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node21\}\{\}\n\ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ #\ Subsequent\ siblings\ need\ serialization.\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node22\}(node22\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node23\}(node23\ \{ref\ root\}\{\}\ \{ref\ node2\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ))\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \{ref\ node3\}(node3\ \{ref\ root\}\{\}\ \{ref\ root\}\{\}\ \{\})\n\ \ \ \ \ \ \ \ \ \ \ \ )\n\ \ \ \ \ \ \ \ )\n\ \ \ \ )\n\ \ \ \ \n\ \ \ \ #\ Subsequent\ siblings\ were\ already\ serialized.\n\ \ \ \ \{ref\ node22\}\{\}\n\ \ \ \ \{ref\ node23\}\{\}\n)\n\n======\n\nReferenced\ structures\ are\ never\ serialized\ more\ than\ once\ (the\ referenced\ value\ is\ serialized\ at\ the\ first\ occurrence).\ That\ way,\ structures\ with\ circular\ references\ can\ be\ serialized\ in\ finite\ space,\ expressed\ as\ strings,\ and\ compared\ for\ equality.\ For\ a\ simpler\ example,\ consider\ this:\n\n======\nset\ l\ \{ref\ a\}(a\ \{ref\ a\}\{\})\n======\n\nThe\ above\ structure\ is\ a\ list\ whose\ 2nd\ element\ is\ a\ reference\ to\ itself.\ Hence:\n\n======\n%\ lindex\ \$l\ 1\;\ #\ Or\ \$l\[1\]\ using\ Cloverfield\ conventions.\n\{ref\ a\}(a\ \{ref\ a\}\{\})\n%\ expr\ \{\[lindex\ \$l\ 1\]\ ==\ \$l\}\n1\n======\n\n----\nReferences\ can\ be\ combined\ with\ argument\ expansion.\ For\ example:\n\n======\n\{ref\ root\}(a\ \{*\}\{ref\ root\}\{\})\n======\n\ngives\ an\ infinite\ (flat)\ list\ of\ \"a\".\ The\ following\ code\ loops\ indefinitely:\n\n======\nforeach\ v\ \{ref\ root\}(a\ \{*\}\{ref\ root\}\{\})\ \{\n\ \ \ \ puts\ \$v\n\}\n======\n\nThe\ argument\ expansion\ word\ modifier\ must\ be\ placed\ before\ the\ reference\ so\ that\ the\ referenced\ value\ be\ expanded\ in-place\ in\ the\ containing\ list.\ Placing\ `\{*\}`\ after\ `\{ref\}`\ has\ no\ effect\ since\ the\ reference\ word\ modifier\ tags\ a\ single\ word.\n\n----\nNotice\ also\ the\ use\ of\ parentheses\ in\ place\ of\ braces,\ in\ order\ for\ the\ reference\ IDs\ to\ share\ the\ same\ scope.\ For\ example:\n\n======\n\{ref\ a\}(a\ \{ref\ a\}\{\})\ \n\{ref\ a\}\{a\ \{ref\ a\}\{\}\}\n======\n\nIn\ the\ first\ case,\ both\ references\ `a`\ designate\ the\ same\ reference,\ contrary\ to\ the\ second\ case\ because\ words\ are\ not\ evaluated\ at\ the\ same\ time.\ The\ first\ case\ is\ a\ self-referencing\ structure,\ whereas\ the\ second\ is\ equal\ to\ the\ list\ `\{a\ \{\}\}`\ value-wise.\n\nThe\ consequence\ is\ that\ the\ following\ works\ as\ expected:\n\n======\n%\ set\ l1\ (\{ref\ a\}1\ 2\ \{ref\ a\}\{\})\n%\ set\ l2\ (\$l1\ \{ref\ a\}3\ \{ref\ a\}\{\})\n\{\{ref\ a\}1\ 2\ \{ref\ a\}\{\}\}\ \{ref\ a\}3\ \{ref\ a\}\{\}\n#\ The\ above\ equals\ \{\{1\ 2\ 1\}\ 3\ 3\}\ value-wise.\n======\n`l1`'s\ references\ don't\ pollute\ `l2`'s\ unexpectedly.\n\nIn\ summary,\ references\ don't\ cross\ brace\ boundaries,\ because\ braces\ delay\ the\ evaluation\ of\ their\ content.\ That\ is\ the\ main\ reason\ why\ parentheses\ are\ necessary\ for\ references\ to\ work.\n----\n****\ See\ also\ ****\n\n\ \ \ *\ \[Reference\]\n\ \ \ *\ \[TclRef\]\n\ \ \ *\ \[Tcl\ references\ in\ Tcl\]\n\ \ \ *\ \[Jim\ References\]\n\ \ \ *\ \[Abstract\ Data\ Types\]\n----\n\n\[CMcC\]\ I've\ been\ persuaded\ to\ lift\ the\ RecentChanges\ suppression\ of\ Cloverfield,\ which\ I\ have\ done.\ \ I\ still\ think\ it's\ a\ walled\ garden,\ but\ ...\ let\ a\ thousand\ flowers\ bloom.\ \ Please\ don't\ feel\ that\ it\ can't\ be\ refactored,\ should\ people\ want\ to\ do\ that,\ and\ be\ aware\ that\ I'm\ considering\ some\ kind\ of\ repetition\ filtering\ for\ RecentChanges\ which\ would\ preclude\ all\ the\ Cloverfield\ changes\ appearing\ on\ the\ same\ day,\ just\ a\ signal-to-noise\ issue.\ \ But\ above\ all,\ I\ behaved\ inhospitably,\ and\ that's\ not\ good\ for\ a\ wiki.\n\n\[FB\]:\ Thank\ you.\ I\ understand\ your\ concerns\ about\ the\ amount\ of\ noise\ generated\ on\ RecentChanges.\ It\ is\ true\ that\ there\ have\ been\ a\ lot\ of\ Cloverfield-related\ changes\ on\ the\ feed,\ but\ the\ reason\ is\ simply\ that\ the\ project\ has\ just\ started\ and\ that\ I'm\ indexing\ and\ cross-linking\ a\ lot\ of\ information\ on\ the\ Wiki.\ Anyway\ I'll\ going\ to\ start\ moving\ some\ of\ the\ above\ pages\ back\ here\ so\ as\ to\ limit\ the\ impact.\ Rest\ assured\ that\ my\ goal\ is\ not\ to\ hijack\ or\ disrupt\ anything\ but\ to\ generate\ discussion\ and\ push\ Tcl\ forward.\n\n\[CMcC\]\ I\ absolutely\ believe\ that's\ your\ intention,\ and\ I\ regret\ imposing\ my\ will\ on\ the\ way\ you\ were\ articulating\ it.\ \ Don't\ rearrange\ it\ to\ suit\ my\ particular\ aesthetic,\ please.\ \ Carry\ on\ as\ you\ were\ and\ we'll\ let\ the\ wiki\ community\ work\ its\ magic\ on\ the\ content.\ \ I\ overreacted,\ and\ was\ wrong\ to\ do\ so.\n\n\[FB\]:\ Oops,\ too\ late!\ Ah,\ never\ mind.\ Maybe\ it's\ better\ that\ way!\ \ I'll\ recreate\ these\ pages\ only\ if\ this\ one\ really\ gets\ too\ large.\ If\ I\ had\ to\ make\ a\ suggestion\ for\ a\ Wikit\ improvement,\ that\ would\ be\ page\ anchors\ (e.g.\ \[\[Cloverfield#References\]\]\ or\ something\ like\ that).\n----\n!!!!!!\n%|\ \[Category\ Language\]\ |\ \[Category\ Cloverfield\]\ |%\n!!!!!!} CALL {my revision Cloverfield} CALL {::oo::Obj4410972 process revision/Cloverfield} CALL {::oo::Obj4410970 process}

-errorcode

NONE

-errorinfo

Unknow state transition: LINE -> END
    while executing
"error $msg"
    (class "::Wiki" method "render_wikit" line 6)
    invoked from within
"my render_$default_markup $N $C $mkup_rendering_engine"
    (class "::Wiki" method "render" line 8)
    invoked from within
"my render $name $C"
    (class "::Wiki" method "revision" line 31)
    invoked from within
"my revision $page"
    (class "::Wiki" method "process" line 56)
    invoked from within
"$server process [string trim $uri /]"

-errorline

4