Is it ok if we reorganize this into sections:

• Things that need to be done ...and no one disagrees). Bug fixes fall into this category. (That doesn't mean it will be done, alas. For example, perhaps no one understands how to fix something.)
• Things that probably can be done if the proposal was fleshed out further. ...but won't be if no one picks up the ball.) Drag and drop is an example.
• Things over which there is fundamental disagreement over. (And always will be.) This doesn't mean such a change won't happen, however. The name and structure of the list command is a good example.
• Things over which there are fundamental incompatibilities. ...in other words, everyone agrees this was be a good thing but it is currently incompatible with Tcl as it stands.a
• Things that don't need to be done. ...for any of a number of reasons. Example: bad proposal, better alternative, incompatible with Tcl way, not much payoff for the work, etc.
• There may be other such categories or perhaps a different organization. Feel free to contribute. - Don Libes

LV:Yes, I think this would be a valuable reorganization. Anyone with an opinion should feel free to step in and start. It might be easier to actually create pages for these categories and move things to the new pages, to prevent this page from getting much bigger...

DKF: Note that some of the items here are the subject of TIPs, which should be referred to for definitive suggestions.

MGS: Would it make sense to split this page up into several pages? It would make it easier to track changes ...

DKF: Longer items should be placed on their own page with a link (preferably including the currently bold text!) from here.

Lars H: And if several items discuss closely related matters, then they could be collected together on a separate page. This was how expr shorthand for Tcl9 came about.

DGP: Better idea: Anything mentioned here that is not registered as a Feature Request at SourceForge, register it. Then make this page nothing more that a list of links to those reports.

There has been a lot of discussion on what version control system to use for Tcl 9 development. There is a general consensus not to use CVS but to opt for git or fossil.

Sourceforge

A tcl9 project was started by FB:

FB: I've created a project on SourceForge dedicated to everything Tcl9-related:

http://sourceforge.net/projects/tcl9/

Currently it only holds the mailing list for Cloverfield but I'd be very happy to host other projects as well.

Fossil

Is there a public fossil repository for Tcl 9 yet?

DKF: See the novem branch, but it doesn't have particularly large changes yet.

Git/GitHub

A tcl repository was created by das on GitHub: https://github.com/das/tcl . This is a copy of the SourceForge CVS repository, with all history, updated every hour.

Any other Git repositories?

The use of rename for this purpose is so bizarre that I can't think of anything else that has copied that model. And it has spawned a myriad of ways to destroy things which is quite unhelpful. Consider these:

rename proc {}
unset var
destroy .widget
close channel
after cancel id
bind tag {} 
file delete filename

(there's probably more)

FW contributes:

image delete imagename
array unset arrayName key

FW, months later, thinks of another:

namespace delete namespace

Larry Smith 2010-10-30: Actually, these are all symptoms of having missed a fundamental paradigm. Ideally, we shouldn't even declare procs. A proc should be just a variable containing a block of code.

 set add1 { init varname ""; if {$varname eq ""} return; upvar $varname var; incr var }

Thereafter, executing [add1 foo] should increase $foo by one. This scheme does require some better way to deal with arguments, here I used init. In theory, you'd really need [$add1 foo], but some command-dispatching logic could deal with that. This system also permits truly anonymous procs:

[{init varname ""; if {$varname eq ""} return; upvar $varname var; incr var } foo]

...would work as expected. Couple this with Rexx-style stem variables:

set .this.name "John Q. Typical"
set .this.address "123 Somewhere Street"

...so $this becomes the dict { {name} {John Q. Typical} {address} {123 Somewhere Street} } Actually, .this.name is probably not Tcl-ish: { this name } is probably more so, but requires more typing. Less than ::this::name does, though. The concept of namespaces folds in. this is a namespace containing the vars name and address. You would execute code there:

in .this {
    set swap {
        set temp $name
        set name $address
        set address $temp; unset temp 
    }
}
.this.swap

would define a method unique to this that would swap name and address (to pick a moronic example). (Yes, you could also just do set .this.swap ..., but I wanted to show the basic idea.) This produces a protoype-based class system like Self. unknown, faced with a var name it doesn't know, would look up the class of this (if it had one) to dispatch class-based method calls. Since these could happen a lot, that logic would probably be hardwired in C rather than being a Tcl procedure, though you could still extend or replace it in Tcl.

If you were to use Tokyo Cabinet or other such database to manage the symbol table, you would effectively get a completely check-pointable interpreter for free.

wdb 2018-09-30 – Larryʼs idea of procs just being vars containing block of code – currently approached with apply – Patchlevel 8.6.1:

 % set dup {x {+ $x $x} ::tcl::mathop}
 % apply $dup 12
 24
 %

The drawback is use of apply; you canʼt just say "$dup 24".

According to lisp, a proposition could be "lambda" as first list element of the string. Then you could say:

 % set dupTcl9 {lambda x {+ $x $x} ::tcl::mathop}
 % $dupTcl9 24
 % 48
 %

KJN adds

unload foo.dll
interp delete $interpreter

On a related note, AMG finds this to be insane:

% proc "" {x} {expr {$x * $x}}; "" 5
25
% rename "" ""                ; "" 5
empty command name ""

KJN: It is a quirk of Tcl that the name of a variable or proc may be the empty string. The variable {} is quite often used as an array in a namespace. The proc {} is not often used, though it would have made widget name processing easier if it had been adopted instead of . as the Tk root window.

KJN: The unfortunate thing here is the unhelpful error message empty command name, which occurs whether or not proc {} was once defined. Since an empty command name is permitted, the usual error message for an undefined command, 'invalid command name ""', would be more appropriate.

% "" 5
empty command name ""
% proc "" {x} {expr {$x * $x}}
% "" 5
25
% rename "" ""                
% "" 5
empty command name ""

WJP: Indeed, better hope that no Perl advocates see this or Tcl will never live it down. They'll be delirious at an example of something even more obscure than Perl syntax.

LV: please Please PLEASE PLEASE if you find bugs, or at least strange behaviors, submit a bug report, and, at the very least, hopefully you will get an explanation for the strange behavior...

LV: Note that if you look at the code in init.tcl , you will see where the error message is raised. It looks, to me, like the situation is this:

When, in interactive command mode of tclsh, an attempt to access a command occurs, tcl first looks to see if it is defined. If not, there is some code there in init.tcl that tries to locate an existing command for which the name is an abbreviation. So init.tcl calls info commands $name* (where name is set at this point to the empty string). The call to info commands results in a match against all tcl commands. Then it goes to look for the first match in the list of hits. So init.tcl executes:

string first $name $candidate

expecting an index into the list of the first command that matches the first characters of name.

In this case, however, string first returns a -1 . None of the items begin with a null character. To keep from hitting an error, specific code was added to init.tcl to avoid this edge case.

Perhaps what should be done is a change in the error message - I suspect the reason for the unique error message was to distinguish the case.

This is one of the reasons I would really love to see a catalog of Tcl error messages, what they are trying to convey, and what the programmer should do about it. That's a very useful catalog in Perl, and I suspect Tcl developers would also benefit.

tonytraductor: Yes...What is "Error: Invalid command name 10", etc.

KJN: Thanks for the explanation, Larry. (In this case you've saved us from reporting a strange behaviour that isn't really a bug). I agree that we need a catalog of Tcl error messages - will you add this as a separate feature request? If error messages were managed by msgcat this would also help with i18n.

LV: There are several feature requests that are relevant here - 1744149, 1214322, 1182109, 1182108, 218345. Well, the first is specific to the idea. The others relate to KJN's mention of i18n.

Note to passers-by: There are currently over 190 open feature requests. Surely there is at least one there that you could take a crack at. There are items which are merely a request for a change in the docs, some that are new docs to be written, others that are tcl code, and others that involve changes in C code. That should provide a wide spectrum of opportunities for the community to show their appreciation.

Duoas thinks not. Currently he can say things like

% proc p args {puts $args}
% set p 42
42
% p $p
42

Moreover, being able to rename a procedure is often essential when creating Megawidgets:

proc megawidget {name args} {
    ...
    frame $name ...
    rename $name ...
    proc $name ...
    ...
    return $name
}

He also thinks that most other delete/destroy/abort/rename commands are appropriately named, and have never caused him confusion.

PYK 2016-01-24: If there was no proc, and procedures were just values containing the literal content of the procedure, how would a command that is not a procedure be surfaced ad the script level?

[Isn't there a TIP for this?] [ DKF - Yes, #114[L1 ] ]

This will only be a problem for code handling Unix permission words, and in that particular case we can still accept old-style octals without compromising things too badly; I don't know anyone setting permissions using numbers parsed out of time strings! (Personally, I quite like the 0o01234567 way of handling octals, and we could add support for binary numbers (0b01) at the same time. But a more general mechanism is probably just "cute" bloat... - DKF) [L2 ]

I second adding support for binary numbers along the lines of 0b01. This is a curious gap in programming languages in general. The only language I can think of that supports such binary constants is Pike. Addendum: I now see that Ruby also has binary constants. WJP

MJ - Smalltalk has had arbitrary base numbers from the beginning with the notation XrY where X is the base and Y the number in that base. So 0b, 0o and 0x would be 2r, 8r and 16r respectively.

While there is a strong case for removing octal support in things like expr, the octal character escape should be preserved. The hex escape has the (sometimes undesirable) property of being "greedy" whereby a "\x" sucks up all successive valid hex digits. Why does this mater? I present the following situation: There is a Tcl enabled application (Modelsim [L3 ]) that fails to preserve quoting of shell arguments when they get passed into the interpreter. The result is that any argument with embedded spaces is seen as multiple arguments by the internal Tcl based argument processor (tclapp "arg1 arg2" is same as tclapp arg1 arg2). The solution is an escape character for the space. The natural inclination to use hex presents a problem. tclapp "arg1\x20arg2" produces an embedded STX,newline instead of the desired space. Because octal character codes are restricted to no more than three digits we can safely use tclapp "arg1\040arg2" or even tclapp "arg1\0401234". - KPT

ET: How about allowing constants to include an underscore for readability. For example, 0x07fff_1000. This would be especially useful if binary were allowed as mentioned above, 0b10101010101010101001010101 is pretty hard on the eyes, where 0b10_1010_1010_1010_1010_0101_0101 is decipherable. With 64 bit here, it will help even more. Seems like a trival lexical scanning change. And no reason to not allow these in decimal numbers also, including in the fractional part: 123_456_789.111_222 (FB: I second that. Eiffel allows this, and it greatly improves readability). And while we were at it, maybe augment the format command to include something like, %08_4x or %,3d or some such (and how about a %b for binary output).

WJP: If we're going to allow separators to be specified in format strings as FB suggests, we should go whole hog and provide for specification of the following information: (a) decimal point; (b) group separator; (c) low group length; (d) other group length. By "group length" I mean the number of digits between separators. The usual value is 3, but in the Sinosphere one sometimes finds 4. The reason for the distinction between "low group length" and "other group length" is that in India the traditional way of writing numbers is with a group length of 2 but with three digits in the low group, e.g. 12,34,567.89.

[What is the TIP number for this one?] [ DKF - #103[L4 ] though I don't think it goes far enough... ]

I think we need to think about the syntax of this a bit more, but the semantics are really good. The original proposal is one that doesn't break things, but isn't (IMHO) beautiful enough, so perhaps using something that is more beautiful but which breaks existing code would be better? Since it is a major version change, we could do this... (Maybe something with parentheses?)

RS: In the Tcl chatroom, dkf at one time proposed just to use back-apostrophe (`) as escape character, e.g.:

 pack `[winfo children $w] -side left

I like it for its minimality (and probably few people used it in code so far), so I second that motion.

Lars H: An alternative is to use $, with $$var for the expansion of the list $var. Then one could write e.g.

  lappend listVar $$moreItems
  destroy $[winfo children .]
  button .b $$stdargs -text $mytext -bd $border
  exec $prog $$opts1 $[getMoreopts] $file1 $file2

The meaning of $$ or $[ is not defined in the Endekalogue (it says $ means there will be substitution, but it doesn't say what happens in these cases). An advantage over ` is that there is no need to change the string representations of lists, since $ is already quoted.

Donald Arseneau: Ack! No! Any naive user who writes $$var wants double dereferencing (how do i do $$var - double dereferencing a variable), so if $$var is used for anything, that is what it must be.

Please don't hunt for scraps of unused syntax to implement new features, because that's how you create a frankenlanguage.

DKF: See also TIP#144[L5 ] as #103 was rejected.

Done in Tcl 8.5. See TIP 157 [L6 ] and {expand}. Also see the new spelling, {*}.

Again, Feather is an important project (Feather is a set of dynamically loadable extensions which have been created to try and exploit the Tcl object to its full potential. It provides an interface mechanism and uses it with various types including some mutable ones), but one which is unlikely to break existing code (well, modulo new commands created, of course!) Most of this doesn't involve backwardly incompatible changes and should hopefully be going into Tcl 8.4. However there are some areas of this which would benefit from some mild backwardly incompatible changes mainly to do with the way in which Tcl variables are structured.

LV: is there a page describing Feather somewhere that we can point to here? Wish lists without descriptions are useful to the proposer only...

KBK: You can get a link to the Feather page at [L7 ] The page itself is currently at [L8 ] but Paul's changed ISP at least once, so tracing through purl is probably the better option. Oh yeah, I also ought to mention that Usenix members can get his paper from [L9 ]. Does anyone know if Paul Duffin is still actively pursuing this?

DKF: He keeps talking about Feather on news:comp.lang.tcl , so I suspect he is still (at least semi-)active on it.

LV: From info from this past summer, Paul is no longer purusing this, and the original source code is no longer available (it was lost during a move). If people want this functionality, then someone needs to read over Paul's paper and start either from there, or come up with their own design.

LV (later): The source code for Feather is now available at http://tcl.sf.net/ , but said to be a pretty gnarly hack that really needs re-implemented.

LV 2011: I wonder whether, with Tcl 8.6, how much of the feather concepts (not implementation) are possible to implement. Paul talked about creating vectors, maps, hashes, structures, generics, curry, lamda, overrides, as well as several other concepts in the paper.

(something for the future …)

This is related to both of the previous topics, and would make supporting things like commands with variable contexts much easier. This would break code that has commands with spaces in the names, but that's pretty rare.

This will break lots of scripts since the natural outer command is currently a list constructor, but people are already proposing ways to (semi-)automatically fix this. Everything else can be handled by existing mechanisms.

Carsten Zerbst summarizes the pros and cons of this as

 + cleans up the interface
 + for every newcomer easier to learn

 - breaks compatibility very much
 - there is no chance to save things by a compatibility package.
        e.g..
        tcl8.1                          tcl9
        set l [list create test]        set l [list create test]
        {create test}                   {test}

A proposed solution for this is a converter script which should help with many things.

Lars H: What's wrong with using List as the base command? In the distant future it might require people to think a little more, but it will save an enormous amount of scripts from breaking. Then perhaps around Tcl 10 you could consider switching from

interp alias {} list {} List create

to

interp alias {} list {} List

as default if people have really started to prefer the subcommand form.

TV: What's with the constructor idea as of higer order (cf. 'object') than the fundamental primarily 1 dimensional datastructure list, and eval of that ?

AMG: See my comments on #67, below.

Fabricio Rocha: 2010-04-10: Incompatibilities like this are expected in new major versions, aren't them? If it comes for the sake of evolution and natural coherence (we already have array and chan working with subcommands), it is worth the change for sure.

bll 2014-2-23: I think it's a really bad idea to break backwards compatibility. How about:

        tcl8.1                          tcl9                         tcl9
        set l [list create test]        set l [list create test]     package require Tcl 9.0
        {create test}                   {create test}                set l [list create test]             
                                                                     {test}

The package require is now overloaded with two meanings. (a) The tcl version that is required to run the script and (b) whether or not backwards-compatible features will be enabled.

DKF: I think that changing list is too big an incompatibility; we want as many existing scripts to work (with no/minimal changes) as possible, saving the incompatibilities for areas where it is strongly justified. Making list be an ensemble is a beautiful plan, but one that runs into the rocks of reality as it forces too many changes. (Doing magic with package require is just scary; it's already too complicated for its own good, even if necessarily so…)

HJB: I agree with the others here suggesting that the list commands be migrated to an ensemble/subcommand form.

Regarding Lars H's suggestion: Why not instead alias the command with a bang, e.g., "list" as "list!"? Bang would not be a new keyword or anything, rather, just a naming convention for pre-ensemble commands which have been aliased. The benefit is that users can still use the outer commands with minor adjustment ( and this adjustment could be performed trivially by a script ) while still being gently reminded to update the command to its proper ensemble/subcommand form. For example:

        Old Command                     New Subcommand               Subcommand Alias
        concat                          list concat                  concat!
        join                            list join                    join!
        lappend                         list append                  lappend!
        lindex                          list index                   lindex!
        linsert                         list insert                  linsert!
        list                            list create                  list!
        llength                         list length                  llength!
        lrange                          list range                   lrange!
        lreplace                        list replace                 lreplace!
        lsort                           list sort                    lsort!
        split                           list split                   split!

...a similar change could be made to the io commands, e.g.:

        Old Command                     New Subcommand               Subcommand Alias
        close                           io close                     close!
        eof                             io eof                       eof!
        fblocked                        io blocked                   fblocked!

...etc.

bll 2016-5-3: Breaks backwards compatibility. Something like this would be better:

        tcl8.6                          tcl9                         tcl9
        % set l [list create test]      % set l [list create test]   % pragma use-list-ensemble
        {create test}                   {create test}                % set l [list create test]             
                                                                     test

Backwards compatibility is preserved, yet the language behaviour can be adjusted through the use of pragmas. Or:

        tcl8.6                          tcl9                         tcl9
        % set l [list create test]      % pragma no-list-ensemble    % set l [list create test] 
        {create test}                   % set l [list create test]   test            
                                        {create test}                

In this case, upgrading an old program to tcl-9 would require the addition of some pragmas, or using a simple wrapper script to set the pragmas and start the program. The pragmas could also be set via command line options in tclsh.

HJB: I could be in the minority, but I'd prefer your latter pragma option, or, the bang aliases, to the former, "use-list-ensemble" pragma. Users should have to "opt-in" to the old outer list commands. The reasoning is that Tcl will be easier to "sell" to new learners if the language is presented as consistently as possible by default. Ensembles are beautiful IMHO; easy to memorize, tidy, and bring structure to Tcl's command set. The old outer list commands ( llindex, llength, lsort, etc. ) are discordant; much like Common Lisp's car and cdr, they bring to mind "cruft" and will likely turn off new users who could be tempted by the relative consistency of Python 3, Lua, Ruby, etc.

Having said all that, I think your earlier suggestion, "package require Tcl 9.0", might be the best option listed. Also, having the compiler/interpreter give a warning/error if it detects the "list" command used without one of the legal subcommands ( concat, join, append, etc ), or, if one of the superseded outer list commands are used ( concat, join, lappend, etc ). The warning/error should provide a suggestion to use the proper list ensemble command, and thereby help the end user to ween themselves off the old outer commands.

bll 2016-5-5: DKF did not want to overload the package require. The second option I outlined would work out fine. The various pragmas could be turned on to ensure compatilibity, but display a warning. Then after a year or two or three, turn them off.

bll 2018-7-26: This is essentially impossible without converting *all* other scripts. All of the list sub-commands become completely unusable as the first argument to the older form of the list command (e.g. after 1000 list set timerpopped 1). That's too much breakage. So the only way to do this is to create a new command: nlist (for new list), or ? And that's just a simple package anyone can write.

Support for a general context mechanism which can be used as the basis for all existing contexts (e.g. procs, namespaces, global namespace/interpreter context) and is exposed at the Tcl and C API level so that it can be used as a building block for [incr Tcl] and other OO frameworks.

Zarutian 2005-07-03: would stackframes fall under general context mechanism? (when you invoke a proc an new stackframe is created which holds the values of local variables)

There are many C functions which are no longer necessary. There are four ways to set a variable from C. (Tcl_SetVar, Tcl_SetVar2, Tcl_SetVar2Ex, Tcl_ObjSetVar2) The only difference between these is the type of parameters they take and could easily be replaced by two a string and a Tcl_Obj version. They also need to be generalised to support multi-dimensional containers.

MAK: See also: [L10 ] regarding using format/printf() style arguments instead of Tcl_AppendResult().

There have been some good arguments for changing namespace syntax to be more Tclish. [L11 ] The namespace separator :: was borrowed from C++, but a more natural Tcl expression would use the space character (" ") to separate namespaces, so fully qualified names are actually lists. So instead of a::b, you would use {a b}. Namespaces become a sort of dispatching mechanism, and major/minor commands become trivial. It has also been pointed out that namespace resolution presently ignores parent namespaces. - RWT

LV: So what, one would say

{blt table} -flag -arg stuff -otherarg morestuff

? That's rather ugly isn't it?

DKF: That is ugly. The aim of TIP #112 is to also have each namespace be a command, so that you could write the following:

blt table -flag -arg stuff -otherarg morestuff

Integrating this with ensembles is also a goal (i.e. this and the next point are really one and the same IMHO.) Extending (or, conversely, restricting) core commands becomes massively easier like this.

MGS: How about mapping namespaces to a file-like hierarchy? So that tcl procs are like programs? For example:

/blt/table -flag -arg stuff -otherarg morestuff

Larry Smith: Although this idea from Unix is readable, it's likely to cause cognitive dissonance in the Windows world, where they can't use \blt\table because of backslash's built-in meaning. I will suggest an idea from Multix/GCOS which used > and < for pathnames. >blt>table would be the pathname in this case. The < character served the same purpose as /.. in Unix (and far more elegantly, in my opinion) so to refer to a sibling namespace, one would write <sibname>and>so>on

Each namespace could then have a PATH variable for finding commands, instead of namespace importing and interp aliases.

set PATH [list / /blt]

Chang: The "command subcommand args ..." is more tclish. Is it easy to replace :: by space?

I disagree on this, the namespace denotation looks relatively clean, is easy to spot, and references a similar functionality in another language making it easy to pickup. the file system look is confusing when looking at unfamiliar code.

Also, imnsho, forcing a namespace to be a command makes assumptions on the usage of namespaces for the coder. Some might use a particular namespace to hold state only.

I would say this is one area that tcl is currently doing well, and attention should be focused elsewhere.

LV: Please make sure that as terms are tossed out one defines what you mean, so that we make sure that we are are agreeing, or disagreeing, on the same things ;-) .

AK: For that see (and extend/change) our Glossary of terms.

DKF: Commenting on cross-relevance. Namespace rationalisation interacts interestingly with leading word expansion which interacts interestingly with ensembles.

Done in Tcl 8.5. See TIP #112 and namespace ensemble.

It should be made easier to separate out features that are unlikely to be needed in embedded systems. Ideally, only a handful commands like set, while, proc, uplevel, upvar, catch, expr and a reduced version of load should be in the interpreter, with everything else linked in as static packages. The reduced load command could then be used to initialize all the other stuff as needed. Discussion moved to MicroTcl for Tcl9

Static packages should not need a pkgIndex.tcl somewhere on disc. And, instead of [load {} PkgName], perhaps a special command could be provided. Discussion moved to Better Static Package Support for Tcl9

An OO system provided by the core should use Tcl_Objs, so that objects get deleted automatically when they go out of scope. Discussion moved to OO System for Tcl9

discussion moved from #26 below

A thread actually a bit forgotten, the (optional) OO extension

8.6 includes a general system for shipping external packages with Tcl, and there is a version of itcl in it.

DKF: So long as it is not integrated with the core but merely shipped in Sumo/BI, this is not an incompatible change.

TV: One may want to think about what such buzzword in effect means, for instance in terms of list operators, maybe half a dozen things like constr, data and functions grouped togehter, a hierarchy of functions/data and inheritance are more a matter of style of implementation than a fundamental issue. Having procs which are also lists and vv. is a fundamental and relevant issue.

Done in 8.6. See TIP #257: Object Orientation for Tcl (spec at [L12 ]) and TclOO.

Arrays should be objects that can be passed around and returned. Discussion moved to Better Arrays for Tcl9

Sounds like dicts, already in Tcl 8.5. See TIP 111 [L13 ] .

14a. Get rid of arrays, and use the name(index) syntax to refer to dictionary variables. Discussion moved to Better Arrays for Tcl9

It should be possible to use shared libs without any wrappers written in C. Discussion moved to Direct Shared Library Use for Tcl9

This appears to be addressed by TIP 239 [L14 ] (and, somewhat, TIP 357 , but isn't really done yet.

Such as case, puts $chan $msg nonewline and the old style of pack

Volker: Seconded. Also, for the C-Interface. Perhaps introduction of a tcldeprecated.h. Or, what about versioning the headerfiles like the libs? tcl83.h?

DKF: I'm not really in favour of versioning the C interface too much (for one thing, it goes strongly against the principles of stubs if you then force everyone to hard-code in a particular version of header file!) but doing so on a basis of just the major version might work (i.e. tcl8.h) I'd rather not though if possible, especially since the version is already #define-ed.

If the stack trace is a true Tcl list, it is much easier for wrappers to properly make themselves transparent by removing their entries from that list. As is, the burden of parsing $::errorInfo as a string is so hard few people bother to try. See [L15 ]. This is conceptually a simple change, but it touches many portions of the core. -- DGP

This is also related to TIP#90 [L16 ]. Why not make a list with the entire "error context"?

Done in Tcl 8.6. See TIP 348 [L17 ] and info errorStack.

Most, if not all, global variables (see [L18 ]) should be replaced with either commands ($tcl_version --> [package present Tcl]) or properly namespaced variables ($tcl_platform --> $tcl::platform). Compatibility support can be provided through Tcl 9 with variable read traces. -- DGP

TV: What's the fundamental problem with global variables? When something is stored programwise, one stores it in a global variable, and when the program doesn't have too many names to distinguish, what's the point of using seperate name spaces? Jeez, those OO-ers have a religion of their own. What became of the hardware java machine, was it so much more efficient we all buy it nowadays?

DKF: 64-bit platforms are becoming ever more common, and at the moment Tcl only supports them insofar as it works on them, as opposed to actually making best use of the platform. This can be broken down into several sub-jobs:

• Reorder common structures to get better alignment
• Make string and list lengths "long" which will allow references to data structures containing far more data than is currently possible.
• Make ckalloc() etc. use "unsigned long"
• (I know there was something else here...)

All of these should have no ill-effect on 32-bit platforms, other than (at worst) completely breaking binary compatability. They all definitely break binary compatability on 64-bit platforms, but that's the point. On the other hand, they should be largely source-compatbile, since static structure declarations (like Tcl_ObjType) will not need to be changed; if there are any alignment issues, they won't need tackling because the number of structure instances should be quite small. The impact on the core (and anyone using things declared in tclInt.h) will be larger, but should affect fewer people and less code.

See TIP#115 [L19 ]

Sebastian Wangnick: The socket command should support UDP sockets so that datagram-style communication (e.g., with DNS servers) can be done.

sergiol 2006-09-13: One of my wishes is to add UDP sockets to the core of the language in the style [socket -type udp]. i am beginning my experimentations to implement that.

DKF: For SOCK_STREAM, that would fit nicely with socket. For SOCK_DGRAM, that would fit with TIP #409 . All that would be needed would be a mechanism to recognise that a socket name is a filename and not a hostname/port number.

I don't know if Windows has anything similar.

RM: How about speeding up the I/O.

TV: Why, is it slow?

DKF: I'm told that there's been quite a bit of effort put into improving 8.6.2. Is it faster now? I've not tested thoroughly, but if it is, dgp is the one to thank.

Carsten Zerbst: Putting all Tk related things into the namespace Tk, make it a usual loadable extension.

 + cleans up the namespace
 + via namespace import won't break old apps
 - ?

DKF interjects: Was agreement ever reached about what effect this should have on the termination behaviour of scripts? (tclsh terminates at end of main script, wish terminates after end of script only when all windows are deleted.) Whatever happened to the [package require Wish] suggestion?

Carsten Zerbst continues: This is part of TIP 248 [L20 ], approved for Tk 8.5.

DKF: Note that the namespace cleanup is definitely more of a 9.0 thing.

#24 was a duplicate of #6

Generally a possibility to extend the "command subcommand" syntax on the script level with own subcommands

DKF: This sort of thing might follow from TIP#112.

Done in 8.5. See namespace ensemble.

#26 was a duplicate of #13

Carsten Zerbst: I'd like to see more extensions shipped with Tcl (Tcl Sumo Distribution Kit)

DKF: Again, this is not incompatible.

Done in 8.6. (At time of writing, shipping with itcl and tdbc; plan also to have thread and maybe sqlite.)

Another important thing (IMHO), a unified DB interface

DKF: See the news:comp.lang.tcl thread on tcldbi. [L21 ]

EG: Since 8.6 a unified database interface (TDBC) is bundled with Tcl.

Done in 8.6. See TDBC.

GPS: I would like support for more flexible indentation of curly braces (Allman style indentation). Then I wouldn't have to use \ to get the curly braces the way that I want them.

This discussion, which ended with

GPS: I would like to note that I have changed my mind and don't currently feel that this aspect of Tcl should change.

has been moved to Why can I not start a new line before a brace group.

RLH 2005-12-22: I would settle for uncuddled else statments. I don't want to hack around it. I want it to just be a part of Tcl.

PSE: command line options to support package loading and one-liner behaviour a la perl.

DKF: that's just a little Tcl script. Got one handy?

RS: How about owh - a fileless tclsh ?

FW: A switch for the interpeter executable (-c?) to emulate Tcl 8.6 (or whatever the last version is before 9.0) for backward compatibility. This would solve the "list" subcommand issue, as well as any other incompatibilities

FW: Here's a radical (and probably dubious) idea: Rename a common command (like "puts") to something else, and then as the interpreter compiles the script, if it notices a use of the "puts" command, revert to compatibility mode. (DKF i.e. autodetect backward compatibility mode)

DGP: Both of the last two comments assume that emulation is a satisfactory solution. I don't think it is, because it erects a wall through which no integration of packages is possible, and being good at package integration is fundamentally important for Tcl.

glennj: Speaking of packages, [package require Tcl 8.x] will fail in Tcl 9.x, will it not?

DGP: I haven't TIPped the idea yet, but in Tcl 8.5 I want to extend [package require] to accept ranges of acceptable version numbers, not just a single minimum acceptable version. () See package BoF: PACKAGE REQUIRE support for RANGES of acceptable versions. With that in place, then [package require Tcl 8.2] would fail in a Tcl 9 interpreter, but [package require Tcl 8.5-10] or [package require Tcl 8.5-] would not. That provides the bridge for a script that runs in both Tcl 8.5 and Tcl 9.x.

glennj: I would suggest the syntax:

package require ?-exact? package ?version1? ?version2?, where version1 is the minimum acceptable package version, and version2 is the maximum acceptable package version. version2 can be the string "-" to represent "at least version1". The option -exact is ignored if version2 is specified.

DGP: A maximum acceptable version is a maintenance nightmare. Instead, the max end of the range needs to be the first version not supported. It's the same as the distinction between an open and a closed interval in mathematics. Also, having string arguments that look like ranges allows for multiple ranges in a single [package require]. And the less said about -exact the better. Before we continue to repeat things already gone over, be sure to do a Google search for the many comp.lang.tcl discussions in this area.

GPS: I would like the math commands + - * / in the standard Tcl 9.0.

Discussion moved to expr shorthand for Tcl9.

Done in 8.5.

jcw 2002-10-30: I would like An objectifying hook for Tcl 9

DKF 2002-11-18: Asymmetric channel close so that you can send an EOF on a socket or pipe and still read data back. This is useful with Unix commands like sort...

Done in 8.6

2002-11-28: I'd be happy to see the removal of the special boolean strings: "true", "false", "on", off", "yes", "no". Tcl's if command could be cleaned up: remove "then" and remove the optionality of 'else' before an else-block. Contraversial i'm sure ;)

DKF: I don't like this idea at all.

Breadcrust: Are you crazy? that would totally break backwards compatability as well as just end up making (tcl) code that would be not as clean. And not all people like using 0 and1 boolean. I prefer using true and false.

wdb: Please NOT! I really like the way of saying "yes" and "no" instead of 1 and 0. Really!

RS 2002-11-28: Extend expr to vector/matrix operations.

If one operand to a binary operator is a (possibly nested) list, and the other a scalar, perform the operations element-wise on each. If both operands are (nested?) lists of same structure, apply operator pairwise. Else throw error (like before).- Also, allow "and", "or", "not" keyword aliases for "&&", "||", "!".

DKF: This is a really simple parser change. Prototyping it should take about half an hour or so; you just need to alter the lexer part of the expr parser to detect the words and return the correct token...

DKF: Note that extending the domain of expr's operators is quite difficult as they're implemented directly in the bytecode engine.

AM 2014-07-11: A very good candidate for this is Christian Gollwitzer's VecTcl package

FW 2003-01-04: Make it do nothing when you attempt to give the empty variable name a value rather than actually setting it so you can use {} as a dummy variable when things like regexp require a variable argument without worrying about memory consumption.

AJD: Perhaps this would be better as just a special case in regexp. In this case i would suggest that a dash indicates 'do nothing'. eg. [regexp {((a?)b)(c?)} $s - - A C]. Saves a keystroke (- versus ""). It also probably reflects current practise. Maybe even allow "->" as a 'do nothing' in the case of the matchVar. AJD: Just tried a simple patch to tclcmdM-Z.c to do the above:

% time { regexp {(a)(b)} ab mv A B } 10000
31 microseconds per iteration
% time { regexp {(a)(b)} ab -> - - } 10000
26 microseconds per iteration

OK, so it's not a big win, but as Paul Daniels would remark, every microsecond counts.

AJD: Jeff Hobbs has pointed out on tcl-core that this optimisation is already available to the submatchVar's within the re_syntax as "(?:)". Learn something new everyday! The "->" optimisation for the matchVar is still a valid possibility eg. regexp {^([^:]*):} $str -> x.

Lars H: OTOH, to generally declare that the empty name variable doesn't exist could be useful with other commands than regexp. It is not uncommon to see procedures with optional arguments that are names of variables in which to return extra data. If the empty string was not a variable name, then one could declare that upvar {} somevar simply makes somevar a local variable, and thus simplify the common idiom

proc do_something {a b {optVarArg {}}} {
    if {[string length $optVarArg]} then {upvar $optVarArg optRes}
    ...

to

proc do_something {a b {optVarArg {}}} {
    upvar $optVarArg optRes
    ...

without leaving stray variables all over. Another possibility this opens up is that $(...) can be used as an expr shorthand for Tcl9, but that is a different topic.

ulis 2004-01-10: Empty name variable is already heavy used by stooop, hugelist and aliens, and surely many others. Changing its use will be a great incompatibility.

Multi-assign (several closely related issues)

Done in Tcl 8.5 with TIP 57 .

Larry Smith wanted a special syntax to invoke expr; discussion was moved to expr shorthand for Tcl9.

(Item moved here on 2003-01-10 from Tk 9.0 WishList.)

Larry Smith wanted built-in stacking

(Item moved here 2003-01-10 from Tk 9.0 WishList.)

Larry Smith: I want something like getparams

(Item moved here 10 jan 2003 from the Tk 9.0 WishList.)

Larry Smith: Building-in Thingy: a one-liner OO system may be the most minimal change that can be made that turns Tcl into an object-oriented language without the massive politics of blessing one of the many oo extensions.

MS asks: isn't interp alias enough - as described in the new version of Thingy?

DKF: Tcl 8.6 includes TclOO which is thoroughly object-oriented.

(Item moved here 2003-01-10 from the Tk 9.0 WishList.)

Lars H 2003-01-14:

A script level interface for defining commands that get byte compiled

One application of this could be to remove the performance penalties on many types of syntactic sugar. Consider a macro command which has syntax like proc, but whose body (i) returns the code that is really to be evaluated and (ii) is evaluated when the command is to be byte compiled. Then one could define things such as the let suggested in item 40, and have the advantage of byte compilation, without having to add them to the Tcl core. If <BODY> is such that

proc let {args} <BODY>

would return

foreach {a b c} [info commands] {break}

in response to

let a b c @:= info commands

(or whatever, but this is the syntax on the let page), then

macro let {args} <BODY>

should have the effect that the byte compiler, upon seeing this let in a procedure, will evaluate the <BODY> and instead of the let command compile whatever script the <BODY> returns (presumably the foreach construction above).

Roy Terry 2003-04-02: The Tmac [L22 ] package implements this concept as "filter" macros. It also provides a block macro to substitute a fixed block of text/code with parameter insertions. Alas, the macro invocation must be delimited so that the preprocessor can find it. Exception could be made for this requirement but the result would not be very robust. Also, as JCW notes below, the macro doesn't have access to valid variable values since it's called before the code runs.

This macro command is however not sufficient for many control structures. I would like to see some interface that would let me implement breakeval!

jcw 2003-01-15: This is similar to Forth's "immediate" command, which tells the parser to evaluate at compile time. It's not possible to get this right for the general case in Tcl: ... [$var ...] ... cannot do anything at (byte-)compile time, since it would have to see what $var contains, which in turn may not have been set yet. But for the limited case of verbatim macro names, it can probably be done.

One could do even more. The moment the word "let" is seen as command, and it is recognized to be defined as compile-time action, it can take over the rest of the parsing. This means that notations such as "while a != 3 do { ... }" (no, there are no typo's in there) could be dealt with. In other words, seeing "let" with a space after it in the place where a command name is expected, could cause the compiler to turn control over to "let", and have it deal with parsing its own args. Taken to extremes, the compiler "vanishes" into a number of macro's called proc, set, if, expr, etc.

Lars H: It would definitely be necessary for macros to deal with arguments that will be subject to substitution. E.g.

let a = 2 * $b

must be possible to translate to

set a [expr {2 * $b}]

if desired. I was thinking that the macro body should receive the command parsed into words, but that these should not have been subjected to substitution (nor stripping off braces or removing of quotes).

Roy Terry: Yes Tmac offers this kind of parsing by default and does not use the Tcl parser generally. Your suggestion goes a bit further, and is more powerful, but it would require that the macro body can make use of the parser. (It would BTW be a good thing if scripts could do that in general.)

DKF 2010-10-20: KBK presented GSoC work on TAL which makes it look like it may be possible to establish enough of a set of safety guarantees to actually allow for cool stuff to go in the core. Very interesting work; watch this^Wthat space?

moved to Getting rid of the value/command dichotomy for Tcl 9, 2003-01-15, jcw

FW: Add -nn as an alternative to the -nonewline switch of puts for brevity.

Donald Arseneau: I find -nnl more mnemonic.

DG: Similar to 20. and 21. but up the requirements to include the ENTIRE WinSock-2 API on windows to bring us not only UDP, but also NetBIOS, IrDA, IPX, AppleTalk, Vines, IPv6, and even allow CLSIDs for protocol providers not yet known. Also improve the WinSock I/O model as the current Win3.1 interface of WSAAsyncSelect has performance issues and limits Tcl currently. Overlapped I/O using completion ports would be a sweet performance improvement (on the NT OSes) and allow WinSock to keep going on accept'ing and read'ing should the event loop in tcl be running a little slower than the network events coming in. Currently things crash in an ungraceful manner when the input is faster than Tcl is able to service it and is a pure I/O model issue. And for the fun of it, maybe add all ioctl's in existance as fconfigure's like -keepalive, -nagle, etc.. And the way socket error codes are returned needs some work. When a connect fails, all we seem to get is "invalid argument". What a PITA. Give me the real drawn-out one, please. Tell me "host not found" and was that the authoratative one or not?, or WSAEPROVIDERFAILEDINIT and stop the info loss in the POSIX error code translation.

This needs to be for 9.0 as the generic part of the socket command is going to have to take some changes to provide additional protocols with an understanding or passing off of the new forms.

 set sock [socket -ipx 0xFEDCBA98 0x1A2B3C5D7E9F 0x0453]

IPX args would be:

• A 4-byte network number (server)
• A 6-byte node number (server)
• A 2-byte socket number (server process)

A beta implimentation of this is ready @ [L23 ]

DKF: What's involved in setting up an IPX server? Can we discover the network and node numbers for people? (I presume you'd have to still specify something like a socket number, as it'd seem very limiting to only be able to open one socket per process.)

DG: Name resolution in winsock can include NDS [L24 ] and other dynamic types. I haven't played around with this stuff yet. Looks interesting... I think they call it process (for socket) by convention and isn't a per-process limit [L25 ].

We could also do with a decent abstraction for packet-oriented sockets; they don't really fit too well with the current channel-based model, and that'd be useful for all sorts of protocols (notably UDP.)

DG: The lolevel part of the driver should work just fine with UDP. I haven't tried attaching a UDP implimentation to it yet, but the gutts are there and waiting [L26 ].

DKF: I'm not worried at all about the low-level stuff. I'm worried about the high-level abstraction! In TCP, a lot of effort is done (behind the scenes, of course) to make sure that if the sender sends a byte, the receiver will get that byte once and only once (or a detectable failure happens.) You don't get that guarantee with UDP, which merely offers "best-effort" semantics (plus port numbers and CRCs.) This means that the natural level of abstraction for a UDP (or any other packet-oriented protocol) is that of the packet, and not the stream. Tcl's channel abstraction matches virtually perfectly with streams, and not at all well with packets.

DG: We do have the difference between gets and read. Could tcl channels be oriented toward message-base by somehow manipulating -buffering to be message-based? Appletalk and IPX: your days are numbered! [L27 ] [L28 ]. Just a reference.

(If anyone wants to add [fconfigure] support for the various things you can fiddle with via ioctl(), please be my guest. All it takes is a little TIP and a little coding work.)

FW: How about an 'event_loop' command that, given a single boolean argument, enables or disables the event loop. Not given an argument, it returns 1 or 0, for whether the event loop is currently activated. This would allow one to do this:

event_loop on

Rather than the significantly weirder and less logical

vwait forever

I've yet to receive an answer on whether it's even technically possible to turn off the event loop, so comments are encouraged.

DG I don't think it's possible to turn it off mainly because from where you might call event_loop off would most likely be from an event loop invocation itself. See tk/generic/tkEvent.c for Tk_MainLoop():

void
Tk_MainLoop()
{
    while (Tk_GetNumMainWindows() > 0) {
        Tcl_DoOneEvent(0);
    }
}

Which translates to english to mean "keep the event loop running until all windows go away". When Tk_MainLoop() exits, so does wish. What's wrong with tclsh and being able to fall into the event loop and control it with vwait?

DGP: I'd like to see this one addressed via an (ab)use of package. Have those packages like Tk that startup an event loop also package provide eventloop, and have any script that wants an event loop running do a package require eventloop, with a simple fallback "eventloop" package that does nothing more than the equivalent of a vwait. This way we could avoid some of the event loop nesting that happens sometimes. This might help Tk, which is a bit less happy running within a vwait than it is within its own Tk_MainLoop().

PYK 2015-03-12: Could vwait, update, and update idletasks be removed? I've never seen those commands put to legitimate use. Perhaps a command to start the event loop if it isn't already running is all that's needed. For routines that really feel the need to block the loop, there's after 0 .... Obligatory link to Update considered harmful. I guess the first step might be to eliminate use of update idletasks from Tk, as a proof of concept.

DKF: Very unlikely to get rid of vwait; vwait forever is a common idiom in Tcl that isn't doing anything with Tk at all. (And yes, I have used both update and update idletasks correctly. Not recently, but I've not been writing complex GUI apps recently.)

AJD 2003-02-03:

It would be nice if the core included commands for the stack functions lpop, lshift and lunshift (lprepend?). Of course these are simple to code in Tcl (remembering to use the K-combinator with lreplace/linsert!), but i would argue these are common enough to justify inclusion in the core (8.5?)

Lars H: Agreed, but this is perhaps rather an item for the What feature or functionality is missing in Tcl page? Indeed, the lshift command is already mentioned there.

MSW 2003-02-17: expr to understand bare-names

Mmmm. I am personally disturbed by having to use $ in expr. Maybe change it so that if expr can determine that something can be a variable reference (i.e. variable of the print-name in scope), you can omit $. When it cannot determine that it's a var, it assumes it's a string. Particular behaviour can be forced by appropriate quotes or a $.

% set a 1; set b 2; expr a + b 
3
% expr "a" == b
0
% proc x { } { expr ::a + ::b }; x
3
% set b "a"; expr $a == "b" 
0
% unset a; expr a == b; # a is string, b is var
1

See also expr shorthand for Tcl9.

MSW: Well, not exactly :p my suggestion is not to get rid of expr, but to get rid of dollars in expr :) (but see also fits well tho i guess)

Lars H: I don't think any of the suggestions there are incompatible with expr. Several are precisely shorthands for expr.

But let's try to be formal about this. Suppose there was a command like

proc echovar {name} {
   if {[uplevel 1 [list info exists $name]]} then {
      return [uplevel 1 [list set $name]]
   } else {
      return $name
   }
}

(if it is a variable, then return its value, otherwise echo the name) is it then correct to interpret your suggestion as: a stray word in expr expressions should be parsed as [echovar word]? This would mean

expr a + b

is parsed as

expr {[echovar a] + [echovar b]}

which is equivalent to

expr {$a + $b}

if a and b are variables.

MSW: Yup.

Donald Arseneau: I think this proposal is terribly inconsistent. OK, I know that expr already has its own syntax, but that is no excuse for abandoning what little consistency remains between it and the rest of Tcl.

slebetman: Please no. I would be personally disturbed by not having to use $ in `[exprBarewords should always be treated as a string, even by exprs. Unfortunately currently in some cases you need to quote for words to be treated as a string. I personally prefer to be able to write:

if {$a = test} { ...

That would be more consistent with the rules of Tcl but currently the code will complain about a syntax error. I can see how this is a good thing in avoiding bugs so I'll live with it for now.

wdb: I agree with slebetman. The differences between Tcl and expr should be kept to a minimum. I'm disturbed by the enforcement of quotes or braces with strings, because in Tcl, you can "forget" them sometimes ... but before any improvement proposal, I must think of the side-effects.

MSW 2003-02-25: Some more sugar for my poor teeth. Currently we have

if { condition } [then] { then } [else] { else}

with then and else being syntactic sugar. A do would be nice for all those sh programmers :)

while { condition } [do] {body}
for {init} {test} {update} [do] {body}
foreach {var(s)} {producer} [do] {body}

or even (sh de luxe)

foreach {var(s)} [in] {producer} [do] {body}

supposedly easy to be done, wouldn't hurt people who don't want it, make others happy, cost about nothing. woohoo :)

DKF: There are problems with foreach because the current syntax allows for iteration over multiple lists at once, which is sometimes very useful indeed. Suggestion is feasable for while and for though.

AJD No need for any core change... this is Tcl remember ;)

rename while __while
proc while { args } {
    if {[llength $args] == 2} {
        uplevel __while $args
    } elseif {([llength $args] == 3) && ([lindex $args 1] == "do")} {
        uplevel __while [list [lindex $args 0] [lindex $args 2]]
    } else {
        error "usage: while EXPR \[do] BODY"
    }
}

MSW: DKF, works fine for foreach too, check if the number of arguments to it is even- or odd-numbered (including the block). If it is even-numbered, it is either a syntax error or includes the do.

foreach {a b} {cmd1} {d c} {cmd2} do {block} <-- even numbered

@ AJD: Who says I'm waiting for a core change to do this, this is a wishlist of things we'd like to see in 9.0 - if it is there already, no need for every programmer to sweeten his programs on his own :)

Lars H: Note that with wish #45, syntactic sugar such as this could have the advantage of byte-compilation without any need to go into the core proper.

MSW 2003-03-18:

I think this is self-explanatory -- SCNR:)

DKF: Hard to mix this with command and execution tracing. But it'd be cool if it could be done all the same.

MSW: You know, if a misleading errorinfo (only one stack-entry where there should be a couple of them) and command/execution tracing is the price for getting tail call optimization, I'm happy to pay it. Maybe follow some suggestion on the Tail call optimization page and have it be a runtime tunable option from within tcl, even default off so you have all the tracing etc. People who bomb their stacks (gotta love to see bus error (core dumped)) can then turn it on and be happy :)

Lars H: The returneval command could provide a conceptually clean way of asking for tail call optimization (even though the current script level implementation will cause the stack to grow, thus missing the point). I wonder if this could be improved ...

Done in 8.6. See tailcall; you have to use it explicitly, but you can do it.

TV 2003-04-02:

I'm not sure how to date this request is, but long ago when I started with Tcl/Tk, I found it possible to take the configuration information of components in Tk, check whether they are all default value, and automatically scan the whole hierarchy of windows and store their state. More or less, that worked. I don't have access to the work I did then, and I know that there is a list of values for each configuration property of which roughly the last value is the current active value, but I didn't check recently whether it is reasonably possible to save the state of a graphical interface by neatly walking a window hierarchy and intelligently storing all config info. I guess the request is to allow that in an organised manner, which to me, apart from porting that information, would technically seem reasonable. Maybe one should purchase TclPro instead?

TV: Maybe just talking to myself on this item, but I just found there is no distinction possibility between

$widget conf -text
$widget conf -textvariable

possible without checking the widget class. Or in other words, the handy shortcut idea cannot be overridden to force a complete match.

rmax proposed to allow an in operator for expr, so expr {$val in $list} is equivalent to expr {[lsearch $list val] >= 0} ?

RS: I would sure welcome that very much - it is a bit down Perl road, but the savings in writing, and the gain in readability, is just so much. It also reminds me of an earlier proposal: allow in as second arg for for, which causes an error now, so for i in $list acts like the simple case foreach i $list ..and allowing and, or, not as aliases for &&, ||, and !.

I think Tcl 9 can well stand a facelift, driving it closer to natural language.

FW: Many people will claim trying to be a natural language isn't in the "Tcl Way" (which they claim is to try to be so simple it doesn't have to be like a natural language)... I think this should be true of the command syntax, but the usage of the actual commands can be as sugary as you want and still feel like Tcl, IMHO.

RS: See TIP 133 for a comparison of

if {[lsearch -exact $list $element] >= 0} {#current Tcl} ;# and
if {$element in $list} {# proposed new operator}

I think the second is more natural, more simple, and still in the spirit of Tcl.

FW: MOOcode, by the way, uses this method as the exclusive means of doing case-insensitive list searching, so it's not like we've just invented the idea. Python just introduced this feature as well.

RLH - I like that. It is very clean.

TV 2003-04-13:

Would it be reasonable to wish a lsearch which, like lset, is recursing into sublists ? I'm sure I'd find it powerfull, though I'm not sure what the gains would be wrt doing such 'by hand' using a script (e.g. in Recursive list searching). And of course there'd be the compatibility issue.

DKF: You'd need to specify a depth bound because a term like "a" is both a singleton list and a leaf; Tcl syntax doesn't distinguish between the two.

On the other hand, we could turn this to our advantage a bit since specifying the search depth provides a fairly natural way to enable deep searching. And we could trivially retrofit lsearch to return a index path as a list; the current result is indistinguishable (i.e. it is a degenerate case.) Thus, we might have:

set list {
   {a b c}
   {d e f}
   {g h i}
}
lsearch -depth 2 $list f   ;# => 1 2
lsearch -depth 2 $list g   ;# => 2 0
lsearch -depth 2 $list no  ;# => {} or -1

The main issue is what to do in the not-found case. Could be an empty list, or could be -1.

Is this covered by TIP 127 ?

wdb: As I understand, no. The search criterion is here that the element is anywhere in the list items, but in TIP 127, it is at a special index.

GPS 2003-04-24:

It seems that 99% of the time people don't care about the case of a command. I think that Tcl should not care whether a command is written as Cmd or cmd or CMD. This approach has worked for Ada and I think it will help prevent bugs, and settle silly arguments over style.

LV has had the same desire about package require names - I never remember whether I need Oratcl, oratcl, OraTcl, OraTCL (and similar variations for any other packages).

Donald Arseneau: Yes Yes! For package names, but not for command names.

Lars H: Caselessness isn't all that simple anymore, now that we've gone international. Which of the following four commands would you want to identify?

I      ; # LATIN CAPITAL LETTER I
i      ; # LATIN SMALL LETTER I
\u0130 ; # LATIN CAPITAL LETTER I WITH DOT ABOVE
\u0131 ; # LATIN SMALL LETTER DOTLESS I

A "convert command name to lower case before looking it up" approach probably would work, but the results wouldn't be universally intuitive. Another matter is that caselessness would break things. For example, Alphatk needs to distinguish between Bind and bind, and also between Menu and menu, but such incompatibilities could happen at a major version switch.

My real objection is however that I feel this would be rather unTclish; why should one identify command names which are different strings, when (almost) any string can be the name of a command? Nor am I that convinced that it would do that much in the matter of style, since case doesn't seem to be the most common stilistic variation in current use.

Package names are another matter. Package names have a strong connection to file names, and file names are often caseless. The style confusion also seems much more severe in that context.

DKF: FWIW, I think that Tcl ought to stay case-sensitive (why break stuff that doesn't need breaking? And I'm a UNIX/C/Java hacker too, so case sensitivity makes sense to me.) Perhaps we need some additional recommendations on package/command names to keep them fairly easy to type...?

AJD: The effect could be achieved by changing unknown to look for a command which differs only in case (would also need to redefine proc to check for other commands whose names only differ in case and either warn or redefine). But personally i would find it more confusing having xyz being identical to XYZ.

DKF: There are a few other complexities too. Case mapping is locale-sensitive IIRC (Turkish uses different, and when you look at it from a textual point of view, more sensible rules than English), so the code to do it needs to be a lot more complex than otherwise. Given that case-sensitivity equates to cheap binary comparison (as well as simple hashing), there's genuinely no reason to do it any other way. (Anyone who wants can modify unknown, of course.)

ulis 2003-05-09: AtExit functionality at the script level. unset traces fired during exit.

PYK 2015-09-01: Hear, hear!

Luciano ES 2003-07-02: Tcl bindings for the GNU PCRE library.

DKF: GNU eh? Bet that means that, for licensing reasons, we can't incorporate this into Tcl. However, it can be done with a separate extension/package, and it would be interesting to see what this ends up looking like.

JH: PCRE is BSD licensed and not GNU.

ekd123: See also TRE , a BSD-licensed regex. IIRC, PCRE has got a JIT compiler!

Lars H 2003-07-24:

Expose bytecoded math operations as Tcl commands

(This is an offspring from expr shorthand for Tcl9, and I suspect it might not be necessary to wait for Tcl 9 to implement it.) It would often be convenient to instead of e.g.

string range $s [expr {$idx + 1}] end

write something like

string range $s [+ $idx 1] end

and of course this can be arranged on the script level; Importing expr functions, part 2 defines ::exprlib::+, which can be imported into another namespace and then used this way. That currently carries a factor 16 (or thereabout) performance penalty however, so people tend to avoid using it.

My idea is that if ::exprlib::+, ::exprlib::-, etc. are made byte-compiled commands that compile to precisely the bytecode operation they anyway boil down to, then this performance penalty could be lifted. The effect on the language as such is extremely small, since all the new command would initially only exist in a separate namespace.

A generalisation of the current exprlib implementations that might be in order is to make the associative operations (+, *, &, etc.) n-ary rather than merely binary.

FW: I personally love this idea. Does anyone else agree with it?

wdb: Neither no nor yes, but consider that this breaks the (philosophical) decision to keep arithmetics out of the core. It could be desirable to keep the language small.

Lars H: Unlike most of the things discussed in expr shorthand for Tcl9, these things would not give arithmetic any more central position in the language than it already has. The bytecodes already exist because of expr. Also compare how TIP 232 deals with functions.

DKF: We've got all the operators (except for the shortcut ones; use if as a replacement for those) in the core. All are variadic if that is at all sensible. Look in the tcl::mathop namespace.

Done. See TIP 174 .

TV 2003-06-04: KILL or <control-C> equivalent

In wish/tclsh. I just hang a shell by some unknown reason, and it would be good and reasonable I guess to have some way to gain back some control. Maybe there are already ways to do that (an command eval-ing entry with recurring after or so), but it would seem nice to be able to simply have the wish or tclsh capable of being forced into some form of responsiveness when debugging gets hot.

DKF: This might be possible with a combination of a TclX signal handler (which is how you find out about the Ctrl-C in the first place), a suitable master interpreter, and some TIP #143 magic...

ES 2003-07-11: Threads a core feature

It's difficult to develop with threads when you know that the users's interpreter is not particularly likely to be compiled with thread support. Make this a core feature, always there.

FW: Yes, yes, yes. Tcl is completely unable to, say, make use of multiple processors with an event loop. It's impractical to have both threads and an event loop in a Tk application, but the wider-spread usage of threads for console apps as an alternative to the event loop is not a bad idea.

DKF: Threads have a whole bunch of practical problems on assorted strange platforms, but there's no reason we can't move ASAP to making threaded builds the standard (letting people turn it off at the build stage if desired.) I'm not so sure about packaging the thread extension with Tcl for now (2003-08-08); it's still in a bit too much flux for my taste. But who's to say that that'll be the same as we move into the 9.0 release timeframe...

DKF: We're very close to having this for 8.6.

DKF and JYL: Object Type Control

See Object Type Control Discussion for details.

exec -exact or exec -pure or exec -nohinder.

I wish there was an option to make exec only exec whatever it is given as argument, just the way it is. No evaluation, substitution, attempts to slash or de-slash paths, nothing. Just pass it to the underlying shell and get the output.

Lars H: Isn't one of the points with exec that there is no "underlying shell"? (This is BTW probably a Very Good Thing, because shells tend to wreck far more havok on the data they process than Tcl does; almost every non-alphanumeric character is special!) Everything is done on the level of kernel calls in terms of setting up processes, creating pipes, etc. Of course, for those who want to feed things through another shell, TclX provides system.

SLB there's some discussion of the problems with exec in exec ampersand problem

merge from #94

In comp.lang.tcl, Kevin Kenny writes:

"Expect and BLT both do their own forking rather than use exec or Tcl_CreatePipeline. They do this for very good reasons: it has become quite clear that exec is rather a botch. This immediate problem can't be fixed until a better alternative is available, so a first project might be for someone to develop one. blt_bgexec might be a good starting point. The developed alternative should use posix_spawn() in preference to fork()/exec(), in order to make sure that a threaded build on QNX doesn't completely take away the ability to launch subprocesses."

DKF: posix_spawn() is described in [L29 ] and is probably just right for redoing exec. PYK 2015-09-01: Maybe the command option could be exec -raw

FW: Static variables in the core, byte-coded.

DKF: Suppose you could create a stack frame as a separate named entity. At that point, you could quite easily use upvar and uplevel to access it. And everything would be nice, neat and conceptually backward-compatible (though I'm less sure about actual backward-compatability. But I don't think it would be a vast issue.) And that would let you get something very close to C's static modifier. The other alternative is to use a namespace.

KPV: the ability to instantiate separate copies of a namespace.

This touches on somewhat on classes and OO, but what I want is much simpler. I don't want inheritance, lifetime management, polymorphism, etc. Rather, I just like more core support in managing global variables, but I'm not sure exactly what I want. I view global variables as a big bag of variables, and namespaces as bunches of littler bags of variables. Now, I'd like some support in working with those bags, such as copying, temporarily replacing, etc.

DKF: Would the ability to clone a namespace be sufficient (or a restatement of what you want)?

KPV: Possibly, even probably. I have to admit I not quite clear on what exactly I want.

MGS 2003/09/15: I've been working on a generic clone package, which can clone almost anything, namespaces included. Drop me a line if you'd like a peek at the code.

NEM: I wrote some notes on creating a [namespace inherit] command[L30 ]. The idea is still in the early stages of baking, but I think it's basically sound.

Sarnold: On the french Wiki, I have created a page for Cloner des espaces de noms [L31 ]

PYK 2015-09-01: ycl::ns::duplicate implements such namespace duplication. See the notes at namespace as well. One limitation is that it can't duplicate commands that are not procedures. It would be nice to see some core feature that enables that at the script level.

SLB: A concise, simple syntax for constructing a list from a collection of values

with command and variable substitution performed.

Consider the bind command. If you don't need variable substitution you can write:

bind all <1> {foo xyz %W}

If you do need substitution you should write:

bind all <1> [list foo $x %W]

but this is a bit verbose, so many Tcl developers (and some Tcl books) would write:

bind all <1> "foo $x %W"

which works, providing $x does not contain white space.

This variety of forms is complicated for beginners and encourages bad habits in some of us who should know better.

For creating longer lists, there's a slightly different problem, if the list spans multiple lines, you need to write:

set l [list $a $b $c \
            $d $e $f \
            $g $h $i]

and it's easy to forget a backslash.

I'd like to use, say, parentheses to group a sequence of values into a list, with substitution performed and multiple lines permitted. The above examples could then be written:

bind all <1> (foo xyz %W)

bind all <1> (foo $x %W)

set l ($a $b $c
       $d $f $g
       $h $i $j)

It would also mean you could write procedures as:

# Argument list in parentheses.
proc foo (x w) {
   ...
}

KPV interesting idea. So the difference between using list and () would be how it handles new lines?

SLB list is a command and so is terminated by an unescaped newline or semicolon. The parentheses syntax would not be.

Lars H Please no! The savings are marginal, whereas the risk of messing things up is huge. Consider instead writing a command substlist which does the above on an ordinary list -- something like

proc substlist {L {level 1}} {
    set res [list]
    foreach word $L {
        lappend res [uplevel $level [list ::subst $word]]
    }
    return $res
}

Then you can go

proc sbind args {
    eval [list bind] [lrange $args 0 end-1] [list [substlist [lindex $args end] 2]]
}
sbind all <1> {foo $x %w}

all you want without messing up the general syntax. The foreach in substlist similarly takes care of occational newlines between list elements and treats them as mere spaces.

SLB: That's an interesting alternative. However, the idea behind the proposal was to have a uniform solution wherever you want a list. You'd therefore need a lot of wrapper functions beside sbind and some, like the configure sub-commands of button, radiobutton and checkbutton, would be harder to wrap than bind.

When you talk of the huge risk of messing things up, are you talking about compatibility with existing scripts? As one measure of how much would be broken, I did a quick search through tcllib, looking for parantheses at the start of a word that are not within a comment, quoted string or an infix expression. This showed up only stoop with its use of the empty string as an array name:

set ($this,size) $size

This might have to change to:

set \($this,size\) $size

So yes, this change would break some code but maybe little enough code to justify the change in a major release. It's then a question of whether the benefits justify it.

Lars H: tcllib is not a survey of Tcl coding practices, so the chances that you have missed some are fairly large. Also the "not within a quoted string" condition can do wonders at hiding characters, since some people quote everything that isn't "syntax", whether it is necessary or not.

But as you write, it is a question of whether the benefits justify breaking code and complicating the syntax rules of Tcl. Since the only benefit would be a "uniform solution wherever you want a list" and list already is completely uniform and general, it is hard to see that the benefit could justify anything at all. Typing "list" and some backslashes is extremely little work.

AMG: I was about to add this same suggestion, but of course it's already here. :^) Yes, this is exactly how I'd like for list construction using parentheses to work. Also, () would construct an empty list, like how [list], "", and {} construct an empty string. (By the way, why doesn't the result of [list] already have a list representation? Lars H: See discussion in the concat page.)

If lists were formed using parentheses instead of the [list] command, then [list] could be used as a list manipulation ensemble command, subsuming [lindex], [lreplace], and the rest.

And since I've already broken compatibility in a most unforgivable way, let's get rid of [concat], replacing it with:

• ($a $b) to concatenate lists $a and $b
• "$a $b" to concatenate strings $a and $b

Adding [list concat] and [string concat] might be alright, too, but the above two syntaxes are more flexible. In the case of lists, the components are only optionally expanded. (By ` I mean {*}. I'm just copying from AMG's language ideas.) In the case of strings, the separator character is stated explicitly and can be left out entirely. If this change is made (as if!), we should deprecate using "$a $b" to concatenate lists because of the type shimmering involved.

Other tricks:

• set a (`$a $b $c) == lappend a $b $c == appending two elements
• set a ($b $c `$a) == set a [concat [list $b] [list $c] $a] == prepending two elements

PYK: See also scripted list, which implements precisely this, but with no new syntax. It would be nice to have a more performant version of it in the core, perhaps as slist, for scripted list, or dlist, for dynamic list.

Lars H: Proper integers for Tcl 8.5 or 9.0, preferably the former.

<discussion>>

Done in Tcl 8.5. See TIP #237

DOCUMENTATION !!!
DOCUMENTATION !!!
DOCUMENTATION !!!
DOCUMENTATION !!!

Decent documentation for the existing Tcl would be far more useful than Tcl 9.

The existing Tcl documentation SUCKS!!!

Actually, it's very good. EXCEPT for the fact that too much basic simple stuff is omitted. Like EXAMPLES in the man pages, for example.

DKF: That's not a suggestion for 9.0 though. More and better docs are always welcome. Just write them so that the core people can figure out what they're saying and how to format them (yes, you don't need to write nroff) and submit them as a "patch" on SourceForge. :^)

RLH: As someone new to Tcl and just learning I would agree that the docs could be better. Examples would be the first best thing to include. One good thing about the Perl docs is that in almost every case they give a short example to get you thinking. Case in point, I was looking at using Tcl to modify some registry settings and I finally wound up in the test scripts to find my answer.

RLH: Although I have not had any issues with speed, it is pretty much always a good idea to speed things up. : )

DKF: There are now examples for almost all Tcl manual pages. Most of Tk is still to be done though. :^/

RLH: Does this mean they will work there way into the ActiveTcl docs as well at some point?

DKF: Yes, at the next release.

A Unified Database Interface

RLH: One nice thing I do like about Perl is DBI. My wish would be for a nice and clean Unified Database Interface.

FW: Couldn't you guess that that's considered library functionality? ;)

RLH: Actually I probably couldn't. I just started with Tcl/Tk and so am not up on its nuances. Maybe you could explain it to me a little more in depth or point to someplace that does? DBI in Perl is just a library as well. Then other database specific libraries use the DBI to connect to the databases. Someone has thought of it [L32 ]. So what does your statement mean?

FW: What I'm saying is a common database interface, much like an object-oriented programming system, is something that's not part of the basic set of language features (like, say, a new data structure would be), and would work fine loaded in a package, as it is in Perl. The TCT itself rarely implements features themselves; a member or of course an independent Tcl'er will implement one, then it can be proposed as a TIP to be included. But if you look, even basic stuff like threads and more extensive image handling are relegated to packages (Thread, Img). The closest thing to "official" inclusion for something like a DBI would be if it was one of the packages that came with ActiveTcl, and that wouldn't fall under a Tcl 9 request. Regardless, it's not a bad idea.

Lars H: There might actually be some TCT support for this "in the core" idea. Kevin Kenny wrote in [L33 ] that

I'm coming to believe that, while the Core is not the correct place for any specific SQL database extension, it would be a good idea for the Core to provide a (very thin) wrapper layer by which such extensions are to be accessed, and to specify the API that such an extension should use.

2007-11: There is now TIP #308 , for with this. See also TDBC.

Done in 8.6. See TDBC

CMcC: Direct, script-level access to low level tcl core, such as bytecode.

• Inline generation of bytecode within procs,
• bytecode objects as first-class objects (there is a TIP for this),
• ability to script command compilation.
• introspection for bytecode - pull apart and inspect first-class bytecode objects.

All the ideas that've come up around bytecode bear thinking about and inclusion in tcl9. It will facilitate a micro-tcl, as it would permit more of tcl to be generated in/by tcl without loss of performance. It will facilitate better compilation. It will facilitate low-level development of tcl core by opening it to easier experimentation.

Zarutian: I support this idea/wish/suggestion. It would make it easer to make speed critical cross-platform extensions than linking with native libraries. And it would make Tcl even more portable than it is today. hmm... a little redundancy there

Possible syntax:

    bytecode eval <binary string> ?version number?
    bytecode compile <script body> ?version number?
    bytecode register <bytecode symbol> <command> ?version number?
    bytecode unregister <bytecode symbol> ?version number?
    bytecode registered ?version number? ; # lists registered bytecode symbols
    bytecode registered-body <bytecode symbol> ?version number?

    callback:
        <command> <bytecode symbol> <stack>

 <stack> is maybe a list. (Implemention detail)
 <bytecode symbol> is maybe a index into a binary tree (Implemention detail?)

asuming that the bytecode interpreter would be stack based

from the Tcl'ers Chat

Zarutian has entered the chat 
Zarutian: Are inlined bytecodes good or bad idea? 
antirez: may be a good idea 
antirez: but it's better to work on the compiler to get it smarter
Zarutian: check out #70 on the Tcl 9.0 WishList (https://wiki.tcl-lang.org/883). Your opinion on the 
  syntax please 
antirez: Zarutian: the problem with this is that I tried to build a bytecode
  interpreter for Tcl, stack based, similar to what you proposed 
antirez: and observed a speed increment that is not as high as it can be expected 
antirez: in most code, 2x or 3x 
antirez: this is of course, preserving the Tcl semantic of typelessness 
antirez: a typized bytecode interpreter can go much faster, but it's a pain to glue with Tcl 
Zarutian: but how much speed increase do you get with native compiled extension? 100x? 
antirez: in math code, I think something like 50x 
antirez: if you play with Tcl objects, maybe 5x or 10x 
antirez: algorithmical code is the one that gets the major benefits 
Zarutian: suspected that 
antirez: probably it may be an alternative to provide a very fast math-only stack machine as a command 
antirez: so, something designed to run algorithmical code that can't deal with strings and so on 
antirez: this way it's possible to go as fast as FORTH for example, and at the same time to be
  cross-compatible 
Zarutian: which I am exactly looking for
antirez: this can be interesting, expecially if it's possible to convert some Tcl type,
 like a Tcl list of numbers, in an array (C-like) that can be handled by the virutal machine directly
antirez: so the VM can be used to speedup image processing and other stuff like this
antirez: ok, time to go for me, need a break. Ciao, later
Zarutian: cia

Zarutian: TAL would maybe a better idea

Sarnold: I made a prototype of that in Numerical RPN. This might be tricky to implement.

DKF 2007-11-11: Tcl 8.5 has ::tcl::unsupported::disassemble which lets you inspect the bytecode for a procedure, script or lambda.

2009-05-11: KBK has been working on a bytecode assembler too. Apparently, it is also helping to shake out bugs in the existing compiler

Zarutian: lookup variable in current namespace then in the parent if not found and so on recursively. Should not be that much of a hassle. Ah before I forget let the global namespace be the top most parent.

NEM: I've wanted this behaviour before, too. IIRC, the reason we don't have this now was for speed. Perhaps an easier suggestion though, would be to provide a per-namespace means of specifying the lookup path? This way, you have a generalised way to do things like inheritance too. I wrote some notes on this a while ago at [L34 ] which could do with some work. See also TIP 181[L35 ] for a possible way to accomplish this (if it passes), although a more direct approach may be better for this particular problem.

NEM: This is somewhat related to item 66, too.

I think this is solved in Tcl 8.5 with namespace path. See TIP [L36 ] .

DKF 2007-11-11: namespace path only controls command lookup.

DKF: Also note that this request reduces to the current behavior in the case of namespaces that are direct children of :: and there it is actually more likely to be a bug when encountered than not. Consequently, I do not support this at all.

Lars H: I'm not certain this is something I wish for, but it's an idea I got, and I think it should be recorded in a suitable Wiki page, just in case someone comes around who might like it.

The idea is to change the interpretation of "variable names" (as command arguments and in $ substitution), so that it wouldn't necessarily refer to a variable as a whole, but equally well to a part of the value stored in the variable. This would provide alternatives to lindex using $-substitution and alternatives to lset using ordinary set.

As syntax, one could require that these "variable part names" are lists of length two or greater (this leaves most existing code unaffected). The first element of that list would be a "container type" specifying what kind of object the referenced "variable part" is a part of, the second element would be the name of the proper variable being subindexed, and additional elements would give the exact index. Standard container types could be list, dict, array (just providing alternative syntax for array variables), raw (don't do any "variable part" interpretation of this name), and upvar (variable has this name in another stack frame).

Here are some examples of code using variable part name syntax and their current counterparts.

cmd ${list L 3}            ; cmd [lindex $L 3]
set {list L 3} 17          ; lset L 3 17
cmd ${dict D foo}          ; cmd [dict get $D foo]
set {dict D foo} 19        ; dict set D foo 19
cmd ${array A foo}         ; cmd $A(foo)
 cmd ${raw {I, Me, Mine}}   ; cmd ${I, Me, Mine}   ; # This would probably be an error, since "I," isn't a known container type

So far it hasn't been anything significantly simpler than the current syntax, but variable part names naturally admit some extensions. What if the second element could itself be a variable part name? Then the above constructions would be nestable

set {dict {list L 2} bar} apa

The current counterpart of this is

lset L 2 [dict replace [lindex $L 2] bar apa]

which undeniably is more complicated in that the variable L and index 2 has to be specified twice. The second form also does unnecessary copying of the dictionary, since it cannot be modified in place when it is part of the list. To work around that, one would have to say

set temp [lindex $L 2]
lset L 2 {}
dict set temp bar apa
lset L 2 $temp

which is even longer. The same problem does not occur with lists nested within lists, as lset can do replacements in place several index levels down.

Another advantage is in interaction with commands such as binary scan which require names of variables to store scanned data in. Currently you have to do something like

binary scan $data c4sss v0 v1 v2 v3
set numbers [list $v0 $v1 $v2 $v3]

but with variable part names you could do

binary scan $data c4sss {list numbers 0} {list numbers 1} {list numbers 2} {list numbers 3}

Note that none of this changes the fact that everything is a string any more than the introduction of lset did (i.e., not at alll). Everything is rooted in a variable name, and setting something changes the value of that variable only. (Shared objects containing the thing to set will have to be duplicated on the way there.) By explicitly specifying the type of the container to access, one forces it to assume that type in the same way as lindex forces values to be lists. The string representation contains all the information needed.

A possible extension of this idea could be to provide a kind of structs in Tcl. What if there was a command

typedef record list {first middle last address telephone e-mail}

which would define record as a new "container type", stating that it is really just "list", but with symbolic names instead of bare indices. Then after the above one would be able to write

puts "Please phone ${record R first} ${record R last} at ${record R telephone},\
  or at least e-mail him at ${record R e-mail}. This is important!"

One could even make such type definitions nesting, so that if there are more than one index then the second index gets interpreted according to a container type implicit from the first. Something like

typedef Node list {left right key value} {Node Node "" ""}

could mean that the left and right parts of a Node are themselves Nodes. A command such as

set {Node someTree left left right left value} 53

would be equivalent to

lset someTree 0 0 1 0 3 53

but less mysterious.

A difference between this kind of symbolic list-indices and nested dicts is that the symbolic names of list indices could cache (in the internal rep, of the Tcl_Obj) the exact numeric value of this index (similarly to how subcommand name interpretations are cached today), whereas in a dictionary one must always do a lookup in the hash table (the dictionary might have changed size since the last access, and then the previous index may be invalid).

Lars H, much later: For the record, it later occurred to me that it is perfectly possible to do a lot of this idea as an extension (even as a pure Tcl package, but then one cannot cache record indices anymore). What one would do is simply to define a command tset that works as set would according to the above! Writing [tset {record R telephone}] is not quite as nice as ${record R telephone}, but already

tset {dict D foo} bar

is not so much worse than

dict set D foo bar

. The big wins (from a code clarity point of view) would be nesting and typedefs. The latter needs a bit of thought to make generally useful, since one wouldn't want the types of one package to collide with types of another, but it is quite possible to such definitions relative to a namespace. The infix package demonstrates one way of doing that: for every namespace ns using tset, one would create a child namespace ns::tset holding all type definitions that should be available from within the ns namespace.

Jean-Luc Fontaine: this is something I have been missing since the beginning:

expr {max($a, $b, $c)}

and obviously min() as well...

FW: Minimalist procedure version:

proc max args {lindex [lsort $args] end}

NEM: This seems like a fairly simple small addition. Why wait for 9.0, though? Seems like a straightforward TIP for 8.5.

Done in Tcl 8.5. See TIP 255 .

CMcC: Analogous to variable traces, the idea would be to trigger a trace when an undefined variable is referenced or written. I suggest the syntax of trace be amended to 'trace add variable op cmd

Lars H: Sounds like something that could be TIPed for 8.5.

CMcC: A procedure which, if defined, would be evaluated before any procedure name resolution. Its result would be substituted for the name to be resolved, or the entire command evaluation replaced with the [known] result if it was of return code break.

The [known] command would be given, as arguments, the entire command.

NEM: Cool idea! Likely to be very slow, though.

Lars H: This might already be possible. What if one creates an interpreter without commands (or with all commands hidden), and then makes the [unknown] command of that interpreter an alias for the desired [known]?

PYK 2015-09-01: Pass the entire command to known, substitute the entire command with the output, and you've just added LISP macros to Tcl.

Review the defaults for commands such as lsearch - why do we have glob style by default?

AM 2005-10-15: Commands like lsearch support various types of pattern matching. But lsearch in particular uses -glob as its default - to get the simplest and quickest (and likely the most used and expected (*)) searching method you need to specify -exact.

Is there a compelling reason to keep -glob as the default?

(*) I quite agree that without proper measurement data this is a rather personal qualification, but I do know that I seldom realise that the default mechanism is -glob and that I almost always want -exact.

DKF: If you want just simple existence testing, use the expr in operator.

SLB 2005-02-02: the variable $args only has a special meaning when used as the last argument to a proc. However, many Tcl command have a variable number of leading switches followed by a fixed number of arguments. It would be helpful if you could write:

proc myPuts {args text} {
    ...
}

myPuts -nonewline stderr "Hello World"

Such a change is not hard to implement (involves changing TclObjInterpProc()) but it does cause an incompatiblity, the current behaviour is:

proc foo {f args} {
    puts "args = $args"
}
foo "" ""
# reports: args = {}

proc bar {args f} {
  puts "args = $args"
}
bar "" ""
# reports: args = 

My proposed change would alter the behaviour of bar.

Steve Bennett: I believe that this (and more) is covered by TIP 288 , which is implemented in Jim and is very convenient. Additionally, I plan to add a suggestion I saw somewhere which allows args to be renamed, which leads to more informative error messages.

. proc bar {{args options} f} { puts "options=$options, f=$f" }
. bar
wrong # args: should be "bar ?options ...? f"
[error] . bar -first -nocase thing
options=-first -nocase, f=thing

AMG: An alternative to reserving the name "args" would be to support prefixing the catch-all parameter name with, say, an asterisk.

proc bar {*args f} {
    puts "args = $args"
}

Just a suggestion. This hopefully avoids breaking code that takes advantage of the fact that args only has special meaning when it's at the end of the parameter list. But it does break anything that actually uses *'s at the start of parameter names. :^) And it breaks all current use of "args" at the end of the parameter list, unless we make the asterisk optional in that case.

I'm not actually proposing we do this in Tcl; I'm just copying the idea over from AMG's language ideas.

To make args (or whatever it's called) truly general, it should also be usable in the middle of the parameter list. Tcl would first assign all fixed parameters before and after then pass what's left via args.

RLH: I want Tcl to be the fastest dynamic language in the whole universe! Is that so wrong to ask? :-)

wdb: not yet thought enough about this idea -- but: what if the byte-code compiler were replaced by an implementation in a really fast version of Forth?

Willi 2005-02-23: It should be possible to explicitly tell expr which operand type to use for all operands, instead of auto-detecting every operand separately. This would be especially useful whenever one wants to calculate a term containing integer values with floating-point precision. It could, on one hand, speed up the execution (because things like the multiplication with 1.0 or the double() operation could be omitted), and, on the other hand, it would allow writing arithmetic expressions that are much less prone to errors. Just consider the following:

expr {5/3 + 1.42}

Here, one always has to remember that 5/3 is computed as integer, because the division operator takes precedence. The ability to simply write something like

expr -f {5/3+1.42}

could immediately eliminate most of those oversights and allow writing much shorter code.

Lars H: A cleaner solution would be to make this expr -f a separate command, e.g. exprf. It's shorter too. And it can be done as an extension.

...like it is done in Jim. These are new semantics that would allow to create new OO systems with maybe better performances, certainly fewer lines of code. As in Jimulation there is a backward-compatible implementation shown, we should upgrade quietly to Tcl 9.0. But as this (combined with dict) makes OOP easier, it would also make non-OO programming of better quality.

RS: Yes, please!

DKF: duplicate?

Many (most?) scripting languages say: "error such in line x of file Y". You can't miss it.

Tcl, instead, presents an unpleasant labyrinth: "error such in line x of proc foo, called by proc bar, invoked by [ source cheese.tcl ] in proc ham which in turn was brought upon your poor soul by [ source doom.tcl ], called by wow, you're still reading it!"

And many Tk apps have error message windows in two styles: one with a text widget that we can scroll and another one that is just a dialog that will not show the entire message if it is too long. Even in those that I can scroll, I often close it without even bothering to read it and try to find my mistake with some other method.

How is "error in line x of proc foo" supposed to be useful? Which editor will show the line numbers relative to the context of a proc?

LV: It's my understanding that one of the arguments for the current situation is "Given the fact that Tcl is such a dynamic language, trying to identify a 'line of code' in a file is frequently difficult, if not impossible. If the tcl code executing was transmitted to the application from another Tk app - then what line number is it? If the code is a dynamically generated proc, executed from a callback, what does one use?"

Seems to me, however, that keeping track of line number information, when available, and using it would be a useful code change. Finding someone with the ability and motivation to code it is a different issue.

Lars H: When the procedure is actually found in some file, one can often find out which file from the auto_index array:

  % set auto_index(foo)
  source /usr/lib/someapp/somefile.tcl

RS: Also, if you keep your procs short (less than 20 lines, say), you can either see the offending line at a glance, or (with an editor that displays line numbers) perform simple arithmetic: If proc foo starts at line 234 in the file and the error is reported at line 7 of proc foo, inspect line 234+7= 241 ... :)

LES disagrees with RS:

1: Keeping one's procs short is not ALWAYS possible or even interesting.

2: Even if the proc is short, finding it in a long file with many procs is tedious.

3: Machines should be adjusted to human beings' convenience, not the other way round.

SLB: This was addressed by TIP 86 , though it has not yet been approved.

DKF: I think that TIP needs a little more love-and-attention. Possibly also subdividing into several proposals; it's grown a lot in scope since the early revisions.

DKF: info frame has the info, but it's not currently being used in error messages.

Steve Bennett: Error reporting is one of the best features of Jim. I strongly recommend this feature for Tcl. Here is a typical error from Jim:

 Runtime Error: errors.tcl:18: can't read "bogus": no such variable
 in procedure 'error_caller' called at file "stacktrace.test", line 17
 in procedure 'error_generator' called at file "errors.tcl", line 50 
 at file "errors.tcl", line 1

With "Error: filename:line" on the first line, it even works nicely with vim "quickfix" mode when part of a build process.

The desire to be a daemon is under POSIX systems, of course... I do not speak about Windows NT Services.

DKF: Does TclX provide the facilities required? (NT Service support is available commercially from ActiveState IIRC.)

RLH: I think that Twapi allows you to do that. It has the facilities to do it but I am not sure if it pertains to Tcl apps (although I don't see why not).

AMG: I never understood why so many programs insist on forking to the background or even offer such an option. The caller can always do the fork itself, and this way the code isn't duplicated in a million places. Want to daemonize a Tcl script? Type "script.tcl &". Auto-fork behavior gets really annoying when using something like supervise [L37 ].

Hmm, actually we can already do that with [exec]--- if the -d option is given, have the script re-[exec] itself in the background sans the -d option, then exit without waiting on the child.

DKF: I'd rather have a "daemonized tclsh" that starts itself in daemon mode and which installs default listeners for the usual "daemon signals". Like that, it would be analogous to what's needed for windows services.

DKF: I mentioned this on the tcler's chat the other day, and jenglish responded by saying that current best practice is to have a separate program that does the daemonizing. Maybe this has just ceased to be a relevant thing to want?

PYK 2014-11-14: For reference, tcllauncher includes ::tcllauncher::daemonize

A better documentation facility like docstrings in Python or javadoc type comments. Preferably the latter of the two as it is more robust. - RS: see docstring for a DIY solution...RLH: Isn't this about including it in "the core"? There a probably many DIY solutions on this page.

See TIP 193 .

Switching to lisp-like syntax together with introducing a new type "list" distinguished from "string". The example of NewLisp [L38 ] shows that it is possible without the needs of a cons procedure, and it would perfectly fit to the spirit of Tcl.

NEM New syntax and new semantics? Probably it'd be wise to give this new language a new name then, as it would be quite different from Tcl.

New subcommand string append str1 ?str2 ...? such that I can get rid of that ugly return statement such as 'return part1${result}part2'. (Prefer functional style.)

RS: Or maybe string join arg arg.., acting like join $args "" ?

NEM: As in,

proc concatenate args { join $args "" }

?

wdb Yes. It's just syntactical sugar if the last line of a proc is stuck to return otherwise. Until now, I used set result ..., and sometimes I wrote # return this value one line above which of course works but is ugly.

Lars H: This wish seems to be precisely for the TclX cconcat command.

RS Functionally thinking, why not do it like this?

proc identity x {set x}
...
identity part1${result}part2

wdb

• The name "identity" does not refer to the string-concatenating function. A better name would be "string-append" (like in Scheme) where the name tells what the function does.
• The solution "concatenate" has this problem: the name is too near at "concat" where lists are appended, the difference is just the grade of abbreviation -- not very intuitive to distinguish.
• The solution join [list str1 str2 ...] {} factually matches best, but not very intuitive to understand.

I prefer the procedure composition string append or string join. Btw, anybody here who feels remembered to improvements to list command? Like just at moment, me?

RS: My point was: Tcl itself does the string concatenation (before calling identity). The general-purpose identity is just to have a command that faithfully returns its input.

wdb: Received that idea. It's interesting but a takes me a night to sleep about it. -- My point was this: It's doubtlessly nice of Tcl to concat the strings voluntarily. But a functional programmer has the quite human habit to call regular function even at end of a proc. -- wdb (again) -- night is over -- the solution of RS is more general than my approach, but I prefer the more understandable name just instead of identity, usage e.g.:

just $foo$bar

RS: identity is the generic technical term (also used in Common Lisp), but call it as you like :) I myself sometimes prefer the name I, as identity is called in combinator theory.

Done in Tcl 8.6.2. See TIP #429

I have always found it nice in other scripting languages that you can do a syntax check by passing a switch to the interpreter:

$ tclsh -c file.tcl
$ Syntax ok

DKF: We have historically preferred to leave it to external tools like nagelfar.

WJP]: Currently (under Unix - I don't know about MS Windows) msgcat determines the locale by looking at LC_ALL,LC_MESSAGES, and LANG in that order. This differs from GNU gettext in one respect, namely that GNU gettext looks at LANGUAGE first. I suggest conforming msgcat's behaviour to that of GNU gettext. In addition to eliminating an unnecessary inconsistency for the user between Tcl programs and C programs, LANGUAGE has the virtue that is value need not be a single locale. Instead, it can be series of colon-separated locales which function as a series of defaults. A value like: ja_JP:ko_KR:zh_TW means "I prefer Japanese, but if Japanese is not available, use Korean, and if Korean is not available use Taiwan Chinese". This is a very nice feature.

I decided to have a look at implementing this and for the most part it is easy. Here is a revised version of the first bit of msgcat::Init that does the trick:

foreach varName {LANGUAGE LC_ALL LC_MESSAGES LANG} {
    if {[info exists ::env($varName)]
            && ![string equal "" $::env($varName)]} {
        set LocaleList [split $::env($varName) ":"]
        foreach loc $LocaleList {
            mclocale [ConvertLocale $loc]
            if {[mcload "msgs"]} {return}
        }
        return
    }
}

The problem is that mcload needs to know what directory to look in. For my test I used a fixed "msgs" directory, but that won't do for general use. Off the top of my head I don't see why the init code can't be included in mcload, so that the directory would be known when it is called. WJP

schlenk: There are some issues with doing it the way you propose. One would be consistency, what about a program that has modules with independent msgcats (msgcat depends on the namespace it is called in for mcset and mc)and differing support for various locales. (say module A is available in japanese, B in korean but not japanese, C fully in chinese but only partially translated in korean and so on. What should happen in that case? With a single locale setting its easy, but fallback for multiple locales has to be defined.

WJP: Good point. This doesn't arise with GNU gettext because it isn't modularized. I'll think about it.

HaO 2015-07-07: If you think, initialising msgcat::mclocale by 'LANGUAGE', please add a TIP. I see no reason to wait for 9.0.

Initializing in the source command is a complete structure change and not helpful IMHO.

See also TIP 412 .

e.g., consistent error reporting + a few more built-ins like parray + console on unix

Discussion moved to Interactive features in Tcl9 discussion.

RLH How about multi-line comments? Anything between the marks (no matter where they show up) is ignored by the interpreter:

(*
    This is a multiline comment. What makes this nice is that other documentors could then extend it so that 
    you can embedd other types of doctool stuff like (@author, @param) and stuff.

   (* I changed the brace type because I like the ocaml style better. *)
*)

(*
   If you going to implement this...
   (* do it all the way *)
*)

if {some condition} {
   some conditionally executed script.
   (* some other code that I don't want to run *)
} else {
   some script to execute if the condition isn't satisfied.
}

I see multi-line comments crop up fairly regularly.

MG Multi-line comments would be nice (particularly if they worked in the way most people new to Tcl would expect, so that mis-matched braces and so forth could be put inside without a problem). Is using #- ... -# the best thing, though? That could quite easily break existing code - I've seen people use #- in normal comments, which would comment out the whole script instead of a single line. Can't think of anything which might be better, off the top of my head, though - I doubt most Tcl'ers would want C-style /* */ comments?

Larry Smith: In line with suggestions above from myself and others about using parens, I would take a page from Pascal and suggest (* and *). I would also suggest making them nestable, Modula-style.

Lars H: It is not possible to have comments override brace matching, regardless of whether the comments are ended by a newline or some other character sequence. (The reason is that brace-delimited words are primarily data in Tcl, whereas comments should only be ignored if that data also happens to be code, and there's no way to know beforehand whether that will be the case.)

Larry Smith: I disagree. There is really no reason why Tcl's 'source' command cannot be extended to perform some pattern-matching and translation before handing the script off to the evaluator. An extension to 'source' to provide comments exactly like this would be quite trivial. There is really no reason why the evaluator should ever see a comment, and therefore no reason why we should have to explain about balancing delimiters over and over and over and...

Lars H: In other words you propose introducing a new comment mechanism totally separate from the old one.

Larry Smith interjects: You say that like it's a bad thing. In my view, this is merely retrofitting something into the language it should have had to begin with. Yes, the "#" convention is handy and cute and very easy to implement - but's a pain in the neck in the real world.

Lars H continues: Sure, at that level you could do it (and as a matter of fact I've done something that could be used to this end, in the tcllib docstrip module [L39 ], even though that would be a rather corny application of it), but it wouldn't make things any simpler. Rather, you would have to spend at least as much time explaining over and over again why the two comment mechanisms are so very different (one is in force everywhere, the other one isn't).

Larry Smith interjects: Or we could deprecate the #-comments and provide automated translation for 10.0. Or 9.1.

Lars H continues: With a pre-eval comment mechanism there is also the not-so-tiny problem of how to work around it when you want the particular character combination that would normally start a comment. Hashes can always be backslashed, but since your (* *) would be active even where no substitution is taking place you'll have to come up with something else for that.

Larry Smith interjects: That's why we have backslashes.

Lars H replies: No, at that point in the processing, you don't have backslashes as escape char, since they work by triggering substitution.

Lars H continues: As for multi-line comments as such, I don't really see any need for that. With editors that have commands for commenting out or uncommenting blocks of code, until-end-of-line comments are as easy to work with as multiline comments (and IMHO much easier).

Larry Smith: Editors cannot be made aware of Tcl's balanced delimiters rule.

Lars H interjects: Yes they can, if programmable. Whether it is worth the effort is another matter, as the user typically already has to be aware of this rule.

Larry Smith continues: You cannot use such a feature without often producing something syntactically unacceptable.

Lars H: Well, yes, it is an age old problem with editors and computer languages that you can write programs that don't compile. Multi-line comments only scratch the surface of what one can do to address this, however; a far more effective measure is to disable the keyboard, as this not only reduces the number of balancing errors, but also the number of typos, uses of uninitialised variables, and pretty much any kind of programming error there is. ;-)

Larry Smith: It's been tried - a number of visual languages do do this in effect. Problem is, such languages are just not as expressive as real programmers need. And, even worse, they require artistic ability.

Larry Smith: Ideally, it would be nice if we could get away from the whole program-as-text concept and set up tcl programs so they can be handled as data structures by an editor. An editor like that could be told "this is a comment" and "this is a command" and so on and it would store this meta-information with the file somehow. Of course, having done that we need some way to extract the actual program from the file, but this is something a preprocessor as described above would be able to do.

NEM: A problem there is that Tcl code isn't very analysable in this way. In many cases you can only guess whether a given piece of text is intended to be Tcl code or something else. You could maybe set up the editor to be an actual Tcl interpreter with dummy commands that build up a syntax tree rather than evaluating (some form of abstract interpretation). So, e.g. the "proc" command in the editor would simply add the proc name to a list of commands, and then syntax-analyse the body. I'm not sure how well this would work in practice, but it could be a fun exercise (custom C commands wouldn't work, for instance).

RS: For example:

set   it "is this data or code?"
puts $it
eval $it ;# will work well after an [is] command is defined :^)

NEM: Well, anything passed to eval is obviously code, so in that case it would be fairly simple to infer. I was thinking more along the lines of stuff like:

someCollection iterate item { puts $item }

To most people looking at this, it is fairly obvious that the { puts $item } is some code. However, for an editor it is much harder to work that out -- it could have just been some documentation, or some text, or even some C-code. To be able to do this in general, you'd have to do some form of type inference over pretty much the entire source code of a program, and even then it would still be guesswork in many cases.

NEM: Regarding multi-line comments, why not simply use a command? XOTcl includes an "@" command for this:

 @ { This is a multi-
 line comment. }

It works well and costs practically nothing. XOTcl uses the command to process meta-data comments too.

techmisc: Re: Style/Syntax: Since Tcl has # for comments and { } for multiline, maybe the new syntax can be based on those characters to make it feel Tclish:

#{ This is a multiline
   comment or discarded data/code

        #{ Nested multiline comment or discarded data/code }#

}#

Not sure if }# or #} is a better terminator. The former feels cleaner and more symmetrical/consistent with #{ while the latter may be easier to implement and remember ie. "all Tcl's comment symbols begin with a #": #, #{, #}.

sergiol Can i ask for something simple like to remove the version number from after the tclsh & wish binaries? it is very stupid to download scripts calling for wishX.Y and it does not execute because user's version is wishZ.W .

NEM: There may be multiple versions of tclsh and wish installed on one system, so having each one called just "tclsh" or "wish" wouldn't be very helpful. Instead, it is up to system administrators to decide which version they want to call by the short name. e.g. on Mac OSX /usr/bin/tclsh is a symbolic link to tclsh8.4.

RS: Also, scripts on Unixoid systems should begin like this:

#!/usr/bin/env tclsh
package require Tk ;# if you need it
...

dizzy: I often write code like this to set a default value if some variable is not existing:

if {![info exists var]} {
    set var $default
}

and also:

 if {$var eq ""} {
     set var $default
 }
 
 if {$var1 ne ""} {
     set foo $var1
 } elseif {$var2 ne ""} {
     set foo $var2
 } else {
     set foo $default
 }

This code could be simplified if we had commands returning a default value if a variable is not set or is empty, for example:

set var [get var $default]                 ;# like [set] but returns default if var not existing

set var [or $val1 $val2 ... $default]      ;# return first non empty arg

set var [and $val1 $val2 ... $result]      ;# return last arg if all args not empty

MJ: I don't see a need to add this in the core as it is easily implemented as a proc:

proc get {varname default} {
    if {[uplevel 1 info exists $varname]} {
         return [uplevel 1 set $varname]
    } else {
         return $default
    }
}

proc or args {
    foreach val $args {
        if {$val!=""} {break}
    }
    return $val
}

proc and args {
    foreach val $args {
        if {$val eq ""} {break}
    }
    return $val
}

Make llength accept additional arguments such that it returns the length of a nested sub-list'''.

In this example, lllength does what I wish:

% proc lllength {l args} {
   llength [eval [list lindex $l] $args]
}
% lllength {{a b c} d e f}
4
% lllength {{a b c} d e f} 0
3
% lllength {{a b c} d e f} 0 0
1
%

This TIP is a cheap one as it does not break any compatibility.

LV 2007-01-03: Note that adding the idea on this page doesn't create a TIP - to do that, visit http://tip.tcl.tk/ and follow instructions there.

(Later that month, this wish became TIP [L40 ].)

Extend expr such that it can calculate complex numbers such as 4+5i.

DKF: That's mostly not too hard conceptually, but is a lot of work. The one tricky bit is the issue of what to do with the likes of the sqrt() function, where currently code might expect a failure when the argument is negative.

wdb: A possible solution: if the argument is written as complex number, then it does not throw an error. Example:

% expr {sqrt(-4.0)}
domain error: argument not in valid range
% expr {sqrt(-4.0+0.0i)}
0.0+2.0i

This behaviour is similar to integer division where integers are treated differnt from floats.

DKF: OK, that's a reasonable suggestion. Now all that's needed is doing it. (Which is actually the real problem; extending the domain of expr operations is difficult because it means working with the bytecode engine. Alas. It's "only" a practical difficulty, but it is definitely there.)

KJN: Many glyphs have now been assigned outside the 16-bit Base Multilingual Plane, e.g., for mathematical symbols. Fonts can be assembled from the Computer Modern (TeX) fonts, or (when released) from http://www.stixfonts.org/ . It would be useful if Tcl could process any Unicode character.

Lars H: As I understand it, there is nothing in the public C APIs which prevent this; in particular, the string representations can already encode arbitrary Unicode since it is UTF-8. What is troublesome is the internal representation of String Tcl_Objs, which encodes a string as a C-style vector of Tcl_UniChar's in order to provide constant time string index and friends; since a Tcl_UniChar is 16 bits, it can only cope with BMP characters. Since many string operations (even string length) cause shimmering to a String internal representation, you have to obey its restrictions. (Is there also something in Tk which enforces the BMP restriction? As I recall it there was at some point a problem with some Linux distro where Tcl had been patched to use a 32-bit Tcl_UniChar.)

Just allocating 32 bits per character in the internal representation of String Tcl_Objs would double the size of strings, which is probably not a good idea. It has been discussed on the tcl-core mailing list to fix this by switching to UTF-16 for the internal representation. Since this would mean not all characters are the same width anymore, constant time access becomes nontrivial (if at all possible).

An alternative would be to keep the current internal representation for strings within the BMP and add a new Tcl_Obj type which is used instead of it for strings containing at least one non-BMP character. This parallels the current situation with respect to ByteArrays and Strings, as the former uses 8 bits per character and can only encode characters \u0000?\u00FF.

LV 2007-11-20: I wonder whether someone might be clever enough to enhance Tcl to handle things this way. Have typed strings. Strings are utf-8 by default. Special commands are used to allocate utf-16 data to a list. Once done, that string is now marked internally as a type of utf-16 string. That forces that string only through optimised code intended to deal with utf-16. If a string doesn't need to be processed slowly, isn't. And those that need the extra flexibility pay the price for that, but only when necessary.

Over in comp.lang.tcl, Kevin Kenny writes (in an article about problems with Tcl, threads, and Unix-like systems) [L41 ]:

"[...] Signal delivery is secondary, since Tcl doesn't do a competent job of that to begin with. (That's an issue for another day.)" TclX was one of the first extensions I recall that made some attempt to provide some sort of interface to signals. It may very well be time for someone who needs reliable signal delivery - particularly in relationship to threading - to begin work on a conceptual model for how things should work, followed, hopefully, by ideas by those who know Tcl's internals best on how such a module could work.

#94 was a (near-)duplicate of #64

FM As I come from the unix world, I'm often writing things like that :

• lsearch -help
• string -help
• split -help

Of course, it doesn't work, so I write, for instance, lsearch kfldmfdk, to get help, so I can get the subcommands in their exact order. Maybe it should be interesting to get such a functionality ?

LV: In Tcl 8.5, at least, by "accident of design", what you request is already there in some cases:

% lsearch -help
wrong # args: should be "lsearch ?options? list pattern"
% string -help
unknown or ambiguous subcommand "-help": must be bytelength, compare, equal, first, index, is, last, length, map, match, range, repeat, replace, reverse, tolower, totitle, toupper, trim, trimleft, trimright, wordend, or wordstart

split, however, doesn't work like that. It doesn't take any arguments, allowing you to split a string of -help abc if you needed to...

The only way to get split to work like the others is to "break compatibility" by requiring the developer to add a "--" before the string to split. With that change, you could then add a -help flag...

FM I saw in TIPs that a new subcommand of info will be created. "info help <command>". It suits me quite well.

MJ: Currently most Tcl commands accepting options also accept a non-empty unique abbreviation in interactive mode, unfortunately this is the default for Tcl_GetIndexFromObj*. This would seem to promote a usage by extension writers where code using the extension (and using partial options) is fragile to change (for instance adding of options). Therefore I think it will be a good idea to make TCL_EXACT the default in Tcl_GetIndexFromObj*

FB: OTOH this is very handy for interactive sessions. So let's make TCL_EXACT the default everywhere but on the command line.

FM: Sometime I'm dreaming of something like this :

delta ?-polar2D ?-parametric2D ?-cartesian2D ?-parametric3D ?-cartesian3D ?-spheric3D ?-cylindric3D ?-spheric4D ?...? {intervals} {expressions} {increment}

for instance :

But maybe this should be put in package math

SArnold: I would like to write :

if {$x = $y or $test > 0 and not $z}

instead of:

if {$x == $y || $test > 0 && !$z}

and I would like to see both syntaxes supported, of course. This could attract people that find Tcl's syntax obscure. I think syntax has a lot to do with the success of Python (which I do not like BTW).

Lars H: Me like. I see no reason why this shouldn't be possible already in Tcl 8.x. You'd probably face opposition though ("too Pascal-like" from those who prefer C), especially regarding = (which some want to use for assignment, although they have yet to sort out the syntax). A disadvantage could be that people only learn one way to do it, so they will have difficulty reading code employing "the other style".

DKF: Indeed. You can do this just fine with an add-on pas::expr command, though when used with if it would need to be like this:

 if {[pas::expr {$x = $y or $test > 0 and not $z}]} ...

Well, you could also do pas::if etc.

The advantage of putting it in a separate namespace is that it is then easy to use it for your own code sans qualification without disturbing library packages.

ferrieux: But don't forget the speed bump. If'fed expressions get inlined just before the conditional jump. Not so with a vanilla function. Back to the macro/parse-hook dream ;-)

KJN: The ideas behind package and tm are sound, but we have too many mechanisms for discovery of library commands. We have the tclIndex mechanism, which examines the directories in $::auto_path; the pkgIndex.tcl mechanism, which examines the directories in $::auto_path and their subdirectories; and the ::tcl::tm mechanism, which examines the directories in the module path [::tcl::tm::path list]. To simplify these arrangements we could:

• eliminate tclIndex - take the autoloader data from tcl9.x/tclIndex, tk9.x/tclIndex and hardcode it in tcl9.x/init.tcl, tk9.x/init.tcl; require packages to do similar initialisation instead of using tclIndex.
• eliminate pkgIndex.tcl and move all packages to the module path - if a package is not in single-file module format, tolerate the presence of files other than *.tm in the module path, and use *.tm simply as the loader for such packages.
• ::auto_path is now redundant
• the module path already has an inconveniently large number of directories. Instead of exposing the module path to the programmer, expose the list of module roots, and hardcode the module path as standardised grandchildren of the module roots (the tcl9/tcl9.0 etc).
• it is inconvenient that paths can be added only to the head of the module path list. More general manipulation (of the new list of module roots) would make the list as flexible as the existing ::auto_path. The list of module roots must be checked for sanity, but this test could be performed lazily when ::tcl::tm uses the variable. The number of root directories will be much less (currently 7 times) than the number of module paths, so a complete test of module roots will be 49 times faster than a complete test of module paths.

tombert 2010-03-11: A new for me very useful feature would be an enhanced vwait: Currently it's an first in, last out algorithm. I would love a switch to control the behaviour like: first-in-last-out, first-in-first-out, first-free-gets-out;

Motivation: waiting for events synchronously is mostly done with vwait e.g. used to programm SCPI enabled devices:

proc write_read data {
    write to socket data
    wait till data is ready with vwait
    read data from socket
    return data read
}

set deviceID [write_read *IDN?]

Now multiple stacks of write_read can happen (controlled with events) whereas the first write_read can only complete if the last write_read finishes i.e. thats the behaviour of vwait: first in, last out.

I'am afraid of wishing this in TCL8.5 or TCL8.6 ... but if you think so ... I would love it. thx

AM: I think that with coroutines you can achieve exactly that - they are implemented in Tcl 8.6

tombert 2010-03-12 06:57:45:

Yes - thx - co_vwait really does ...

Add another element to the callback, being the old value of the thing traced. Easy enough to do, makes read-only variables simpler to implement.

interp produces a command, call it $interp. Currently (8.6) $interp is an ensemble-like form, where the first argument is interpreted as an operation on the interpreter. There are (or could easily be) identical functions available in the form [interp $verb $interp]. I think all [$interp] forms should be treated as [$interp eval] is now, and all the ensemble forms should be removed. This would allow interps to simulate all other general functional elements of Tcl.

Lars H: Bad idea, but I'll humor you; how would an interp simulate if? (Hint: Having some sort of eval simulate another command gets into trouble when that other command expects data in the first argument, i.e., it gets into trouble for every non-ensemble-like command.)

tombert 2010-07-30 14:01:54

clock milliseconds -ref referenceTime

This would allow easy time measurements.

Package Tk split into Tk-Base, Tk-Widgets; with package Tk becoming a virtual package that loads both for backwards compatibility.

Essentially, I'd love to have the option to not load the old non-themed Tk widgets (which I don't use much anymore; I use Ttk/Tile) when I load the package providing grid, place, pack, winfo and so on.

DKF: You want to be without toplevel, canvas and text, too? They have no equivalents in Ttk at all, and frame is required too (if you want to support wm manage at the very least).

#105 was duplicate of #103

I have a couple ideas that may cut down on the need for complex procs to accept args, internally assign arguments to variables, and internally detect and generate error messages.

1. Don't require optional parameters and args to be at the end. Instead assign all required parameters first, then assign all optionals (left to right), and lastly fill in args. This approach is backward compatible with existing code, except that it will recognize the parameter name "args" as being special even when not at the end of the list.

proc sproc {name arguments statics {init ""} script} {...}
proc range {{start ""} stop {step ""}} {...}
proc test {a {b ""} args {c ""} d} {...}
test              ;# wrong # args: should be "test a ?b? ?arg ...? ?c? d"
test a d
test a b d
test a b c d
test a b arg1 c d
test a b arg1 arg2 c d

2. Support having an omitted optional parameter be initially unset instead of defaulted. This will probably require some magic syntax. I suggest appending a question mark to the parameter, or maybe prepending or surrounding with question marks. The question marks won't actually be part of the variable name.

proc puts2 {chan? msg} {
    if {[info exists chan]} {
        puts $chan $msg
    } else {
        puts $msg
    }
}

3. Allow args to have any name. Instead use some other notation to indicate that it should collect extra arguments. I suggest an asterisk. I'm not sure how this is useful, but it goes along with #1 and #2 above.

proc list2 {elems*} {
    return $elems
}

See Internationalization and the Tcl C APIs

DKF: It just occurred to me that it would be nice to support nested scopes, i.e., local variables that have scope that is less than the enclosing scope (proc, namespace). They would of course need to be unset on leaving the scope, but it's not just that: they'd become unreachable outside the scope too. Note that it would also be possible to have an inner-scoped variable override the name resolution so as to hide an outer-scoped variable:

apply {{} {
    set x "outer"
    puts x=$x
    do-an-inner-scope-somehow {local vars: x y} {
        puts "x exists: [info exists x]"
        set x "inner"
        set y "foobar"
        puts x=$x,[info exists y]
    }
    puts x=$x,[info exists y]
}}

Which will print:

x=outer
x exists: 0
x=inner,1
x=outer,0

PYK 2015-09-02: What would be the advantage over a nested apply?

apply {{} {
    set x "outer"
    puts x=$x
    apply {{x} {
        puts "x exists: [info exists x]"
        set x "inner"
        set y "foobar"
        puts x=$x,[info exists y]
    }} x
    puts x=$x,[info exists y]
}}

tclbliss: have an empty var name (or some similar mechanism) behave like a bit bucket or /dev/null (under Unix/Linux) for a clean, easy way to discard unneeded data:

set "" somedata

It would be useful in regexp or foreach statements etc, where you need to make sure that potentially large amounts of unneeded data are not being stored (even temporarily) in memory.

Andrew Bell: Forgive me if this exists, but I haven't found one. We have a lot of different vendors in EDA that embed or extend Tcl Interpreters and occasionally break Tcl functionality when doing so. I have always had to discover and send one-off bugs where I'd prefer if there was a simple compatibility suite of tests I could point them to.

AMG: Try using Tcl's own test suite. I recently used dict.test to validate forward-compatible dict.

DKF: The Tcl test suite isn't public domain. It's BSD-licensed, identical to the rest of Tcl. It's use is recommended, and has no licensing implications for anything else in the world, especially as you wouldn't usually ship it with a downstream product anyway. (If you've broken the test suite runner itself, you can be pretty sure not many standard Tcl scripts will execute well either.)

AM I think it would be wise to reconsider the default behaviour of such commands as switch and lsearch. Right now the default matching method is -glob for these two, whereas most people do not expect that. I always forget it myself and if my string to be found happens to contain glob meta-characters, the results are surprising.

DKF: The default for switch is -exact, which we (try to) compile to a hash-based jump table.

APN: I would like the default for switch to be the same semantics as the expr == operator. So 0 would match 0x0 by default. The semantics for -exact would stay as they are.

AMG: [switch] does not have a numerical comparison mode. The only way to make it have 0 match 0x00 would be to set up a regular expression that matches every acceptable form of numerical zero, which would be costly both in terms of programmer time and CPU time.

What is the benefit of a (default or optional) numerical match mode for [switch] over and above what is already available with [if]?

If the thought is a hash table could improve performance, be aware the Tcl hash is optimized for strings and not numbers. What's more, implementing numerical matches using a hash table would require canonicalizing numbers even to the point of forcing 0.0 to 0. These considerations may eat up any theoretical gains provided by a hash table. But you can experiment with this right now by implementing a canonicalization procedure and invoking it on the argument to [switch].

If the thought is [switch] is easier/faster than [if] because it avoids repeated evaluation of the term to be matched, a [switch]-like wrapper for [if] can be written. And as above, this can be done right now in Tcl.

APN: Creating a new interp takes ~ 5ms which seems a lot slower than it should be. This prevents their use in some scenarios like web servers where they would otherwise be an ideal fit. It would be nice if Tcl9 was faster in this regard. Ongoing work to move the init scripts libraries into CVFS might help some. Also possibly sharing the core structures for built-in commands (with copy-on-write when they change) might also help.

bll: I firmly believe in backwards compatibility. The use of pragmas can alter the behaviour of the base language such that the old "cruft" (as some put it) is not visible and newer interfaces are, but the language is completely backwards compatible.

pragma noabbreviations    ; # turn off unreadable stuff: pu {hello, world}
pragma listsubcommandform ; # doesn't break backwards compatibility as 
                            # you must specify the pragma to get the new form.
pragma nodeprecated       ; # provides a method to turn off deprecated commands.

PYK 2015-03-14: A namespace read command that registers a script to be called in order to process any script evaluated in the specified namespace before it is parsed by Tcl. IOW, Lisp read macros.

HaO 2015-07-07:

source uses the system encoding by default, making many scripts non portable on computers with different system encodings.

I propose to source with utf-8 encoding<<discussion>> by default and not with the system encoding.

This would make scripts more portable and allow them to start with a BOM.

osch It would be convenient if the glob-command would be able to search the subdirectories too. But please bear in mind the potential number of subdirectories so that I advise not to use a recursion.

RLE (2015-09-11): You need to be more specific. Glob does include subdirectories when it scans a directory. What it omits is recursion into those subdirectories. For that (recursion into subdirectories), there is ::fileutil::find and ::fileutil::findByPattern in the fileutil module of Tcllib.

osch (2015-09-12): Thank you for your answer. I assume fileutil is just what I meant and I will have a look at it. Sorry for my needless wish but I didn't knew about the fileutil till now.

HJB Assuming it's even possible given Tcl's dynamic nature, bundle Tcl 9 with a convention-based code formatter similar to golang's "gofmt" utility. More info here.

DKF: We can do that at any time. “Just” requires someone to write such a tool.

FM The aim to enhance expression syntax show some difficulties to deal with array (see [L42 ]). My view about it was that the fact that the array variable syntax uses parens create some ambiguities with the expressions syntax that uses parens also.

I then determined general principles : The toplevel parser syntax (Tcl) should not interact with any sublevel parser syntax (ex : expr). The significant symbols of a sublevel parser should not have any meaning for the toplevel parser (Tcl), wheras the significant symbols of the toplevel parser should have a constant meaning for any sublevel parser (let's avoid "quoting hell"). Let's qualified this design : Orthogonal grammar.

The fact that the Tcl parser has very few significant symbols make it perfect for beeing a macro-langage, embeding some other micro-langages, the first word becoming the context of those little-langages. But the ascii set of characters is too limited to achieve those Orthoganal Gramar, so I came to introduce unicodes symbols. Along those thoughts, I came to those suggestions :

* As the array value access must be reviewed to make it coherent with the expr syntax, I generalise it a little, to introduce new symbols to access part of every kind of Tcl variable :

   $Dict«$K»; # Value in dict variable Dict at key $K
   $Array“$K”; # Value in array variable Array at key $K
   $List‹$I›; # element at index $I of the list List
   $String‘$I’; # char at index $I of the string String

* A dedicated symbol to denote nested part of a variable '⊙' :

   ex : 
       dict set D«$k0 ⊙ $k1» ;# access to the value of the dict : [list $k0 [list $k1 V]]
       list set L‹i0 ⊙ i1› ;# 

This should adress some limitations of the interface to work with nested dict and list [L43 ]

* "↔" symbol to indicate a range for lists and strings, "⋮" to denote discontinuous indexes :

   set E [list {*}[lrange [lindex $l 0] 2 3] {*}[lrange [lindex $l 1] 3 4]]; # old syntax
   set E $l‹0⊙2↔3⋮1⊙3↔4› ; # new syntax

* Change the function syntax in expr : no need to use parents. Parents are used in expr to denotes subexpression only, folowing a strict analogy : Parens and Comma are for the Expr parser what Bracket and space are for the Tcl Parser. To say :

Tcl nested subscript is <Open Bracket> <command> (<space> <arg>)* <Close Bracket>

Expr nested expression is <Open Paren> <function> (<comma> <arg>)* <Close Paren>

       expr {cos,$a}

The parens are then used only to group elements. Ex :

       expr {sqrt,(cos,$a)**2 + (sin,$a)**2}

* This way, we can use expressions into every kind of variable component, since the parens 'only' denote subexpression.

  expr {
    A“0” = "even" 
    A“1” = "odd"
    V = $A“($N%2)” 
  }

* EXPR_BRACED_WORD : Now, let's imagine we have introduce into expr a "if" control construct, just exporting the "if" Tcl command into expr. As for every function in expr, the arguments must be comma separated. But we see that the argument are interpreted along a Tcl syntax.

   expr {
      if, {$i%2 == 0}, {
         set L‹$i› "even"; # Tcl syntax
      }, else, {
         set L‹$i› "odd"; # Tcl syntax
      }
   }

I'd like it exists a kind of "EXPR_BRACED_WORD" - similar to the "TCL_BRACED_WORD" - except it will denote an Expression. I choose the chars "⦃" and "⦄" to achieve this :

   expr {
      if, {$i%2 == 0}, ⦃
          L‹$i›="even"; # Expr syntax
      ⦄, else, ⦃
          L‹$i›="odd"; # Expr syntax
      ⦄
   }

It may be extend at the Tcl toplevel :

   for {set i 0} {$i < 2} {incr i} {}  
   for ⦃i=0⦄ {$i < 2} ⦃i=$i+1⦄ {} 

* EXPR_BRACKET_WORD : chars "⟦" and "⟧" are used to make a shorthand for expr.

    set y [expr {$x+2}]  BRACKET_WORD
    set y ⟦$x+2⟧          EXPR_BRACKET_WORD

* EXPR_LIST : chars "⟪" and "⟫" are used to build list of expressions (comma separated)

    set L [list [expr {$a+1}] [expr {$b*2}] [expr {double($c)/3}]] ; # old syntax
    expr {
        L = ⟪$a+1, $b*2, (double,$c)/3⟫
    }; # New syntax

On the left part of an assignation, it is a kind of lassign :

    expr {
        M =⟪⟪0,0,0⟫,⟪0,0,0⟫,⟪0,0,0⟫⟫
        ⟪⟪a,b,c⟫,⟪d,e,f⟫,⟪g,h,i⟫⟫ = $M
    } # $a,$b,...etc are all assigned to 0

What do you think about those proposals ?

---

LAM There are some pure-script code but for large images are too slow and memory killers.

---

General Discussion

It seems like as if Tcl9 will be a modern Scheme implementation with a compact but very nice and desirable practical library, great portability, and namespace support. Once lambdas, macros and the elimination of the variable/command dicothomy are set, we're done. I'm not kidding, Tcl really feels a lot like a cross between Scheme language constructs and a command shell pragmatic way of doing stuff.

Nice to see one of the best implementation for that language coming from non-Lisp folks. :)

mailto:[email protected]

MHo: Please, please: I respect all the good ideas for future enhancements, but: don't overload the core with each and every possible solution! Keep the core command set simple! Keep it upward compatible! It's better to load code for rare and special requirements as packages (extensions)...

WJP I'm sympathetic to the general point of not overloading the core, but I don't see why that is relevant here. msgcat already exists. I'm just walking about modifying the way it works. And I don't see how the modifications could be included in a separate extension. Even if they could, I don't think that would be desirable since one point of the proposed modifications is to make Tcl's behaviour consistent with that of GNU.

tombert: Since the changelog files are no longer updated by the developers I could provide two maintenance scripts for GIT. One that creates a debian compliant changelog out of any GIT history, and another file that creates a file history between two commits in CSV formatting. ... if that helps any decision making towards using GIT ...

Links

• An objectifying hook for Tcl 9
• Better Arrays for Tcl9
• Better Static Package Support for Tcl9
• Direct Shared Library Use for Tcl9
• expr shorthand for tcl9
• Getting rid of the value/command dichotomy for Tcl 9
• interactive features in tcl9 discussion
• MicroTcl for Tcl9
• OO System for Tcl9
• Tcl9 and annotated strings
• tcl9var
• The Mystical Tcl 9.0