PWQ 31 Jan 04
From comments made in Dictionaries as Arrays.
The purpose of the aforementioned page was to suggest a way of addressing differing views on how Tcl would allow concepts to be expressed that currently cannot be. Prior to this post ,the general discussion was of the form:
Dear TCT, Can we please have arrays as first class variables?
Dear Geek, No but here are dicts that do the same thing.
Dear TCT, But dicts dont use the same syntax.
Dear Geek, We are not breaking Tcl backward compatibility just to
support your desire to have this feature.So Philip put his thinking cap on and spent some of his valuable time thinking about this issue and came up with the post as referenced above.
(DKF: That's not entirely fair. The problem is that arrays-as-first-class-entities breaks a number of other cherished features; to make it happen requires deep insight into what exactly needs to be broken. The phrase "non-trivial" is quite apt here, and breaking backward compatability always draws plenty of flak from Tcl users in my experience. They're a naturally conservative lot! As it is, I suspect that breaking traces and making upvar much hairier will do the trick.)
(MS): That dialogue sounds unreal to me. Arrays are just a type of variable, and variables themselves are not "first class variables" (whatever that may mean). Letting arrays be first-class-entities (let's call them "values") cannot happen consistently without letting variables in general be values. Backwards compat may indeed be a barrier, but the lack of a proper spec of what is meant by "variables are values" is the main one. IOW, I do not really understand what it is that is being asked.
Rather than take sides and debate which option is 'better' and use majority rules and political manuvering, I looked for a proactive solution (a solution that is better than the current one when applied to issues outside the current problem scope).
I looked at the issue from the point of view What is the least we can do that would allow all parties to go forward?.
The solution I put forward proposed a small change to the core that had nothing to do with dictionaries or arrays (nothing about them enters into the discussion other than as examples for clarification).
As I am not a Tcl CORE developer I am not aware of the ramifications of the changes I put, so I left the specifics of my suggestion to be commented on by those more familiar.
Instead of comments like:
A) It is more complicated than that, but put in a TIP so that this thread can be captured formally ...
B) Please supply more information on ... so that we can comment.we get remarks on the tone of the post and who pays who for what.
While I share some of the burden for the waste of bandwidth, this is due more to cultural differences rather than a deliberate desire to antagonize those referenced.
Why do we need to change the TCL core?
There are two parts to this; the first is that the language is not complete at the script level. Some things can only by done as a 'C' extension but some people have a desire to invoke said features from the script level.
The second reason is that Tcl is a solution looking for a problem. Every Tcl programmer applies Tcl to an instance of a problem to generate a solution (read application). In doing this, most programmers find that they have to expend more effort than first thought. If an analysis is done, it may point to some design decision made at some point in Tcl's creation (or extension/modification). The analysis further shows that if a different decision had been made, then the extra effort would not have been needed. So the programmer desires the original decision be revoked and a new one instantiated so that similar jobs in the future require less work.
[LWV writes: of course, if a different decision had been made, there are several possible results. 1: the problem the developer is attempting to solve might still not be possible in Tcl (depends on the implementation, ripped effect into other changes, etc.), 2: even if that problem might now be easier to solve, other problems that are currently easier to solve might become harder to solve. There is seldom a silver bullet that benefits all past, present, and future developer needs...]
From the Q Paradigm, a new system will be adopted only when it achieves a lower entropy than the present system and the change process provides the activation energy to enact such change of state. To express this crudely.
People are lazy and will always take the easy way out.
Q. Why don't I just submit a TIP then?
A. First, I would like to know why 50% of the TIPS dating back to 2000 have not been finalised (archived, rejected, voted on, suspended et al). They have been raised, discussed, and then no further action has been taken by the TCT. It seems a waste of time to contribute insightful input to have it sit around until is is no longer relevent.
(answer: it is not the TCT's job to implement tips. They only agree to provide a forum for discussing the proposal with a framework of knowledge, and concerns, about ensuring that new releases of Tcl minimize breakage, and then, when an implementation of the code is presented, to vote on whether to include or exclude the manifestation of a tip into the core. The tips that aren't in the core have not yet had an advocate either willing to implement them or willing to contribute in a discussion, obtain a TCT advocate to conduct a vote, and then to work with the TCT to integrate the code into the distribution).
[LWV writes: also, technically, 71% of the TIPs at http://tip.tcl.tk/ are in a finalised state (using your definition of final, rejected, withdrawn, etc. So only 30% are not yet acted upon. Since the TIP is an administrative body, if one wants to see action take place on a TIP they should contact the original author as well as others who have an interest in seeing the TIP finalized to work together on a plan to get code, doc, test cases, etc. written and debugged, then contact the TCT for a discussion on the TIP and the change, and then ask for someone to call for a vote on the TIP once all questions have been answered.]
Q. Don't talk about it, just do it, make some changes to the core and put them on the net get on with life.
A1. Because OSS relies on the collective input from developers. If no one contributed to a collective goal then there would be no OSS. My goal is to write code once, not have to change it every time a new version of Tcl is released. This is why I don't contribute extensions to Tcl.
A2. This again is reactive development. It also sets in motion a path of least resistance. That is, because someone has already written the code we will adopt it. While the change proposer should be expected to contribute substantively to the coding effort, this should be done only after discussion on the change has been concluded. Put more simply, changes that have already been written stand more chance of being adopted.
A long intro but necessary. So back to the title - If I were to complain about the TCT I would write thusly:
I would prefer to see the TCT as a body that sets the standards and framework for change rather than implementing the change. For example if someone wants a change, the TCT sets the ground rules for how it would be implemented in both the C and script api's without ruling on its inclusion or exclusion from the core. This is a big stick (to beat you with) that if in the future the TCT were to include this feature in the core, this is how they would do it. The carrot is that if you adopt their way you will not have to expend effort in reimplenting your change to stay compatible with future versions of TCL.
This means that if the change is adopted to become part of the core then the all the proposers work will be of immediate use to other users.
In Closing
Anyone that submits opinions in the public domain must accept criticism; this includes myself and the TCT. By nature, criticism is expresed in the negative.
Also when we wish to prove something, we test the assertion by attacking it with examples to disprove it. We consider the assertion proved when we run out of examples to disprove it. In the same way, when someone suggests that a given change results in a better system, the only way we can test this is though repeated argument against it.
This argument by its very nature must be expressed in a negative way. When I have expressed negative comments posted in the public domain, it is because my goal is to help make TCL All it can be. I don't ask that software be easier, I just ask that it be better.
And thanks to the person that adds the category links that I am too lazy to add.
FW: My criticism about the TCT is that Tcl progress is so ... slow. Yes, in a language used by so many, precision and consistency are valued. But somehow I just don't think the TCT are pondering and laboring continuously for the 3 years it takes to get Tcl features added. There's got to be some padding there.
[LWV writes: I don't think that the TCT is the reason that it takes years for a new Tcl release to occur. I think the reason is that the people writing code have lives outside of the freely distributable programming language, and so changes take longer to write and test. Perhaps, however, if the TCT release coordinator for a particular release chose less features to go into a release, Tcl could be up to version 124.395 or whatever... There have been cycles where the TCT thought about being more agile in the way that releases occurred.]
My guess as to how the TCT works is that the actual pondering is done on the team members' individual time, e-mails are sent back and forth, and a vote is held once a majority of members have developed some kind of opinion. Complaints are raised, but the amount of people, all individually busy, who have to discuss something, can hardly result in a fiery, back-and-forth debate that ultimately convinces a majority toward one side, since the information flow is slow enough that a debate that tackles all issues takes years. Plus, anything that raises so much controversy in the first place would confabulate and set off even more slow-motion debate on whether it's worth changing the language for something so controvertial. And by the time it hits the voting block, surely not every member will have a strong opinion, and many will vote against change by default merely due to the degree of controversy. And the same process of distraction and skepticism, even after approving a change, fillibusters the eventual actual implementation of it as well.
Tcl is run by a comittee, a system notorious for, well, y'know. A slow-motion, long-distance comittee. It's amazing anything has changed at all. I'm not blaming anyone. This is all speculation verging on fantasy, so I'm open to any TCT members who'd like to correct me here on what exactly the process is like.
RS: I have used Tcl since 7.4/4.0 and have seen many changes over the years (for a deeper look back in time, see Tcl 2.1). I liked it then, I like it now, and have over eight years not found a language that better fits my needs. Of course, history has left some nooks and crannies in the architecture and syntax details (e.g. llength vs. string length - but C and C++ have them as well...). But most important, I think of Tcl as a "gift horse" - take it as it is, and make the best of it. You can't "ask your money back" for free software. You can't tell unpaid volunteers, as the TCT is, how they better run the shop. - And re backwards compatibility: a little more than a year ago, I had to modify a script distributed as example (Euro converter) to run under an early 8.0 where even the :: syntax for globals was unknown. It took me just a few lines, and made me appreciate the overall stability of Tcl over years, which is great for an inhomogenous environment. Thanks to the TCT for their involvement, thanks to AS for providing ActiveTcl! Changes to the language can be a good thing, but even if Tcl were frozen now forever, I think I could still live with it for a long time, and discover new possibilities...
FW: I'm clearly pro-Tcl, I'm here after all ;) I love all the progress the core team has made. I didn't tell them how to run the shop, and even suggested that they may be running it in the best way practically possible. I just speculated on what exactly causes the percieved shortcomings that causes complaints such as PWQ's, and invited commentary. I don't see why that requires a refutation.
ulis, 2004-1-01-31. All the days I code I thank J. Ousterhout, the TCT and the Tclers. But I complain. Because if I think Tcl the best, I want it better. I've a gripe about some weirdness of Tcl: namespaces/environments, string/list/script Hell, expressions... But I can do a larger list of the jewels of Tcl: list handling, control constructs, input/output, interpreters(!)...
For the evolution of Tcl I find it substancial but opaque. For me it lacks bright guidelines. If I understand well, the evolution of Tcl follows the availability of code from the Tclers. So, yes I think that the role of the TCT is to be revisited: to decide of the future of Tcl, to define the goals of Tcl, the parts that need to evolve and the overall taste of Tcl, to prepare the evolutions decided and to call for design and coding. All that with the help and the approbation of the Tclers.
RS: I wasn't refuting FW's opinion, just giving mine... and I'd love a grand ironing out of warts, maybe for Tcl 9.0. But my experience is: it's easy to do things different from Tcl, but it's hard to do it better.
FW: Sorry, the fact that it was in the same line-area (whatever it's called) made me think you were addressing me. <g>
PT 2004-JAN-31: Personally I have found the TIP process reasonably effective. The main thing I have noted is that if you raise a TIP you really do have to drive it through yourself. If TIP's are languishing in the archive, it's mostly because the author hasn't pushed it along IMO. Perhaps TIPs need an expiry date like draft RFCs? It also seems to me that there are not that many of the TCT who are active in sponsoring TIPs so you are looking at some quite thinly spread people. As usual, OSS developers like to code and not politic so there is more activity in the CVS tree than in management of requests.
Frankly, it seems to me that it would be irresponsible to reject a TIP because no one had time to work on it currently.
Instead, why don't people who complain about TIPs sitting around for so long step up and take one and see it to finish.
I think the biggest problem facing the Tcl community in general (and not necessarily those on this page) is expecting others to do the work of improving Tcl.
schlenk I don't know if PWQ reads the Tcl-Core list. There is little activity there, but last year a big TIP cleanup was done, revisiting most of the TIPs that didn't move.
But i agree with PT, the TIP process relies on the submitter of the TIP to move forward. The TCT looks at it, discusses for a week probably, sometimes in c.l.t, the chat, Tcl-Core and than forgets about it, unless the original submitter asks whats up. If he does and there is a reference implementation, the feature is in or out a month later usually. Look at the Tcl-Core archive for examples.
I think PWQ would be right about TCT as change managers, if they were overwhelmed by TIP's, Java and Linux suffer this overload sometimes. Many TIP's are proposed and created by TCT members, and so they do code a reference implementation. Do you propose they should stop coding? Write a TIP and hope someone else does the coding? It simply won't happen.
If i visit the chat and read on the wiki, i get a clear impression that the TCT (or better said some TCT members) has visions and ideas for the future development of Tcl. From memory i remember anonymous functions (Lambdas), cleaned up namespace variable name resolution, more abstract list interfaces, converting Tk to use Tcl_Objs, lots of internal improvements regarding byte compilation and Tcl_Objs that would break binary compatibility etc. But most of this cannot be done in the Tcl 8 release cycle. It has to wait for Tcl 9 to be created as a development branch in CVS which has not happened. See at the long new features list for Tcl 8.5, it could have been Tcl 9 if enough developers would work at fixing bugs and maintain Tcl 8.
PWQ 1 Feb 04, I don't read news groups, or subscribe to lists. I may have missed the boat, but I don't ever recall the TCT asking for help fixing bugs. In fact the very nature of the TCT means that this responsibility is squarely in their court.
Unlike other OSS projects where anyone can submit a patch and have it added to the core, Tcl has its own internal processes.
I went to the Tcl home page, no wait, do I mean source forge or tcl.tk.com, or was that tcl.tk. Well wherever I went, I looked for the page "Sign up as a TCL contributor", and "Submit Bug Fix" forms. I looked through the TIP's but did not see a "How to contribute" TIP. (PT Well right at the bottom of the TIP index page -- TIP 1 -- there is a section on how to submit new TIPs). PWQ TIPs are not for bug fixes. ''
Somewhere there is a list of names of who maintains what part of tcl/tk , I assume you could email them with a fix. In what format I am not sure. (PT These will be TIPs 16 and 23)
Most other OSS projects make it easy to contribute fixes and modifications to the project as the "Community owns" the project. In the case of TCL we are all just observers. (PT It is quite simple to contribute patches through the sourceforge trackers. See [L1 ] and [L2 ])
FW: Well, nothing's stopping us, it's all OSS so we could make our own fork and run it as a community. So all you're saying is that there's a team that's unsatisfactory to you, that's making a product which they brashly choose to call the official Tcl. Which is true of all the langauges. It's really exactly the same as all the languages, but a bit more organized. I definitely wouldn't call Perl, Python, or Ruby community languages either - they have bodies that advance the official version of the language too. But since OSS is so fluid, the "official" bit doesn't really mean much for any OSS language. All the TCT does, if you want to downplay their "ownership" role, is moderate and improve what is called the official version. All of the other languages are the same way, and in a way Tcl is a bit better in that it's apparent how the leadership is done rather than unseen.
PWQ I do not support or criticise the structure of the development team for TCL. I have found that with other OSS projects that submitting bug fixes is straight forward and easy. From finding their site to finding either a contact / form / process and to creating the fix/change. TCL on the other hand has had half a dozen home page, still has at least three sites with claim to being the home page for TCL.
And finally for the record, I am doing my own thing. Like 100's of other TCL developers, since that's the essence of TCL programming. If you want change, do it yourself (x 100) and forget about improving the language for the community as a whole.
FW nods
LWV writes: maybe I've just been around too long, or just have a different (i.e. wrong) understanding of how things work. Here's how I have thought the Tcl community worked:
There are probably others. My point here is that sometimes there are discussions - on this page and elsewhere - that expect that the TCT is writing code, or that there is some group who is designing the future of Tcl. I have certainly seen members of the TCT who also hold a role as Tcl maintainer or Tcl developer, they do so not as TCT members, but as maintainers or developers. There are many hats, and the community is small. Hopefully there are enough people in some of the above groups that the community of others are still served.
What do you want to call an official home for Tcl? The location of the source code? That is http://tcl.sf.net/ - at least , that's the location of the source code, bug tracking, new feature request mechanism, and mailing lists for the TCT, the code which forms the basis of ActiveTcl, etc. This is the place to go if you want access to the very latest source code (via the CVS repository), submit a request to become a source code maintainer (via the TCT mailing list), etc.
Any other web site is auxilary to that web site. For instance, https://www.tcl-lang.org/ is a web site which has tried, over the years, to be the community center for Tcl developers.
The news:comp.lang.tcl USENET group is a tool for public discussion in a newsgroup forum.
This wiki is a user maintained repository of information relating to tcl and its community.
I don't know any other web sites claiming to be the official home of Tcl (there are several URLs which all map to the same web page and machine as www.tcl.tk - that's to help people using older books to learn Tcl). But if they are out there, let us know and we can figure out what they are talking about...
FW: Here's my other issue (besides the usual complaint of development sluggishness) - does the TCT actually have an opinion as to what Tcl's goal is? By which I mean: is Tcl primarily a glue/automation language that's very usable for developing entire apps, or is it at this point consciously being developed for both ends - does anyone on the team have an opinion on this issue? You might say the devel team's goal is to just keep improving, not decide how people use it, but IMO it wouldn't hurt to have an idea as to how you intend the tool to be used so you know, for instance, which TIPs to accept.
DKF: I used to have plans for Tcl, but every time I go to the Tcl conference I find out about cool stuff that I would never have thought of and which take the language and library off in other directions. :^) So instead I have a long-term goal of making Tcl really good for doing things like writing network servers. Aside from that, I also like to "scratch itches", fixing things that have been wrong for a while... :^)
LV I'm not on the TCT - so I won't speak for them. But, it appears to me, a person who has been around Tcl since Tcl 6, that Tcl doesn't have a goal, nor do I see the TCT as providing a goal for the language.
Instead, what appears to me to be the case is that individuals using Tcl have goals. And each of those individuals (whether they are a member of TCT, a member of the Tcl maintenance team, or just interested developers in the community) have the ability to submit TIPs and see them through.
I see above various comments over the years, wondering about the large number of TIPs that seem to languish. If you read the TCT mailing list, you will see that often someone submits a TIP and then participates in discussions to resolve questions and objections, works on writing code or finds someone to do the coding, etc. Those TIPs seem to move right along, get voted upon, and if passing the vote, get incorporated into the core and everything works as expected.
Other times, however, people either submit things and we never hear from them again, or discussion proceeds but the submitter doesn't work to resolve objections, or no code is written. And guess what - those TIPs just sit. Should the TCT hurry the vote along and reject the TIP? That's certainly one way to do things. However, letting things sit waiting for someone with enough interest/time/whatever to come along and take care of things seems to work as well.
The main thing missing in the Tcl community is involvment in more than talking; getting people to write code, doc, test cases, and examples is something that happens, but perhaps not with as much volume as always needed. If you want to see that change, step up and start writing!
FW: The TCT has some editorial power, though, doesn't it? The idea of [incr Tcl] in the core had a following and people willing to implement it, for instance. So to me it seems pretty clear that some ideas are (in that case wisely) rejected not due to lack of interest, but due to a conscious decision that it doesn't jibe with the overall direction of Tcl.
LV Actually, you have it backwards from my perspective - while the idea of incr tcl had public proponents, no one has yet stepped forward willing to implement it. The TCT actually would like for this to happen, if someone would do the work.
FW: Heh, didn't know that. At this point I'd be happier if someone proposed XOTcl in the core instead though, since it's superior.
DKF: Right now, getting an OO system in the core is tricky. The iTcl people never supplied the effort to back up the vote to move it into the core. Heck, that vote was taken in-person at a conference and was not even discussed on the mailing list before it was voted through; I was on vacation at the time, and I got back to find that iTcl had "won". This wouldn't be so bad, but it uses quite a different code structure and build system (iTcl is very tangled with iTk and iWidgets) and nobody from the iTcl community would enter into any kind of discussion in relation to how to get things integrated.
None of this would matter except the iTcl people still feel that they have "won" and that it is up to them to prevent there from being multiple OO systems in the core. TIP#6 was Rejected (that was direct integration) and TIP#50 has foundered (the set of people who want it to really happen seems to be disjoint from the set of people doing the work). Nobody's formally proposed what to do next.
FWIW, I will not work on integrating iTcl into the core distribution in any way. I tried, and I put a lot of effort in. I got nothing out (except a draft proposal which is gathering dust, TIP#81). Given that, the next "OO in the core" proposal had better have someone lined up to do the work because otherwise I'll be very negative...
GPS: Note: I'm not a core team member, but I do commit to the core.
"1. The TCT should stop coding and become a change management team."
There aren't enough developers that submit patches to SF. Having the TCT only take patches would mean that Tcl would probably become stagnant.
"2. The TCT should get of their collective arses and action all outstanding TIPS. If the proposers have not got there stuff together then reject the TIP. No one forced them to become TCT members, step up to the responsibility."
Some people elected to the core team do very little, or nothing. It's their choice whether or not they work on Tcl or Tk. With the licensing of Tcl and Tk, developers could keep their changes private, but the developers are willing, and want to share their changes with you. They give you a gift. If you want to pay someone at ActiveState to add a feature, then you may find the response you want. We have a great free project that lots of money and hours have been put into, and it's provided at little to no cost.
Anyone is able to take the Tcl and/or Tk sources and create their own distribution, as long as the license information is left in the sources, and the other simple conditions stated are met. A person can sell it or create their own core team. But, why do that? The Tcl and Tk cores are worked on by people that generally know how to engineer code properly, and donate their code to the public repository. It would be better to focus complaining energy towards improvements. There are bugs to be fixed, and tracked down. There are millions of code paths, and possible state changes that can occur. (I hope that some of this will help you realize that it's an exciting time, and Tcl and Tk are as good as you help make them. :)
Peter Newman 5 January 2005: The root cause of everything wrong with Tcl/Tk is the core. Cores are the root of all evil (in programming languages). The problem is that as any programming project grows in size, it becomes more and more difficult to maintain and expand. The Tcl core has long ago reached the point that it's just way too big for any single programmer to really do much with - apart from fiddle here and there. It's a dead horse, stuck in the mud and going nowhere.
However Tcl/Tk development is still very much alive and well as far as new package releases are concerned. That's because a package is a small self-contained entity, that a single programmer can easily develop and document.
The only way to fix the core stagnation is to break the core up. Essentially, every command should be a single, stand-alone, self-contained module. And there would probably be a make file that went something like:-
# My very own personal Tcl/Tk interpreter... module require set module require lists module require namespace module require string module require allTraditionalConditionals module require justTheTraditionalIf module require justTheTraditionalWhile module require justTheTraditionalFor module require joeBloggsRedesignedIf module require schemeTypeLists module require thisFantasticObjectSystem module require thatFantasticObjectSystem module require myObjectSystemIsTheOnlyWayToGo module require allObjectSystemsSuck_LocalGlobalAndPerlOurVariablesOnlyPlease etc etc
And ditto for Tk.
Programmers could then select the modules they want, to get the programming language toolkit of their choice.
I personally don't believe this would be too difficult to do. The Tcl/Tk sources are generally one module per source file already. It's just have a case of having a re-think as to how they can be compiled as separate, stand-alone, script-level programmer selectable entities - instead of into one monolithic core, as is now the case.
DKF: You have my sympathy, but I'd like to note that there'd be a lot of interconnectedness between those modules/packages. It always seems to end up that way... :^/