APW 2007-05-16
Inspired through the (comp.lang.tcl) discussion on "Tcl on Rails", I had some thoughts on how to make that (or at least a part of it) happen.
As I was thinking about some Tcl tools to help me create some personal web pages, I think that could be the starting point of going deeper in that area. Knowing nothing about "Ruby on Rails" (RoR) until some days ago I have now had some contact with it through downloading it and looking for some of the features.
This page shall be a container to collect ideas, wishes and concepts on something similar to "Ruby on Rails," but done with Tcl and adding features for use of XML and XSLT additionally (I don't know whether RoR also supports that). Comments are welcome.
My idea is to build something using Tcl/Itcl/TclXML/TclXslt and starkits for use with Apache 1.3 and Rivet or tclhttpd or wub.
apw 2010-05-09 The implementation of all the ideas collected here and a slightly different approach can be found in project ATWF
LV Please DON'T use XML for configuration files though. I've used several applications that did - hated every one of them...
JMN 2007-05-28 Could you or someone please explain this distaste for using XML in configuration files? I can understand that if the more complex features of XML such as namespaces are used - it can get very ugly very quickly from a human readability & editability viewpoint.. but I would have thought that if the configuration file was specified to be constrained in the XML features used it would be a much better format than the myriad ad-hoc & somewhat inconsistent formats that seem to get invented for configurations.
In particular.. it seems to me to be a great win for a configuration file parser to be able to pass off a chunk of configuration to some other body of code to deal with, so some sort of tree structure to the data makes a lot of sense.
What are the big objections to XML for configuration?
LV In my case, both technical and non-technical users dealing with these applications found reading and modifying the XML files difficult. Perhaps if there were absolutely positively NO chance a user had to modify such a file directly, then it would be okay. In the cases I've experienced, applications required the users to code the XML files directly - that was nasty. There were also times, with the same application, where some hardware or OS type problem would occur, causing one or more characters in the file to be changed. Trying to figure out what the right value should be might be obvious to some, but not always.
Then there are the applications who tried to embed logic into the XML - attempting to turn it into nearly a programming language. In the end, that application ended up moving from XML into another format in hopes of gaining functionality.
BLG Unless you really think your config files need semantics, XML is pretty much overkill. In essence you should be serializing the configuration object (or objects depending on how you roll things). Unserializing via a format like YAML reduces the volume of parsing necessary to recreate the object. XML is, and for the foreseeable future, will be all about some form of iterating over the DOM tree. Absolutely essential, I suppose, for a strongly typed language, but as you note, it quickly devolves into something not human readable. I'd go so far as to say that XML/XSL/XSLT are the least preferable method of serializing data unless you really like complexity and need semantics. Very few applications of this sort require that level of semantics, IMHO. Define and solve the problem first, then select the tech to support it and you'll be happier and *much* more successful.
I know that I am too late to fight against RoR- that is not my intention, but I see it more as a fun project.
Everybody who is willing to help is invited (I know it will be very few :-) ), but you can also help me with your comments.
When I am really starting to implement the project, I will give it the name ToW (Tcl on WebFreeWay). That was inspired by the term, which I saw in the USA, "car in tow" and it should reflect the picture: take that package as a "car in tow" which you just add to your application and you have access to the Web freeway.
I hope this page gets a lot of comments on the general idea and on requirements/features you would like to have.
This is the starting point to get it somehow done instead only discussing about it!
RLH I would have some suggestions. Either use Tcl+Snit to have a pure Tcl solution that is easy to install since it is all Tcl, or use XOTcl. I personally would use Snit for the all-Tcl version. I would also program it to target the CGI space and scale well up to FastCGI and above. It will be much easy to get Tcl installed for CGI and you would have a better chance of more people using it. I know that NearlyFreeSpeech [L1 ] gives you Tcl 8.4.14 to work with as a CGI process. So maybe that could be a proving ground because it is a pay-as-you-go service and would be "cheap" to use.
APW I like Itcl much more then XOtcl, so I don't know if that will be the way ..., and I will think about Snit.
MR If you're going this way, you may be interested in a short e-book called 'Rubyisms in Rails' (available on Safari and elsewhere). It talks about how some of the particular language features and philosophy in Ruby were taken advantage of to give Rails some of the convenience and expressiveness that people have liked. Some of the techniques would translate nicely to Tcl, some would not, and of course there are some Tcl idioms that might well make a lot of sense in this domain that aren't present in Ruby. The point is, doing a web framework that respects Tcl idioms would be quite different, and probably more interesting and useful, than one that simply tries to clone Rails in Tcl, which would likely result in a weird impedance mismatch.
APW It was not my intention to just clone RoR, but to build some Tcl-ish framework and get a little bit inspired from RoR.
LV Amen! One of the complaints that I have frequently heard about pTk (a perl effort to make use of the Tk C interface without including any Tcl) has been that the interface was not perl-y enough. I would think there isn't really a need for a straight port of Ruby on Rails to Tcl. Instead, evaluating what people using Ruby on Rails love most - which, from the brief browsing I've done, is the sheer ease of specifying minimal specs and getting generated a web site which takes intelligent defaults (gee, sort of sounds like what some people love about Tk!).
APW I am not planning a straight port, that is why I am asking for features which are useful. My direction will be to build some generators to make it easy as LV suggests to build at first simple web pages and later on web pages with more comfort.
For example I am thinking about having an XML structure (maybe similar to a XML-Schema specification) and to use that to generate XSLT templates, which then can be filled. But that is an additional feature besides "normal" HTML files. I am also thinking about a little tool which helps you defining CSS files (maybe something in that direction already exists). Another tool would be a simple generator which use a XML, XSLT and CSS file and generates the html file (that's very easy using the tools already there but for beginners that might be hard). I would also like to have as one of the databases a starkit/meta DB and I think some of the tools or all should be in a starkit to make it easy to install and start. There should be also included one of the Tcl http servers to be able to generate a small application to begin with with minimal effort.
RS Starting with Schema, XSLT, CSS doesn't look like minimal to me :^) I'd suggest to start from one use case, see what's needed for that, and how it can be implemented with most convenient for the user. Then the next, etc. ... APW you are right, XSLT, CSS etc. are not planned for the start, that is only planned for a later version :-).
escargo 16 May 2007 - I've looked at this problem from a couple of viewpoints before. One question that's central is, "Is the user trying to have access to an existing data base store, or is the user wanting to manipulate some population of objects that he or she wants to be persistent?" In the latter case, one could have Snit objects, something like the Snit object inspector, and a Snit persistence layer through a Snit data base connector. (Instead of a Tk interface to the Snit object inspector, one would need to generate a form that presented the same information.)
slebetman You guys are already starting to talk about implementation here. Whatever the implementation ends up being, itcl, XOtcl, Snit or plain ensemble, I hope from the user's point of view it looks and feels like Tk. For me, the user, I don't want to be forced to work in an object oriented manner. I guess snit is the most flexible choice. Anyway, the key is the features. What I'd like ToW to have is:
APW I had the same in mind as slebetman suggests.
JE RS' suggestion -- start from one use case, implement that in a way that's convenient for the user, then do another one, and another, and another, is a very good approach. You can't tell if a Web App framework is useful until and unless you've built actual Web Apps with it. So: does anyone have any ideas about what sort of applications they want to build?
DK77 Here's an uninformed opinion: Be as inflexible as possible. If you think OO using Snit with an SQLite backend is the way to go, don't worry about making allowances for those who see things differently. It's too complicated and they're probably wrong anyway.
M If you start with NaviServer and ns_db, you'll be half-way there. You can borrow extensive code and ideas from OpenACS. Most important things, at least to me, seem to be the mechanisms that ties the database to code, and the code to the UI. Do these right, and you have a winner.
APW 2007-05-17 I am happy that the discussion now gets a little bit more concrete, seems it is possible to get more people involved in a starting project, then I was expecting. Please continue to write down your ideas and thanks for the ones which have already been written down!
RS: re hype psychology, I'm not sure that "On Tow" has all positive connotations - doesn't it suggest "Here's a defective vehicle being slowly towed to repair or demolition"?
APW I had more people in mind which are driving there trailer when having holidays and had their "normal" car in tow. Don't know, if that is still true, my experience is 25 years ago. Nevertheless I am open to a better name for it, please make suggestions. I think it should be a really "cool" name, to help it to become well known, if the project succeeds.
stevel Tcl on Trax ?
APW What does "Trax" stand for?
stevel Trax, Tracks, Rails .... never mind, I was ranting.
MR FWIW, most of the suggestions about what to include so far would correlate to building yet another web framework without most of the conveniences that Rails brought (not that there is anything wrong with that, just saying). If you think the big thing about Rails is the scaffolding, you're missing the point. The other meta-note is that one of the reasons Rails was interesting is that (being extracted from a particular type of application), it handles a certain class of application very well, and becomes just gross for many other types of applications. The Tcl community historically has worried a lot about generality and doing things to fit many different situations, which is the antithesis of this approach. Frankly, I'm not about to knock anyone who wants to go about writing a Rails-or-any-other-framework-in-Tcl knockoff from scratch "just because", but my expectations for anything of the sort would be quite modest.
Also of interest might be a framework that targets a different type of web (or other) application. Are there niches that existing solutions aren't filling well? Are there any such niches that Tcl seems to do well in? Are there existing Tcl apps in the niche to use as inspiration (extracting the framework from them, or at least core ideas)? Is the niche large enough to support a bit of a community of users, and can enough be put in place off the bat to interest that community? And could people be restrained enough to stay focused and not try to turn it into a general purpose solution capable of doing anything, but nothing well?
APW Just one question to MR: what do you mean with "conveniences that Rails brought"? Maybe that also can at least partially be included, if I know what you are speaking about.
NEM Like others, I'd say keep it simple and concentrate on a single particular application you want to build. I'd also suggest that the very first thing you should do is figure out why we need this project at all:
APW Answers:
Additionally: my intention is not to compete with other applications, as I mentioned already, I see it as a fun project, so one of the interesting points for me is, does it help me and some others to have a tool which saves work in building websites with or without a connection to a database. If more poeple later on like it it is a good side effect and maybe it can be enhanced be additional features.
CMcC I wish I understood what RoR actually does - what its core value is.
LV http://www.onlamp.com/pub/a/onlamp/2006/12/14/revisiting-ruby-on-rails-revisited.html is part 1 of an O'Reilly intro to Ruby on Rails.
MR Note that most of the intro Rails articles you'll find focus on the scaffolding (ooh look, basic application up quickly!), which is cute but not the real highlights, which have to do with how many things the frameworks do for you without intervention at all (which as I said, by design makes it ideal if you're building the type of app it's designed for, and a nightmare if not...). Needless to say, if things are done for you, you can't make another choice! You have to dig a bit deeper than the quick tutorials to get a good sense of this, but what's really going on is that it's raising the level of abstraction for building those types of applications higher than you'd get with, e.g., Aolserver/ACS. Beyond that, it's the layers of convenience routines and add-on modules that over time get added to any popular tool.
escargo 17 May 2007 - As a base for a browser-based application framework, there needs to be a web server and a data base back end. The choice of the web server could be anywhere from the small (castle) to large (AOLserver). I keep thinking that, for many uses, having to deal with SQL in the back end is wrong because mechanism (SQL) is getting in the way of policy (persistence). If the scaffolding took care of object-relational mapping rather than connecting SQL tables and fields I think it would be a more convenient framework.
LV For cases where the data in question is ONLY going to be used in the web application, then requiring a SQL database might be a bit pushy. However, many web apps that I see also involve interacting with existing databases, or at the very least databases which are used for other purposes (purchase orders, email lists, whatever). In those cases, having a common format, with fast reliable access and scalable support, seems to be a reasonable requirement.
What alternative (to sql tables and fields) did you have in mind in terms of having the scaffolding taking care of the object-relational mapping?
APW What about one step further in abstraction and instead of speaking of databases use another abstraction level and speak of containers, parcels and elements where a container can be a database or a computer, a parcel is a table/tablerow in a database or a file on the file system and an element is a field in the table or a "part of a line" (however that is defined) in a file. There are other definitions as database and computer (systems) possible. The the controllers would just be common interfaces to whatever system is used (computer/database etc.). I am also thinking about some layer for verifying the incoming data within the controllers before writing (handing over the data) to the destination system.
escargo - My minimal application is a contact manager. I have people that I need to track. Each person has some data (name, address, e-mail address, phone number, notes) that I need to maintain. I want to be able to produce reports (sorted by name for example). I need to transform the data (produce e-mailing lists). I don't care how the data are made persistent.
From a practical standpoint, every object needs a presentation layer; every object needs a verification layer (to ensure that constraints on values of fields are enforced); every object needs a persistence layer (where data and metadata can be preserved). All this could be in a metakit data base, an SQLite data base, an XML file, a Tcl file. To a certain extent, I don't care.
When it comes to building access to an existing data base, then the scaffolding needs to find a way of exposing data base rows as editable forms. It does depend on which way data will be moving.
FF - from my modest point of view, I can say that, no matter how much you want to abstract concepts, you still have to think eventually in low level terms. The biggest deal in doing high level features is the lack of building blocks (the low level stuff); a practical example would be (in user interface land) that one imagines using entries, treeviews, listviews, date-selectors, as they are used to working with in Tk, and the Tk-like widgets are not there when spooling output in HTML format. The list of missing stuff can be quite big, but in my opinion database is a feature which comes prior XML, but a template engine - maybe it's every day bread and it comes first of all if one want to deal with markup languages (but that's an opinion; I can't imagine the infinite uses of such an application). (p.s. I like the idea of using Snit). Theoretically this project could spawn more than one subproject to fulfill these lacks. I would look outside to see how other projects/frameworks cope with that, but try to not just clone them; instead keep in mind the real life problem and work on what's missing in Tcl to achieve it.
In the past I worked with PHP, doing web/db stuff. I used to code my own framework, but that was concerned with a too specific problem. After I've seen numerous web frameworks make use of a common set of features and concepts, I selected one at random [L2 ]... that's an interesting feature list to read.
Maybe a feature list is a place to start?
APW 2007-05-20 the reference [L3 ] of FF is very close to the ideas I had in mind. It is also very similar to the approach of RoR (if I have understood that correctly).
To conclude what I plan to do is: use MVC concepts to have additional layers below model, view and controller, allow template html forms which are "preprocessed" by Tcl (on the server side) using for example:
<% some code %>
or:
<%TCL some code %>
and to let Tcl build the final html output on the fly when called via some cgi-bin script interpreting the above parts within "".
What I also plan is to have a TK GUI which represents the file system structure of the views, templates etc. as a tree so you can navigate and open the files in a text widget or with your favorite text editor. But if you only want to maintain the structure using the GUI, it should be possible to directly edit the files (containing for example the templates) within the file system.
I want to put the commands/application for maintaining the structure (the scaffolding) into a starkit, to make it very easy to start with the tool, just download the kit file, open a new app and start building the structure and filling in the templates.
RLH I would go with <% some code %>. Less to type.
APW that is also what I would prefer, as it is also what Tcl Rivet uses, which perhaps some users are familiar with.
LV These last two comments puzzle me. I am not certain to what they refer and the comment's don't seem to tie into other things on the page now. Did something get deleted? APW did fix the missing/deleted stuff, don't know how they did disappear during editing from somebody; does anyone have an idea how that can happen and how you can avoid that?.
I have looked around in Symfony, Propel, Creole, Phing - all parts of PHP/written using PHP for doing similar things like RoR and I like very much the strategy used there to put all relevant information into XML/XSL ... files, using DTD and Schema files for validation. Not that I want to start with that directly, but I want to keep that in mind, because that can make automatic validation of input later on relatively easy. As there are already tools in Tcl for handling that sort of things tDOM including XPath, tclXML etc. the parts are already there to be combined for an application.
APW inspired from stevel Tcl on Trax what about using TRAX (Tcl Rapid webAccess eXtension) as the name instead of ToW
escargo 21 May 2007 - Would there be any benefit to making the templating compatible with what's available as part of AOLserver Dyanmic Pages? (Or is that assuming the answer too soon?) Or is the templating in Wtcl adequate?
APW 2007-05-21 after having a more detailed look at symfony [L4 ], propel [L5 ] and phing [L6 ] (the PHP analog on to RoR) I like some of the stuff they provide for example having XML files for storing all the information and for automatically having the possibility to validate all input. What I don't like is that the application(s) is(are) too big and on some places too complicated, but I think if one would strip down that and also have a look at RoR that would be the right direction to go. After a short look at AOLserver ADP for me that has too many hard wired features which could make it hard for programmers to be flexible. Wtcl needs to many calls in my opinion (but that is a question of taste :-)) ). What I want to get is a generator which makes the "low level stuff" and produces files which I can modify if I am not satisfied with the generated stuff. By the way the ideas for templating for RoR and symfony are after all very similar, I think both have been influence by the Apache Torque project (for symfony I know it).
escargo - I thought part of the point of Rails was "convention over configuration" so that you could do away with XML files. Given that Tcl has such great introspection and dynamic code creation, I think there might be useful ways to avoid XML.
MJ - This is probably the most important strength of RoR. When you have a collection of users, the associated table in the db will (by default) be users. You can change it, but normally you don't have to. This makes configuring your application very straightforward, there is no need to spell out every detail of your schema in configuration files. Another thing RoR does systematically is ditch XML in favour of YAML which is a human readable configuration file format.
APW interesting arguments, will have a closer look at YAML. Nevertheless I was thinking about let a GUI generate the XML files with minimal input from a user and only if you want to make special things you would modify and not create the XML configuration files.
escargo - Consider what the problem is that use of XML would solve, and then consider whether there might be other solutions. I might consider XML to be suitable for use in importing or exporting data from a system that uses a data base as a back end. For a Tcl-based system, we have many examples of where a Tcl application saves its state directly in Tcl files (sometimes read into a safe interpreter for processing).
APW 2007-05-23 One advantage is, that there exist a lot of Tcl tools to work with XML files and using XSLT makes it easy to convert it and using XSD-SCHEMA allows you to verify input easily. Not that I am fixed to XML but at least there is no Tcl Parser for YAML and the existing C-parser is not easily integrated to Tcl (I have looked at it and it also not easy (not difficult but a lot of work) to write a Tcl native parser for YAML. Until now all that stuff are just ideas and I am still open to change my mind and use a different technique (hopefully the best for Tcl environment).
slebetman - Personally I'd prefer a dict. It's natively Tcl and is far more human readable compared to either XML or YAML.
APW I was also thinking about (nested) dicts, because I like them, but would there be a problem if we not take one of the "mainstream" configuration file formats as other scripting languages are using them? For me it would be ok. Any comments?
LV The problems I can foresee are this:
APW The configuration files will not be editable over the net, but the designer of the application will be the only one which edits them. So there seems to be no need to use a safe interpreter. The user of the application gets only forms to enter data via a HTML page sent to the browser using a POST request for the return. The execution of a configuration file or a file produced from a configuration file will (on the server) generate a HTML form which is forced by a cgi-bin request which in turn starts a request to a generator for a form page and the generator produces a HTML page as response (eventually interpreting Tcl code in the generic file for generating the page). Hope that explains it a little bit clearer. If the designer of the configuration file opens the door that is a different issue.
APW Using dicts, do you think a configuration file could look like the following:
databases {
customer {
tables {
person {
columns {
person_id {
attributes {
type integer
required yes
primary_key yes
size 10
}
}
department_id {
attributes {
type integer
required yes
size 10
}
}
first_name {
attributes {
type string
required yes
size 25
}
}
last_name {
attributes {
type string
required yes
size 35
}
}
}
}
department {
columns {
department_id {
attributes {
type integer
required yes
primary_key yes
size 10
}
}
company {
attributes {
type string
required yes
size 25
}
}
name {
attributes {
type string
required yes
size 35
}
}
}
}
}
joins {
person:department {
from {
table person
column department_id
}
to {
table department
column department_id
}
}
}
}
}That could be the "abstract" description of a little database (container). The indentation does not matter; it's just for readability!
escargo - When humans can edit the file, to some extent all bets are off. At least with Tcl we can execute the file in a safe interpreter to extract the information. Even with XML, if humans are allowed to edit the file it could still wind up being broken. APW You are right!
DKF: That reminds me of dealing with users of some software I wrote that were complaining that "your software is broken", when in fact they'd written ill-formed XML configuration files (despite all the examples being correct) and the software was giving very clear error messages in the log (it was server software, so that was the right way to report problems). The moral? Many users are stupid and will not read even the clearest of instructions. :-(
FF: but XML structures also have DTDs. I guess you can perform DTD validation prior to using the XML file; so you'll know if the XML tree is not valid before parsing/using it. Instead I don't like Tcl config files. Yes, they look cool, but it's too easy to break an application and difficult to find such bugs (if whoever has edited the config files was so good at hiding the error, and your syntax highlight can't help).
escargo 25 May 2007 - I had a bit of an "aha" moment and realized that there is some code out there that related to this discuss (that's the good news): Orb Spinner. (The bad news is that web site that hosted it is gone.) Orb Spinner was, in some sense, more of a Content Management System than a web application framework because it dealt with static pages rather than dynamic pages. That duality might be considered somewhat artificial. Many of the pieces that Orb Spinner used (data base for values, a templating engine to convert a template + data into an HTML page) could be reused in a more dynamic environment. I'd certainly like to see the possibility of Tcl being used for both static and dynamic pages.
APW Do you have any code of Orb Spinner which you could send to me so that I could get some ideas (it sounds interesting from what you are describing), as there seems to be a dead link as you have mentioned. At the moment I am thinking about using a metakit database for the values and files for templates for generating HTML pages with a simple GUI representing the project (this is mainly the root directory), the storage (database or filesystem), the collections (tables in the database) and fields (columns in the database).
So the tree structure I am planning could be:
projects
wiki # this is the project name
storages
adapter # which database to use
sqlite
:
collections # the tables in the database
pages # this is a table for storing wiki pages
fields # the columns in the database
name # one of the pages columns used for storing the page name
: # here the attributes will be filled in
contents # one of the pages columns used for storing the page text
: # here again space for the attributes
:
:
authors # this is a table for storing author infos
fields
name
:
:
project2
storage
adapter
mysql
:
:More details to follow. The adapter could be "none", that would be for web pages without a database in back, if you just want to handle the layout using the tool. The collections represent the model (a la MVC (Model/View/Controller)) concept, the views and layout will be below that topic, the controllers will be below the storages.
Any comments?
escargo - I don't have an OrbSpinner download on the machine I'm using at the moment. I'll have to get a different machine (rarely used) back on-line to look.
escargo 27 May 2007 - I had a few more thoughts about this as I was taking a walk today.
APW 2007-05-28 what you are writing sounds very interesting, nevertheless I am not sure if I have completely understood all you want. Could you perhaps make some small examples to show me what you want to have. You can also contact me personally, if you want, because I am not sure if that would be too much code to put it into this thread (see APW).
APW a little bit later after having spent some thoughts on the above.
If I understand it right, the output would always be HTML, as I don't know of any browser which directly can handle XML and XSLT etc.
So it is more the layout of the templates from which generator(s) produce the HTML output, where you think that should be more general to allow all the different inputs you mention. Am I right? And you think the generator(s) (of the HTML output) should be flexible enough to make calls via some interfaces to the data sources to get the input they need.
In RoR, if I have understood it correctly, they have something like "abstract" calls to the data sources with the "CRUD" (Create, Read, Update and Delete) layer, which is differently implemented for each data source for example by having the calls as abstract virtual calls in a base class, which must be implemented by derived classes.
FF I'll spend some words here, first about Model/View, then about the template system.
In previous examples is not clear where (if any) borders are defined/enforced. In the past (in work environment) I was pushed very strongly to enforce the MVC paradigm. Although the C(ontroller) often was disputable, there were no doubts about M(odel) and V(iew). In second analysis, the Controller can be (and IMHO, it should be) the funniest part, where you can put your best creativity to solve common problems, to reuse code, to avoid repetitive stuff, to automate some tedious works... But let's focus on the M/V part: imagine you fetch data from an hypothetical Page table (along with its foreign table references):
Page {
id {15}
title {my page}
description {an example page}
text {....}
Author {
id {346}
name {some guy}
email {[email protected]}
}
modification_date {20050123}
access_counter {474}
Comments {
{id {4657} text {cool!} Author {id {655} name {some other guy} email {}} date {20070605}}
{id {4659} text {sounds interesting} Author {id {125} name {a developer} email {[email protected]}} date {20070606}}
}
}Ideally this data comes directly from the Page model, which knows which operations to perform on read (e.g. increment the access_counter, perform join across the Author and Comment table, ...) like:
set data [MyPageModel::read 15]
So that $data holds structured data, but we don't know how to make it appear to the user (work for the template engine).
In my opinion a template engine shouldn't know anything about HTML, XML, WAP, PostScript, TeX or whatever, but should be flexible enough to allow mixing both plain text, pseudo-code, loops, and so on, allowing almost a plain representation of the data. If we want to spool the $data (of the Page type/model) both in HTML and XML, we have to do two different views, and we have to know both HTML and XML. From my experience, a template system would work this manner:
<!-- Begin of document --->
<h1>{$title}</h1>
<p class="pageDescription">{$description}</p>
<hr>
<div id="pageText">{$text}</div>
<p class="pageAuthor">{$Author:name} (<a href="mailtto:{$Author:email}">{$Author:email}</a>)</p>
<!-- comments
<table border="0" width="300">
<tr>
<th bgcolor="green">Comments (<a href="{$SCRIPT_NAME}?action=add">add a comment</a>)</th>
</tr>
{foreach from=$Comments item=entry}
<tr bgcolor="{cycle values="#dedede,#eeeeee" advance=false}"><td>
posted by {$entry:Author:name|escape} on {$entry:date|date_format:"%e %b, %Y"}<br />
{$entry:text|escape}
</td></tr>
{foreachelse}
<tr><td>No records</td></tr>
{/foreach}
</table>
<!-- End of document --->I hope this will be understandable by most people knowing a bit of HTML or XML (or other markup language). Sorry for being so verbose: I wanted to show some practical (everyday) issues while showing a generic example. The code above I borrowed from PHP Smarty [L7 ] (a well known template engine)... it could be useful look on its doc to see which features needs a good template engine. (sorry I do not cite RoR cause I never looked at it)
A personal note that comes from my experience: it is very important having the ability to nest templates, to both reduce markup-code duplication, and help modularize stuff.
Looking for comments and (constructive) criticism...
APW For RoR that would look similar to:
<!-- Begin of document --->
<h1><%= @page.title %></h1>
<p class="pageDescription"><%= @page.description %></p>
<hr>
<div id="pageText"><%= @page.text %></div>
<p class="pageAuthor"><%= @author.name %> (<a href="mailtto:<%= @author.email %>"><%= @author.email %></a>)</p>
<!-- comments
<table border="0" width="300">
<tr>
<th bgcolor="green">Comments (<a href="<%= :SCRIPT_NAME %>?action=add">add a comment</a>)</th>
</tr>
<% foreach |entry| @comments %>
<tr bgcolor="<%= cycle values="#dedede,#eeeeee" advance=false %>"><td>
posted by <%= escape(@comments.author{entry] => :name) %> on <%= date_format("%e %b, %Y", @comments[entry] => :date) %><br />
<%= escape(@comments[entry] => text) %>
</td></tr>
<% foreachelse %>
<tr><td>No records</td></tr>
<% end %>
</table>
<!-- End of document --->Please don't nail me down on the exact syntax, I have only had a shorter look at RoR and have never used it :-) Nevertheless I think the idea is very similar in both cases, so that seems to be the right direction. The <%= ... %> and <% ... %> syntax is borrowed from ASP (Active Server Pages) to the best of my knowledge
As I see it after all the discussion, the ideas of symfony, RoR, Smarty seem to relativly similar in using the MVC concept, having an abstraction from the data source, using a generator to convert an input template in some format be it HTML, XML whatsowever with spread in Tcl code to generate (at least for the 3 systems mentioned) HTML output.
The spread in code should not only be able to handle one static input of the data source, but to be able to make additional requests to the data source for getting additional input to handle. The data source should be accessed via the abstract CRUD interface (shortly described above), the generators should be able to handle different kinds of templates (i.e. HTML, XML ...). There are controllers which handle the requests (actions) in calling generators for producing the output. The output is driven by the use of templates (the views) which describe how the output has to be formatted and by use of CRUD requests for getting the appropriate data.
As HTML and XML have both the < ... > tags there is no problem in parsing the stuff if no validation of the semantic is done, the validation could be done by some different program which could be added later on to the tool. For other stuff WAP, Postscript, TeX etc. one would need a closer look to see, if that is also possible with the same generators or with different ones (at least parsers for the input). As I only know Postscript of these others should comment that.
APW 2007-05-30 What about doing the templating the other way around and let the template be some Tcl code which is called like a preprocessor and produces output (for example a HTML page):
the above RoR example could then look like:
<%= {
<!-- Begin of document --->
<h1>} ; <%= {[@page getTitle]} ; <%= {</h1>}
<%= {<p class="pageDescription">} ; <%= {[@page getDescription]} ; <%= {</p>}
<%= {<hr>
<div id="pageText">} ; <%= {[@page geText]} ; <%= {</div>}
<%= {<p class="pageAuthor">} ; <%= {[@author getNname]} ; <%= {(<a href="mailtto:}
<%= {[@author getEmail]} ; <%= {">}
<%= {[@author getEmail]} ; <%= {</a>)</p>}
<%= {<!-- comments -->
<table border="0" width="300">
<tr>
<th bgcolor="green">Comments (<a href="} ; <%= {[set SCRIPT_NAME]}
<%= {?action=add">add a comment</a>)</th>
</tr>}
set rows [@comments getRows]
foreach entry $rows {
<%= {<tr bgcolor="}
<%= {[cycle values "#dedede,#eeeeee" advance false]}
<%= {"><td>
posted by} ; <%= {[escape [[[@comments getEntry] getAuthor] getNname]]}
<%= { on } ; <%= {[clock format [clock scan [[@comments getEntry] getDate]] -format "%e %b, %Y"]}
<%= {<br />}
<%= {[escape [[@comments getEntry] getText]]
<%= {</td>
</tr>}
}
if {[llength $rows] == 0} {
<%= { <tr><td>No records</td></tr>}
}
<%= {</table>
<!-- End of document --->}@page: the @ marks it as a "page" object (from the model of that application). getText, getAuthor etc. are methods of that object which return the field value of the appropriate field of the current row of the object. <%= is a proc which is returning its argument "<%= {hallo}" returns "hallo" or if it starts with "[" is evaluating the argument "<%= {[@page getTitle]}" returns the title field contents of the current row of the page object. the escape proc would do the HTML escaping.
The generator knows what to do with the results (for example it just outputs it to the client using the result as a HTML page).
Comments are welcome!
escargo 30 May 2007 - If we intend to keep the template syntax close to Tcl syntax, and template data access is understood to always be relative to the current data source, then I could see some things simplifying to "<%$page(title)%>" (if data are stored in tables) or maybe "<%@page title%>" if data are stored in dicts, (where @ is syntactic sugar for "dict get $").
APW Good idea, I would prefer the dict usage!
This would also allow to change the foreach loop above to:
while [@comments nextRow] {
@comments getAuthor
<%= {<tr bgcolor="}
<%= {[cycle values "#dedede,#eeeeee" advance false]}
<%= {"><td>
posted by} ; <%= {[escape <%@author name%>]}
<%= { on } ; <%= {[clock format [clock scan <%@comments date%>] -format "%e %b, %Y"]}
<%= {<br />}
<%= {[escape <%@comments text%>]
<%= {</td>
</tr>}
}FF 2007-05-30 - honestly I liked much the first syntax for template (the classic ASP-like). I developed a very minimal template engine at TemplaTcl. I hope you guys get interested in it, and share comments, ideas, and code if needed.
APW There has been a very interesting suggestion from jcw Templates and subst on that page. I think that would be a good approach for a template syntax (and an implementation is already there, some lines below :)) )
APW 2007-06-07 There seems to be a solution for templating (see above), so the next point is a database abstaction layer. For that I plan to use nstcl. After a closer look at it, it seems to have all the features needed. Nevertheless there will be the need for an additional layer to separate object relations from database table relations. I am looking around to see, if there exists already something in that direction.
escargo - Note that nstcl includes a templating system, nstcl-templating [L8 ], that addresses some of the same issues discussed here.
APW - I have seen that, but I think Templates and subst has more "power" and allows you to add much more Tcl constructs, if needed. And it also works for example for postscript output, which nstcl-templating does not (it's bound to HTML).
escargo - I had asked above (27 May 2007) if something more general than an HTML templating system was wanted. HTML Generator is getting a bit far from real HTML. -- APW I was remembering that you asked for that, so I think Templates and subst is a good solution to not only have an HTML Generator, but a more general solution.
LV I would be concerned, I think, about whether nstcl is an actively supported extension. I visited the sf.net page for it, and see dependencies on at least some software which has minimal support and note that the last release of nstcl is dated 2003... -- APW just read on the nstcl page, that MC is awaiting Tcl 8.5 to make a new release. Besides that, it is written in pure Tcl, has about 13000 lines of code, but I think 30% of that is the inline documentation, and if I have understood that correctly, the documentation is generated from the sources. At the first glance it does not look too complicated; at least the code I looked at is straight forward.
DomingoAD I'm thinking on an application server like this NaviServer/SQLITE3 (nsdbilite) with possible TK interfaces, make an application like an ERP with WEB presence, the application code goes on the server, TK can be used for specialized interface like POS and REPORTS, WEB Iinterface access for full functionality, public web presence and shoping cart, the TCL/TK interfaces are like JAVA'S WEBAPPLICATIONS they are downloaded fron the server and autoupdate itself. The server will be static linked for easy deployment (one executable and one database file) when running it creates static version on disk for fast deployment. Development is done trough the web interface allowing to modify extend tcl/tk code. Something like ADRIA [L9 ]
SDW - Adding ToW to the Wiki to describe the implementation "in progress" of these ideas
APN Whatever happened to this project? Is it being worked on or does it now belong to the bursting-at-the-seams open source orphanage?
apw 2008-12-10 at the moment there is no action from my side, as I am working on 3 other Open Source projects itcl-ng, ntkwidget and reportingTools. itcl-ng work will go down (I hope) when the 1st beta version will be released this week. If I have some spare time and ideas, I will continue with the small parts I have started to code. So don't expect to much from my side within the next few months. Nevertheless I don't see it as a dead project, only delayed.
CMcC 2010-2-22 opines: it is a major error to copy the name of a competing product. It represents a reactionary stance, which is never a good look. It also unconsciously hems you in, so you miss all the opportunities to be different and better, and you set your sights so low that all you set out to achieve is to become a knock-off.
apw 2010-04-02 my implementation project has now started with a different name, it is called ATWF.