Name for the concept of placing the contents of a whole website into a single file and serving it directly from there. By [Neil Madden]. ---- He, Neil, already uses something near to this for his personal website, based on [Metakit] and [tDOM]. I.e. he stores XML and converts that to HTML. This is currently done offline. The moment this is changed to online generation the StarSite is complete. [AK]: Related to this is a notion by [BrowseX]. This browser allows the retrieval of a website and its storage into a zip-archive. This is usually meant for easy transfer of a website, but also allow display of the archive via BrowseX, without unpacking it (IIRC - [AK]). ---- [AK]: ''Obvious'' extensions of the concept. * A ''local mode'', i.e. running the Starkit containing the StarSite, or providing access to it [[*]], in a non-web environment pops up a Tk based display which allows browsing the site without web-browser. * An extension of such a local mode would be to enable the editing of pages in the site. * Allow the starkit to run not only as [cgi]-type application, but as its own web server. Depending on the exact nature of how web pages are stored in the StarSite this can have significant overlap with the code base providing the Tcl'ers Wiki. Especially as some discussed extensions to it would allow the storage of not only Wiki markup, but other types of data as well, like images, HTML, or XML. The similarity should be clear by now. [AK]: [[*]] I should explain. My first association was that the code providing access to the contents of the StarSite was part of the StarSite Metakit, in a VFS, making the StarSite a [StarKit], or [StarPack]. As the Wiki codebase shows that doesn't have to be the case. The StarKit containing the code can be distinct form the Metakit database containing the StarSite. Regarding editing: For HTML this might have to be a free-form editor. For XML we can use [StarDOM]. Also note that the [AlphaTk] editor already has a Wiki mode. Extending this to HTML and XML modes might be simple. This implies that the StarSite access StarKit does not need to have the editor embedded into it, although that is an option too. It just has to have way of invoking editors we can hook our prefered editor into. [NEM] - Yes, I was thinking along these lines. Some other things I am considering are: * Storing data with a mime-type association (text/html, text/xml, image/gif etc). I don't believe that mk4vfs does this presently. * Allowing viewing the database as a metakit database, or a filesystem (both are useful at times). * Some sort of authentication/access-control built in. Wiki type applications with universal access are useful for some things, but often, you want more security. This needs to be designed in from the start, to be effective. * Versioning/Archiving (just like the wiki, but maybe more fine-grained?) * Ability to run as standalone HTTP or as CGI, with a consistent scripting API in both environments (ie a script shouldn't care). * Some mechanism for plugging in XML/XSLT transformations. * Ability to query database using XPath??? * Ability to group items together (for instance, grouping identical pictures in different formats: a .gif/.jpg for web grouped with a bitmap for WAP). Lots to think about. I think that getting the authentication/access-control stuff right will be the toughest bit. Looking at zope, everything seems to be an "object" (including users, scripts, static content), which can have access permissions granted to it. I know adding security features might seem like overkill, but it needs to be there from the start if anyone wants to use it (which I will). Adding it in later would be a bit hackish. ----- [NEM] - Excellent summary. This is exactly what I was planning. My main interest was in XML/XSLT generation of content, but really anything should be possible. The StarSite would sit on the server, and intercept requests using the PATH_TRANSLATED variable. So, for instance, in my website currently, the script xml.cgi can be invoked like http://www.tallniel.co.uk/cgi-bin/xml.cgi/home.xml which grabs the home.xml file and applies necessary stylesheets to it. Likewise, images could also be requested and returned from the database. The fact that MetaKit is the backend, allows for sophisticated searching and user interface options (session management, personalisation etc). Mirroring a site would be a case of copying one, highly-compressed MetaKit datafile. I find this concept quite exciting. ---- Note that the [Wikit] is a case of putting the _contents_ of a website into a file. I see above that Starsite would include a web server and, rather than using a markup style and conversion like the wikit does right now, would use xml as the markup and tdom or tclxml as the conversion software. Another difference appears to be that wikit is about content management, in a sense, in that visitors to the web site have the ability to update the pages. What other differences are envisioned? Well - as far as I am aware, [wikit] only allows the inclusion of the textual content in one file. The StarSite concept takes this a bit further, by allowing images, media etc to be stored in the same file, as well as other information (e.g. a user database). The idea of a starsite, as I ([NEM]), envision it, is that it should be able to do whatever a normal website can do, but with the added advantage of having ''everything'' in one file. So, you could, theoretically, put a wikit ''inside'' a StarSite. That is how I see it developing. At the moment it is nothing but this collection of ideas. When things start to reach a more coherent state, I (and any others who wish to join me) will sit down and start making it. The ability to update a StarSite (or parts thereof) over the web, is a feature I would like to include. The XML references are just there as that is what I like to create my site in. However, I feel StarSite should be broader than that. It should be a means of encapsulating a whole web site, with various common functionality available to make things easier (collaborative editing, authentication, session management, data storage etc). In the simplest case, a person would fire it up at home and use the Tk GUI to add static content (HTML, pictures etc). When finished, they would simply ftp the file to the webspace they use (in a cgi directory), and it would just work (just like starkits - no hassle installation). Alternatively, it could run as its own webserver, for intranets and the like. StarKits solve installation problems for regular applications. StarSites would solve it for web applications. [AK]: * Look at [Ideas for Wikit enhancements] and [Christophe Muller] to see the overlaps. * Using mime/type association for the content: Exactly as proposed for the wiki. Note that the wiki stores its pages directly in Metakit tables. It does not use the mk4VFS for its contents. * mime-types / mk4vfs: Interesting idea. Generalized: User-defined attributes for files. I am not sure, but I believe there are even native filesystems which might support this. Needs research. * Authentication/Security: Agree with building this in form the start. * Authentication/Security: Has to allow deactivation. Example: Wiki * Versioning/Archiving: The wiki codebase itself remembers the times of any change, and also saves out any change to a directory, if so configured. It only does not remember the exact changes/diffs in the internal database. The history of the Tcler's Wiki itself is a daily CVS import of the current state, making this more coarse-grained than the wiki codebase is able to support. * Regarding plugins: Ties to mime-types in my view. Based on the mime-type of a content page, and the chosen output medium we can choose which renderer to use, which editor to use, etc. The wiki already has several Wiki Markup renderers chosen automatically upon 'format' flag and medium (Tk vs. Web). ----- [NEM] 30Nov2002: Latest brainstorming on this (flow of control of a request coming into a starsite): * The whole system sits on a special virtual filesystem, with some differences: * Files have a mime-type associated with them * As well as directories, there is the concept of ''sections''. These are mounted on to directory points, and control access to all files from that point down (until a new section starts). * These ''sections'' are essentially directories, but with some procedures associated with them - namely a ''handle request'' procedure, and a ''handle error'' procedure. (Possibly others). * Sections have an access-control list associated with them. This consists of a list of ''groups'' and a set of permissions. Initially, I think the following permissions: * '''page''' - create, delete, read, edit. * '''subsection''' - create, delete, read, edit. * ''groups'' are like they are on [UNIX]. ''users'' are people viewing/editing the site. Users belong to groups. There are two special users: ''anonymous'' is a non-logged-in user, ''webmaster'' is the super-user. There are like-wise two such-named groups which contain these users. The webmaster (or admin, or root, ...) group has complete access to everything, while the anonymous group typically would only have read-only access (notice, though how a section can override this in its access control list, so a wiki could work). A user who has edit permissions on a section can alter the permissions (?? - maybe). * Access to this VFS is through a special API (probably not the standard Tcl VFS API, due to the need for mime-type associations). * Right, now onto how this all works: * A request comes into the starsite (either through the built in webserver, or CGI or...). The first stage is to authenticate the user. A seperate (replaceable) module handles this. It simply does all it has to do to determine who the user is. It removes any trace of its mechanism from the input (so, if it used a cookie, it would remove the cookie from the list passed in). It returns the username of the person making the request, or ''anonymous'' if they are not logged in. This module could work in any way, and so will be replaceable. It only works out the user name, it does not do access-control. * Next step, the starsite runtime looks at the requested URL, and figures out which section it falls in (as sections are mapped onto directories, this will be by just finding the most specific directory which is a section map point). * The starsite works out the format that the client wants the result in. It will use a (customizeable) algorithm based on specific request (e.g. if .html was request then return HTML), accept-type headers, and finally, as a last resort user-agent strings. * Access-control: The star-site then looks up the access control list of the section in question, and compares it against the groups which this user belongs to. If the user has access to this section, then we call the request handler for this section, passing in the requested URL, the requested mime-type, and the arguments passed (from ?blah=foo&a=b stuff, and from POSTed data etc. PATH_INFO/PATH_TRANSLATED stuff will not be passed in here - it will be used to figure out the requested URL). * The request handler retreives the file, and performs whatever processing it needs to do (e.g. dynamically generating the file, applying style-sheets, etc), and returns the file contents. The mime-type etc will already have been set. There may be an API for adding extra headers etc. * If an error occurs at any time, or if the user doesn't have the correct permissions, then the section's error-handler proc will be called with a mime-type and the error message. It should format a nice error message and return it. * How to enforce access-control within a section handler? Well, here I thought the best way, would be to only allow access to the VFS through a special API. When a request comes in, the request handler is called in a ''new interp'' (or one from a pool). This interp is a safe interp with access to the VFS API setup through aliases. These aliases incorporate the username into them, so that they can check access control, without the content-handler having to pass through the name of the user (that would be open to attack). * All access to the VFS and StarSite internals would be through these safe interps with checked access control. This keeps the starsite secure (at least, I think so, but I'm not a security expert - '''comments appreciated'''). * Versioning/history could be activated on a per-section basis, by adding more information to the interpreter aliases for the API - if an argument is flaged in the call then a versioning routine is called. In fact, the sections could each have an ''update-handler'' which handles edits of files, and can store away the old version. This is just some brainstorming, and hasn't been thought through to the bitter end. I quite like the design, but I'm willing to take criticism to perfect this in the design stage. Consider this, a '''request for comments'''! [Neil Madden]. ----- [Category Internet]