**Command Option Parsing**

[Napier] 10/24/2017 


***Preface***

This article is an a (largely opinionated) overview of option / argument handling in Tcl, many solutions and packages for this can be found in the [command options] page.  The goal here is to take a look at a proposed solution for the universal handling of options within Tcl Commands (whether they be [proc]s, [method]s, [apply]s ([lambda]s), or whatever else).  This proposal may not be the solution to every problem - or anyones problem for that matter.  Discussion is absolutely encouraged.  Suggestions will be added to the '''optcmds''' implementation behind an option for testing and comparison.

There are so many opinions on the "best way" to handle all of these topics.  Due to this, the [TIP]'s for [named arguments] seem to constantly be a hot topic (and at an almost complete stand still).  When introduced - some will say "ahh no default values - trash", others "it needs to be all local variables", others "it needs to be an array", others "none of that is needed - it needs to be simple."  So our goal here is to start to hone in on a universally (good luck) accepted solution.

'''https://github.com/Dash-OS/tcl-modules/blob/master/docs/TIP-480.md%|%optcmds%|%''' package was created with the specific intent of improving the handling of options passed to the command without trying to get too fancy.  These options already are not positional by nature, they allow giving a simple "switch" (-opt), as well as a "named value" (-opt value), and go along with the built-in Tcl syntax for taking options.  There is really nothing new to learn, and it can be implemented with very little changes to the way we define our commands (maintaining backwards compatibility).

While the https://github.com/Dash-OS/tcl-modules/blob/master/docs/TIP-480.md%|%(currently) unofficial TIP%|% provides the detailed information, this page was created to provide examples and allow discussion of various aspects of the proposal.  It's pure-tcl implementation provides various options for testing different possible methods that the final solution could be implemented if accepted. You should also view the useful links section below to take a look at the other proposals for named arguments.  

****Keep It Simple****

While the author is actually a supported of the named argument TIP implementations, they can make code hard to read and understand. The goal of the optcmds syntax is to instead keep it simple.  Focus on the parts that can be improved (especially when it comes to performance) and leave the rest of the logic to the script.

To illustrate the point being made, below is an example of the definition for named arguments.  If you are not familiar with the specification & syntax itself, this will likely serve as a great example.  Can you tell exactly what this command is expecting and how to call it?  Sure, documentation will help - but should it be required to find some external documentation when working with your source?  This is even a simple example with a single argument.  It can also get much more complex with more arguments as they are needed.

****Separating "options" from "arguments"****

We are specifically targeting the parsing and handling of options, which share many traits with named arguments.  Personally I see `options` and `arguments` as different things.  Options are "modifiers" that serve as instructions to our command as-to''''' how it should act'''''.  Arguments, on the hand, '''''serve as the input to our command'''' which are then acted upon based on the given options.  Therefore, while named arguments generally end up merging both concepts into one, keeping them separated as two concepts can make code easier to understand and work with.

Below is an example of the definition for [named arguments] via TIP-459.  If you are not familiar with the specification & syntax itself, this will likely serve as a great example.  Can you tell exactly what this command is expecting and how to call it?  Sure, documentation will help - but should it be required to find some external documentation when working with your source?  This is even a simple example with a single argument.  It can also get much more complex with more arguments as they are needed.

======
proc p { { level -name lvl -switch {{quiet 0} {verbose 9}} } {
     list level $level
}
======

What about when we need more than one argument?!  While it is the authors personal opinion here, it just becomes hard to understand the intent of the argument definition without either being the author of the code and/or spending time with the documentation to grasp what is going on.  Sure, its awesome to save a few lines of code in the body of the proc and use our little "arg scripting language" to do some logic, but it just seems like its trying to do too much.

======
proc p { { level -name lvl -switch {{quiet 0} {verbose 9}} { inc -name i -upvar 1 -required 1 } { v -name var } a } {
     list level $level
}
======


----


**optcmds**

The author will be the first to admit there is no perfect solution (and optcmds absolutely is not the perfect solution either).   

Lets say you wanted to provide a command which models its option/argument handling after the built-in tcl commands.  Lets mirror a well-known built-in commands argument handling.   The focus here is not "lets find the most efficient and best way to parse this."  Lets simply illustrate the amount of logic that can be necessary to properly parse options. We will be looking at the '''[glob]''' command.

glob defines various switches/options.  Some of these take a value and others are simple toggles:

    :   '''glob''' ?''switches...''? ''pattern'' ?''pattern ...''?

Supported ''switches'' are:

    :   '''-directory''' ''directory''
    :   '''-join'''
    :   '''-nocomplain'''
    :   '''-path''' ''pathPrefix''
    :   '''-tails'''
    :   '''-types''' ''typeList''
    :   '''--'''


----

***without optcmds***

In order to handle this syntax today, we will need to provide our [proc] with a single args ([args]).  We are going to need to iterate args, noting any values that appear to be switches (start with a dash (-), are one of the possible options, and come before the optional double-dash (--)).  Some of these switches are provided by themselves (-join, -tails, -nocomplain) while others take a value of some kind.  

If you look through pages like [command options] you will find various implementations mostly specific to `argv` (but similar in nature) which generally end up taking the form looking like the below (modified for our case):

======
proc newglob args {
  set arglen [llength $args]
  set index -1
  while {$index < $arglen} {
    set arg [lindex $args [incr index]]
    switch -exact $arg {
      -directory - -path - -types {
        set opts($arg) [lindex $args [incr index]]
      }
      -nocomplain - -tails - -join {
        set opts($arg) 1
      }
      -- {
        break
      }
      default {
        # validation is going to be required here -- did they provide an invalid
        # switch or is it simply that we have gotten to the actual arguments and
        # the optional -- was not provided?  this can be a source of errors and
        # more verbose code required.
        # have to be careful that the globed file isnt something like -directory 
        # without the user passing -- first!
        break
      }
    }
  }

  set args [lrange $args $index end]

  # now we can handle our opts and see what we need to do next
  # -- we may need to validate and/or confirm values exist and/or
  # that they are the what we expect.

  if {[info exists opts(-directory)]} {
    puts "-directory switch set"
    # handle switch
  }
}
======

As you can see, we have not even begun to actually run our procedure and its already getting quite verbose.  We could move this into a utility proc to parse for us, of course, but either way - in the case above we need to write this code for any proc that needs to have switches/options associated with it.  This can quickly slow our applications down as we add more and more of these throughout.  

'''Wouldn't it be nice if handling of options and switches like this was a native solution which was efficient and stupid simple?  Not to mention re-useable?!'''

----


***with optcmds***

Remembering that the reference implementation does '''not modify proc''', it is shown here that way to give a better idea for how the implementation would look if brought into the Tcl Core.

======
proc newglob {
  -directory directory 
  -path  filepath 
  -types filetypes 
  -nocomplain 
  -tails 
  -join 
  -- args
} {
  if {[info exists opts(-directory)]} {
    puts "-directory switch set"
    # handle switch
  }
}

======

The line-breaks shown are completely optional and meant to show the near-perfect correlation to the way the command itself is documented on its manpage.  We can look at this and instantly understand what it expects.  We know which options will expect a value provided and which are toggles.  This specification handles the "--" with switches before it in a special manner.  All switches are optional, but parsed when provided and added to a special `opts` array that our local procedure has access to when the syntax is used. 

Since we provide value names to indicate an option expects a value, it aids in providing intuitive error messaging and introspection of commands.

There are lots of ways this could be implemented of course, the $opts could be a dict or the options themselves could simply become local variables (*shrugs*).  The reference implementation allows for most of them for testing.  However, as implemented it makes it easy to separate our options from our arguments and continue on with our procedures actual purpose.


...more to come -- in the meantime - links for more information and related pages are below! 


----


****Useful Links & Information****

   * https://github.com/Dash-OS/tcl-modules/blob/master/optcmds-1.3.2.tm%|%optcmds Reference Implementation%|% - A pure-tcl implementation of the options parser discussed above.
   * https://core.tcl.tk/tips/doc/trunk/tip/479.md%|%TIP 479%|% - Add Named Procedures
   * https://core.tcl.tk/tips/doc/trunk/tip/457.md%|%TIP 457%|% -  Add Support for Named Arguments
   * https://github.com/Dash-OS/tcl-modules%|%Tcl Modules Micropackages%|% - An ever-growing collection of pure-tcl micro packages 


<<categories>> Options | Named Arguments | TIP