by [Reinhard Max], August 2003.

This little script (originally written in early 2003) fetches emails from [T-Online]'s web mailer [http://webmail.t-online.de]. I wrote it, because T-Online charges too much extra money (currently 2.95 Euro/month) for [POP3] access from outside their own dialup networks.

The name is a merger of the three words '''POP3''' (although it doesn't actually talk POP3), T-On'''line''', and '''Popeline''', the German word for poplin. I chose it because T-Online leaves you out in the rain when you want to move on to another access provider, but keep your email address.

Fortunately the webmailer allows for downloading of selected emails in the traditional mbox format. This makes it easy to re-gain the original emails, including all the header fields, and pass them on via smtp or local delivery on Unix systems.

To use it on Windows, you can either add support for storing the configuration in the registry, or simply set the variables at the beginning of the script instead of sourcing the config file.

Required extensions are '''[http]''', '''[Tls]''', and [tcllib]'s '''[mime]''', and '''[smtp]''' packages.

----
 set config ~/.pop3line
 if {[file exists $config]} {
    source $config
 } else {
    set fd [open $config w]
    file attributes $config -permissions 0600
    puts $fd "# Die T-Online E-Mail Adresse, von der die Mails geholt werden."
    puts $fd "set login {@t-online.de}"
    puts $fd ""
    puts $fd "# Das Paßwort für den T-Online Webmailer."
    puts $fd "set password {mysecret}"
    puts $fd ""
    puts $fd "Der SMTP-Server, über den die Mails weitergeschickt werden."
    puts $fd "set smtpserver localhost"
    puts $fd ""
    puts $fd "# Der lokale Benutzer, an den die Mails weitergeschickt werden."
    puts $fd "set recipient $tcl_platform(user)@$smtpserver"
    puts $fd ""
    puts $fd "# Sollen die Emails nach dem Abholen"
    puts $fd "# gelöscht werden (1) oder nicht (0)?"
    puts $fd "set delete 1"

    close $fd
    puts "Created configuration file: $config ."
    puts "Please edit and run me again."
    exit 1
 } 

 package require http
 package require tls
 package require mime
 package require smtp

 http::register https 443 ::tls::socket

 proc formail data {
    set next -1
    puts -nonewline stderr "Passing emails " 
    while 42 {
        set start [string first \n $data [expr {$next + 1}]]
        incr start
        set next [string first "\nFrom " $data $start]
        set end [expr {($next == -1) ? "end" : ($next - 1)}]
        set mail [string range $data $start $end]
        set T [mime::initialize -string $mail]
        set cmd [list smtp::sendmessage $T -queue 1 \
                     -servers $::smtpserver -recipients $::recipient]
        foreach {key value} [mime::getheader $T] {
            mime::setheader $T $key "" -mode delete
            foreach item $value {
                lappend cmd -header [list $key $item]
            }
        }
        eval $cmd
        mime::finalize $T
        puts -nonewline stderr .
        if {$next == -1} break
    }
    puts stderr ""
 }

 proc debug {descr string} {
    # puts stderr "===> $descr: $string"
 }

 proc main {} {

    set error 0
    set BASE https://modem.webmail.t-online.de

    # login and redirect
    debug URL $BASE/index.cgp
    set token [http::geturl $BASE/index.cgp]
    debug STATUS [http::status $token]
    set data [http::data $token]
    debug DATA $data
    http::cleanup $token

    regexp {name="sessionid" value="([^\"]*)">} $data -> id
    regexp {input type="text" name='([^']*)'} $data -> logfield
    regexp {input type="password" name='([^']*)'} $data -> passfield
    debug SID $id
    debug NAME $logfield
    debug PASS $passfield
    set query [http::formatQuery js 0 \
                   sessionid $id \
                   $logfield $::login \
                   $passfield $::password]

    # message overview
    debug URL $BASE/main.cgp
    debug QUERY $query
    set token [http::geturl $BASE/main.cgp -query $query]
    debug STATUS [http::status $token]
    set data [http::data $token]
    debug DATA $data
    http::cleanup $token

    # Redefine BASE according to the current session
    regexp {<A HREF="([^\"]+)/main.cgp">} $data -> BASE

    # find the emails
    set RE {name="(MAIL[^\"]*)" value="([^\"]*)">}
    set query ""
    while 42 {
        debug URL $BASE/main.cgp
        debug QUERY $query
        set token [http::geturl $BASE/main.cgp -query $query]
        debug STATUS [http::status $token]
        set data [http::data $token]
        debug DATA $data
        http::cleanup $token
        set mails [list]
        foreach {_ name value} [regexp -all -inline $RE $data] {
            lappend mails $name $value
        }
        set count [expr {[llength $mails]/2}]
        if {$count} {
            puts stderr "Downloading $count mail(s)."
        } else {
            puts stderr "No (more) mails to download!"
            break
        }
        set query [eval http::formatQuery Speichern.x 1 $mails]
        debug URL $BASE/mailfolder.txt
        debug QUERY $query
        set token [http::geturl $BASE/mailfolder.txt -query $query]
        debug STATUS [http::status $token]
        array set meta [set ${token}(meta)]
        set http  [set ${token}(http)]
        set ncode [http::ncode $token]
        set data  [http::data  $token]
        debug DATA $data
        http::cleanup $token
        if {$ncode == 200 && $meta(Content-Type) eq "x-download/rfc822"} {
            formail $data 
            if {$::delete} {
                lappend mails Loeschen.x 1
                set query [eval [linsert $mails 0 http::formatQuery]]
            } else break
        } else {
            puts stderr "$http"
            set error 1
            break
        }
    }
    exit $error
 }
 main

----

Coincidentally in a recent article German c't magazine [http://www.heise.de/ct]
showed a solution written in [Perl] for the same issue
[http://www.heise.de/ct/03/15/158/default.shtml].

----

[rmax], January 2004: I replaced the code above with my current version
that should be somewhat more robust and includes debugging support, which
becomes handy if T-Online changes the page structure, and you have to adapt
the script.

----
[[ [Web Scraping] | [Category Internet] | [Category Application] ]]