About
=====

telnetlib3.py is an ISC-licensed Telnet Server library.

Implemented using the "tulip" module of PEP 3156, the proposed Asynchronous I/O framework for Python 3.4 and forward. This framework should be compatible with existing frameworks such as Twisted.

Development is currently in progress, Telnet Server is complete, and under testing. Feel free to make use of github's fork and "Issues" service to report any bugs or greivances.

Status
======

TODO: Charset Negotation
TODO: TelnetClient using same TelnetServerStream
TODO: fingerprinting Client & Server
TODO: nosetests
TODO: example MUD server
TODO: example wunderground.com client

Telnet
======

The Telnet protocol is over 40 years old and still in use today. Telnet predates TCP, and was used over a wide array of transports, especially on academic and military systems. Nearly all computer networking that interacted with human interfaces was done using the Telnet protocol prior to the mass-adapation of the World Wide Web in the mid 90's, when SSH became more commonplace.

Naturally, Telnet as a code project inevitably must handle a wide variety of connecting clients and hosts, due to limitations of their networking Transports, Terminals, their drivers, and host operating systems. 

This implementation aims to implement only those capabilities "found in the wold", and include, or chose not to include, mechanisms that are suitable only for legacy or vendor-implemented deviations. It even makes one of its own, wiring the 'encoding' to the value of replied by the CHARSET (rfc 2066), see UTF-8 below.

Kludge mode
-----------

With no MMU and crude process control, it was necessary to differentiate the "command channel" from "inband data" from the start. This is done by enforcing that, unless negotiated otherwise, the following core Rules:
  * Each end transmits only 7-bit ASCII, (except as used in the interpreter).
  * A line prompting input be followed by the 'Go-Ahead' (GA) command
  * Client signals end of input (send) by CR LF (Carriage Return + Linefeed)

Half-Duplex
-----------

These basic restrictions are modeled after a Teletype 33 which runs in "half-duplex" mode. While data is being printed, any keypress is buffered internaly in the teletype input buffer. The teletypist may press 'BREAK' at any time to signal a control line, the telnet Synch mechanism forwards this to the remote host.

This terminal is also strictly 7-bit ASCII. A supervisor connecting a teletype to a telnet stream would release ownership of all I/O except when the 8th bit is high, could then release further I/O by calling to an IAC interpreter.

"Synch" Mechanism
-----------------

NOT SUPPORTED.

Consider the description of a PDP-10 session in rfc139 (May 1971), presented here as a faux naif unix session:

    1. Teletype sends command input,
          find /usr -name 'telop.c'<send>

    2. Server begins output -- perhaps many rows of 'Permission Denied',
       meanwhile, user has already filled the input buffer, then decided
       to abort the previous program:
          ed /usr/local/s^t/tel^t^c

At this point, the only way to signal the attention of the IAC interpreter of the client host serving the teletype, which is currently blocking the half-duplex transmission with output and has not yet sent GA (Go-Ahead), is by a special line signal using the 'BREAK' or 'ATTN' key.

This notifies the Terminal driver, signalling the 'supervisor' (rfc139), which then sends 'INS'. Remaining control codes from the teletype (^t^t^c) continues to the remote end, but is discarded until the Data-Mark (DM) is received. This ensures the ^t and ^c characters are not received by the remote program.

TCP Implementation
------------------

In the TCP implementation of telnet, where presumably a half-duplex terminal may still interconnect, the 'INS' marker referenced in pre-TCP documents is marked by sending the rarely-used TCP Urgent option, marked by socket option MSG_OOB to send:

    socket.send(IAC, socket.MSG_OOB).

The value of the byte does not seem to matter, can be of any length, and can continue sending socket.MSG_OOB, though, the bsd server sends only a single byte:
            /*
             * In 4.2 (and 4.3) systems, there is some question about
             * what byte in a sendOOB operation is the "OOB" data.
             * To make ourselves compatible, we only send ONE byte
             * out of band, the one WE THINK should be OOB
               (...)

Where "Telnet Synch" mode is to end, DM should be inserted.  All input is discarded by the IAC interpreter until DM is received. This was used to some abuse, as greived about in rfc 529, for vendor-implemented sub-protocols:

      The TELNET SYNCH mechanism is being misused by attempting to give
      it meaning at two different levels of protocol.

The BSD client may be instructed to send this legacy mechanism by escaping and using the command "send synch":
    telnet> send synch

This sends IAC marked MSG_OOB, followed by DM, not marked MSG_OOB. The BSD server at this point would continue testing "SYNCHing = stilloob()" which continues to test errorfds (third argument to select.select). As only IAC is marked oob, it correctly ends SYNCHing at the data-mark, DM.  A modern implementation may use sockatmark(),

BSD Telnet Server sets "Packet mode" with the pty driver:

        (void) ioctl(p, TIOCPKT, (char *)&on);

And when TIOCPKT_FLUSHWRITE is signaled by the pty driver:

#define         TIOCPKT_FLUSHWRITE      0x02    /* flush packet */

awaiting data buffered on the write transport is cleared; taking care to ensure all IAC commands were sent in the netclear() alogorithm, which also sets the neturgent pointer.

Some work in progress on a correct discard_output() impl,

Not Implemented
===============

Telnet SYNCH mechanism is not implemented.

RFC-1416 "Telnet Authentication Option", RFC-1411 "Telnet Authentication: Kerberos Version 4", and RFC-1412 "Telnet Authentication: SPX" are supported by the BSD telnetd.c, but there are no plans to implement any of them.

RFC-861 "Telnet Extended Options List", May 1983. describes a method of negotiating options after all possible 255 option bytes are exausted by future implementations. This never happened (about 100 remain), it was perhaps, ambitious in thinking more protocols would incorperate with Telnet (such as FTP does).

RFC-927, "TACACS User Identification Telnet Option", describes a method of identifying terminal clients by a 32-bit UUID, providing a form of 'rlogin'. This system, published in 1984, was designed for MILNET by BBN, and the actual TACACS implementation is undocumented, though partially re-imagined by Cisco in rfc1492. Essentially, the users' credentials are forwarded to a TACAS daemon to verify that the client does in fact have access. The UUID is a form of an early kerberos token.

RFC 933, "Output Marking Telnet Option", describes a method of sending "banners", such as displayed on login, with an associated ID to be stored by the client. The server may then indicate at which time during the sesssion the banner is relevant. This was implemented by Mitre for DOD installations that much, for instance, need to display various levels of "TOP SECRET" messages each time a record is opened, preferably on the top, bottom, left or right of the screen.

RFC 946, "Telnet Terminal Location Number Option", only known to be implemented at Carnnige Mellon Univ in the mid-80's, this was a mechanism to identify a Terminal by ID, which would then be read and forwarded by gatewaying hosts. So that user travelling from host A -> B -> C appears as though his "from" address is host A in the system "who" and "finger" services. There exists more appropriate solutions, such as the "Report Terminal ID" sequences CSI + c and CSI + 0c for vt102, and ESC + z (vt52), which send a terminal ID inband as ASCII.

RFC 1041, "Telnet 3270 Regime Option", Jan 1988
RFC 1043, "TELNET Data Entry Terminal Option", Feb 1988
RFC 1143, "The Q Method of Implementing .. Option Negotiation", Feb 1990
RFC 1097, "Telnet Subliminal-Message Option", Apr 1989
RFC 1205, "5250 Telnet Interface", Feb 1991
RFC 1411, "Telnet Authentication: Kerberos Version 4", Jan 1993
RFC 2217, "Telnet Com Port Control Option", Oct 1997

Implemented
===========

* RFC 779 "Telnet Send-Location Option", Apr 1981
+ RFC 854 "Telnet Protocol Specification", May 1983
+ RFC 855 "Telnet Option Specifications", May 1983
  RFC 856 "Telnet Binary Transmission", May 1983
+ RFC 857 "Telnet Echo Option", May 1983
+ RFC 858 "Telnet Supress Go Ahead Option", May 1983
  RFC 859 "Telnet Status Option", May 1983
+ RFC 860 "Telnet Timing mark Option", May 1983
* RFC 885 "Telnet End of Record Option", Dec 1983
  RFC 1073, "Telnet Window Size Option", Oct 1988
  RFC 1079, "Telnet Terminal Speed Option", Dec 1988
+ RFC 1091, "Telnet Terminal-Type Option", Feb 1989
  RFC 1096, "Telnet X Display Location Option", Mar 1989
  RFC 1184, "Telnet Linemode Option (extended options)", Oct 1990
+ RFC 1123, "Requirements for Internet Hosts", Oct 1989
* RFC 2066, "Telnet Charset Option", Jan 1997
  RFC 1372, "Telnet Remote Flow Control Option", Oct 1992
  RFC 1408, "Telnet Environment Option", Jan 1993
  RFC 1571, "Telnet Environment Option Interoperability Issues", Jan 1994
  RFC 1572, "Telnet Environment Option", Jan 1994

* = Not implemented in bsd telnet (rare!)
+ = Required by specification (complies!)

Linemode
--------

This project is the only known Server-side implementation of Special Linemode Character negotiation with BSD Clients. This allows the server and client to agree on Backspace, Interrupt Process (^C), Repaint (^R), Erase Word (^W), etc.

In kludge mode, SLC characters are simulated for remote editing, provide an almost readline-like experience for all telnet clients, except those that perform only local editing, which are unaffected.

UTF-8
-----

CHARSET (rfc 2066) specifies a codepage, not an encoding. It is unimplemented in bsd client, and generally found implemented only in recent MUD client and servers, and possibly some vendor implementations. Where implemented, the a client replying "UTF-8" has been found, and is presumed utf-8 encoded.

The default preferred encoding for clients that negotiate BINARY but not CHARSET, such as the bsd client, is defined by the TelnetServer keyword argument *default_encoding*, which is 'utf-8' by default. Setting binary for only a single direction (outbinary or inbinary) is supported, and client support of one does not immediately toggle the other, it must be negotiated both ways for both UTF-8 input and output.

Carriage Return
---------------

There are five supported signalling mechanisms for "send" or "end of line" received by clients.  The default implementation supplies remote line editing and callback of ``line_received`` with all client-supported carriage returns, but may cause loss of data for implementors wishing to distinguish among them.

Namely, the difference between 'return' and 'enter' or raw file transfers. Those implementors should directly override ``data_received``, or carefully deriving their own implementations of ``editing_received`` and ``character_received``.

#An overview of the primary callbacks and their interaction with carriage returns are described below for those wishing to extend the basic remote line editing or 'character-at-a-time' capabilities.

CR LF: The Telnet protocol defines the sequence CR LF to mean "end-of-line".  The default implementation strips CL LF, and fires line_received on CR.

CR NUL: An interpretation of rfc854 may be that CR NUL should be sent when only a single CR is intended on a client and server host capable of distinguishing between CR and CR LF ('return' vs 'enter' key).  The default implementation strips CL NUL, and fires line_received on CR.

CR: CR alone may be received, though a client is not RFC-complaint to do so.  The default implementation strips CL, and fires line_received.

LF: LF aline may be received, though a client is not RFC-complaint to do so.  The default implementation strips LF, and fires line_received.

IAC EOR: In addition to line-oriented or character-oriented terminals, IAC EOR is used to delimit logical records (e.g., "screens") on Data Entry Terminals (DETs), or end of multi-line input on vendor-implemented and some MUD clients, or, together with BINARY, a mechanism to signal newline vendor-implemented newline outside of CR+LF during file transfers. MUD clients may read IAC+EOR as meaning 'Go Ahead', marking the current line to be displayed as a "prompt", optionally not included in the client "history buffer". Its meaning as received by server is not known ..

Resources
=========

These RFCs predate RFC 854, but are often relevant (or not)

RFC 137 TELNET Protocol.
RFC 139 Discussion of TELNET Protocol.
RFC 318 Telnet Protocol.
RFC 328 Suggested Telnet Protocol Changes.
RFC 340 PROPOSED TELNET CHANGES.
RFC 393 Comments on TELNET Protocol Changes.
RFC 435 TELNET Issues.
RFC 513 COMMENTS ON THE NEW TELNET SPECIFICATIONS.
RFC 529 A Note on Protocol Synch Sequences.
RFC 559 Comments on the new TELNET Protocol and its Implementation.
RFC 563 Comments on the RCTE TELNET Option.
RFC 593 Telnet and FTP Implementation Schedule Change.
RFC 595 Some Thoughts in Defense of the TELNET Go-Ahead.
RFC 596 Second Thoughts on Telnet Go-Ahead.
RFC 652 Telnet Output Carriage-Return Disposition Option.
RFC 653 TELNET OUTPUT HORIZONTAL TABSTOPS OPTION.
RFC 654 TELNET OUTPUT HORIZONTAL TAB DISPOSITION OPTION.
RFC 655 TELNET OUTPUT FORMFEED DISPOSITION OPTION.
RFC 656 TELNET OUTPUT VERTICAL TABSTOPS OPTION.
RFC 657 TELNET OUTPUT VERTICAL TAB DISPOSITION OPTION.
RFC 658 TELNET OUTPUT LINEFEED DISPOSITION.
RFC 659 Announcing Addtional Telnet Options.
RFC 698 TELNET EXTENDED ASCII OPTION.
RFC 701 AUGUST, 1974, SURVEY OF NEW-PROTOCOL TELNET SERVERS.
RFC 702 SEPTEMBER, 1974, SURVEY OF NEW-PROTOCOL TELNET SERVERS.
RFC 703 July, 1975, Survey of New-Protocol TELNET Servers.
RFC 718 Comments on RCTE from the TENEX Implementation Experience.
RFC 719 Discussion on RCTE.
RFC 726 Remote Controlled Transmssion and Echoing Telnet Option.
RFC 727 TELNET Logout Option.
RFC 728 A Minor Pitfall in the Telnet Protocol.
RFC 732 Telnet Data Entry Terminal Option (Obsoletes: RFC 731)
RFC 734 SUPDUP Protocol.
RFC 735 Revised TELNET Byte Macro Option (Obsoletes: RFC 729, RFC 736)
RFC 749 Telnet SUPDUP-OUTPUT Option.
RFC 818 The Remote User Telnet Service.

"Telnet Protocol," MIL-STD-1782, U.S. Department of Defense, May 1984.
"Mud Terminal Type Standard," http://tintin.sourceforge.net/mtts/
"Telnet Protocol in C-Kermit 8.0 and Kermit 95 2.0," http://www.columbia.edu/kermit/telnet80.html
"Telnet Negotiation Concepts," http://lpc.psyc.eu/doc/concepts/negotiation
"Telnet RFCs," http://www.omnifarious.org/~hopper/telnet-rfc.html"
"Telnet Options", http://www.iana.org/assignments/telnet-options/telnet-options.xml

Others
------

It should be said, that as far as historical source code of note, BSD 2.11's telnet source of UCLA, or NSCA Telnet client of Univ. of IL for MacOS is most notable.

There are a notable few modern Telnet Servers, many implementations are minimal, linemode only, or kludge mode only. Many modern Telnet Clients do not care to recognize latency in a modern network, and are kludge mode only. 

Twisted is the only other known asynchronous callback implementation of note, though it makes little distinction between protocol and stream, providing very few default handlers for intelligent negotiation, but instead would require overriding many bare IAC interpretation methods for many of the functions implemented in telnetlib3.


RFC 495
-------

RFC 495, NIC #15371 "TELNET Protocol Specification." 1 May 1973, A. McKenzie Lists the following attatched documents, which are missing.

[...] specifications for TELNET options which allow negotiation of:

            o binary transmission
            o echoing
            o reconnection
            o suppression of "Go Ahead"
            o approximate message size
            o use of a "timing mark"
            o discussion of status
            o extension of option code set

   These specifications have been prepared by Dave Walden (BBN-NET) with
   the help of Bernie Cosell, Ray Tomlinson (BBN-TENEX) and Bob Thomas;
   by Jerry Burchfiel (BBN-TENEX); and by David Crocker (ULCA-NMC).

If anybody can locate these documents, please forward them along.

License
=======

telnetlib3 is (c) 2013 Jeffrey Quast <contact@jeffquast.com>.

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

SLC functions were transcribed from NetBSD.

 Copyright (c) 1989, 1993
      The Regents of the University of California.  All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
 3. Neither the name of the University nor the names of its contributors
    may be used to endorse or promote products derived from this software
    without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
 ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
 FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 SUCH DAMAGE.