Network Working Group R. Droms Internet Draft Bucknell December, 1989 The Knowbot Information Service 1 Status of this Memo This Internet Draft specifies the architecture and protocol of the Knowbot Information Service (KIS), a directory or ``white pages'' service that provides a uniform interface to existing Internet directory services. This service and protocol are experimental; comments and discussion should be directed to the kis-users@nri.reston.va.us mailing list. This draft document will be submitted to the RFC editor as an Information document on February 28, 1990. Distribution of this memo is unlimited. Please send comments to kis-users@nri.reston.va.us. 2 Abstract Within the metanetwork of networks that exchange electronic mail, there are many directory services that provide partial coverage of network users; that is, directories with information about some subset of a particular network's user population. Searching the collection of available directories is time-consuming and requires knowledge of each directory's user interface. Although X.500 is currently under study as a basis for an Internet-wide directory service, it is unlikely that a universal user registry will be in place in the near future. The Knowbot Information Service provides a uniform interface to heterogeneous directory services that simplifies the task of locating users in the combined network. 3 Introduction The loose confederation of networks that exchange electronic mail, including the Internet, CSNET, BITNET and UUCP(1) provides services to hundreds of thousands of users. Locating a specific user in this collection of networks is a difficult problem; there is no single directory service in which all users are represented. Searching for a user requires that queries be directed to a collection of more-or-less well-known directory services, ____________________________ (1)Quarterman and Hoskins have written a summary of contemporary networks[8] that includes descriptions of the Internet, CSNET, BITNET, UUCP and many other public and commercial networks. Internet Draft The Knowbot Information Service December, 1989 including whois@NIC.DDN.MIL [4], whois@SH.CS.NET [1], Profile [5] and X.500 [12]. Of course, each of these services accepts different styles of queries and returns its responses in a unique format. While the White Pages Pilot Project[10] is exploring the use of X.500 as the basis for an Internet-wide directory service, we believe that the current diversity of directory services will persist for some time. There are also users of other networks not a part of the Internet that are unlikely to become part of any Internet-based directory service. The likelihood that users will want to query multiple directory services through a single, uniform interface motivates the development of our new tool. We have developed the Knowbot Information Service (KIS) that uses Knowbots to automate the process of searching multiple directory services. KIS is a directory service user interface that integrates heterogeneous directory services into a single, uniform service. In response to user input, KIS forwards queries to multiple directory services. The responses are collected and formatted by KIS into a standard format for presentation to the user. The design of KIS is flexible and extensible, allowing convenient integration of new directories into KIS as they become available. 4 Knowbots and KIS The Knowbot model of distributed computation supposes a network of computational resources, into which Knowbots are launched by network users. These Knowbots then access network resources on behalf of the user, perhaps bringing remote data back for local processing or migrating to a remote location for processing of remote data. Knowbots visit multiple network sites and access multiple network services during their lifetime. Each Knowbot acts as an independent agent acting on behalf of a user and reporting results back to the user as computations are completed. Knowbots are part of the ``Digital Library Project'' [3], which will make a variety of library services available on the National Research Network. Knowbots are essential to the Digital Library, in that the project is of large enough scale that automated assistants -- Knowbots -- will be required to make effective use of the diverse and distributed services and resources to be provided by the Digital Library. The Knowbot Information System is a distributed service that represents an early prototype of the Knowbot paradigm. KIS uses two Knowbots: one that performs user interface activities and a second that provides a distributed user directory ``meta-service''. 5 Architecture The underlying model used in KIS is the client-server model, where the KIS services are provided through a reserved TCP/IP protocol port [6, 9] by a server Knowbot. The server fields incoming queries and hands those queries R. Droms [Page 2] Internet Draft The Knowbot Information Service December, 1989 off to sub-processes, which, in turn, send queries to remote directory services. The responses are formatted by the sub-processes and returned through the TCP/IP connection. Users gain access to the KIS services through client Knowbots that communicate with the server through TCP/IP. In the simplest case, the user can use telnet [7] to connect directly to the KIS server. There are client Knowbots that accept both interactive and electronic mail KIS queries. The client/server Knowbot architecture allows the incorporation of new interaction mechanisms through the development of simple front end Knowbots. In addition, the use of TCP/IP as a transport protocol allows access to KIS services from remote sites, which need only implement the interface 5.1 Client A user gains access to KIS services through a local user interface Knowbot, which is run as a user program. The user Knowbot establishes a TCP connection with a server Knowbot, through TCP port 186. After the connection has been established, the user Knowbot accepts user input, forwards commands to the server Knowbot and relays responses back to the user. Currently, there is only one implementation of the user interface Knowbot, which uses a simple ``dumb terminal'' display model. An X Window System [11] interface is under consideration. 5.2 Server After the server Knowbot accepts a TCP connection on port 186, it starts a sub-process as hands the connection to the sub-process for further processing. The sub-process accepts commands from the client Knowbot, sends requests to other directory services and returns the responses back to the client Knowbot. While the KIS server is currently available on host nri.reston.va.us, it could easily be made available at other Internet sites. Because the directory services are available to all Internet hosts (with the exception of the MCImail directory service), any KIS server can provide identical function independent of any other server. 5.3 Converting to KIS Format KIS uses AWK [2] to convert queries and responses from KIS format to the formats used by other directory services. Each directory service has two AWK conversion programs, one for queries and one for responses, in a library of KIS conversion programs. KIS passes the following arguments to each query conversion program: R. Droms [Page 3] Internet Draft The Knowbot Information Service December, 1989 BEGIN { host = "nic.ddn.mil" } END { cmd = "/usr/ucb/whois -h " host if (ident != "") cmd = cmd " !"ident else { if ((i = index(last\_name, "*")) != 0) last_name = substr(last_name, 1, i-1) "..." cmd = cmd " ." last_name if (first_name != "") cmd = cmd "," first_name } print cmd } Figure 1: AWK program for whois@nic.ddn.mil Service. Korb, John T. (JTK1) JTK@CS.PURDUE.EDU Purdue University Department of Computer Science West Lafayette, IN 47907 (317) 463-3644 Figure 2: Response from whois@nic.ddn.mil. first_name :The first name of the query subject. last_name :The last name of the query subject. ident : The service-specific identifier. org : The organization name qualifier. host : The host from which the directory service is to be requested. For example, the AWK program in figure 1 generates a request for the whois@nic.ddn.mil service. Given input last_name=korb, first_name=john, host=ddn.nic.mil, the server Knowbot will issue the request: /usr/ucb/whois -h nic.ddn.mil .korb,john Similarly, the server Knowbot uses an AWK program to reformat responses from remote directory services into a standard KIS format. Appendix A shows the AWK program for responses from whois@nic.ddn.mil. Figure 2 shows the input to the program in appendix A and figure 3 shows the resulting response in KIS format. R. Droms [Page 4] Internet Draft The Knowbot Information Service December, 1989 Name: John T. Korb Phone: (317) 463-3644 E-Mail: JTK@CS.PURDUE.EDU Source: whois@nic.ddn.mil Ident: JTK1 Last updated: (unknown) Figure 3: Response from whois@nic.ddn.mil in KIS format. 5.4 Client--Server Protocol The client and server Knowbots communicate through a simple ASCII protocol. Each message in the protocol consists of a string of ASCII characters terminated by an EOT character. Messages from the client to the server consist of a single command word, followed by command parameters, as in the following example: "service profile@nri\n" Messages from the server to the client may include multiple newline characters, as in the following example: "Name: Ralph E. Droms\n E-Mail: rdroms@nri.reston.va.us\n Source: profile@nri\n Last Updated: (unknown)\n Name: Ralph Droms\n Organization: NRI\n Address: 1895 Preston White Drive, Suite 100\n City: Reston\n State: VA\n Country: US\n Zip: 22091\n Phone: (703) 620-8990\n E-Mail: rdroms@nri.reston.va.us\n Source: profile@nri\n Last Updated: July 12 1989\n\004" The protocol is strictly synchronous -- essentially an RPC mechanism. Each message from the client to the server is followed by exactly one response from the server. If the command word in the client message is thought of as the name of a remote procedure, the additional words on in the client message can be thought of as parameters, and the reply from the server as a returned value, completing the RPC analogy. R. Droms [Page 5] Internet Draft The Knowbot Information Service December, 1989 % kis droms Searches the default list of directory services for user droms % kis -s profile@nri.reston.va.us dromsSearches the Profile service at nri.reston.va.us for user droms % kis 'alison brown' Searches the default list of directory services for a user with first name alison and last name brown % kis > service profile@nri.reston.va.us Searches the Profile service at nri.reston.va.us for user droms (using interactive interface % kis Is identical to the previous example > service profile@nri.reston.va.us > query droms % kis Searches for user alison brown > service profile@nri.reston.va.us > alison brown Figure 4: Examples of the KIS user interface. 5.5 Implementation One of the primary goals of this project was simplicity. The implementation itself is constructed, as much as possible, from existing Unix(2) tools and application programs. Unix applications such as whois, profile and finger provide access to the remote directory services. The AWK programming language performs the format translations between the remote directory services and KIS. The bulk of the server code (400 lines of C) and the client code (250 lines of C) deal with command line argument parsing and TCP-based inter-process communication. The key to the flexibility and extensibility of KIS is the implementation of the query and response translation functions. KIS uses a series of tables (really, AWK programs) to specify the set of remote directory services and to describe the translations required for the queries and responses exchanged with each remote directory service. New remote directory services can be incorporated into KIS by simply adding the name of the service to the set known to KIS, and defining a new translation description. For example, KIS has gained access to the X.500-based White Pages Pilot project by simply adding configuration information describing the X.500 DUA interface to the KIS configuration database. Similarly, KIS provides access to MCImail ____________________________ (2)Unix is a registered trademark of AT&T. R. Droms [Page 6] Internet Draft The Knowbot Information Service December, 1989 directory services through an interface application program described in the KIS configuration database. 6 Using KIS There are several interfaces available through the KIS client Knowbot. The first is modeled after the whois service, in that it accepts switches on the command line and returns a single response when the request is complete. The second interface, described in more detail in section 6.1, allows the user to formulate queries and receive response interactively. Other interfaces, using electronic mail and telnet, are provided through simple extensions of the interactive and command line interfaces. 6.1 Interactive User Interface The model used by KIS is one in which the user forms a search context that is used by KIS to query the remote directory services and search the responses. Any entries that satisfy the search context are returned to the user. When used interactively, the user can iteratively refine the search context and query KIS to successively narrow the search to the desired entries. In non-interactive mode, the user forms a single search context (e.g., through the body of an electronic mail message directed to the KIS server) from which KIS then performs a single search. The user commands in interactive mode include: service directory-service :Add directory service to the list of directory services to be searched by KIS. If directory service is not specified, or if the service command is not used, KIS uses a default list of directory services. services :List all services available through kis. org organization :Use organization as a qualifier to constrain the KIS search. ident identifier :Search for the service specific identifier, rather than a user name. query user-name :Issue a query for user-name. Equivalent to entering user-name with no command. print : Print the current list of services, organization, user name and identifier. help, ? :Print a short command summary. man : Print a longer command description. quit : Terminate the KIS session. R. Droms [Page 7] Internet Draft The Knowbot Information Service December, 1989 Figure 4 gives a series of examples of the command line and interactive KIS interfaces. 6.2 Current Services KIS has translation programs for the following services: whois (as provided by nic.ddn.mil) whois (as provided by sh.cs.net and identified by KIS as csnet profile mcimail quipu finger KIS uses the following list of services at the specified sites by default: whois@nic.ddn.mil csnet@sh.cs.net profile@nri.reston.va.us profile@megaron.arizona.edu profile@gwen.cs.purdue.edu mcimail@nri.reston.va.us 6.3 Mail Interface Host nri.reston.va.us maintains a mail interface to KIS. Mail addressed to user kis is processed by a KIS client Knowbot as interactive input. The response from the KIS client is returned to the mail sender in a return mail message. Multiple commands and queries can be packaged into one mail request, and all responses will be returned in a single mail reply. 6.4 Telnet Interface Host nri.reston.va.us also provides a telnet-compatible interface to a KIS client Knowbot through port 185. Telnet connections to nri.reston.va.us, port 185 are fielded by the Unix inetd program, and handed off to a KIS client Knowbot. 7 Status KIS is implemented as two C programs, a collection of AWK programs and other support files on a Sun 4/280 running SunOS 4.0. The KIS client Knowbot has R. Droms [Page 8] Internet Draft The Knowbot Information Service December, 1989 been ported and run on a VAX 11/780 running 4.3BSD Unix. The code should be compatible with any version of Unix that supports the Berkeley-style socket interface. 8 Acknowledgement This work was undertaken with the support of the Corporation for National Research Initiatives, Reston, VA. 9 Author's address The author's current address is: Ralph Droms Computer Science Department Bucknell University Lewisburg, PA 17837 (717) 524-1145 droms@sol.bucknell.edu References [1] User Name Server: Introduction. CSNET Coordination and Information Center, October 1988. (Available as NS--1 from info-server@sh.cs.net). [2] Alfred Aho, Brian W. Kernighan, and Peter Weinberger. Awk -- A Pattern Scanning and Processing Language. Technical report, Bell Laboratories, Murray Hill, NJ, 1974. [3] Vinton Cerf and Robert Kahn. The Digital Library Project. Technical report, Corporation for National Research Initiatives, 1895 Preston White Drive, Reston VA, 1988. [4] K. Harrenstien, M. Stahl, and E. Feinler. NICNAME/WHOIS. RFC 954, NIC, October 1988. [5] Larry L. Peterson. The Profile Naming Service. ACM Transactions on Computer Systems, 6(4):341--364, November 1988. [6] J. Postel. Transmission Control Protocol. RFC 793, NIC, September 1981. [7] J. Postel and J. Reynolds. TELNET Protocol Specification. RFC 854, NIC, May 1983. [8] John S. Quarterman and Josiah C. Hoskins. Notable Computer Networks. Communications of the ACM, 29(10), October 1986. R. Droms [Page 9] Internet Draft The Knowbot Information Service December, 1989 [9] S. Romano, M. Stahl, and M. Recker. Internet Numbers. RFC 1062, NIC, August 1988. [10] Marshall T. Rose and Martin L. Schoffstall. An Introduction to a NYSERNet White Pages Pilot Project. Technical report, NYSERNet, Inc., Troy, NY, 1989. [11] Robert W. Scheifler and Jim Gettys. The X Window System. Digital Press, 1988. [12] CCITT Recommendation X.500. The Directory -- Overview of Concepts, Models and Services. International Telegraph and Telephone Consultative Committee, December 1988. R. Droms [Page 10] Internet Draft The Knowbot Information Service December, 1989 A AWK Program to Translate Response from whois@nic.ddn.milinto KIS Format BEGIN { collecting_address = 0; last_rec = ""; next_to_last_rec = ""; found = 0; read_multiple = 0; } /No match/ { # name not found if (first_name != "") print "No matches found for " first_name " " last_name " from service " service "@" host else print "No matches found for " last_name " from service " service "@" host found = 0; exit; } /timed out/ { # network down print "Unable to connect to " service "@" host found = 0; exit; } /refused/ { # server down print "Unable to connect to " service "@" host found = 0; exit; } /host unknown/ { # network down? print "Unable to connect to " service "@" host found = 0; exit; } /unreachable/ { print "Unable to connect to " service "@" host found = 0; exit; } NR == 1 { found = 1 R. Droms [Page 11] Internet Draft The Knowbot Information Service December, 1989 last_name = substr($1, 1, length($1)-1); given_name = $2; i = 3; other_name = " " while (substr($i, 1, 1) != "(") { other_name = other_name $3 " "; i++; } ident = substr($i, 2, length($i) - 2); i++; e_mail_address = $i; # Possible phone number all on one line organization_phone = ""; for (i++; i <= NF; i++) { if (organization_phone != "") organization_phone = organization_phone " "; organization_phone = organization_phone $i; } } NR == 2 { # Is this the start of multiple responses? if (substr($0, 1, 1) != " " && substr($0, 1, 1) != " ") { print "Name: " given_name other_name last_name; if (organization_phone != "") print "Phone: " organization_phone; if (e_mail_address != "") print "E-Mail: " e_mail_address; print "Source: whois@nic.ddn.mil"; if (ident != "") print "Ident: " ident; print "Last updated: (unknown)"; last_name = substr($1, 1, length($1)-1); given_name = $2; i = 3; other_name = " " while (substr($i, 1, 1) != "(") { other_name = other_name $3 " "; i++; } ident = substr($i, 2, length($i) - 2); i++; e_mail_address = $i; # Possible phone number all on one line organization_phone = ""; for (i++; i <= NF; i++) { if (organization_phone != "") organization_phone = organization_phone " "; organization_phone = organization_phone $i; } read_multiple = 1; R. Droms [Page 12] Internet Draft The Knowbot Information Service December, 1989 next; } else if (substr($0, 1, 1) == " ") { organization_phone = ""; for (i = 1; i <= NF; i++) { if (organization_phone != "") organization_phone = organization_phone " "; organization_phone = organization_phone $i; } read_multiple = 1; next; } else { organization_name = substr($0, 4, length($0) - 3); collecting_address = 1; next; } } read_multiple == 1 { if (substr($0, 1, 1) != " " && substr($0, 1, 1) != " ") { print "Name: " given_name other_name last_name; if (organization_phone != "") print "Phone: " organization_phone; if (e_mail_address != "") print "E-Mail: " e_mail_address; print "Source: whois@nic.ddn.mil"; if (ident != "") print "Ident: " ident; print "Last updated: (unknown)"; if (substr($0, 1, 1) == "\r") exit; last_name = substr($1, 1, length($1)-1); given_name = $2; i = 3; other_name = " " while (substr($i, 1, 1) != "(") { other_name = other_name $3 " "; i++; } ident = substr($i, 2, length($i) - 2); i++; e_mail_address = $i; # Possible phone number all on one line organization_phone = ""; for (i++; i <= NF; i++) { if (organization_phone != "") organization_phone = organization_phone " "; organization_phone = organization_phone $i; } read_multiple = 1; } else if (substr($0, 1, 1) == " ") { organization_phone = ""; R. Droms [Page 13] Internet Draft The Knowbot Information Service December, 1989 for (i = 1; i <= NF; i++) { if (organization_phone != "") organization_phone = organization_phone " "; organization_phone = organization_phone $i; } read_multiple = 1; } } collecting_address == 1 { if (next_to_last_rec != "") { if (organization_address != "") organization_address = organization_address ", "; organization_address = organization_address next_to_last_rec; } if (substr($1, 1, 1) == "(") { split(last_rec, last_rec_fields, " "); organization_city = ""; i = 1; while (index(last_rec_fields[i], ",") == 0) { organization_city = organization_city last_rec_fields[i] " "; i++; } organization_city = \ organization_city \ substr(last_rec_fields[i], 1, length(last_rec_fields[i]) - 1); i++; organization_state = last_rec_fields[i]; i++; organization_zip_code = last_rec_fields[i]; # Some responses have "\r\n" line terminators... if (index($0, "\r") != 0) organization_phone = substr($0, 4, length($0) - 4); else organization_phone = substr($0, 4, length($0) - 3); collecting_address = 0; } else { next_to_last_rec = last_rec; # Some responses have "\r\n" line terminators... if (index($0, "\r") != 0) last_rec = substr($0, 4, length($0) - 4); else last_rec = substr($0, 4, length($0) - 3); } } END { if (found == 1 && read_multiple == 0) { print "Name: " given_name other_name last_name; if (organization_name != "") R. Droms [Page 14] Internet Draft The Knowbot Information Service December, 1989 print "Organization: " organization_name; if (organization_address != "") print "Address: " organization_address; if (organization_city != "") print "City: " organization_city; if (organization_state != "") { print "State: " organization_state; print "Country: US"; } if (organization_zip_code != "") print "Zip: " organization_zip_code; if (organization_phone != "") print "Phone: " organization_phone; if (e_mail_address != "") print "E-Mail: " e_mail_address; print "Source: whois@nic.ddn.mil"; if (ident != "") print "Ident: " ident; print "Last Updated: (unknown)"; } } R. Droms [Page 15]