Introduction

The work of the author centred around the design and use of web pages involving elements of database connectivity, search engines made with proprietary Microsoft products and the setting up of newsgroups.

The output of this work was the development of a prototype Intranet site which was accessed form the URL http://www.Intranet.man01.icl.co.uk/Why/, this effort has been captured as a number of printed screens which are available in Appendices 3-5 at the rear of this document. It is assumed that the reader has a basic knowledge of HTML, there being several very readable texts on the subject available if they are not [30].

Rather than analyse every last bit of HTML, the purpose of this chapter is to discuss the underlying elements used in the design of these pages as a great deal of hidden complex technology is now involved in implementing the features of these pages.

As has been previously stated, the concept of the world wide web is relatively new, in addition, the progress of the development of the HTML standard, and the capability of the browsers that access HTML is a constant process.

The author was involved amongst other things in the development of web pages which linked to Powerpoint presentations which had been converted to HTML, in addition, the actual presentations files themselves were made available for download. As well received as these pages were by those who attended the presentations given by the author at Kidsgrove and Ashton-Under-Lyne, such pages are static and deviate little from the original capabilities of HTML and browsers.

It is now however possible to build a great deal of dynamism into web pages, such dynamism might extend to displaying animated sequences on-screen by using Java scripting or even linking into company databases.

There are various solutions available with which to add dynamism to pages, some of which were used by the author during his time at D2D, such solutions can be broken down into those which are standards based and those which are not, the standards based solutions are examined in the following section.

Please note that a great deal of the effort of the manufacturers of such solutions has been aimed at providing for database access, whilst as will be seen, CGI is reasonably implemented for solving small scale implementations of server side dynamic applications, the inefficiencies of the concept mean that it is not suitable for the kind of heavy processing that one might expect in a well used company Intranet. That being the case, the more complex approaches to these tasks are presented as well.

The Common Gateway Interface (CGI) [31], [32a, 32b] is a standard for interfacing external applications with information servers and is supported by virtually every web browser known, it is limited but can prove effective in many circumstances.

CGI implementations are commonly encountered when filling out on-line forms [33] seen commonly on feedback pages on the web, the form is filled in and its contents are then posted to a CGI program for processing.

CGI programs may be written in any language executable on the system where the server resides, typical languages used are C/C++, Fortran, Visual Basic. It is usual to find the source code for these programs in an additional directory called cgi-src/ or, if a scripting language is used, then all that will be seen is the script in the cgi-bin directory since no source code is of course associated with the script.

There are a number of so-called scripting languages available e.g. PERL, TCL or UNIX shells which may be used [34]. People quite often prefer to write scripts because they are easier to debug and maintain and modify than are programs written in a programming languages.

The main drawback with the CGI approach to interactivity is highlighted by the following diagram.

The drawback to the CGI approach lies in the round-trips of interactivity being made between the client, server and CGI program, there is the real danger that the viewer will simply be waiting at their screen for something to happen. What is more fundamental to this issue however is that the server is burdened with an additional overhead of processing to do.

Another drawback with the CGI approach is that security is poor since one is allowing the world to run an executable program on the server, this can be circumvented by placing CGI programs in a special cgi- directory to which only the systems administrator has access.

Development tool vendors have recognised the shortcomings of the CGI approach and have developed the application programming interface (API) as a solution. These API's come in vendor specific flavours, thus Microsoft offer the Internet server application programming interface (ISAPI) whilst Nestcape offer the Netscape server application programming interface (NSAPI).

The most crucial aspect of this approach is that the extension, as API's are referred to, run on the client side, cutting down on wasteful loading of the server resources. Such processing is enabled by the use of JavaScript or VBScript client scripting, note that this solution is only a half way house as the client scripting does not allow for database access.

ISAPI' are dynamic link libraries (DLL's) not executable files, this being in contrast to CGI programs, in addition, API's tend to be specific to the vendor's server software. Any client may connect to a Netscape web server, but the back end service written using NSAPI will only run with Netscape servers, needless to say, a similar state of affairs applies to Microsoft's ISAPI. This is one area where any outcome of the battle for Internet supremacy currently being waged between Microsoft and Netscape will have possible far reaching effects.

It is important to introduce the concept of ODBC, a variant on the themes discussed so far, as this particular API was used heavily in the author's web pages.

Open DataBase Connectivity (ODBC) [35], currently version 3.0 is an API 'buffer' that allows a programmer to abstract information from a database. The term buffer is used because the use of ODBC removes the need for the programmers of server software to have to code for every available database using a proprietary language.

Web pages can be hooked up to databases by using ODBC as the intermediary, communication is achieved by using a combination of HTML and a version of structured query language (SQL) [36], the standard for database querying. The term 'version' is used because it is certainly the case when using the 32-bit ODBC driver by Microsoft that the query syntax is subtly different from that of standardised SQL. The ODBC Manager (driver) figures out how to contend with the type of database being connected to.

This is a more expensive approach to the implementation of database interrogation, the distributed objects approach is more complex and was not a solution implemented by the author, several excellent references are available however for further consultation [37, [38].

One point to be note here is that the use of Microsoft Front Page as a web authoring tool was officially encouraged by the Intranet Development Council.

It should also be noted however that this was not the author's favourite program as he finds a lot of Microsoft applications to be too feature laden and bloated in their composition. It has to be said in Front Page's favour however, that it does provide a lot of extensions which would have to be coded manually otherwise.

Front Page utilises extensions either as external programs accessed as .exe files or by the use of the dynamic link libraries as previously discussed, these are referenced as .dll files. There is no bar to using custom CGI scripts or programs and there are many of these available [55], it should be noted that somewhat unsurprisingly, custom scripts do not run as fast as native Microsoft equivalents. The extensions are referred to as 'WebBots' .. short for Robots. Analysis of the HTML generated by Front Page appears to show that a lot of the extensions utilised by the program are written in Java script.

There can be no doubt that the use of databases has revolutionised the way companies can work. Manual processing and storage of customer information is feasible for small scale implementations, but for larger scale concerns such as insurance companies and banks, it is impracticable to rely on human processing skills alone to carry out large volume business transactions.

Computer databases have evolved from their initial crude beginnings into powerful relational and object oriented models, the field of database research is a subject in its own right39. It is important only to recognise that in the context of Intranet development, that the ability to hook up web pages to such potentially huge stores of information makes for a powerful company tool.

One of the initial targets of the work at D2D was to create a stockpile of standard material for the Sales and Marketing team and a database of such material was created using Microsoft Access. When working with databases, it is usual to want to carry out one of three basic operations.

1 Read from the database

2 Add to the database

3 Modify an existing database article

It was possible to implement all of these processes using HTML and ODBC, the resulting pages and their associated HTML can be found in Appendix 3:, parts 4, 5 and 6. Please note however that unfortunately, there were problems with a clash between the ODBC driver and the Windows NT Workstation software which was used in the latter part of the project, as a result of this, it has not been possible to show a sample set of return screens for the database queries. Nonetheless, those who saw the workings of the database interrogation pages were impressed with them.

The ODBC manager communicates with the database by means of setting up a source, to accomplish this, the sever administrator sets up the database from the ODBC Source Administrator window found within the confines of the Windows NT administration tools, the ODBC manager is an integral part of the Internet Information Server suite. In the case of the author's work, the database used was set up using Microsoft Access 2.0 and was stored within the confines of the development web-site, 'Why's' directories. The data source was called 'Search', this source being accessible by any number of client requests.

Using such technology is not as straightforward as using normal HTML, there are a number of extra operations which have to be carried out to be able to access the data held in the database, these are summarised by the following diagram.

The client makes a request for data to the server by means of a form submitted from the HTML file, this stream of information is fed to an .idc file which resides on the server machine. The .idc file is little more than an SQL statement which feeds the parameters sent from the original form to the ODBC driver. The required operation is carried out by the database and any results are returned to the ODBC and passed back to the server. The server returns this information to the client for display after it has been formatted by an .htx file, this file contains information about how to display the data.

Appendix 3: 9 gives some sample shots of the Microsoft Front Page view of setting up a connection with the Access database using a so-called 'Wizard' (template), comparison of the two window views will show that they both equate to the actual file 'update2.idc'. This set of files were used to update the contents of one column of information 'Subject', in the database, the complementary .htx file 'addreturn2' is shown too.

The previous work dwelt upon has introduced some state of the art concepts, whilst such technology is very impressive, it should not be allowed to detract from the underlying principles of using WWW technology, i.e. the easy presentation of information to the user.

To that end, the development web was used to give out general information, as an example, the choice of 'option 6' in the 'control panel' featured in Appendix 3: led to a screen containing references to several key Internet sites devoted to the promotion of Intranet technology [40a, 40b]. It would have required a substantial printed manual to have replicated this kind of information, information which might well have quickly gone out of date but which would no doubt be kept up to date by those maintaining the web sites referred to.

Another striking example of the use of the web to save on company time was encountered whilst dealing with the setting up and administration of newsgroups.

A large number of settings have to be made within the controls of the newsreader, there was also the problem that a lot of people don't understand the concept of newsgroups particularly well. To solve this problem, the author made a series of on-line tutorials for newsgroup users to consult, see Appendix 3: 'part 7'.

Such a business is not to be taken lightly, it took two days to think through all of the possible routines and options, capture appropriate screen-shots, write the accompanying text and convert it all to a form readable by a browser. There is the anecdote that software designers spend 5% of their time writing the engine of their software and 95% writing the interface. Such a topic, human computer interaction (HCI) [41], [42] is the subject of much research, after all, what use is a sophisticated web interface if no-one can use it ?

Company Intranets soon fill up with a whole host of information, much of it held in different formats over many servers. It is not uncommon to find Powerpoint presentations, spreadsheets, still images, video clips and sound files as well as the more conventional text based documents all on the system.

Most people familiar with the Internet will have used search engines to aid them in their search for material on the Internet. Such search engines are tools provided by third party companies on the WWW, those provided by Yahoo [43] and Webcrawler [44] are just a few examples of the many available, these facilities usually offer some degree of specialisation, Info-Seek [45] for example tends to concentrate on the sites of interest to the business community.

There is some mystique surrounding the use of search engines, enter a key word and as if by magic, the page returns a list of web pages bearing references to the chosen subject. In fact, Search Engines are not as clever as they might seem, the WWW is simply far too big to be able to allow free searching and the search results are drawn from information stored on dedicated databases maintained by those who offer the facility. It is usually necessary to register the details of a page with one of the search engines, acceptance leads to registration of the page details and web-page designers are usually careful enough to put key words such as the subject etc. at the top of their pages to ensure their reference in any search.

The author made two enquiries to those who maintain two of the best know search engines asking whether they offered a more sophisticated searching mechanism than this, since such searches are really little different to searching an Access database, unfortunately, these enquiries went unanswered.

The reason for the enquiries was to try and make a comparison between the search tools of those institutions and the facility offered by the Microsoft Corporation in the guise of their Index Sever tool, currently version 1.1. This is a powerful if difficult to properly use feature which operates in conjunction with the Microsoft NT 4.0 Workstation and Internet Information Sever 3.0 software.

What makes this a truly useful tool for the Intranet is that it is capable of searching within formatted documents, that is to say that it could pull out text from within a Powerpoint presentation referenced from an HTML page, this represents a powerful means of searching a companies database.

Chronologically speaking, the development of the search facility was the last piece of work taken on by the author and was not fully implemented. Microsoft provide example scripts for Index Server though the author found that some of these worked spuriously or not at all, nonetheless, the 'Simple' search facility provided was sufficiently powerful to be adaptable for use in the D2D Intranet.

Appendix 4: shows some example code for setting up a page using Index Server 1.1 and the results of a search of the Intranet server using the search term 'D2D'. One can see how all formats of document are retrieved by this tool, making it very handy for those who wish to search out presentations or other formatted documents. Again, as with all Microsoft products, a certain dependence upon the use of Microsoft products then develops in using this tool since it only indexes Microsoft formatted files, nonetheless, it remains a powerful Intranet tool.

The utilisation of Index Server 1.1 is very similar to that used when executing database connectivity as seen previously.

As seen for ODBC querying, a series of intermediate files, in this case with .IDQ and .HTX extensions handle the querying and data return side of processing