WinBatch Tech Support Home

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

WinBatch
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus
plus

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

Re-publisher's Note

This article, republished with permission of the author, details the place of WinBatch and other scripting languages in the software programming food chain.. WinBatch fits this definition of a scripting language.

Scripting: Higher Level Programming
for the 21st Century

John K. Ousterhout


Sun Microsystems Laboratories
2550 Garcia Avenue, MS UMTV-29-232
Mountain View, CA 94043-1100
john.ousterhout@eng.sun.com

Abstract
Scripting languages such as Visual Basic and Tcl represent a very different style of programming than system programming languages such as C or Java. Scripting languages are designed for "gluing" applications; they use typeless approaches to achieve a higher level of programming and more rapid application development than system programming languages. Increases in computer speed and changes in the application mix are making scripting languages more and more important for applications of the future. Whereas object-oriented approaches have largely failed to increase software reuse or reduce development costs, scripting languages have succeeded.

1 Introduction

As we near the end of the 20th century a fundamental change is occurring in the way people write computer programs. The change is a transition from system programming languages such as C or C++ to scripting languages such as Visual Basic or Tcl. Although many people are participating in the change, few people realize that it is occurring and even fewer people know why it is happening. I am writing this paper to call attention to the change and to explain why scripting languages will handle many of the programming tasks of the next century better than system programming languages.

Scripting languages are designed for different tasks than system programming languages, and this leads to fundamental differences in the languages. System programming languages were designed for building data structures and algorithms from scratch, starting from the most primitive computer elements such as words of memory. In contrast, scripting languages are designed for gluing: they assume the existence of a set of powerful components and are intended primarily for connecting components together. System programming languages are strongly typed to help manage complexity, while scripting languages are typeless to simplify connections between components and provide rapid application development.

Scripting languages and system programming languages are complementary, and most major computing platforms since the 1960's have provided both kinds of languages. However, several recent trends, such as faster machines, the increasing importance of graphical user interfaces and component architectures, and the growth of the Internet, have greatly increased the applicability of scripting languages. These trends will continue over the next decade, with scripting languages used for more and more applications and system programming languages used primarily for creating components.

2 System programming languages

In order to understand the differences between scripting languages and system programming languages, it is important to understand how system programming languages evolved. In the earliest days of computing, computers were programmed by entering instructions and data directly into the computer in binary form. This approach was quickly replaced by symbolic assembly languages. Assembly languages represent a program textually with one statement for each machine instruction. The fields of the instruction are specified symbolically: for example, the symbol ADD might be used to represent the opcode for an addition instruction and SP might be used to specify the register that serves as stack pointer. An assembler reads the statements, converts symbols to the corresponding binary values, and generates a binary program suitable for execution.

The use of textual symbols in assembly language makes programs substantially easier to write than binary approaches while still providing complete control over the machine. However, assembly language suffers from three defects. First, it requires the programmer to write one line of code for each instruction or variable, which becomes tedious as programs grow in size. Second, assembly language forces programmers to deal with many low-level details such as register allocation and procedure calling sequences. Third, assembly language provides almost no structure; for example, all instructions have roughly the same appearance and integer and floating-point variables are declared in the same way. As a result, it is difficult to write and maintain large programs in assembly language.

By the late 1950's higher level languages such as Fortran and Algol began to appear. In these languages statements no longer correspond exactly to machine instructions; a compiler translates each statement in the source program into a sequence of binary instructions. Over time a series of system programming languages evolved from Algol, including such languages as PL/1, Pascal, C, C++, and Java. System programming languages allow large applications to be developed much more quickly than assembly languages while providing nearly the same level of efficiency. As a result, they have almost completely replaced assembly language for the development of large applications.

System programming languages differ from assembly languages in two ways: they are higher level and they are strongly typed. The term "higher level" means that many details are handled automatically so that programmers can write less code to get the same job done. For example:

On average, each line of code in a system programming language causes about five machine instructions to be executed, compared to one instruction per line in assembly language. Various studies have shown that programmers can write roughly the same number of lines of code per year regardless of language, so system programming languages allow applications to be written much more quickly than assembly languages.

The second difference between assembly languages and system programming languages is typing. The term "typing" refers to the degree to which the meaning of information is specified in advance of its use. In a strongly typed language the programmer declares how each piece of information will be used and the language prevents the information from being used in any other way. In a weakly typed language there are no a priori restrictions on how information can be used: the meaning of information is determined solely by the way it is used, not by any initial promises.

Modern computers are fundamentally typeless at heart: any word in memory can hold any kind of value, such as a character, an integer, a pointer, or an instruction. The meaning of a value is determined by how it is used: if the program counter points at a word of memory then it is treated as an instruction; if a word is referenced by an integer add instruction then it is treated as an integer; and so on. The same word can be used in different ways at different times.

Today's system programming languages are strongly typed. For example:

Typing makes large programs more manageable by clarifying how things are used and differentiating between things that must be treated differently. Compilers can use type information to detect certain kinds of errors, such as an attempt to use a floating-point value as a pointer.

To summarize, system programming languages are designed to handle the same tasks as assembly languages, namely creating applications starting from scratch. System programming languages are somewhat higher level and much more strongly typed than assembly languages. This allows applications to be created more rapidly and managed more easily with only a slight loss in performance.

3 Scripting languages

Scripting languages such as Perl, Tcl, Visual Basic, and the Unix shells represent a very different style of programming than system programming languages. Scripting languages assume that there already exists a collection of useful components written in other languages. Scripting languages aren't intended for writing applications from scratch; they are intended primarily for plugging together components. For example, Tcl and Visual Basic are used to arrange collections of user interface controls on the screen while Unix shell scripts are used to assemble filter programs into pipelines. Scripting languages are often used to extend the features of components but they are rarely used for complex algorithms and data structures; features like these are usually provided by the components. Scripting languages are sometimes referred to as glue languages or system integration languages.

In order to simplify the task of connecting components, scripting languages tend to be typeless: all things look and behave the same so that they are interchangeable. For example, in Tcl or Visual Basic a variable can hold a string one minute and an integer the next. Code and data are often interchangeable, so that a program can write another program and execute it on the fly. Scripting languages are often string-oriented, since this provides a flexible representation for many different things.

A typeless language makes it much easier to hook together components. There are no a priori restrictions on how things can be used, and components and values are represented in a uniform fashion. Thus any component or value can be used in any situation; components designed for one purpose can be used for totally different purposes never foreseen by the designer. For example, in the Unix shells, all filter programs read a stream of bytes from an input and write a string of bytes to an output so any two programs can be connected together simply by attaching the output of one program to the input of the other.

The strongly typed nature of system programming languages discourages reuse. Programmers are encouraged to design a variety of incompatible interfaces. Each interface requires objects of specific types and the compiler prevents any other types of objects from being used with the interface, even if that would be useful. In order to use a new object with an existing interface, conversion code must be written to translate between the type of the object and the type expected by the interface. This in turn requires recompiling part or all of the application, which isn't possible in the common case where the application is distributed in binary form.

To see the advantages of a typeless language, consider the following Tcl command:

button .b -text Hello! -font {Times 16} -command {puts hello}
This command creates a new button control that displays a text string in a 16-point Times font and prints a short message when the user clicks on the control. It mixes six different types of things in a single statement: a command name (button), a button control (.b), property names (-text, -foreground, and -command), simple strings (Hello! and hello), a font name (Times 16) that includes a typeface name (Times) and a size in points (16), and a Tcl script (puts hello). Tcl represents all of these things uniformly with strings. In the button example the properties may be specified in any order and unspecified properties are given default values; more than 20 properties were left unspecified in the example.

The button example requires about 25 lines of code in three procedures when implemented in C++ with Microsoft Foundation Classes. Just setting the font requires 7 lines of code:

LOGFONT lf;
memset(&lf, 0, sizeof(lf));
lf.lfHeight = -16;
strcpy(lf.lfFaceName, "Times New Roman");
CFont *fontPtr = new CFont();
fontPtr->CreateFontIndirect(&lf);
buttonPtr->SetFont(fontPtr);
Almost all of this code is a consequence of the strong typing: variables lf and fontPtr must be declared, fields in the font descriptor structure lf must be filled in individually, then a new object of type CFont must be created from lf in order to match the API of the button's SetFont method. In Tcl, the essential characteristics of the font (typeface Times, size 16 points) can be used immediately with no declarations, intermediate structures, or conversions. Similarly, in C++ the behavior for the button must placed in a separately declared method, whereas in Tcl it can be included directly in the command that creates the button.

It might seem that the typeless nature of scripting languages could allow errors to go undetected, but in fact scripting languages are just as safe as system programming languages. For example, an error will occur if the font size specified for the button example above is a non-integer string such as xyz. The difference is that scripting languages do their error checking at the last possible moment, when a value is used. Strong typing allows errors to be detected at compile-time, so the cost of run-time checks is avoided. However, the price to be paid for this efficiency is more conservative restrictions on how information can be used: this results in more complex and less flexible programs.

Another key difference between scripting languages and system programming languages is that scripting languages are usually interpreted whereas system programming languages are usually compiled. Interpreted languages provide rapid turnaround during development by eliminating lengthy compile times. Interpreters also allow powerful effects to be achieved by generating code on the fly. For example, a Tcl-based Web browser can parse a Web page by translating the HTML for the page into a Tcl script using a few regular expression substitutions. It then executes the Tcl script to render the page on the screen. Or, an application can be extended by downloading code into it.

Scripting languages tend to be less efficient than system programming languages, in part because they use interpreters instead of compilers but also because their basic components are chosen for power and ease of use rather than an efficient mapping onto the underlying hardware. For example, scripting languages tend to use variable-length strings in situations where a system programming language would use a binary value that fits in a single machine word, and scripting languages often use hash tables where system programming languages use indexed arrays.

Fortunately, the performance of a scripting language isn't usually a major issue. Applications for scripting languages are generally smaller than applications for system programming languages, and the performance of a scripting application tends to be dominated by the performance of the components, which are typically implemented in a system programming language.

Scripting languages are higher level than system programming languages, in the sense that a single statement does more work on average. A typical statement in a scripting language executes hundreds or thousands of machine instructions, whereas a typical statement in a system programming language executes about five machine instructions. Part of this difference is because scripting languages use interpreters, which are less efficient than the compiled code for system programming languages. But much of the difference is because the primitive operations in scripting languages have greater functionality. For example, in Perl it is about as easy to perform a regular expression substitution as it is to perform an integer addition. In Tcl, a variable can have traces associated with it so that setting the variable causes side effects; for example, a trace might be used to notify a control whenever the value of a variable changes, so that the variable's value can be displayed continuously on the screen.

Because of the features described above, scripting languages allow very rapid development for applications that are gluing in nature.

The scripting versions typically required 5-10x less code or development time than the system programming versions. Applications that involved more complex algorithms or data structures, such as the last example in the table, benefit less from a scripting language than applications that are mostly gluing.

To summarize, scripting languages are designed for gluing applications. They provide a higher level of programming than assembly or system programming languages, much weaker typing than system programming languages, and an interpreted development environment. Scripting languages trade off execution efficiency for speed of development.

4 Different tools for different tasks

A scripting language is not a replacement for a system programming language or vice versa. Each is suited to a different set of tasks. For gluing and system integration, applications can be developed 5-10x faster with a scripting language; system programming languages will require large amounts of boilerplate and conversion code to connect the pieces, whereas this can be done directly with a scripting language. For complex algorithms and data structures, the strong typing of a system programming language makes programs easier to manage. Where execution speed is key, a system programming language can often run 10-20x faster than a scripting language: the flexibility provided by the scripting language results in more run-time checks.

In deciding whether to use a scripting language or a system programming language for a particular task, consider the following questions:

"Yes" answers to these questions suggest that a scripting language will work well for the application. On the other hand, "yes" answers to the following questions suggest that an application is better suited to a system programming language:

Most of the major computing platforms over the last 30 years have provided both system programming and scripting languages. For example, one of the first scripting languages was JCL (Job Control Language), which was used to sequence job steps in OS/360. The individual job steps were written in PL/1, Fortran, or assembler language, which were the system programming languages of the day. In the Unix machines of the 1980's, C was used for system programming and shell programs such as sh and csh for scripting. In the PC world of the 1990's, C and C++ are used for system programming and Visual Basic for scripting. In the Internet world that is taking shape now, Java is used for system programming and languages like JavaScript, Perl, and Tcl are used for scripting.

Scripting and system programming are symbiotic. Used together, they produce programming environments of exceptional power: system programming languages are used to create exciting components which can then be assembled using scripting languages. For example, much of the attraction of Visual Basic is that system programmers can write VBX components in C and less sophisticated programmers can then use the components in Visual Basic applications. In Unix it is easy to write shell scripts that invoke applications written in C. One of the reasons for the popularity of Tcl is the ability to extend the language by writing C code that implements new commands.

5 Scripting is on the rise

Scripting languages have existed for a long time, but their importance has increased in recent years because the application mix has shifted more towards gluing applications. Three examples of this shift are graphical user interfaces, the Internet, and component frameworks.

User interfaces are fundamentally gluing applications: the goal is not to create new functionality, but to make connections between a collection of graphical controls and the internal functions of the application. Scripting languages have always been the primary vehicle for creating user interfaces, whether the user interface was a card deck as in OS/360, a command line on a terminal as in Unix, or a graphical user interface as with Tcl or Visual Basic. With the arrival of graphical user interfaces (GUIs) in the 1980's the richness of user interfaces increased dramatically, as did the development effort required to create them. The GUI accounts for half or more of the total effort in many programming projects.

I am not aware of any rapid-development environments for GUIs based on a system programming language. Whether the environment is Windows, Macintosh Toolbox, or Unix Motif, GUI toolkits based on languages like C for C++ have proven to be hard to learn, clumsy to use, and inflexible in the results they produce. Some of these tools have very nice graphical tools for designing screen layouts, but things become difficult as soon as the designer begins writing code to provide the behaviors for the interface elements. All of the best rapid-development GUI environments are based on scripting languages: Visual Basic on the PC, HyperCard on the Macintosh, and Tcl on Unix. Thus scripting languages have risen in popularity as the importance of GUIs has increased.

The growth of the Internet has also popularized scripting languages. The Internet is a gluing environment. It doesn't create any new computations or data; it simply makes a huge number of existing things easily accessible. The ideal language for most Internet programming tasks will be one that makes it possible for all the connected components to work together, i.e. a scripting language. For example, Perl has become popular for writing CGI scripts and JavaScript is popular for scripting in Web pages.

The third example of scripting-oriented applications is component frameworks. Although system programming languages work well for creating components, the task of assembling components into applications is better suited to scripting. Without a good scripting language to manipulate the components, much of the power of a component framework is lost. This may explain in part why component frameworks have been more successful on PCs (where Visual Basic provides a convenient scripting tool) than on other platforms such as Unix/CORBA where scripting is not included in the component framework.

Another reason for the increasing popularity of scripting languages is the ever-increasing speed of computer hardware. It used to be that the only way to get acceptable performance in an application of any complexity was to use a system programming language. In some cases even system programming languages weren't efficient enough, so the applications had to be written in assembler. However, machines today are 100-500 times faster than the machines of 1980. Many applications can be implemented in an interpreted language and still have excellent performance (for example, a Tcl script can manipulate collections with hundreds or thousands of objects and still provide good interactive response). As computers get faster, fewer and fewer applications will need to be implemented in a system programming language to get adequate performance.

One final reason for the increasing use of scripting languages is a change in the programmer community. Twenty years ago most programmers were sophisticated programmers working on substantial projects. Programmers in that day expected to spend several months to master a language and its programming environment, and system programming languages were designed for such programmers. However, since the arrival of the personal computer, more and more casual programmers have joined the programmer community. For these people, programming is not their main job function; it is a tool that they use occasionally to help with their main job. Examples of casual programming are writing macros for a spreadsheet or writing a simple database query. Casual programmers are not willing to spend months learning a system programming language, but they can often learn enough about a scripting language in a few hours to write useful programs. Scripting languages are easier to learn because they have simpler syntax than system programming languages and because they omit complex features like objects and threads. For example, compare Visual Basic with Visual C++; few casual programmers would attempt to use Visual C++, but many have been able to build useful applications with Visual Basic.

Even today the number of applications written in scripting languages is much greater than the number of applications written in system programming languages. On Unix systems there are many more shell scripts than C programs, and under Windows there are many more Visual Basic programmers and applications than C or C++. Of course, most of the largest and most widely used applications are written in system programming languages, so a comparison based on total lines of code or number of installed copies may still favor system programming languages. Nonetheless, scripting languages are already a major force in application development and their market share will increase in the future.

6 The failure of object-oriented programming

Scripting languages have received little attention in the programming language community or in the press; instead, most of the attention has been focused on object-oriented programming languages such as C++ and Java. Object-oriented programming is widely believed to represent the next major step in the evolution of programming languages. Object-oriented features such as strong typing and inheritance are often claimed to reduce development time, increase software reuse, and solve many other problems including those addressed by scripting languages.

Unfortunately, there is not much evidence that object-oriented programming produces the benefits claimed for it. Some projects have claimed success with object-oriented languages, but others have reported that languages like C++ actually made software management problems worse. I do not know of any studies showing substantial reductions in development costs due to object-oriented programming, and some language experts are now beginning to speak out against object-oriented programming.

The reason why object-oriented programming has not lived up to expectations is that it doesn't raise the level of programming or encourage reuse. Programmers still work with small basic units that must be described and manipulated in great detail. In principle, powerful library packages could be developed, and if these libraries were used extensively it could effectively raise the level of programming. However, not many such libraries have come into existence. The strong typing of object-oriented languages encourages narrowly defined packages that are hard to reuse. Each package requires objects of a specific type; if two packages are to work together, conversion code must be written to translate between the types required by the packages.

Another problem with object-oriented languages is their emphasis on inheritance. Implementation inheritance, where one class inherits part of its implementation from another, is simply a bad idea. It binds the implementations of classes together so that neither can be understood without the other: the subclass cannot be understood without knowing how the inherited methods are implemented in the superclass, and the superclass cannot be understood without knowing how its methods are inherited in subclasses. In a complex class hierarchy, no individual class can be understood without understanding all the other classes in the hierarchy. Even worse, a class cannot be separated from its hierarchy for reuse. Multiple inheritance only makes these problems worse. Implementation inheritance causes the same intertwining and brittleness that have been observed when goto statements are overused. As a result, object-oriented systems suffer from complexity and lack of reuse.

Scripting languages, on the other hand, have actually generated significant software reuse. They use a model where interesting components are built in a system programming language and then glued together into applications using the scripting language. This division of labor provides a natural framework for reusability. Components are designed to be reusable, and there are well-defined interfaces between components and scripts that make it easy to use components. For example, in Tcl the components are custom commands implemented in C; they look just like the builtin commands so they are easy to invoke in Tcl scripts. In Visual Basic the components are VBX or ActiveX extensions, which can be used by dragging them from a palette onto a form.

7 What will happen to system programming languages?

It might seem from the above arguments that scripting languages will eventually replace system programming languages entirely. After all, a similar effect occurred with assembly language. In the early 1960's, most serious programming was done in assembly; higher level languages like Fortran and Algol were widely viewed as too slow for serious programming tasks. But the quality of compiled code improved and machines got faster, so the performance differences were less important. At the same time, it became clear that programmers were substantially more productive with high level languages than with assembly. Over time, more and more people decided that it was worth giving up a little speed to get faster application development. Today, assembly language is rarely used, even for the most performance-critical code. The same arguments might be made for scripting versus system programming: scripting does not use the computer as efficiently as system programming, but it enables much faster development.

However, I doubt that system programming languages will disappear to the same extent that assembly languages have. Scripting languages are not designed to serve all purposes. They work particularly well for gluing together components, but they are not as suitable for creating the components. The strong typing of system programming languages makes it easier to build complex data structures and algorithms; for example, trees and other pointer-rich structures are easier to implement in a system programming language than a scripting language. A scripting language is great for writing a database query, but if you're writing a database engine you're better off with a system programming language.

Thus system programming languages will continue to play an important role in computers, but they will be used primarily for creating components and fixed-function applications such as database servers. Many of the more glue-like areas where system programming languages are used today, such as GUIs and enterprise applications, will come to be dominated by scripting languages.

8 Conclusion

Scripting languages represent a different set of tradeoffs from system programming languages. Scripting languages give up execution speed and strength of typing relative to system programming languages but provide significantly higher programmer productivity and software reuse. This tradeoff makes more and more sense as computers become faster and cheaper in comparison to programmers. System programming languages are well suited to building complex data structures and algorithms from scratch, while scripting languages are well suited for gluing applications such as graphical user interfaces and component assembly. Gluing tasks are becoming more and more prevalent, so scripting will become an even more important programming paradigm in the next century than it is today.

9 References

[1] K.Arnold and J. Gosling, The Java Programming Language, Addison-Wesley, ISBN 0-201-63455-4, 1996.

[2] F. Brooks, The Mythical Man-Month, Addison-Wesley, ISBN 0-201-00650-2, 1975.

[3] S. Johnson, Objecting To Objects, Invited Talk, USENIX Technical Conference, San Francisco, CA, January 1994.

[4] B. Kernighan and D. Ritchie, The C Programming Language, Second Edition, Prentice Hall, ISBN 0-13-110362-8, 1988.

[5] Netscape Inc., "JavaScript in Navigator 3.0", http://home.netscape.com/eng/mozilla/3.0/handbook/javascript/atlas.html#taint_dg.

[6] J. Ousterhout, Tcl and the Tk Toolkit, Addison-Wesley, ISBN 0-201-63337-X, 1994.

[7] B. Stroustrup, The C++ Programming Language, Addison-Wesley, ISBN 0-201-12078-X, 1987.

[8] L. Wall, T. Christiansen, and R. Schwartz, Programming Perl, Second Edition, O'Reilly and Associates, ISBN 1-56592-149-6, 1996.



john.ousterhout@eng.sun.com
Tcl Home Page

Last updated: 03/28/97 09:40:23


Article ID:   W12708
Filename:   Scripting White Paper.txt
File Created: 2017:08:29:11:37:14
Last Updated: 2017:08:29:11:37:14