Game Engine Configuration Language Part 4: Visual Programming Languages

Game engine configuration languages should facilitate programming for non-programmers (e.g. game designers) to allow them to create game mechanics and autonomous entity behaviors. This problem has arisen through much of the history of computer programming, and for decades, people have tried to make computer programming accessible. Lately that trend has continued with some interesting new attempts. I want to review these languages with the hope that we can identify patterns -- what works and what does not -- and hope this guides our attempts to make a game configuration language which lets non-technical game developers the ability to control the interaction mechanic and other aspects of gameplay.


We can divide computer programming into at least two major components: data flow and control flow. We can categorize languages in terms of whether they principally focus on providing instructions (so-called imperative languages) or descriptions (so-called declarative languages). Imperative languages read like recipes -- a sequence of instructions (called a procedure) for how to accomplish a task (called a process). In contrast, declarative languages describe what results should look like, what properties they have, and the computer infers how to accomplish that. We should consider which of these is easier to understand and see whether we can construct a language accordingly.

Non-programmers struggle with the syntax of computer programs -- the grammatical rules that dictate whether a given sentence fits into the computer language. Computer language syntax is usually rather unforgiving, pedantic and arcane; even very small, seemingly insignificant changes in punctuation can drastically alter the meaning of a program. People have therefore made tools which strive to eliminate the possibility of writing programs with bad syntax. Such languages are often visual, in which authoring programs feels like constructing a physical machine. So in addition to constructing a game configuration language, we should consider how to construct a UI that assists authoring programs in that language -- since perhaps that will guide us on how to write the language.

In the study of formal language theory, the Chomsky hierarchy identifies connections between kinds of languages and the machines that can process them. Some machines are simpler than others, and perhaps we can exploit this fact to construct simpler languages. The resulting language should be easier to understand and use. The drawback is that the processes might be restricted but perhaps that is a Good Thing; perhaps non-programmers should have some restrictions on what they can do. Or perhaps the language can expose more sophisticated features only as-needed, such that non-programmers can use only the more restrictive, easier aspects if they choose -- then graduate to more complicated features when they become familiar with how they work.

After reviewing other language, I will propose adopting certain paradigms for game configuration languages: For AI, combine a finite-state machine (FSM) with steering behaviors. The language should facilitate both declarative and imperative paradigms. The authoring GUI should let user decide which to use, and provide a user-friendly visual dataflow interface with limited capabilities in addition to a text-based interface that exposes more sophisticated functionality for more advanced users.


Background


This section lists a sampling of computer programming languages intended for non-programmers or non-traditional programmers.

Alice: "Alice is an innovative 3D programming environment that makes it easy to create an animation for telling a story, playing an interactive game, or a video to share on the web. Alice is a freely available teaching tool designed to be a student's first exposure to object-oriented programming. It allows students to learn fundamental programming concepts in the context of creating animated movies and simple video games. In Alice, 3-D objects (e.g., people, animals, and vehicles) populate a virtual world and students create a program to animate the objects." Apparently Alice accomplishes its goal but its approach seems inappropriate for game configuration. I also find the GUI extraordinarily busy and distracting. Before you can learn to "program", you need to go through a rather long tutorial on how to use the Alice GUI. So even if Alice makes programming concepts more accessible, it does so at the cost of having to learn a complex GUI. Alice is fashionable in academia and worth knowing about but I do not find it inspirational for designing game configuration languages.

Kodu: "Kodu is a new visual programming language made specifically for creating games. It is designed to be accessible for children and enjoyable for anyone. The programming environment runs on the Xbox, allowing rapid design iteration using only a game controller for input." Although I have not read it described this way, it seems to use subsumption architecture -- the same scheme used by the "fast, cheap and out of control" robots. The GUI is elegant, pretty and engaging. The GUI only reveals elements that make sense in the context of what the user is doing at that precide moment, therefore is quite uncluttered. The user would therefore not get distracted by irrelevant aspects of the GUI. Kodu also effectively uses an FSM-plus-steering architecture, which I advocate. I would, however, change at least one aspect of the Kodu GUI: The fact that the user is effectively creating an FSM is never made visually explicity; each "state" (which Kodu calls a "page") can have a rule that would cause a transition to another "state" -- but there is no visual representation that shows the transitions between states. Also, the states have no meaningful name; they're simply numbered pages. So I would add the ability to label states with a meaningful name. More on that later. But in summary, I suspect that a person who had never seen Kodu before could walk write up to Kodu and, just by playing around, figure out how to write rather sophisticated programs. That's amazing.

LabView: "LabVIEW is a graphical programming environment used by millions of engineers and scientists to develop sophisticated measurement, test, and control systems using intuitive graphical icons and wires that resemble a flowchart. LabVIEW offers unrivaled integration with thousands of hardware devices and provides hundreds of built-in libraries for advanced analysis and data visualization. The LabVIEW platform is scalable across multiple targets and operating systems, and since its introduction in 1986 has become an industry leader." Programming LabView resemblies creating a "factory floor" where you have "stations" that process data and "conveyer belts" that shuttle data between stations. (These are my terms.) A lot of dataflow lanaguages use this same idea.

Pure Data: "Pd (aka Pure Data) is a real-time graphical programming environment for audio, video, and graphical processing." To me, this resembles LabView.

Unreal Kismet: Yet another dataflow language -- this one specifically written for game configuration. Kismet also includes "events". Kismet control flow follows dataflow. As values pass from left-to-right, so does control flow (which is typical for a dataflow language). In addition, Kismet actions have "variables" which do not "flow" or cause processes to run; they just provide access to data. Resembles Virtools and I think that aside from the complexity of its GUI, this is worth studying further. Tutorials I read do not mention facilities for abstraction i.e. the ability to cluster a dataflow graph into a user-defined graph with fewer inputs. Many aspects of the FIEA game engine configuration language resemble what must be the underlying virtual machine for Kismet.

Virtools: Yet another dataflow language specifically for configuring simulations. Programs consist of "building blocks" (BB's), whose visual representation is arcane. Like dataflow nodes in other languages, BB's have inputs, outputs and variables, but the GUI does not label them so they just look like distracting decorations. Virtools also has "messages" which I believe are like events. I suspect the features of Virtools and Kismet are probably quite similar, but I find the Kismet GUI looks more intuitive -- but perhaps only slightly. Neither is obvious.


Further Reading

Have a look at these resources for more examples along the lines of the above systems:
Boshernitsan & Downes (1997): Visual Programming Languages: A Survey
ARK: Alternate Reality Kit (1986) for simulations, based on Smalltalk.

VIPR: Visual Imperative PRogramming (1994): Object-oriented. Nesting rings dominate diagrams. Inscrutible diagrams for anything more than trivial examples.

Prograph (1982): Object-oriented, imperative, dataflow determines flow control. Commercially available for a while. Shortcomings include messy diagram organization (user controlled) and unlabeled classes and inputs.

Forms/3: spreadsheet.

Cube: 3D programming language.

Nickerson (1994) Missing lots of figures which, for a survey of visual languages, is quite an impediment. Furthermore I was unable to find examples on the web of several topics covered there; one would probably have to obtain print versions of his references, and they are all out of print. What diagrams it contains are unreadable in some cases. It covers a lot of diagraming conventions.

H-graphs: expansion on detail of a state which contains substates.

PICT: resembles typical flowcharts.

Blox: puzzle pieces represent program structure.

Show-and-Tell: iteration results in a list of each value.

Programming in Pictures: Not imperative.

Garden: meta-language meant for constructing visual languages. Can build FSA, dataflow and flowchart languages.

Summary

Dataflow languages provide a visual programming environment which alleviates the problems of understanding and following pedantic computer language syntax. But the "visual grammar" can also be difficult to leanr, and diagrams for complicated procedures can become unwieldy. Loops difficult to implement because each block has no state, so input data needs to contain state (e.g. loop index variable), which is awkward in some dataflow languages, but others seem to solve this problem by discriminating between "variables" (which provide access to data) and "inputs" (which trigger control flow).

Although the hybrid dataflow-causes-control-flow paradigm seems to work, I wonder whether using signals/events/listeners introduces a lot of overhead for processing small signals. When messages point to huge arrays, overhead is acceptable but for single values, overhead costs more than the regular processing.

Dataflow languages can also model blocks as having explicit control-flow sockets in addition to having data sockets. The control-flow sockets depend on data, messages or signals. So each block effectively becomes something like a multiplexor or switch-case statement.