Flexo is meant to facilitate the development, testing and optimization of programs using fluid concepts and high-level perception. Since these models are still evolving actively, its reusability and modifiability are points of great concern, and a component-based design is pursued to make future adaptations easy. Flexo should provide all mechanisms commonly used in fluid concept implementations, and provide means to offer the user maximum control over them (for now, the project is restricted to the mechanisms used in Copycat/Tabletop, however, Flexo's architecture should be flexible enough to add the extensions used in newer models (Letter Spirit, Metacat) without too much trouble). The user should be able to monitor, log and change nearly all of the system's parameters (also formulas), at runtime if possible, and it should be easy to log user-defined parameters and to gather additional statistics and tendencies.
In addition, it should provide components that enable the user to easily enter and alter the details of the domain under investigation:
The Coderack module should be able to load user-defined codelets, and all its parameters must also be easily entered and changed. The user (and codelets themselves) should be able to call, toggle or clamp each codelet.
A Parameter Manager should offer extensive parameter-managing facilities that every parameter should be able to subscribe to, whether they are just values or entire formula's. When designing a new component, the new parameters should be easy to add to this manager. The other way around, when a certain component needs such a value, it should be easily retrievable through this manager.
A Log Manager should enable users to log all system (or user-defined) parameters, and to create reports over several runs in order to study the system's behavior.
Self-watching modules should be easy to add and use.
The Slipnet - a dynamic semantic network
The Slipnet could be envisaged as our long-term memory. It is a network of concepts (nodes) connected by conceptual relations (links). However, these nodes carry an ever changing activation, like a electrical tension reflecting the esteem the program currently has toward the relevance of the concept. When the program deems the concept is relevant (when there is an instance of this concept in the given problem), it supplies its node with a jolt of activation. If the activation passes a certain threshold, the concept even has a probability to jump to full activation, as the program focuses on it. Over time, this activation decreases gradually, as concepts slowly fade away if the program doesn't show renewed interest in the particular concept. Also, some concepts are more directly perceivable than others, which is reflected by their depth-value, shallow concepts being perceived immediately, while for deep concepts this is much harder. It takes a while for these deep concepts to get a lot of activation, but they also lose their activation much more slowly than shallow ones do. When you discover (notice the high activation of) a deep concept, it seems important, containing the essence of the problem, and you want to focus on it for a while.
Correspondingly, links have an ever changing conceptual distance, reflecting the program's current esteem of the closeness of the two concepts connected by it. There are a limited number of link types, and each type has a concept describing it (a label), for example identity or opposite. These labels are treated as any other concept, but they have an extra influence on the conceptual distance of their link type. When a label becomes highly activated, their links shrink, as the current conceptual distance between the two concepts decreases, and vice versa. When opposite becomes activated, all opposite concepts in the Slipnet get 'closer' to each other, since you want diametrical slippages to occur easier while the program focuses on oppositeness. Now, imagine what happens if these links are to conduct activation, at a rate inversely proportional to their conceptual distance (you could see this distance as the electrical resistance of the link). We'll illustrate what would happen in Copycat's Slipnet, depicted in figure 3 (the link lengths in this figure have no meaning).
In the beginning, there only is an activation-free Slipnet. Then some letter-strings are given to the program. The letters in this string are regarded as instances of the letter concepts in the Slipnet. Because of this relation, the nodes of these letters become highly activated. These concepts are shallow, since they are easily perceivable, and their activation decays quite fast. However, they also can spread some activation through their links (for example, from 'a' to 'first'). The letter instances also get related to their position in the string: the first letter gets linked to 'leftmost', the middle letter to 'middle' and the last one to 'rightmost'. These relations with instances are not shown in the figure since they are not a part of the Slipnet (you could imagine them to be perpendicular to the figure's plane). Because of this, these three concepts also receive high activation (they're relevant), and they can spread it further to 'left', 'right' and 'Direction'. These last concepts are deeper, and typically receive little activation at a time (due to their distance), but they also decay slower. After a while, activation has spread toward all (at first sight) relevant concepts. Also, when some successive letters are given, the concept 'successor' will become highly activated (you'll soon know how), and successor-links will shrink considerably, facilitation flow through these links, making some successive letters seem relevant, and forcing the program to focus on successorship. The Slipnet thus shapes itself to the problem. Alas, without more architecture, the Slipnet will quickly lose all its activation by decay, as activation spreading in itself will quickly capitulate for the relentless decay process.
Like most semantic networks, Copycat's Slipnet has different 'classes' of links, which are sometimes used to focus on a certain relationship. Copycat distinguishes Category/Instance links (in fact two links in opposite directions), Property links, Slip links (between concepts that can slip into each other (replacing the other), they're indicated in thicker lines in the figure) and lateral links (for non-hierarchical semantic relationships). Through it all, they all behave like normal Slipnet links, spreading activation.
The Workspace corresponds to the blackboard in certain programs. It can be envisaged as a construction site on which an architecture contest is held. A large number of independent builder teams are given the same set of building blocks (the basic concepts of a problem, e.g. letters), and are challenged to use these to build the most elegant structures, starting at different times, but working completely in parallel. It's even so that when some structures are built by other teams at the time a new teams enters the site, it may use these structures in building a bigger one. There is however a catch, since in the end, all structures should together form a coherent whole, and should be well integrated in the surroundings (the problem's context). Thus, when two teams independently build two elegant, but clashing structures, eventually one of the two must give up and tear the structure down. Typically, the structure which is best integrated into the surroundings survives. Slowly, a consensus emerges, although sometimes an ingenious team can decide to go against the flow, constructing another elegant structure in parallel. When that structure is sufficiently convincing, it could result in surviving and tearing the bigger, established structure down. To give a feeling where this leads us, imagine the view of a city recorded over decades, but played back in a minute or so. At some point a building arises in a new style. If it is appreciated, soon new buildings in the new style appear, transforming the view of the city. The difference here is that, if at some point in time the city is built entirely in a single style, the contest ends, and the solution to the problem is found.
This translates to the notion of viewpoints. First, the Workspace is filled with lonesome concepts. As we mentioned earlier, these are instances of the concepts in the Slipnet, the latter being Platonic, unique, indestructible and not binding with each other. Early workers will link these instances up with their Platonic counterparts, by attaching descriptions to them. When more workers arrive, they first survey the scene. They have a tendency to start working on the most salient instances. The salience value of an instance is a total of the number of descriptions it has, the activation of the Slipnet nodes it is connected to via these descriptions, and its unhappiness. A concept is unhappy when it is hardly used in the existing structures, or is badly integrated. Thus, it cries out for attention. Eventually, the workers bind some instances into the first conceptual structures. What these structures look like, depends entirely on the skills of the workers, and is as such very problem-dependent. We'll discuss these workers and their abilities and behavior in the next section. Descriptions are an exception here, they are in fact a fundamental part of the architecture, as will be discussed right away. However, once a structure is built, it gets a strength. The strength is influenced, first of all by the structure's own properties (e.g. the depth of the concepts used), but also depends on other structures already built (e.g. how well it fits in), and is evaluated by other workers. The stronger a structure, the easier it can beat its rivals, if necessary. Some structures can in turn be used as parts of other structures, and can receive their own descriptions. As more and more structures arise and the Workspace becomes more complex, automatically a drive toward consistency and the use of deep concepts arises, as 'crazy' structures won't survive long. The Workspace now created an (imaginary) viewpoint, and will try to perfect this viewpoint until it is supreme (a solution is found). However, some alternative viewpoints are always present in the background, and sometimes they can gain enough power to topple the established viewpoint. This often yields very unexpected new insights.
Figure 4: Structures emerging in the Workspace.
Time for an example, don't you think? Figure 4 depicts the workspace of Metacat during an actual run. The building blocks here are, of course, letters. A variety of structures has been constructed:
Bonds are not drawn here, but they are the basis for groups. Two letters or groups in the same string can be bonded by a sameness, successor, or predecessor relation.
Groups bind two or more concepts with a common bond. Sameness groups (boxes) "a-a" and "k-k" arose, and also two successor groups appeared, "i-j", and "i-j-k". The latter clashes with "k-k" and will probably not survive, since 'identity' is a deeper concept than 'successor' in the Slipnet. Still, these two could coexist for quite a while. It is drawn with a dashed line because it's still being evaluated, as we'll discuss shortly.
Bridges (correspondences) bind corresponding letters or groups in different strings. They can stand for an identity mapping (sameness sameness, left left), or a conceptual slippage (letter group, left right). The strength of a bridge depends on the ease of the slippage, the number of identity mappings underlying it, and how it relates to other bridges. A deep slippage (left right) is less likely to occur, but also creates a strong bridge once conceived. There is for example one dashed bridge which wants to achieve a group letter slippage, but the "A-A"-bridge will probably be stronger thanks to the identity mapping of two groups, both deeper concepts. Likewise, a vertical "A-K"-bridge is being considered. This one is sworn to become a very strong bridge, because it incurs (leftmost rightmost), and 'opposite' is a very deep concept. However, it could take some time before this bridge is actually built, because the program is not yet focusing on the hard to reach 'opposite'-concept.
Descriptions are attached to instances (and structures) to describe what's known about them. They set up a connection between instances in the Workspace and concepts in the Slipnet. A description consists of a description type and a descriptor (e.g. "object category: letter" and "letter category: m"), and both must be Slipnet concepts. As we shall see, an instance is hereby bound to the fate of those concepts. If the concepts are highly activated, the instance or structure will be deemed important, and vice versa. Figure 5 shows the descriptions (only the descriptors are shown) attached to a mature letter-string with many structures built in. This way of representing descriptions actually comes from a Copycat implementation by Bolland [2], and although it is limited to problems where groups can only be formed between adjacent elements (like in Copycat), it's very instructive. In the column under each instance are its attached descriptions and the rows spanning multiple instances show reified structures existing between the spanned instances. With the two instances of the letter 'r' for example, a group has been built. Now, further descriptions can be attached to this group, such as group category: sameness group, length: 2 and string position: middle. As we have mentioned before, the most basic descriptions (here the first three rows), are attached in advance. The other descriptions will be attached by the program.
Rules are structures made to articulate the nature of bridges, such as [rightmost rightmost, letter group] for the "c-K" bridge. They are the actual 'output' of he program, defining the analogy
Figure 5: Descriptions in a letter string.
The Coderack - a probabilistic ant colony
If we would be facing a classical program, the Coderack would be the locus of its set of control algorithms. Rather, it doesn't really impose control, and it hardly is just a set of algorithms. Then, what is it? I already compared the behavior of the architecture with an ant column, we'll now see why. These 'ants' here are 'codelets', small pieces of code doing some small actions (related to building structures) in the workspace. They're so small and so large in number that it wouldn't really hurt if some codelet wasn't executed, or got terribly delayed. Also, they live in a hierarchical society. There are some basic kinds of codelets, and each kind normally has a specialist for each structure, although you could easily invent new breeds as well:
Scouts just wander around in the Workspace and examine instances or structures that are in their interest. If it thinks there's a nice structure to be built with them, the only real action they can do is to propose a certain structure and to call in other codelets to take it from there.
Evaluators come and look for the type of object they were called in for. They are specialized in estimating the promise of a certain structure. Will it be strong? Will it fit in with the other structures? When it deems the structure interesting enough, it calls in another codelet to do the actual work (or, if you like, it leaves a trail of pheromones for the big boys in the column to follow).
Builders then come and reify the structure they're specialized in.
Special codelets will do some other specific tasks. Breakers, for example, will come and tear some (weak) structures down when the Workspace doesn't really seem to go anywhere.
For example, when a description scout runs into an instance that could be linked up to a Slipnet concept, it calls a description evaluator to assess the quality of this possible description. When it appears to be fruitful, a description builder will eventually come and tie up the deal.
This scheme of exploring, evaluating, and finally building is called the parallel terraced scan. It originates (like many concepts discussed here) from Hofstadter's Jumbo program [9, ch.2] (see also section 4.3.2), an important predecessor of Copycat. Imagine falling in love. You don't just see another human being, marry him or her, and file for divorce if it doesn't turn out the way you wanted. You can easily rule out a lot of possibilities just checking them out superficially. Occasionally, a spark shoots out. Probably, this happens a lot, possibly with more than one person at the same time. Then, by getting to know these people better, a flash can occur, although mostly just with a very small number of people. This doesn't necessarily destroy the spark you had with other people, you may still like them, but not "the same way". As you get to know more and more about the most interesting people, perhaps flirting a little, you can get more and more attracted, and eventually you can fall head over heels in love. Then, the time is ripe for the happy couple to solemnize this bond (or another kind of structure). This natural scheme meets the real-time pressures any human is subject to [9, p.107]:
Figure 6: Copycat's Coderack, early during a run. [13]
This is the idea of the terraced scan: a parallel investigation of many possibilities to different levels of depth, quickly throwing out bad ones and homing in rapidly and accurately on good ones. In computational terms, you can use a lot of computationally cheap scouts to check out which structures are even remotely interesting. Then, hopefully having excluded an awful lot of possibilities, the more expensive evaluators can come to size up the situation, leaving just a few structures behind for actual building. Building inappropriate structures can be very expensive, resulting in a lot of wasted time.
Chopping up the construction of a structure in different phases also enables us to virtually do all this processing in parallel, by interleaving these small pieces of code. Two structures can thus be built 'in parallel', and they can seriously influence each other if they have common interests. Mostly the stronger structure gets built, making the other impossible, but that's not always so. Sometimes they can even coexist for a while. In a parallel environment, a process that is very important should be executed faster than its less interesting kin. We do this by giving each codelet an urgency value. Scouts are of little importance, so they are given a low urgency. However, this scout can stumble upon something very interesting and create some other codelets with a higher urgency. These in turn might create some other codelets, either with a high or a low urgency, according to the promise of the structure they're examining. But, doesn't that lead you back to a basically serial system, with a lot of high urgency codelets, forcing the less important scout to wait for a long time, thus destroying the systems parallelism? Indeed, that's why codelets are placed in the Coderack, and the Coderack decides probabilistically which codelet may run next. The high urgency codelets thus still have a large probability to be unleashed soon, but every now and then, an unimportant scout can leave as well. Also, codelets who are neglected for a long time gradually get an increased urgency. Now, say you have 79 codelets with urgency 1, and one codelet with urgency 21, then the high urgency codelet still only has a chance of 1/5 to be chosen. Doing all this, the Coderack can quickly adjust to new needs of the system. Evaluator codelets will assign high urgency values to codelets that will further the current viewpoint of a workspace. However, if a viewpoint gets unstable, on the verge of falling for a new viewpoint, the parallel processing will 'fairly' judge which one survives, and adapt itself quickly to the new ruling force. Figure 6 shows Copycat's Coderack during a run.
There are three ways in which codelets enter the Coderack. We've seen one already, being these follow-up codelets, created by other codelets in order to work further on their findings. Since scouts disappear when they have fulfilled their duty, the Coderack must replenish itself with new scouts, to keep the parallel exploration for new possibilities going. These scouts are bottom-up codelets, since they are independent of the given problem and control. In contrast, there are also top-down codelets, as shall be discussed in the following section.
How these pieces fit together
Like the success of any complex adaptive system (economies, ecologies, swarms,...) depends on the way its different components communicate, our architecture will only be useful when appropriate communication mechanisms are provided between these three main components. Another system in which organization and communication are of utmost importance, are ant colonies. We will use these as an illustration, showing why these mechanisms are sensible.
Imagine a mad scientist who has bred three ant colonies. Colony A has ants that don't leave pheromones, colony B has ants leaving and following pheromones, but not leaving more pheromones while they are following another ant's trail, and colony C has exceptionally shy ants not leaving the nest at all. His following field studies revealed a dreary outcome. Colony A had ants that wandered aimlessly around, occasionally bumping into a food supply, bringing home a tiny bit of it, but unable to retrace it, or to communicate it to others. Soon they starved. Colony B lasted a bit longer, since now ants followed each other to the same food supply, but only until the pheromone trail evaporated and the rest of the supply was not harvested further, unless accidentally found again. The colony perished due to a lack of efficiency. The last colony blossomed temporarily under an evil queen enforcing cannibalism, but it didn't last long. The moral of the story is that there must be some feedback loop between distinguishing good trails and continually testing their quality, between perceptual activity [Workspace] and conceptual activity [Slipnet], as depicted in figure 7.
Figure 7: A feedback loop between perceptual and conceptual activity.
When a discovery is done [food is found] in the Workspace, a description is attached on [pheromones are left toward] the instances or structures playing a role in it. This results in a substantial jolt of activation being sent to the corresponding Slipnet nodes. You can imagine the activation going through the description links (the dashed lines in the figure). This 'discovery' can be any kind of change, like linking up an undescribed instance or constructing a new structure of any kind. The activation then spreads to related nodes [somewhat like pheromone interplay], maybe also changing the conceptual distances. The Slipnet thus adapts itself to new perceptions [trails], putting these to the test in order to see if they can stand up against others. It does so by creating related top-down codelets. These are special codelets, trained to investigate a specific concept, and are given an urgency proportional to the sending node's activation. Suppose a structure in the Workspace was made that led to a high activation of the 'sameness-group' node, then the Slipnet could dump some 'sameness-exploring scouts' into the Coderack, to see if more sameness structures can be found. If so, the 'sameness-group' node will get even more activation, and will further try to steer the perception toward a viewpoint focusing on it. Top-down codelets may seem quite powerful, but they still are far to weak to lead directly to a dominance of its master node. Instead, by the commingling of a lot of top-down codelets, the interests of their masters get blended, like different small pressures resulting in a general large one. Still, any discovery, if interesting enough, can launch the perception down a certain direction, like pheromones attract ants toward a certain trail. This may lead to a new powerful viewpoint [a large food supply], or may 'evaporate' soon if it isn't interesting enough, drifting the perception toward another direction.
As you already may have noticed, colony A lacked the "Workspace Slipnet" link, and Slipnet activity dwindled rapidly to zero. Colony B missed the "Slipnet Coderack" link, not being able to keep the focus on interesting pathways. Finally, colony C was born without the very substantial link between Coderack and Workspace, resulting in no perceptual activity at all. Cutting any of these links thus obstructs the propagation of rational pressures needed to guide the perceptual activity, resulting in a very chaotic Workspace behavior.