01 Systems Analysis History

This web page is associated with a book called called Hermeneutics in Agile Systems Development.

The book can be bought at: https://play.google.com/store/books/details/Dr_Jerome_Heath_Hermeneutics_in_Agile_Systems_Deve?id=mi6VBQAAQBAJ

The book gives a more complete explanation of these issues and includes a number of related topic discussions. The combination develops the understanding of the concepts of Agile software development.

The History of Systems Analysis as a Basis for Context Calculus

Systems analysis was developed to find a solution to the problem of designing and developing software for computers. As computer programs got more complicated they also increased in problems. As we tried to use computers more to solve bigger problems, bugs in the programs would cause strange things to happen.

Systems theory had long been in existence and it provided a rough beginning to the new search for a method. But the systems approach in the computer field took special directions. The idea was to combine systems thinking with an emphasis on trying to limit the production of bugs in code by controlling the development process.

The initial approach involved defining program processes with flow charts. The basic flow chart for computer development was defined as:

The Ultimate Flow Chart

So, at the beginning, systems analysis was flow charting. What they did not anticipate was that flow charts do not amend themselves to levels of abstraction. Developing applications requires levels of abstraction. Also, flow charts did not define functionality. They define process. The computer field wanted a system that could divide the system into levels of abstraction and produce functionality rather than mere processing. It was the functional age in industry.

Functionality relates to the fact that factories are made more efficient by concentrating on functionality. So we train workers by looking at what functions they are required to do in order to accomplish the work needed. So computer programs when compared to the work in factories seemed to need improvement in functionality rather than improvements in processing.

A man named Yourdon developed the concept that data flow diagrams did provide these two needs. As data flow diagrams were developed they could be used to develop layers of abstraction. This made systems design easier than using flow charts. Because the definition of a data flow process was tied to the data process it was related to, the charting process was defined in a way that made levels of abstraction possible (if done carefully) and this also could be used to develop more functional modules in a program.

The overall scheme of the structured methodology looked like this (called Waterfall):

The Systems Development Life Cycle (SDLC) or Waterfall

Leading into this process were ideas that came from developing complex programs for operating systems. In this process a leader named Parnes emphasized the importance of what he called data hiding. The idea is that a given procedure hides how it does something and the only important issue is that the process does the right thing. Yourdon also emphasize and extended this same concept with the concepts of coupling and cohesion. The idea is that the coupling between processes is a minimum (this means very little data is passed between processes – ‘data hiding’). Cohesion adds the concept that a process needs to be cohesive (all the things that go together in the process belong together - functionally).

Yourdon’s methodology was complex and specific in how the process was to be done. These methods needed to be followed in detail. They were meant to provide management control over the software design process, which was a management goal. But as the process was used it was found that the process did not guarantee on-time, successful software. In fact the resultant software was often quite different from what the users desired or were expecting.

Another part of structured design methodology is the Entity Relationship Diagram. As structured design and Yourdon’s methodology were being developed, there was also development in databases. Data had to be stored separate from programs so programs could better use the data. But databases were difficult to use in the beginning. A man named Codd developed the relational algebra and demonstrated that if a database management system and the design of the database agreed with relational algebra then it could be accessed by simple English like statements in a language called Structured Query Language. At that point there were only a couple database management systems that were relational. Now most databases are relational.

But part of the requirement of relational algebra is that the design of the database within the database management system had to follow relational rules. This required designing the database itself using an Entity Relationship Diagram and using that tool correctly. This is how databases are designed today and perhaps always will. Except where design teams decide they did the class diagram so they don’t need to do the Entity Relationship Diagram.

Then there was a new idea from a group of developers, Grady, Booch, and Cummings. The three amigos developed the idea of object oriented software design. This went beyond data hiding to encapsulation. The data is only available through functions of the object (which then owns the data). The functions are called methods (or according to some - operations). To add, retrieve, or change data requires a method (actually a separate method for each).

An advantage of an object in object oriented programming is that program objects seem to act like things, so on using them you get acquainted with how they work and then you can anticipate how they will work as the situation changes. This was never true about functional software. Thus working with objects becomes intuitive in a way that is very similar to how we understand real things. Designing software had a new character.

This new character of object oriented software leads to easily re-using code. An object can be transposed into different applications with perhaps slight changes and can perform there as well as in the original setting. In general this did not happen with functional program modules.

Object oriented design uses a series of drawings you make to develop the software. Each of these captured a certain aspect of the total model. But a single methodology, like the methodology of Yourdon, has not developed in object oriented design. This is partly due to the fact that you can start with different drawings in the series and successfully develop software. Most like to begin with use cases, but others start with the class diagram and some might start with activity diagrams. Some use a methodology similar to Yourdon’s only following object oriented diagrams rather than using data flow diagrams and module hierarchies. In this case we can reference the System Design Life Cycle (SDLC) in Object Oriented Design. One accepted methodology is the Rational Unified Process (RUP) which is still much more flexible than Yourdon’s methodology and also can include iterative development. Another object oriented methodology is UML which is less formalized than RUP. Our object oriented examples, given below, are from UML.

A problem, that is not discussed adequately in the systems analysis being taught today, is the similarity between class diagrams and entity relationship diagrams. These two diagrams look very much alike and have similar characteristics, like relationships and the concept of one to many. Some say the two diagrams can be linked, in other words you draw one and then compare it as you draw the other. My feeling is that they are developing two different things. One is designing the relationship between objects in the program (inheritance and inter-object method calls or messages) and the other is setting up the database so that it can be accessed using Structure Query Language (relational algebra). So there may be some referencing to the other drawing with doing either but the overall goal of each is separate and must be kept separate.

On a recent move I made an address change with one of the banks I deal with. The new address was never right. As I tried to correct the address through many phone calls they could get the address right at first. But when things were sent to me or I called in to check on the address, it was strangely altered. It mixed old and new addresses so the result was a mish-mash. I am of the opinion that this bank designed their database from the class diagram. So the each data table belongs to a class that managed that data as it pleased.

One emphasis in Yourdon’s methodology is documentation. Each step of the process is well documented. Particularly, every discussion with users or managers is covered by a document that explains everything said and done. The documentation in structured system development is awesome. It is not just a report it is a volume or many volumes. Besides the fact that no-one could read all that and understand it, was the fact that the methodology did not work. In most cases software development was late, did not work correctly, and did not do what the users wanted.

As a result of these problems a group of programmers ‘sort of’ rebelled. They declared a group of principles they called the Agile Manifesto:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Here is a list of ten ways that agile development differs from structured methodology:

1. Active user involvement is imperative

2. The (programming) team must be empowered to make decisions

3. Requirements evolve but the timescale is fixed

4. Capture requirements at a high level; lightweight & visual

5. Develop small, incremental releases and iterate

6. Focus on frequent delivery of products

7. Complete each feature before moving on to the next

8. Apply the 80/20 rule (see below)

9. Testing is integrated throughout the project lifecycle – test early and often

10. A collaborative & cooperative approach between all stakeholders is essential

The 80/20 rule is based on an Alan Shalloway quote:

"The easiest way to build something simpler is to not build things you don't need. 64% of our software (by the industry) is barely used. The approach of building the 20% of the system that creates the greatest value (80%) and then re-assessing and building the next 20% has several advantages."

Perhaps the single biggest problem with agile software is that there is ultimately very little or no documentation. Yourdon’s methodology actually produced excessive documentation that was counter-productive. But after an agile project there is little evidence about how the system was developed. But in my own experience this problem is not as bad as it seems. The goal of agile is working software. Because we can often re-use software of this kind, the software itself becomes the documentation. The developer (who is the programmer also) can show users examples of recent software that is quite similar to their needs and get feedback on what changes would make the software acceptable. The user sees how this type of software works and the programmer sees how the user reacts and thus can develop software closer to what the user wants. This is a reasonable form of documentation.

Examples of agile methodology are: Extreme Programming, Scrum, Crystal Methods, Feature Driven Development, Rapid Prototyping, Kanban or Lean Software Development, and the Dynamic System Development Method.

An Example of Iterative Development

So Systems Analysis has become a very complex course. But all this needs to be discussed since most people doing systems analysis are privy to these issues even though their understanding in particular areas may be sketchy. You need to know the spectrum to be involved in system development.

Hermeneutics in Agile System Development

Dr. Jerome Heath