A View of 20th and 21st Century Software Engineering

Abstract:

George Santayana's statement, "Those who cannot remember the past are condemned to repeat it," is only half true. The past also includes successful histories. If you haven't been made aware of them, you're often condemned not to repeat their successes. In a rapidly expanding field such as software engineering, this happens a lot. Extensive studies of many software projects such as the Standish Reports offer convincing evidence that many projects fail to repeat past successes.

This paper tries to identify at least some of the major past software experiences that were well worth repeating, and some that were not. It also tries to identify underlying phenomena influencing the evolution of software engineering practices that have at least helped the author appreciate how our field has gotten to where it has been and where it is. A counterpart Santayana-like statement about the past and future might say, "In an era of rapid change, those who repeat the past are condemned to a bleak future." (Think about the dinosaurs, and think carefully about software engineering maturity models that emphasize repeatability.)

This paper also tries to identify some of the major sources of change that will affect software engineering practices in the next couple of decades, and identifies some strategies for assessing and adapting to these sources of change. It also makes some first steps towards distinguishing relatively timeless software engineering principles that are risky not to repeat, and conditions of change under which aging practices will become increasingly risky to repeat.

SAGE Software Development Process:

Software Engineering Trends Through the 1970’s:

Royce Waterfall Model:

A Full Range of Software Engineering Trends

Conclusions:

• From the 1950’s
  • Don’t neglect the sciences. This is the first part of the definition of “engineering”. It should not include just mathematics and computer science, but also behavioral sciences, economics, and management science. It should also include using the scientific method to learn through experience.
  • Look before you leap. Premature commitments can be disastrous (Marry in haste; repent at leisure – when any leisure is available).
  • Avoid using a rigorous sequential process. The world is getting too tangeable and unpredictable for this, and it’s usually slower.
• From the 1960’s
  • Think outside the box. Repetitive engineering would never have created the Arpanet or Engelbart’s mouse-and-windows GUI. Have some fun prototyping; it’s generally low-risk and frequently high reward.
  • Respect software’s differences. You can’t speed up its development indefinitely. Since it’s invisible, you need to find good ways to make it visible and meaningful to different stakeholders.
  • Avoid cowboy programming. The last-minute all-nighter frequently doesn’t work, and the patches get ugly fast.
• From the 1970’s
  • Eliminate errors early. Even better, prevent them in the future via root cause analysis.
  • Determine the system’s purpose. Without a clear shared vision, you’re likely to get chaos and disappointment. Goal-questionmetric is another version of this.
  • Avoid Top-down development and reductionism. COTS, reuse, IKIWISI, rapid changes and emergent requirements make this increasingly unrealistic for most applications.
• From the 1980’s
  • These are many roads to increased productivity, including staffing, training, tools, reuse, process improvement, prototyping, and others.
  • What’s good for products is good for process, including architecture, reusability, composability, and adaptability.
  • Be skeptical about silver bullets, and one-size-fits-all solutions.
• From the 1990’s
  • Time is money. People generally invest in software to get a positive return. The sooner the software is fielded, the sooner the returns come – if it has satisfactory quality.
  • Make software useful to people. This is the other part of the definition of “engineering.”
  • Be quick, but don’t hurry. Overambitious early milestones usually result in incomplete and incompatible specifications and lots of rework.
• From the 2000s
  • If change is rapid, adaptability trumps repeatability.
  • Consider and satisfice all of the stakeholders’ value propositions. If success-critical stakeholders are neglected or exploited, they will generally counterattack or refuse to participate, making everyone a loser.
  • Avoid falling in love with your slogans. YAGNI (you aren’t going to need it) is not always true.
• For the 2010’s
  • Keep your reach within your grasp. Some systems of systems may just be too big and complex.
  • Have an exit strategy. Manage expectations, so that if things go wrong, there’s an acceptable fallback.
  • Don’t believe everything you read. Take a look at the downslope of the Gartner rollercoaster in Figure 10.