How Many Prototypes Do We Need?

Introduction

This page is to provide guidance on how to calculate how many prototypes are needed to complete a program. This is based on the fact that many development project needs can be addressed using preliminary hardware/software, as long as the unit under test has a sufficient level of maturity to achieve its purpose.

For instance, sales and shipping may need a complete physical unit for fit and packaging checks, but the prototype does not have to work at all, as long as the weight and dimensions reflect the final configuration. Training and Manuals may need the user interface to generate their contribution, but a finished unit is not required to generate much of the content, as long as the prototype responds as the final unit will to user input. It is assumed that the final product will meet the specification, after all, that is what the specification is for.

Often I have seen this as a big lump of time and budget, with no breakdown. That is just lazy. And it is also how things get out of hand. Break it down, figure out how it interlocks. People may grumble up front, but everyone will be happier in the end.

While it may be possible to get your project planning software to generate something like this, I have always found it better to make a spreadsheet to be filled in. Example below.

Warning: Too many prototypes (or releases) can wreck a job. It is not a question of cost, but of focus. The logistics of maintaining numerous prototypes can overwhelm the effort. For an example of how logistics can wreck a job, review the false starts and eventual success of the Panama Canal.

Nobody wants to use prototypes that are not up to date or broken. Too many prototypes can even mask problems during integration and test. Why find and fix a problem when I can get a new unit and continue to "make progress"? I'll find out why the last two units blew up some other time.

Although the title of this article contains the word Prototypes, this method covers all other pre production asset allocations, such as:

• Pilots,

• Market Penetration Units,

• Breadboards, etc.

If you are unfamiliar with the differences between these types of units and their uses, read this before continuing: Prototypes and Breadboarding.

This works well with either Parallel or Serial developments.

Method:

Examine the table below.

When the contract is awarded, build and publish a spreadsheet to allow the users to enter the number and type of each assembly they will need. These people are identified in the "Demand From" column. Once delivered, these assets become theirs, to control, modify and maintain. Besides the prototypes themselves, they may need other support equipment to be built or borrowed. Ask them to enter the maturity they need to perform the testing or other operations they intend. For example, early units may not have to function at all for photographs or a show, while the system test units should be pretty stable, or a lot of people will be wasting time with them. If someone does not respond, place the budget numbers in there with data from the proposal or business plan.

Yes, you can build this into a project planning tool, like MS Project. But these tools are typically analysis tools, and are not good at capturing requirements and value. I recommend starting with something like this to define needs and allocations. Everyone can use and understand a basic spreadsheet, and access can be shared and controlled easily. Once this table is stable and agreed upon, you can enter it into your project planning tool for analysis, and you work with it from there. The spreadsheet then remains a historical record of the basis of the project requirements.

Then see if you have the money for all this stuff, which, of course, you don't. Trim down the numbers to match the budget allocation, and publish with the dates they will be available, from your project tool. The spares are yours to hold and allocate, as new needs arise. They will.

This method provides a concise way of determining who gets how many of what, what the test and maturity will be, and what the cost will be. Then you can sum it up and state clearly that you need 12 processor assemblies, and when you need them. Operations likes that sort of thing. And it will cost $XX. Finance likes that sort of thing. And their functionality/dates are defined. Engineering likes that sort of thing. I have used this in both a Meritocracy and a Colonial system, and it works fine in both. I have used it for parallel and serial developments also.

To indicate delivered units, you can change the background color, pattern, or other mechanism that suits you, such as your project planning tool. This will allow you to provide status for EVM or whatever.

The maturity of the hardware and software is critical. It doesn't do someone much good to get units that do not work well enough to complete their task, or will be changed later and invalidate tests that have been performed. Some people will ask for units they cannot possibly use for the intended purpose, given the maturity the units have achieved at that date. And early units are expensive and fairly low quality, because they are usually assembled and tested by hand either by the engineer or an operations tech.

So ASAP does not mean when the screws are seated, and it is roughly correct. It means when the unit is stable enough for the intended purpose, and will not cause the user to waste time chasing ghosts due to the equipment not really being complete.

The DATE column here is for need, you can add date columns for "need", "promise", "actual" if you are more anal.

You will need a table like this for each "spin" of the hardware, or major release of the software. So if you are planning three spins, then some of the boxes may be empty in some of the sheets, as people would have no need for early assets if updated units will be available later.

It is best to start this at the proposal stage. Estimate the cost of each of the required subassemblies and build that into a spreadsheet. That way, as the cost of these assemblies mature, you can easily update your report, and make adjustments to the budget.

By the way, at some point, more prototypes is not only more expensive, but actually slows things down. More units active means more units to keep up to date. No one wants to use a unit that is not current, and those that aren't, languish. On some of my programs, some units were always behind. Since they were never up to date enough to get relevant results, they were never used at all! And too many units will allow some stay broken without figuring out why, which is also dangerous. Programs where people grumbled about the lack of prototypes generally finished up long before those with plentiful hardware.

Maturity States

Examples of typical Maturity States of Hardware/Software in development. You may have fewer, or more, if you are implementing features on a serial basis (Network, No user interface, or Network, User Interface, no power control). Some examples below, yours may vary depending upon your prototype plan.

Software Maturity Example:

Note that the customer could elect to receive any intermediate software build. As long as you made it clear what they were and were not getting. What was a little humorous was that when we delivered early versions with function X missing of disabled, like we promised, the first feedback was always, "Hey, where is function X?".

Test Maturity Example :

Yours will certainly be different. You may have mechanical maturity, packaging maturity (eng enclosure, proto assembly, final for structural test) or other functional maturities.

It is important to include the cost of testing the early units. Since it will be done by engineers without test equipment, it could be quite expensive, both in labor and capital equipment required. And you will probably want those engineers doing other things, so don't forget to allocate them in resource management. One job I inherited had 6 turns of prototypes, about 15 of each. The prototype testing costs and manpower impacts were huge.