Calculation-based (CB) models range from simple spreadsheet models developed in-house to sophisticated database or spreadsheet-based
offerings from software vendors or consultants. These CB models estimate the rough cost of goods based on a variety of input
parameters. They provide an inexpensive way to estimate the cost of manufacturing for a product while providing flexibility
in the definition of input parameters and the structure of the model.
The model's basic structure can follow any number of formats; a simple and powerful structure is shown in Figure 2. The key
to this and every other model structure is the process definition. To ensure proper estimates for each of the critical components
of cost (materials, labor, and capital), it is essential the process definition be as accurate and complete as possible. The
user interface can be designed to limit the number and range of variables tested or to allow the maximum flexibility and variation
in input parameters.
The greatest benefits of CB models are cost, time, and flexibility. Generally, such models can be built to the level of complexity
needed to answer specific questions within a short time frame. If the modeler uses an off-the-shelf spreadsheet or database
program, no additional software licenses are required to build the model, and it can be as flexible as desired.
All inputs and outputs can be configured to identify key parameters and sensitivities. The user can, with some simple analysis,
not only understand the estimated cost of goods associated with a specific process but also the cost implications of changes
to that process. These advantages make CB models a valuable part of the cost analysis tool-kit throughout the life of a project.
CB models are particularly useful in the early stages of a project when the process is not completely defined and assumptions
must be made about input variables such as gene expression level, purity and yield, binding capacity of purification columns,
and run time for each unit operation. In any modeling exercise, it is important to make sure that the power and accuracy of
the model is consistent with the data that drive it.
CB models can be oversimplified, which can lead to large errors in the outputs. Each assumption and input variable must be
carefully evaluated for relevance and accuracy. So called "rules of thumb" buried in the software may be a source of error.
When complexity overwhelms utility, the CB model must be abandoned in favor of other tools.
CB models are limited when predic-ting the probability of an outcome due to the variability of the many different inputs.
To take the CB model to the next level of sophistication, and to improve upon a simple cost analysis, Monte Carlo (MC) simulations
Monte Carlo Simulations
An MC simulation uses statistical simulation to model a multitude of outputs and scenarios. Many versions are based on spreadsheets
that randomly generate values for uncertain variables over and over to simulate a model. A statistical simulation, in contrast
to conventional numerical methods like the CB model, calculates multiple scenarios of a model by repeatedly sampling values
from the probability distributions for each of the input variables. It then uses those values to calculate the output for
each potential scenario. The result is a distribution of outputs showing the probability of each output over a range of input
variables - rather than the single output of a CB model. This effectively automates the "brute force" approach to running
sensitivities for CB models.
Figure 3: Monte Carlo simulations show that risky projects can have negative payoffs.
In an MC simulation, the process is simulated based on the process description and its input variables as described by probability
density functions (PDFs) rather than discrete values. (Developing a PDF is hard work and beyond the scope of this article.
To learn more, see Reference 8.) Once the PDFs are known, the MC simulation proceeds by random sampling from the PDFs. Many
simulations are performed and averaged based on the number of observations (a single observation or up to several million).
In many practical applications, one can predict the statistical error or variance of this result and estimate the number of
MC trials needed to achieve a given degreeof certainty.
A good application of MC simulation is predicting the range of net present values (NPVs) for a product portfolio based on
a variety of input variables, including the overall yield of the manufacturing process, different process options, and the
probability of successful commercialization of each product. Each product in the pipeline will have a different probability
of success based on its current stage of development and potential clinical indication, as well as fermentation volume requirements
based on the overall process definition, process yield (expression level and purification yield), and expected market demand.
The weaker alternate, a CB model, allows the estimation of the most probable COGS or the required fermentation volume for
each product based on discrete inputs. However, the CB model does not allow a complete analysis of each product's potential
demand based on a distribution of values for each input variable. And it cannot analyze the entire product pipeline simultaneously
or estimate the most probable distribution of portfolio NPVs.
We can combine a simple CB model with an MC simulation, and generate a probability distribution of NPV. Different PDF inputs
lead to markedly different outputs. In both examples in Figure 3, the dotted red line shows the mean NPV of a company's current
portfolio based on the discrete inputs (CB alone). While the mean NPV for the two cases is the same, the distribution of possible
outcomes is very different. In the first example - which shows the probability distribution of a high-risk product portfolio
- there is a significant probability that the NPV will be negative, suggesting that the company may want to re-evaluate the
portfolio, including manufacturing options. The second example shows the output for a more conservative product pipeline.
In this case, the majority of the predicted values for the NPV of the portfolio are positive.
A similar MC simulation using the same data can determine whether the company will have either too much or too little manufacturing
capacity for its product portfolio. In the case of the risky portfolio where the chance of product failure is high, the company
may find that it has too much capacity, resulting in valuable capital being tied up in "bricks and mortar," potentially making
product development difficult. On the other hand, building too little capacity could result in an inability to meet market
demand as in the recent Enbrel shortage.
In addition to the example in Figure 3, MC analyses can also be used to analyze all aspects of the COGS for a specific product,
including the impact of process variables on cost and the risk of not achieving certain process objectives. The timing of
bringing a product to market can impact overall operations and costs, and analyzing project schedules can help explain their
uncertainty. As with the other models, MC simulations are dependent on the probability distributions supplied for the variables.
In short, the better the input data, the more accurate the model. Nevertheless, by using a distribution of values rather than
discrete inputs and by reviewing the results of the simulation, one can begin to understand the possible implications of errors
in the model's assumptions.