Assigning Multi-Attribute Targets under Uncertainty with Multiple Suppliers and Stakeholders

Abstract

A manager must create deliverables satisfying multiple stakeholders. Different stakeholders use the deliverables to satisfy multiple different (and potentially conflicting) objectives. For each criterion, the manager assigns different targets to different suppliers. If suppliers meet their targets, the deliverable score is the sum of the targets. The manager will only be successful if all suppliers meet their targets and if the deliverable scores are acceptable to stakeholders. This chapter solves for the target maximizing the manager’s probability of success both in simple cases and in the general case of multiple suppliers, multiple attributes, and multiple stakeholders. The targets introduce a supplier margin of safety (or buffer) to allow for uncertainty in that supplier’s capability. The targets also introduce a stakeholder margin of safety to allow for uncertainty in what is acceptable to each stakeholder. The assigned targets reflect trade-offs between the margins of safety for each supplier and stakeholder. For simplicity, all uncertainties are assumed Gaussian, and deliverable scores are additive in the targets. Generalizing both assumptions is straightforward.

1. Introduction

A project is a temporary endeavor aimed at creating some deliverable subject to cost and deadline constraints. Stakeholders use these deliverables to achieve their own objectives under various scenarios (e.g., a vehicle enables its driver to commute to work and travel medium distances during normal or increment weather.)

The stakeholders and project manager agree on the multi-attribute targets (on multiple attributes) against which deliverable performance will be compared. Complex projects must be decomposed into work packages by dividing each project target into specialized targets for each work package. For example, designing a vehicle to meet some weight target is decomposed into designing tires, seats, engines, exterior panels, etc. to meet weight targets for tires, seats, engines, etc. In this case, the sum of the work package weight targets equals the overall target. Sometimes several tiers of decomposition are required before each work package is sufficiently granular to be assigned to a small specialized supplier.

There are many technical, raw material, personnel, and other issues, which could prevent the supplier from meeting the target. These issues create uncertainties about whether suppliers can meet their targets. To reduce the risk of supplier failure, the manager tracks the progress of each supplier and reallocates resources when necessary. Despite these efforts, however, there will still be some risk of suppliers not meeting their targets.

The manager is also uncertain about whether the agreed-upon overall target accurately reflects what stakeholders need to achieve their objectives. Use cases, sequence diagrams, activity diagrams, and other tools of model-based systems engineering [1] help stakeholders identify all the specific functions the product must perform to help them achieve their objectives. But when projects are lengthy, stakeholder preferences can change because of changes in family size, competitor offerings, environmental threats and opportunities, etc. This can be especially difficult for consumers. To better estimate consumer needs, conjoint measurements combine stakeholder feedback in hypothetical settings with parametric models of stakeholder behavior. There are many other approaches to requirements elicitation [2, 3, 4, 5]. But even with these models, there is still uncertainty about what stakeholders will need.

In some cases, stakeholders will request changes in the targets for evaluating the deliverable before the deliverable is finished. This “requirement volatility” is a leading cause of project failure, especially with software projects [6, 7, 8, 9, 10]. But if needs change after the project is finished (and the suppliers are compensated), the stakeholder may be dissatisfied and seek an early replacement for the deliverable.

This chapter sets supplier targets given uncertainty about what suppliers can produce and what stakeholders need. When all uncertainties are Gaussian, the solutions will depend on a “safety margin” for suppliers and a “safety margin” for stakeholders.

2. One design team and one stakeholder

2.1 Mass reduction

To reduce the costs of fueling vehicles, customers prefer more fuel-efficient, lower-weight vehicles. While the project team can reduce the weight of a vehicle, there is a limit to how much they can reduce the weight without reducing performance. Suppose this lower limit is a known value, $m$. Suppose that the greatest weight customers will accept is known to equal $M$. Then if $m<M$, the design team can meet customer requirements.

But customer demand for lower-weight vehicles is heavily affected by fuel costs. Fuel costs can change dramatically as the supply of fuel changes. Over the past century, discoveries of new oil fields and the invention of fracking have expanded the availability of fuel. In addition, the development of fuel-efficient engines (including hybrid and electric vehicles) also reduces the number of fuel required to reach a destination. These factors might cause customers to accept vehicle weights higher than $M$. But oil production often varies, sometimes in response to dwindling oil reserves and sometimes to optimize revenues. Taxes on fuel also increase the cost of using fuel to travel to a destination. These factors could make customers more likely to insist on weights less than $M$. Both of these factors create uncertainty about how much the maximum weight customers will accept differs from $M$. Suppose this uncertainty is described by a normal distribution with mean $M$ and standard deviation $S$. To elicit $S$, it is common [9] to ask consumers to specify

• A pessimistic but realistic (“rainy day”) estimate of maximum acceptable weight when fuel costs are high. (The chances of customers requiring an even lower weight are negligible.)

• An optimistic but realistic (“sunny day”) estimate of maximum acceptable weight when fuel costs are low. (The chances of customers being will to accept an even higher weight are negligible.)

This definition presumes a quantitative definition of “negligible chance.” Hypothesis testing in the social sciences typically defines a confidence/credibility interval to be approximately four standard deviations in length¹. As a result, the standard deviation, $S$, is one-quarter of the absolute value of the difference (the range) between pessimistic and optimistic. The expected value, $M$, is the average of the bounds.

Let $\mathrm{\Phi }\left(z\right)$ be the probability a normally distributed value with zero mean and unit standard deviation is less than $z$. Since the customer is more likely to accept lower-mass targets $t$, define:

Stakeholder margin: The difference, $M-t$, between the best estimate of the highest acceptable mass and the target.

The probability of a target $t$, if achieved, being acceptable to the customer is $\mathrm{\Phi }\left(\frac{M-t}{S}\right)$.

Suppose the manager recognizes the variation in customer needs but ignores the variation in what their team can achieve. Then since, they believe their team can design a vehicle with mass $m$, they set $t=m$ and erroneously estimate the probability of success as the probability of $t=m$ satisfying the customer. Then $z=\frac{M-m}{S}$ and $\mathrm{\Phi }\left(z\right)>\frac{1}{2}$ since $M>m$.

In reality, uncertainties in material costs, technology readiness, available personnel, etc. create uncertainty about how much the total project weight can be reduced. To quantify this uncertainty, the supplier is asked to specify:

• A pessimistic (“rainy day”) estimate of the minimum weight that can be supplied if everything goes wrong. (The chances of being unable to provide a weight below this bound are negligible.)

• An optimistic (“sunny day”) estimate of the minimum weight that can be supplied if everything goes right. (The chances of being able to provide a weight below this bound are negligible.)

If the uncertainty is normally distributed, then one-quarter of the absolute difference between these bounds (the range) is an estimate of $s$, the standard deviation of the uncertainty. The average of the bounds is an estimate of the expected value (or $m$). Since the team is more likely to achieve higher targets, define:

Supplier margin: The difference between the target and the best estimate of the least possible mass, $t-m$.

Then the probability of the supplier being able to achieve the target is $\mathrm{\Phi }\left(\frac{t-m}{s}\right)$. Increasing $t$ increases supplier margin but lowers stakeholder margin. To balance these margins, the manager chooses the target to maximize the joint probability of the target being both feasible and sufficient. If the uncertainties in the customer environment are independent of the uncertainties in the supplier environment, this probability is

$$\mathrm{\Phi }\left(\frac{t-m}{s}\right)\mathrm{\Phi }\left(\frac{M-t}{S}\right)$$E1

2.2 Numerical example

Table 1 presents the supplier and customer upper and lower bounds along with computations of their means and standard deviations (denoted by $\sigma$).

	Supplier	Customer
Lower bound	1363	400
Upper bound	1803	2200
Mean	1583	1300
Sigma	110	450

Table 1.

Data: one supplier, one user.

Table 2 presents the results of using excel solver to identify optimal targets with the targets, margins, z-scores, and probability of success.

	Supplier	User
Target	1677	1677
Margin	94	−377
Z-Score	.85	−0.83
Probability	0.80	0.20

Table 2.

Solution: one supplier, one user.

with an overall probability of success of $16\%$. Thus, failure is likely.

3. Multiple suppliers

3.1 General solution

Having a single large supplier work, a single project is often inefficient. For example, a meeting with twenty people often devolves into a meeting between smaller subgroups, with other people uninvolved. As a result, best practices in project management and systems engineering decompose the work into smaller work packages. This decomposition continues into every more granular work packages until each of the work packages is sufficiently specific to be addressed by a small specialized supplier.

Suppose the automobile weight reduction problem is decomposed into weight reduction targets for $N$ different automotive components. The smallest feasible weight for work package $j$ is normally distributed with mean $m_j$ and standard deviation $s_j$ for $j=1,\dots N$ . If weight targets $t_1,\dots t_N$ are assigned to each of the component suppliers, the probability all suppliers can meet their targets and satisfy the customer is

$$\prod _{k=1}^N\mathrm{\Phi }\left(\frac{t_k-{\mu }_k}{{\sigma }_k}\right)\mathrm{\Phi }\left(\frac{M-\sum t_k}{\sqrt{s^2+S^2}}\right)$$E2

3.2 Numerical example

Suppose the vehicle is broken up into three components: Powertrain, Chassis, and Body. Suppose that all component suppliers might have spent 10 hours in a large meeting — with 2 hours focused on discussing the work of each small team, 1 hour focused on the three interactions between each pair of teams, and 1 hour focused on the joint interactions between all suppliers. Then distribution of the work to each separate supplier reduces the time each supplier spends in irrelevant meetings by 4 hours.

If the average mass reduction from reducing meeting time by 4 hours is $\frac{10}{6}$, then the minimum total feasible weight from three more specialized suppliers is $950.$ The proportion of average weight reduction from each supplier remains the same as in Table 1 (with standard deviations unchanged.) Upper and lower bounds can then be computed from summing the mean and adding (or subtracting) two standard deviations.

Based on the first numerical example, Table 3 specifies lower and upper four-sigma bounds on the minimum mass feasible and the maximum mass acceptable for each of the three components and the user. This is used to compute means, standard deviations, and, for an initial set of targets, z-scores, margins, and success probabilities (feasibility probabilities for each design team, acceptability probabilities for the user, and the overall probability of feasibility and acceptability.)

	Powertrain	Chassis	Body	User
Lower bound	175	50	400	400
Upper bound	3254	150	800	2200
Mean	250	100	600	1300
Sigma	37.5	25	100	450

Table 3.

Data: multiple suppliers, single user.

There are uncertainties in what is physically feasible and what the user considers acceptable.

If both uncertainties are ignored, then the manager can set any targets satisfying $t_P>m_P,t_C>m_C,t_B>m_B$, $M\ge t_P+t_C+t_B$.

If only uncertainty about feasibility is ignored, then the manager will minimize weight by assigning $t_P=m_P,t_C=m_C,t_B=m_B$. Table 4 specifies these initial targets, margins, z-scores, and probabilities:

	Powertrain	Chassis	Body	User
Target	250	100	600	950
Margin	0	0	0	250
Z-score	0	0	0	0.7777
Probability	0.5	0.5	0.5	0.78

Table 4.

When supplier uncertainty is ignored.

The suppliers get zero margin and the customer gets a margin of $250$. Thus, the feasibility probabilities for the suppliers are $\frac{1}{2}$_, while the acceptability probability for the customer is $78\%$. Multiplying these probabilities gives an overall success probability of $9.75\%$.

Alternatively, suppose the feasibility uncertainty is considered, while uncertainty about customer acceptance is ignored. In this case, the project manager requires that $t_P+t_C+t_B=M\left(1300\right)$ while setting $t_P,t_C,t_B$ to maximize the probability of feasibility. The targets, margins, z-scores, and probabilities in Table 5.

	Powertrain	Chassis	Body	User
Target	340	164	796	1300
Margin	90	64	196	0
Z-score	2.4	2.6	1.96	0
Probability	0.99	0.99	0.97	0.5

Table 5.

When user uncertainty is ignored.

with an overall success probability of $48\%$.

Setting targets to recognize uncertainty in both feasibility and customer acceptance requires relaxing the constraint that the sum of the supplier targets equals the average customer need, $M$. Table 6 presents the solution without this constraint.

	Powertrain	Chassis	Body	User
Target	325	154	746	1225
Margin	75	55	146	75
Z-Score	2.0	2.2	1.5	2.0
Probability	0.98	0.99	0.93	0.90

Table 6.

Solution: multiple suppliers, single user when no uncertainties are ignored.

with an overall success probability of $87\%$.

4. Multiple attributes and stakeholders

4.1 Multiple attributes

Typically targets are set on multiple attributes, e.g., weight, safety, comfort, convenience, speed, etc. If uncertainty on each attribute $k$ is measured by a Gaussian distribution and all attributes are uncorrelated, then targets will be set to maximize

$$\prod _{k=1}^N\mathrm{\Phi }\left(\frac{t_k-m_k}{s_k}\right)\mathrm{\Phi }\left(\frac{M_k-t_k}{S_k}\right).$$E3

But physical improvements on one attribute often affect the ability to physically improved on another attribute. Suppose these physical interactions are described by a multivariate normal with mean vector $\overrightarrow{m}$ and variance-covariance s.

Likewise, there are interactions between improvements in one attribute and preferences for other attributes. For example, improving a vehicle’s crashworthiness and its ability to avoid crashes both increase the safety of passengers in the vehicle. Hence, dramatic improvements on one attribute reduce the need to make improvements on other attributes, that is. the attributes are partially substitutable. In contrast, improving a vehicle’s reliability increases the amount of time a driver can use the vehicle which increases the need for performance improvements. These attributes are complementary. Suppose these correlations between preference attributes are described by a multivariate normal distribution with mean $\overrightarrow{M}$ and variance-covariance $\widehat{\mathbf{S}}$. Then if there is still no correlation between customer preferences and supplier uncertainties, targets must be set to maximize

$$N\left(\overrightarrow{t}-m,\mathbf{S}\right)N\left(\overrightarrow{m}-t,\mathbf{s}\right).$$E4

If the standard deviation on a requirement is small, then the normal distribution will attach high weight to improving performance to meet the target. Thus, the normal distribution also allows for the prioritization of different attribute targets.

Some suppliers, for example, powertrain will have a significant impact on meeting vehicle emissions and fuel-efficiency targets but not on comfort and convenience. Other suppliers, for example, body will have less impact on emissions and fuel efficiency but greater impact on comfort and convenience. Software, while having a great impact on many performance targets, typically has a negligible impact on vehicle weight. This formulation allows target allocation to reflect these differences among different work teams.

4.2 Multiple stakeholders

In many cases, stakeholders have very different, or even conflicting, objectives. Stakeholders concerned with vehicle crashworthiness might prefer higher-mass vehicles to lower-mass vehicles. Suppose their minimum acceptable mass has mean $M_2$ and standard deviation $S_2$ (with the original stakeholders having mean $M_1$ and standard deviation $S_1$.) If we also use the numbers 1, 2, 3 to represent the powertrain, chassis, and body group, then the manager must set targets to maximize

$$\left[\prod _{i=1}^3\mathrm{\Phi }\left(\frac{t_i-m_i}{s_i}\right)\right]\mathrm{\Phi }\left(\frac{M_1-\sum _i{t}_i}{S_1}\right)\mathrm{\Phi }\left(\frac{\sum _i{t}_i-M_1}{S_2}\right).$$E5

This distinction between stakeholders becomes especially important in setting multi-attribute targets.

Different stakeholders are often only interested in different targets. Government regulators are typically interested in vehicle emissions but not comfort, convenience, and style. Buyers are interested in comfort, convenience, and style but not emissions. There are some attributes, for example, vehicle safety and fuel efficiency where both are interested.

4.3 Example

Suppose there are two engineering groups (Powertrain, Body), two user groups (Customer, Govt), and two attributes (Defect rate, Weight). The customer demands average defect rates lower than the government, while the government demands average weights lower than the customer. The powertrain team provides a lower average defect rate, while the body group provides a lower average weight. For each attribute, Table 7 lists the capability and needs of engineering and users.

	Powertrain	Body	Customer	Govt
Lower bound: Defects	2	10	16	20
Upper bound: Defects	8	26	36	50
Mean: Defects	5	17	27	35
Sigma: Defects	1.5	4	5	7.5
Lower bound: Weight	175	50	350	250
Upper bound: Weight	325	150	750	550
Mean: Weight	250	100	550	400
Sigma: Weight	37.5	25	100	75

Table 7.

Data: multiplier suppliers, attributes, users.

Table 8 presents the results of using an excel solver to identify optimal targets with the targets, margins, z-scores, and probability of success.

	Powertrain	Body	Customer	Govt
Target: Defects	6.8	18.5	25.3	25.3
Margin: Defects	1.8	1.5	1.7	9.7
Z-score: Defects	1.2	0.4	0.3	1.3
Probability: Defects	0.90	0.64	0.63	0.90
Target: Weight	293	136	429	429
Margin: Weight	43	36	121	71
Z-score: Weight	1.1	1.4	1.2	0.95
Probability: Weight	0.87	0.92	0.89	0.93

Table 8.

Solutions: multiple suppliers, attributes, users.

For an overall 33 and 59% probability of meeting the defect and weight targets, respectively. Because devoting time to defect reduction might divert effort from weight reduction, there might be a negative correlation between each engineer team’s capability on each attribute. Since customers might feel willing to accept higher weight in return for few defects, there might also be negative correlation between satisfaction with the attributes. Extensions of this approach could address these issues.

5. Conclusions

This chapter illustrated how targets could be set to balance feasibility and sufficiency for multiple suppliers and multiple stakeholders. Initial examples focused on the case of one supplier and one stakeholder and showed how ignoring either supplier uncertainty or stakeholder uncertainty reduces the probability of success. This illustrated the importance of both stakeholder and supplier margins.

However, even fully considering both uncertainties still did not lead to a high probability of success — because of the inefficiency of one team addressing a complex problem. This motivated the decomposition of a single supplier into multiple more specialized suppliers each focused on a different work package.

This approach was extended to multi-attribute targets with complementary and substitutable attributes. Finally, it was extended to multiple stakeholders, which allowed consideration of differences between stakeholder preferences for different attributes. Thus, this chapter presents a tractable approach of addressing complex interactions between design sub-teams, attribute targets, and the preferences of different stakeholders in the presence of uncertainty.

Behavioral research finds that individuals naturally evaluate outcomes against reference points and that how a product is described can influence individual reference points. Hence, this approach highlights the role of advertising and social media in shaping what individuals find acceptable. This joint use of product design and marketing can then increase the probability of success.

While this paper assumed Gaussian distributions, extensions to more realistic distributions are straightforward. It is also straightforward to generalize this chapter’s assumption of an additive relationship between supplier targets and the overall target.