Perspective Chapter: Program Planning and Management for Defense Advanced Concept Technology Programs

Abstract

The complexity of program planning and management (PPM) for defense programs and related projects depends on the program types and associated budget size. In general, the defense program types can be classified into three categories, namely, normal program of record (POR), new program with traditional and/or well-defined acquisition strategy, and advanced concept technology (ACT) program. This chapter offers a new perspective on the development of an effective PPM plan for ACT programs. For the ACT program type, the traditional PPM is usually not applicable and required to be handled differently according to the uncertainty associated with the technical requirements and associated technology and corresponding cost risks. The chapter provides an overview of typical ACT PPM and associated planning, execution, and management activities from both government and contractor’s perspectives. In addition, the chapter attempts to (i) quantify the risks associated with ACT programs in terms of innovation indicators using simplified Cooper chart, and (ii) develop a set of recommended PPM activities that can be used as a basic framework for conducting the planning and execution of PPM of ACT programs.

1. Introduction

In practice, the complexity of planning, managing, and executing a defense program depends on the budget size and program type. In the US, for a new traditional or a POR with large budget, usually above 100M USD, the government PM plans the program using the DOD acquisition system [1, 2] and DOD Instruction (DODI) 5000.02 [3]. The PM follows the planning and execution of the program according to the DOD guide for PMs [4]. In addition to the DOD guide for PMs, the government PM also uses additional DOD guidebooks to (i) identify the potential technical, management, and related program issues and risks [5], and (ii) investigate the use of modular open system approach to reduce the interfaces technical risk and the associated cost [6]. When the new traditional program with large budget involved with the acquisition of commercial products and/or commercial services, the government PMs seek guidance from the federal acquisition regulation (FAR) Part 12 [7] for the development of a PPM plan. This type of large programs is usually required to go through the normal acquisition process, which leverages mature technology and related technology enablers (TEs). In practice, the level of mature technology is defined using the Technology Readiness Level (TRL) scale ranging from TRL-1 to TRL-9 [8]. TRL-1 is defined as the basic principles have been observed and reported, and TRL-9 is for an actual system is proven through successful mission operations. Practically, TRL-8 is usually considered to be matured, because at this TRL the actual system is completed and qualified through test and demonstration.

Unlike the traditional program and/or POR, for advanced concept technology (ACT) programs with small budgets (less than 100M USD) are usually not acquisition programs. This type of advanced defense programs includes DOD ACT Demonstration Program (ACTD) [9, 10], advanced contract research and development (CRAD) programs from DOD Laboratories (Labs) (e.g., Air Force Research Lab (AFRL), Naval Research Lab (ARL), Army Research Lab (ARL), etc) [11], and Defense Advanced Research Projects Agency (DARPA) programs [12, 13]. In addition to these ACT programs, US government also manages ACT programs with emphasis on the development of advanced TEs in critical technology areas by domestic small businesses [14]. These ACT programs are referred to as Small Business Innovation Research and Small Business Technology Transfer (SBIR/STTR) programs. For these ACT programs, the government PMs are required to use different DOD acquisition process that is different from the normal acquisition process for traditional programs and/or POR. The development of a PPM plan for ACT programs is quite different from traditional/POR programs with large budgets. In practice, when selected as the prime contractor (a.k.a. developer) for executing an ACT program, the contractor program manager (PM) is also required to develop a PPM plan, and execute and manage the plan, according to the government PPM requirements.

The primary objective of this chapter is to provide an overview on the development of an effective PPM plan and executing the plan from both government and contractor perspectives. The chapter also provides a set of recommendations that can be used as a basic framework for the development and executing a PPM plan. The chapter has eight main sections, and it is organized as follows:

• Section 2 describes the type of ACT programs/projects and their characteristics,

• Section 3 presents a typical ACT program acquisition life cycle from both government (a.k.a. US department of defense (DOD)) and contractor perspectives,

• Section 4 presents a recommended tailored Zachman framework that can be used for ACT program planning activities,

• Section 5 recommends an approach to quantify the technology and market risks associated with ACT programs using the innovation indicators and Cooper chart,

• Section 6 recommends a set of PPM activities for balancing cost, technical and program management risks from both government and contractor perspectives,

• Section 7 describes the use of Earned Value Management System (EVMS) for tracking and managing ACT program risks,

• Section 8 discusses the use of machine learning and artificial intelligent (ML-AI) and recommends ways to improve the development and executing of a PPM plan, and

• Section 9 concludes with a summary of the chapter.

2. Characteristics of ACT programs/projects

Figure 1 captures typical DOD ACT program types discussed in Section 1. For US DOD, the defense ACT program types can be classified into four categories, namely, ACTD, DARPA, CRAD, and SBIR/STTR programs. The ACTD programs usually range from a few millions USD to 10+ mils USD [10], which are initiated by DOD to determine a proposed mature technology enabler (TE) that will be used to improve specific defense capabilities before entering the normal DOD acquisition process. The period of performance (PoP) for the assessment of the proposed TE is typically between 2 to 4 years, and the TE under ACTD program implementation is usually at TRL-7 (or even at TRL-8) with a goal to achieve higher TRL before entering formal acquisition process. Note that TRL-8 and TRL-9 indicate low and the lowest possible technology risk level, respectively. From the government PM perspective, ACTD program requires to identify (i) a mature TE that aligns with a priority military need for achieving specific defense capabilities, and (ii) a corresponding government sponsor in urgent needs of these capabilities. From the developer (contractor) PM perspective, ACTD program requires the execution team to be ready and prepare a detailed plan to conduct the demonstrations and/or exercises with required key performance parameters for the military utility assessment. The plan must also address all related risks for the demo/exercises.

Figure 1.

A general description of ACT program types.

The CRAD programs are usually more advance than the ACTD programs, since they are more focus on the advancement of scientific and technical knowledge and apply that knowledge to achieve specific goals set by the sponsored agency and national goals [11]. Practically, most of CRAD programs usually start at TRL-1. Like ACTD programs, CRAD program funding has similar budget ranging from a few millions USD to 10+ mils USD. PoP for CRAD programs also range from 2 years to 4 years. From the government PM perspective, CRAD program requires to (i) find a critical technology area and related TEs that are aligned with the agency needs and national goals, and (ii) supply a clear, concise, and complete statement of work (SOW) or a request for proposal (RFP) describing the area for basic research and the end objectives for development and applied research. The technical and contracting personnel must individually tailor the SOW/RFP to allow for contractor to exercise innovation and creativity while achieving objectives of the R&D [11]. From the contractor PM perspective, the CRAD program requires the contractor execution team to be ready and prepare a detailed PPM plan to address the SOW/RFP requirements and associated challenges. The contractor PPM plan must also provide supporting evidence to demonstrate the contractor’s technical capabilities to achieve the end objectives.

The DARPA program type is quite different than ACTD and CRAD programs because they are focused the development of breakthrough technology [12]. As stated in the DARPA website, the objective of DARPA programs is to transform revolutionary concepts and even seeming impossibilities into practical defense capabilities. Typical DARPA program ranges from a few millions USD funding and up to 100M¹ USD [13]. Practically, DARPA program PoP ranges from 1 to 3 years for proof of revolutionary concepts. From DARPA perspective, DARPA program requires to (i) identify a revolutionary and breakthrough technology that aligns with DARPA needs (or national needs), and (ii) provide a clear, concise, and complete Broad Agency Announcement (BAA) or an RFP describing the area for research and development pushing the leading-edge technology and the end objectives. The BAA/RFP should address how DARPA rewards risk by clearly define criteria for evaluating the proposed DARPA programs using a set of questions known as the “Heilmeier Catechism” [15]. From the contractor PM perspective, DARPA program requires the contractor execution team to be ready and prepare an innovative PPM plan to address the BAA/RFP requirements with emphasis on the answers to Heilmeier’s questions. The plan must clearly describe the innovative features of the proposed solution and provide supporting evidence to demonstrate the contractor’s technical capabilities to achieve the program objective.

Last but not least, the SBIR/STTR programs are usually focused on the critical technology areas that are aligned with the government agencies’ objectives and national goals. Typical SBIR/STTR programs are usually emphasis on the basic and applied research for advancing the state-of-art, increasing knowledge, or understanding of specified technology and related TEs rather than focusing on a specific system or hardware solution. Typically, these programs have three phases, namely, Phase 1, Phase 2, and Phase 3. Phase 1 funding ranges from 150K to 175K USD, Phase 2 funding from 750K to 1M USD, and Phase 3 funding ranges from 2M or higher depending on the commercialization matching funds. Typical PoP for Phase 1 is usually from 6 months to 1 year, Phase 2 is 2 years, and Phase 3 is 2 to 4 years depending on the funding and industry partner’s plan for the integration with existing partner’s products or planned systems.

3. ACT program life cycle: government vs. contractor perspectives

Practically, a detailed ACT program life cycle varies depending on the ACT program types, agency objectives and national goals. Thus, the development of a PPM plan also varies accordingly depending on government and contractor perspectives. This section provides an overview of typical ACT program life cycle and discusses the roles of the PMs and desired PPM activities from both government and contractor perspectives. In general, the ACT program life cycle can be expressed in four phases, namely, concept, pre-acquisition, post-acquisition, and transition phases as shown in Figure 2. The figure is derived from the traditional DoD program acquisition life cycle [1, 2, 3]. It also captures the roles of government and contractor PMs for each phase.

Figure 2.

ACT program life cycle derived from traditional DoD program life cycle [1, 2, 3].

As shown in Figure 2, the government PM role with required PPM activities covers the entire ACT program life cycle from the concept phase to the transition phase. While the contractor PM role with PPM activities begins after the pre-acquisition from the post-acquisition to the transition phases. Theoretically, the contractor PM role starts at the post-acquisition phase after the ACT contract is awarded. But in practice, the contractor PM role starts at the release of the BAA/RFP/SOW. For large ACT programs (i.e., 10⁺M USD), at the release of the BAA/RFP/SOW, the contractor PM is usually working with the contractor capture team (CCT) under the leadership of a business capture manager to prepare and generate proposal and cost volume for the bids. The contractor capture and program managers with support from their program chief engineer will work with the government PM to gain a deep understand of the agency objectives, national goals, and corresponding program requirements to properly address them during the preparation of the proposal and cost volume.

As mentioned earlier, the PPM activities begin at the concept phase and pre-acquisition phase for the government team and contractor team, respectively. In practice, for the contractor team, the PPM activity begins at end of the pre-acquisition phase after the lease of the BAA/RFP/SOW and the execution of the PPM plan starts at the beginning of the post-acquisition phase after the contract award and continues throughout the transition phase. As shown in Figure 2, the role of the contractor PM changes slightly during the transition phase. When the technology and associated TE developed by the program is selected for transition to existing POR and/or planned acquisition program, the contractor PM and his technical team will continue to work with the new government PM to integrate the newly developed TEs into the new program execution plan. When the newly developed TE is selected for commercialization, the contractor PM will continue to work with the government PM and the industry partner to commercialize the products. For SBIR/STTR programs, the funding for the commercialization of the ACT products is usually through a matching fund with support from an industry partner. Note that for ACTD program, if the selected technology and associated TE are successfully demonstrated, the contractor PM will continue to work with the new government PM.

During the concept phase, the government PM works with the government team to develop a PPM plan. The team includes potential stakeholders, technical personnel, contract personnel, and operational users. The government team ensures that the PPM plan will (i) provide required operational capabilities to meet the user needs, and (ii) meet the end objectives and national goals. The user’s needs must be identified and approved by appropriate government decision makers and associated stakeholders. After the approval, the government PM will conduct industry survey (a.k.a. market survey) to collect necessary technical inputs and related data to identify appropriate technologies and related TEs to address the user’s needs. The government PM with support from the team will work with government acquisition authority to make the decision on new ACT programs/ projects based on the collected inputs and data. A positive decision allows to turn on the pre-acquisition process and start the new ACT programs/projects. During this pre-acquisition phase, the government PM with support from government technical and contract personnel will identify and analyze the program risk, including technical performance, cost, and schedule risks and prepare the BAA/RFP/SOW. The government PM generates and releases the BAA/RFP/SOW to public for bids. After the release of BAA/RFP/SOW, the government PM forms the source selection team (SST) consisting of subject matter experts (SMEs) in specific technology areas related to the ACT topics, cost, contract, and schedule. The SST will review and select the best proposal(s) for the contract award. The post-acquisition phase begins after the contract award, and the government PM executes and manages the ACT contract (i.e., executing and managing the PPM plan) to ensure the contractor team meets the contract requirements from technical, cost, and schedule perspectives.

The contractor team begins the PPM activities after the release of BAA/RFP/SOW. The team usually works with the government PM to (i) understand the ACT program requirements, and (ii) prepare the proposal addressing all required requirements and submits the bid. As mentioned earlier, for large ACT programs (10⁺M USD), the contractor PM works with the business capture manager to accurately address all ACT program requirements with high probability of winning the contract award. For this type of program, the contractor capture team (CCT) will develop an effective PPM plan to ensure high probability of win. The CCT team consists of SMEs across contractor’s organization, including engineering, contract, cost, and schedule departments. After the contract award, the contractor PM will work with the government PM to adjust the proposed contractor’s PPM plan to ensure alignment with the government’s PPM plan. The contractor PM will work with his contractor execution team to execute the adjusted PPM plan. The contractor PM reports the program progress and milestone accomplishments to the government PM. The role of the contractor PM will slightly change during the transition phase depending on the transition path. As shown in Figure 2, when the transition path goes to existing and/or planned DOD program that followed normal DOD acquisition process, the contractor PM is required to work with the existing government PM and the new government PM² and the new government execution team to develop an integration plan. At this time, the contractor team is required to (i) gain a good understanding of the proposed system being acquired, and (ii) develop an integration plan to integrate the newly developed TEs into the proposed system. The contractor PM and the contractor execution team are usually required to provide technical support over the life cycle of the DOD program being transitioned into. When the transition path goes to the commercialization path, the contractor PM will work with the ACT government program PM and the interested industry partner to develop detailed plan and associated products using the newly developed TEs. For this transition path, the contractor PM is required to understand the industry partner products and the to be developed products.

The remaining of this chapter provides an overview of the ACT program PPM activities for both government and contractor perspectives.

4. ACT program planning: the Zachman framework

As pointed out in ref. [16], there is a set of twenty multiple PM discipline areas, including (i) Program goals management, (ii) Systems engineering related to the systems/products/services being acquired, (iii) Specialized engineering related to the products and services being acquired, (iv) Contracts and legal dealing with contractors, suppliers, and stakeholders, (v) Program Financial management, (vi) Business and marketing practices for the newly acquired systems/products/services, (vii) System/ product/service technical requirements and associated performance risk management, (viii) System/product/service cost planning and management, (ix) Program schedule planning and management, (x) Program cost planning and management, (xi) System/ product/service³ risk planning and management, (xii) Program risk planning and management, (xiii) System test and evaluation, (xiv) Logistics and supply chain management, (xv) Production, Quality, and Manufacturing (PQM), (xvi) Program and system intelligence and security management, (xvii) Program and system software management, (xviii) Program and system configuration management, (xix) Program and system information technology, and (xx) Other Specialty Program Planning and Management. For ACT programs, depending on the PM’s perspective, the PM is required to select a subset of these PM discipline areas for the development of an effective PPM plan. From the government perspective, at the minimum, the PM must develop a PPM plan that addresses the eleven out of the twenty PM discipline areas listed above, including (i), (ii), (iii), (iv), (v), (vii), (viii), (ix), (x), (xi), and (xii). From the contractor perspective, the PM also requires develop a PPM plan that addresses the same PM discipline areas except Bullet (i), namely “program goals management.” Note that for the PM discipline area (iv), namely the “contracts and legal dealing with contractors, suppliers, and stakeholders,” the contractor PM is required to address the contracts and legal dealing with the government PM and its subcontractors, including the suppliers. Based on our experience working on ACT program planning from both contractor and government perspectives, the Zachman framework can be tailored to effectively develop the PPM plan:

• For Government Perspective: The government PPM plan includes acquisition strategy, execution, program management, and transition plans; and

• For Contractor Perspective: Contractor PPM plan includes bidding strategy, execution, program management, and transition plans.

To develop an effective PPM plan, the Zachman framework can be tailored as recommended in Table 1 to address the PPM activities across the PM discipline areas (i), (ii), (iv), (v), (vii), (viii), (ix), (x), (xi), and (xii) from both government and contractor PM perspectives. Like standard Zachman framework, the recommended tailored-Zachman framework also organizes around the points of view taken by the various players. The players include government PM and contractor PM. From the government perspective, the PM undertakes the planning of an ACT program to ensure alignment with the agency objectives and national goals. Hence, the government PM role is to develop a PPM plan, prepare, generate and release BAA/RFP/SOW that is fully support by the industry. From the contractor perspective, the PM will work with his technical team to identify and apply specific technologies and related TEs to solve the ACT problems described in the government released BAA/RFP/SOW. In summary, each of these players can look at the same PM discipline areas but with different perspectives. The government perspective is to ensure meeting the agency’s ACT development objectives and national goals within allocated budget with minimum program risks. The key program risks are defined in terms of technology and market uncertainties that will be discussed in the next section.

	ACT program data (What)	Newly proposed function (How)	Operational environment/existing network (Where)	People: DOD agency/User (Who)	Time (When)	Motivation /Defense needs (Why)
Objectives/Scope	List of things important to sponsored DOD agency	List of new functional capabilities vs. Existing functions	List of operational environment and locations where to-be system operates	List of agency involved and users	List of ACT program events / mile-stones	List of agency objectives, goals, and strategies
Business & Operational Model for Defense Applications (BOMDA)	Entity relationship diagram (including attributed relationships)	New BOMDA vs. existing BOMDA	ACT Logistics network (nodes and links for to-be deployed ACT system)	Organiza-tional chart with roles, skill sets, and issues.	ACT program master schedule	Business and operational ACT plan
Information System Model (ISM)	Data model associated with ACT program	ACT data flow diagram vs. existing ISM	Distributed ACT system architecture	User interface (roles, data, access)	Depen-dency diagram, program life cycle	Business and ACT system operational rule design
Technology Model (TM)	Data architecture, map to legacy data	ACT system design vs. existing design	ACT System architecture (hardware, software types)	User interface (how ACT system behaves); security design	ACT Control flow diagram (control structure)	Business ACT system operational rule design
Detailed Representation (DR)	Data physical design	Detailed ACT program design	To-be ACT network architecture	Screens, security design (who can see what?)	ACT system timing definitions	ACT system operational Rule specification
Functioning ACT System (FAS)	Existing working system	ACT executable program planning	Existing vs. to-be ACT Communications facilities	Opera-tional user training	ACT program mile-stones	Enforced business and ACT system operational rules
	Data model associated with ACT program

Table 1.

Tailored Zachman framework for ACT program planning [17, 18].

Like standard Zackman framework, the roles of the players are represented by rows in a matrix shown in Table 1, and the columns represent the issues/topics that will be examined by the players. More specifically, the columns represent [17, 18]:

• ACT program data to be used by an DOD agency and/or desired operational users (what),

• Newly proposed functions (how),

• Operational environment and/or existing network where the newly developed ACT capability will be conducted/deployed (where),

• DOD Agency and associated operational users involved (who),

• Operational events that trigger the defense activities (when) – Note that this event related to the “Business Model for Defense Applications (BMDA),” and

• Motivations and constraints which determine how the BMDA behaves (why).

The set of cells shown in Table 1, constructed by the roles of the players and the issues/topics to be examined by the players. These ceels describe all the ACT planning topics/issues that are required to be addressed by the government and contractor PMs. The government PM will use this tailored Zachman framework to conduct the PPM activities under government perspective discussed. The contractor PM can also use this tailored framework but with contractor perspective. Unlike the government perspective, the contractor perspective focuses on meeting the government requirements stated in the released BBA/RFP/SOW with minimum cost (i.e., maximum profit) and lowest ACT program risks. The ACT program risks will be addressed in the next section.

The description of the rows in the first column shown in Table 1 is given below:

• Objectives/Scope: Definition of the organization’s direction and business/agency purpose. From government perspective, this is the DOD agency’s objectives and national goals for the ACT programs in planning. From the contractor perspective, this is concerned with the things that define the contract objectives and business goals under the contract pursuit. The contractor’s objectives and business goals must be aligned with the government perspective.

• Business and Operational Model for Defense Applications (BOMDA): From the government perspective, this defines the existing business/agency and operational user model for defense applications, including its structure, functions, organization, and so forth. From the contractor perspective, this defines the to-be BOMDA that is compatible with existing government BOMDA.

• Information System Model (ISM): This defines the BOMDA, but in more rigorous information terms, where the BOMDA describes business/agency and user operational functions, and ISM describes those things about the to-be developed ACT system to collect and maintain information and begins to describe desired information to be collected by the to-be ACT system.

• Technology Model (TM): This describes the to-be ACT system how the new technology and associated TEs may be used to address the information processing needs identified in the ISM.

• Detailed Representations (DR): This is typically a contractor view of the program listings, database specifications, networks, and so forth that constitute a to-be ACT system that will meet the government’s requirements. Government team usually generates a government reference architecture (GRA) with related DR information for assessment of contractor’s ACT system solution.

• Functioning ACT System (FAS): The to-be ACT system is final implemented and made part of an existing defense system or a commercialized product.

5. Quantification of ACT program risks using innovation indicators

This section emphasizes on the analysis of ACT program risks for the development of an effective PPM plan. From a PM’s perspective, regardless of government or contractor, it is important to understand and mitigate the program risks. As discussed in Section 4, the technology and market⁴ uncertainties are the key attributes for the assessment of ACT program risks. Practically, the technology and market uncertainties can be used to translate into the technology and market risks, respectively. To plan and manage these risks, the PMs are required to quantify these risks based on the technology and market uncertainties provided by the manufacturers. Let us classify the uncertainty (i.e., risk) levels as Low (L, blue color), Medium (M, green color), and High (H, red color), and define the innovation indicators associated with these uncertainty levels as follows:

• Innovation Indicator Level 1 (IIL-1): This innovation indicator level indicates both the technology and market uncertainties that are low risk level (L).

• Innovation Indicator Level 2 (IIL-2): The IIL-2 indicates both the technology and market uncertainties that are medium risk level (M).

• Innovation Indicator Level 3 (IIL-3): The IIL-3 designates high market uncertainty that is high market risk (H) level and technology uncertainty ranging from low-to-medium risk level (L-M)

• Innovation Indicator Level 4 (IIL-4): IIL-4 indicates market uncertainty ranging from low-to-medium (L-M) risk and technology uncertainty at high (H) risk level.

• Innovation Indicator Level 5 (IIL-5): IIL-5 indicates both the technology and market uncertainties are at high risk level (H).

Table 2 summarizes the technology and market risk levels associated with the five proposed innovation indicator levels. Our next step is to associate these innovation indicator levels with the desired innovative solutions required for various types of ACT programs as described in Figure 1.

Innovation Indicator Level (IIL)	ACT program risk				Remark
	Technology risk		Market risk
IIL-1	L		L		Technology and market risks are quantified based on the technology and market uncertainties associated with the selected technology and its TEs. The government PM can use the request for information (RFI) process and tools to collect and assess the uncertainties from industry
IIL-2	M		M
IIL-3	L →	M	H
IIL-4	H		L →	M
IIL-5	H		H

Table 2.

Newly proposed ACT program innovation indicators.

In practice, private and for-profit enterprises (PaFoPEs) are usually invested into their internal research and development (IRAD) projects⁵ to (i) defend and extend their current capabilities to sustain the position in existing market, (ii) prepare for a venture launch by continuously improving existing products, (iii) look-out for a market for their new products (scouting option) by incremental changes of technology, (iv) position for a newly developed radical technology and related products that can transform industry and potentially creating a new market (position option), and (v) develop disruptive technology that can disrupt existing products and market (stepping-stone option) [19, 20, 21, 22, 23]. In fact, these types of PaFoPEs’ IRAD projects are usually classified based on the technology and market risks that can be mapped to the IILs presented in Table 2. For government ACT programs listed in Figure 1, they are classified in terms of TRLs that can be linked to the technology risk shown in Table 2. Thus, ACT programs can also be mapped to the IILs presented in Table 2. Table 3 captures the mapping of the IIL levels to industry IRAD projects and government ACT projects/programs.

Innovation indicator level (IIL)	IIL mapping to IRAD and ACT programs		Remark
	Industry IRAD project	Government funded ACT project/program
IIL-1	Defend and extend	ACTD, CRAD	Low technology and market (T&M) risks
IIL-2	Venture launch	CRAD, SBIR/STTR	Medium T&M risks
IIL-3	Scouting option		Medium technology and High market risks
IIL-4	Position option	CRAD, SBIR/STTR, DARPA	High technology and Medium market risks
IIL-5	stepping-stone Option	DARPA Program	High T&M risks

Table 3.

Newly proposed mapping of IILs to industry IRAD and ACT projects/programs.

As shown in Table 3, the ACTD type of ACT programs focuses the technology demonstration of mature TEs with high TRLs (i.e., L technology risk). The CRAD type of ACT programs can range from low-to-high TRLs (i.e., H-to-L technology risk). Typically, the SBIR/STTR program type focuses on the development of TEs ranging from medium-to-high TRLs (i.e., M-to-L technology risk). Finally, the DARPA programs/projects focus on the development of disruptive technology with low TRLs (i.e., H technology risks). The mapping shown in Table 3 reflects these facts. Note that the mapping of SBIR/STTR, CRAD, and DARPA are based on our experience working on these ACT programs/projects.

The mapping presented in Table 3 can be captured using a simplified Cooper chart approach [19, 20, 21, 22, 23] as shown in Figure 3. In this figure, the chart has two axes, namely, x-axis represents the market uncertainty indicator, and the y-axis is the technology uncertainty indicator with a scale from low to high corresponding to L-to-H risk. As mentioned earlier, the technology and market uncertainty indicators are translated directly to technology and market risks. The technology risk is indicated by the TRL as discussed above.

Figure 3.

Newly proposed simplified cooper chart for quantifying the innovation indicators.

Note that the market risk presented in Table 2 can be translated into the manufacturing readiness level⁶ (MRL) [24, 25]. The five IILs presented in Table 2 are then mapped onto the simplified Cooper chart depending on their assigned risks. The mapping of the industry IRAD projects and government ACT programs/projects shown in Figure 3 is performed using Table 3.

In practice, from the PaFoPE perspective, the industry IRAD projects are usually planned and managed by PaFoPEs to align with the national goals and agencies’ objectives. If this alignment is done properly, it will help PaFoPEs to (i) prepare their proposals for biding the BAA/RFPs/ SOWs to be released by DOD agencies, (ii) effectively address the government requirements described in their released BAA/RFPs/ SOWs, and (ii) increase the probability of winning the bids. From the government perspective, understanding of the technology and market risks will help the program managers to better plan the budget and prepare the BAA/RFPs/SOWs.

6. Program management: balancing cost, technical, and program management risks

As discussed in Section 4 above, there are only ten or eleven out of twenty program management discipline areas that required the program managers to address during the program planning and execution phases. A good program manager must know to balance cost, technical, program management risks during the planning and execution phases. An effective PPM plan must carefully address the three key program management discipline areas including cost, schedule, and program risk planning and management. To do this the program manager must understand the key ACT program risk types and identify all associated risks for each type. Based on our experience, there are four key ACT program types, including technical/technology risk, non-technical risk, people/staffing risk, and program management risk. Based on our research in public domain concerning the risk type [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34], a generic list of risks for each of these risk types is provided in Figure 4. Understanding of these risks will help the program managers to balance the cost, schedule, and program management risks when executing the ACT programs. As an example, if the technical risks associated with technology maturity and system complexity are high, the program manager must (i) identify a subject matter expert and a technical team who are familiar with the identified technology and related TEs, and (ii) allocate appropriate budget to mitigate these risk in the program plan.

Figure 4.

Understanding the key ACT program risk types.

Understanding of the ACT program risk types and associated risks will help the program manager to identify the risks and develop an effective program plan to mitigate and balance out the identified risks. Depending on the PM’s perspective, the ACT program risk types and associated risks (see Figure 4) usually have different impacts on the program planning and execution. Based on our experience, to effectively develop a PPM plan for executing and managing the program during the post-acquisition phase, the program manager requires to understand all PPM activities throughout the ACT program life cycle illustrated in Figure 2. From the government perspective, Figure 5 describes these PPM activities from the concept phase through the pre-acquisition phase with related source selection planning-and-execution and to the post-acquisition phase. For the concept phase shown in Figure 5, the planning actives must address the following tasks: (i) understand the agency objectives and national goals, (ii) understand the user needs and align the user needs with agency objectives and national goals, (iii) identify the required technology and related TEs to provide desired operational capability that meet the user needs, and (iv) conduct the (industry) market survey to understand the technology and market uncertainties on the identified technology and related TEs. For the pre-acquisition phase, the planning activities must address the following tasks: (i) analyze, assess, and quantify the program risks based on the (industry) market survey results, (ii) identify the technical and programmatic challenges based on the market assessment results, (iii) identify and generate required technical tasks in the form of the work breakdown structure (WBS) to address the identified challenges, and (iv) conduct and perform cost, schedule, and program planning and analysis to fit the allocated budget and scheduled timeframe and generate the government program plan to be described in the BAAs/RFPs/SOWs. For the source selection planning-and-execution phase, the activities must address the following tasks: (i) generate and release the BAAs/RFPs/SOWs to public domain requesting the bids from industry, (ii) form a source selection team with subject matter experts on both technical and program management areas, (iii) conduct source selection and select the best contractor for executing the government program plan, and (iv) announce the contract award winner(s) and debrief the losers. Finally, for the post-acquisition phase and program execution- and-management, the activities must address the following tasks: (i) plan and conduct program kick-off and work with the selected contractor to finalize the program requirements and request the contractor to present their final execution program plan at the kick-off meeting, (ii) conduct the program quarterly review (should be bi-monthly for short period of performance program), (iii) review contractor quarterly reports and identify new and/or potential technical and programmatic risks and update the cost/schedule/and program plan to add new risks and retire the old ones, and (iv) conduct final review and evaluate final report to make final decision on the way forward.

Figure 5.

A new perspective on the understanding ACT program planning, execution, and management activities from government perspective.

Similarly, from the contractor perspective, Figure 6 illustrates the desired PPM activities from the pre-acquisition phase through the source selection planning-and-execution and to the post-acquisition phase. Unlike the government perspective, the contractor perspective does not have the concept phase planning activities. For the pre-acquisition phase illustrated in Figure 6, the contractor planning actives must address the following tasks: (i) receive and review⁷ the BAAs/RFPs/SOWs from the DOD agency of interest, (ii) form a CCT⁸ to prepare the proposal and start the bidding process⁹, (iii) conduct the contract requirement flow-down, (iv) identify the key program requirements and associated technical and programmatic challenges, (v) tailored system engineering process with required program tasks and generate a WBS addressing the overall program requirements and challenges, (vi) analyze program cost, schedule, and program planning to fit the allocated budget and schedule within specified timeframe, (vii) conduct program cost, schedule, and program risk assessment, and (viii) revise and finalize the WBS to address cost, schedule, and all technical and program risks. For the source selection planning-and-execution phase, the PPM activities must address the following tasks: (i) generate the proposal and prepare the cost volume for the bid, (ii) work with the government PM to gain better understanding of ACT program requirements, (iii) refine the proposal and cost volume as needed, (iv) submit the proposal and cost volume, and (v) when requested, respond to government source selection team and wait for the contract award decision. Finally, for the post-acquisition phase and program execution- and-management (assuming the contract is awarded), the PPM activities must address the following tasks: (i) work with the government PM to plan and conduct program kick-off and finalize the program requirements and present the final execution program plan at the kick-off meeting, (ii) work with the government PM to prepare and execute the program quarterly review (should be bi-monthly for short period of performance program), (iii) respond to government PM’s requests, and (iv) prepare and execute final program review and address all government PM’s requests.

Figure 6.

A new perspective on the understanding ACT program planning, execution, and management activities from contractor perspective.

Figures 5 and 6 describe the recommended PPM activities that can potentially help the program managers to develop an effective PPM plan with a good balance between cost, technical, and program management risks for both government and contractor perspectives, respectively. From the government perspective shown in Figure 5, the act to balance the cost, technical, and program management risks occurs between the pre-acquisition planning phase (see blue-star) and the post-acquisition phase (see red-star). In practice, the blue-star captures the government PPM plan, while the red-star captures the progress of the plan during program execution phase. This is the time when the government PM can balance the cost, schedule, and program risks based on contractor’s performance. The PM will (i) add new risks and retire the old risks, and (ii) adjust the PPM plan to balance the risks. Similarly, from the contractor perspective, the act to balance the risks also occurs between the blue-star and the red-star shown in Figure 6. It should be noted that, for a small ACT program, the PPM only requires the WBS, and a schedule plan. For large budget program (typically 20⁺M USD), the PPM plan requires an integrated master plan (IMP) and integrated master schedule (IMS) [26, 27].

7. Earned value management system for tracking and managing risks

In practice, both government and contractor PMs use the Earned Value Management System (EVMS) to effective track and manage the program risks. As described in [29, 30], the term “EV” is defined as an objective measure of the work done expressed in terms USD or hours that representing the value of the work done. The “earned value management (EVM) process is defined as the process of defining, planning, and controlling the scope of work, program schedule, and program budget. Thus, the EVMS is the integration of EVM processes, EVM procedures, and related EV tools to comply with the ANSI/EIA Standard 748 [28]. He recommended EVMS is a combination of processes, procedure, and related tools [28, 29, 30, 31], which can be used to measure and track the “earned value” (EV) against an integrated baseline plan (IBP) captured in the PPM plan. The IBP is the baseline IMP and IMS mentioned earlier. In practice, for defense ACT programs with contract value greater than or equal to 20M USD, the contractor PM is required to submit the integrated program management report (IPMR) to the government PM [29, 32]. The IPMR combines and replaces the Contract Performance Reports per DIMGMT-81466 and the Integrated Master Schedule per DI-MGMT81650 [33]. The IPMR report contains the EV performance data. Per DI-MGMT-81861 [32], the report provides required program status of contract cost and schedule performance according to the government required seven formats. The seven formats include Formats 1, 2, 3, 4, 5, 6, and 7. Table 4 summarizes the requirements associated with the seven required formats.

IPMR format	Format requirement description
Format 1	Define and report cost and schedule performance data by a specified program WBS
Format 2	Define and report cost and schedule performance data by the contractor’s organizational structure, e.g., Functional or Integrated Product Team (IPT)
Format 3	Define and report changes to the IPB or Performance Measurement Baseline (PMB)
Format 4	Define and report staffing forecasts
Format 5	Provide a narrative report used to provide the required analysis of data contained in Formats 1, 2, 3, 4, and 6
Format 6	Define and report IMS and changes
Format 7	Define and report time-phased historical and forecast cost submission

Table 4.

Description of required IPMR seven formats to capture EV performance data [32, 33].

As indicated in [29, 30], US government has adopted the standard that defines the EVMS implementation requirements for tracking and managing risks for all defense programs. According to [29], the EVMS implementation is (i) a mandatory requirement for all defense programs with contract value greater than or equal to 20M USD, and (ii) not required for less than 20M USD. Per DI-MGMT81861A, for contract value between 20M and $50M, a simplified IPMR report may be allowed at the government PM discretion based on program risk. The simplified IPMR requires to report EV data according to Formats 1, 5, and 6 described in Table 4. For contract value greater than $50 M, a full IPMR report is required, including all formats described in Table 4. Table 5 captures a summary of the applicability of EVMS to defense ACT programs/projects and associated contract values.

Contract value (USD)	ACT program applicability	Remark and recommendation
<20M	Not required	Based on our experience, it’s recommended the contractor PM to report IPMR Formats 1 and 5 for contract value between 10M and 20M USD. For Format 5, only Format 1 data analysis is required
≥ $20M and ≤ $50M	Required monthly IPMR report	Formats 2, 3, and 4 may be excluded at government PM discretion based on program risk
> $50M	Required monthly IPMR report	All Formats must be included in the IPMR report

Table 5.

Applicability of EVMS to ACT programs/projects [32, 33].

To provide a better understanding of EV data and related cost and schedule performance data, the remaining of this section provides an overview of the recommended baseline EVMS model. The model includes five key EV data captured the Budgeted Cost of Work Scheduled (BCWS), Budgeted Cost of Work Performed (BCWP), Actual Cost of Work Performed (ACWP), Budget at Completion (BAC), and Estimate at Completion (EAC) [28, 31]. The BCWS represents the planned EV of the work planned to be accomplished in a period of time. BCWS can be calculated using:

$$\text{BCWS}=\%\text{Complete}\left(\text{planned}\right)\ast \text{ProjectBudget}$$E1

BCWP is defined as the EV of completed work in terms of the work’s assigned budget and calculated using the following equation:

$$\text{BCWP}=\%\text{Complete}\left(\text{Actual}\right)\ast \text{ProjectBudget}$$E2

The ACWP is the actual cost incurred and recorded for work completed within a specified time period. The contractor PM usually reports the cumulative ACWP for the WBS work package that have been completed. Finally, the estimate of BAC is established by the PMs during the program planning phase. From the contractor perspective, the contractor PM estimates the BAC during the pre-acquisition phase for every specified level of the WBS. The BAC value represents the total budget from which individual period BCWS values are derived and they are the benchmarks for assessing overruns and underruns at the end of the contract. The budgets for all authorized work must be captured within the BAC. The BAC can also be referred to as the “Total Planned Value” of a project. As indicated in [34], there is no formula for calculating BAC. The calculation of BAC requires complex cost estimating methods and associated tools [16, 34]. Popular estimating methods include parametric, analogy, engineering estimate, actual costs, and three-point estimate [34]. Popular estimating tools include SEER, aPriori Cost Estimating Software Tools, and DOD COCOMO Software [16].

The cost and schedule performance data are characterized by Schedule Performance Index (SPI) and Cost Performance Index (CPI), and the SPI and CPI data are generated using the following equations:

$$\mathrm{SPI}=\frac{\text{BCWP}}{\text{BCWS}}=\left\{\begin{array}{c}\mathrm{SPI}>1:\text{Indicates an ahead}-\text{of}-\text{schedule condition}\\ \mathrm{SPI}<1:\text{Indicates}\mathrm{a}\text{behind}-\text{schedule condition}\end{array}\text{plan}|\right\}$$E3

$$\mathrm{CPI}=\frac{\text{BCWP}}{\text{ACWP}}=\left\{\begin{array}{c}\mathrm{CPI}>1:\text{Indicates}\mathrm{a}\text{favorable cost efficiency condition}\\ \mathrm{CPI}<1:\text{Indicates an unfavorable cost efficiency condition}\end{array}\text{plan}|\right\}$$E4

The PMs track and monitor the SPI and CPI data manage the execution teams according to the reported SPI and CPI values. As an example, when both SPI and CPI indices are equal to 1, the execution team performs their work according to schedule time and allocated budget. For instance, if the SPI is .8, it means that the team is behind the schedule and that only 80% of the scheduled work has been completed, not the 80% of the total planned work. When the CPI is .8, it means that for every dollar actually spent by the execution team, only 80% worth of work was performed. For this case, the execution team might have spent 20% of the budget on the re-work.

The EAC is also an important EV parameter that required the PMs to track and monitor. The EAC value represents the forecasted total budget that is required to complete at a given time during a project. The EAC value can be computed using BCWP, ACWP, SPI, and CPI using the following equation:

$$\mathrm{EAC}=\text{ACWP}+\frac{\mathrm{BAC}–\text{BCWP}}{\mathrm{CPI}\ast \mathrm{SPI}}$$E5

Other related EV parameters are the percentage of the Cost Variance (CV%), percentage of the Schedule Variance (SV%), and the Variance at Completion (VAR). The CV%, SV%, and VAR can be calculated from the EV data using the following equations:

$$\mathrm{CV}\%=\frac{\text{BCWP}-\text{ACWS}}{\text{BCWP}}$$E6

$$\mathrm{SV}\%=\frac{\text{BCWP}-\text{BCWS}}{\text{BCWS}}$$E7

$$\mathrm{VAR}=\mathrm{BAC}-\mathrm{EAC}$$E8

From Eqs. (6) and (7), the Cost Variance (CV) and Schedule Variance (SV) can be written as follow:

$$\begin{array}{rcl}\mathrm{CV}& =& \text{BCWP}-\text{ACWS}\\ & =& \left\{\mathrm{CV}>0:\text{Indicatesthecostunderrunplan}|\left|\mathrm{CV}<0:\text{Indicatesthecostoverrun}\right|\right\}\end{array}$$E9

$$\begin{array}{ccc}\mathrm{SV}& =& \text{BCWP}-\text{BCWS}\\ & =& \left\{\mathrm{SV}>0:\text{Indicatesthescheduleisaheadplan}|\left|\mathrm{CV}<0:\text{Indicatesthescheduleisbehind}\right|\right\}\end{array}$$E10

In practice, the level of effort (LOE) cannot have a schedule variance because BCWS and BCWP are always the same. The PMs can use the EV data to check and correct for anomalies associated with the program “health.” Typical anomalies are: (i) actual budget that is required to complete at a given time during a project should never be greater than EAC at the same given time, and (ii) you cannot earn more than what is budgeted, i.e., BCWP cannot be greater than BAC.

8. Conclusion and recommendations

The specific nature of ACT program characteristics requires the PMs to have a good understanding of the PPM activities to develop effective PPM plans. As discussed above, the plan will be developed depending on the PM’s perspective, i.e., contractor vs. government. In general (regardless of the perspective), a PM, who is responsible for the planning and executing an ACT program, must have a good understanding of the challenges and issues associated with ACT program characteristics, acquisition life cycle, program cost/technical/management risks, desired PPM activities for balancing cost, technical and program management risks, and EVMS methods and tools. The above sections, Sections 2, 3, 4, 5, 6, and 7, have provided an overview with sufficient details on each of the challenges and issues mentioned above. To help improve the preparation and development of an effective PPM plan during the concept and pre-acquisition phases, the chapter recommends (i) a tailored Zachman framework for PPM planning, (ii) an innovative approach to quantify the technology and market risks associated with ACT programs using the innovation indicators and simplified Cooper chart, (iii) a set of PPM activities for balancing cost, technical and program management risks, and (iv) the EVMS applicability for tracking and managing the identified ACT program risks.

Recently, based on [16, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45] has proposed an approach to integrate data and decision sciences into PPM. The proposed integration approach for PPM planning and execution leverages big data analytics (BDA) technology with BDA data acquisition and data curation TEs, and ML-AI TEs. The recommended ML-AI TE’s include (i) data mining techniques and tools (DMTT), (ii) data exploitation using multi-objective reinforce learning and adaptive neural network (MORL-ANN) tool, and (iii) predictive analytics techniques using MORL-ANN tool [34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45]. These ML-AI techniques and tools leverage related cost historical data bases, BDA framework with data acquisition and data curation models and tools to develop the program cost estimate and execute the EVMS plan to be included in the PPM plan. As pointed out in [35], there are many researchers and start-up companies developing algorithms, analytical models, and tools to apply ML-AI in program management. When this next generation ML-AI tools becomes popular and widely adapted by the decision makers (such as PMs and acquisition authorities), there will be radical changes that will disrupt the development and execution of a PPM plan [35] predicted that ML-AI will disrupt six aspects of PPM, including: (i) better section and prioritization, (ii) support for the project management office, (iii) improved, faster project definition, planning, and reporting, (iv) Virtual project assistants, (v) advanced testing systems and software, and (vi) a new role for the project manager.

In summary, this chapter provides an overview of the ACT program types and related program characteristics along with program risks and describes the acquisition life cycle from both government and contractor perspectives. The chapter also describes (i) the government recommended ANSI/EIA Standard 748 for the implementation of EVMS framework, procedures, processes, and tools, and (ii) the trends for integrating ML-AI TES into the planning and executing of the PPM plan. In addition, the chapter recommends an innovative approach to quantify the technology and market risks, and a tailored Zachman framework that can be used to enhance the efficiency of a PPM plan. A set of desired PPM activities is also recommended for balancing cost, technical and program management risks. Finally, the chapter discusses the applicability of EVMS to ACT Programs and recommends a simplified EVMS report should be provided for contract value between 10M and 20M USD. The simplified report with Format 1 data analysis is required to capture cost and schedule performance data by a specified WBS along with a narrative report to capture the required performance analysis.

Acknowledgments

The author wants to thank his wife, Thu-Hang Nguyen, for her tremendous patience and support during the preparation and writing of this chapter.

Conflict of interest

The author declares no conflict of interest. The opinions expressed in this chapter are those of the author and they are not endorsed by CSUF nor Aerospace Corporation.