Program Management Integrated with Data and Decision Sciences

Abstract

Program management (PM) complexity depends on the budget size and program types. In general, the program types can be classified into three categories, namely, defense, commercial, and civilian types. This chapter presents and discusses an approach for integrating the PM discipline areas with emerging data science and decision science1 (DDS) for any program type. Additionally, we describe the key PM areas and present a corresponding generalized model consists of a list of multiple PM discipline areas that can be tailored for any program types. To demonstrate the PM-DDS integration approach, we focus on three key PM areas and corresponding PM discipline areas related to schedule, cost, and risk management. These three discipline areas are analyzed to identify appropriate program elements that can be enhanced using existing DDS technology enablers (TEs). We also propose a flexible PM-DSS integration framework by leveraging existing machine learning operations (MLOps) framework. The proposed integration framework is expected to allow for enhancing the program planning and execution by reducing the program risk using a wide range of DDS TEs, including big data analytics, artificial intelligence, machine learning, deep learning, neural networks, and artificial intelligent.

1. Introduction

Traditionally, program management (PM) usually addresses a group of several related projects that are meant to achieve an organization’s goals and business objectives when integrated them together. A project management is usually deals with a single short-term period of performance (PoP) focused on specific objective(s) and related delivery schedules, quality, and cost controls. In contrast, PM deals with a much longer-term PoP with an emphasis on the integration of all the short-term projects to achieve an overall benefit to the organization. In another word, PM addresses the outcomes of all deliverables obtained from a group of short-term projects.

PM for a sizable budget program (i.e., above 50 Mil USD) regardless of the program type (i.e., defense vs. commercial vs. civilian) is a complex task. As an example, it is usually involved with the nine basic PM areas [1], including (i) managing enterprise, organizational, and program goals, (ii) managing program financial goals, (iii) managing program risk (a.k.a. risk management), (iv) managing program schedule (a.k.a. Schedule Management), (v) managing technical/product performance, (vi) developing and managing program team, (vii) managing performance and quality assurance (QA), (viii) managing internal and external communications, and (ix) managing program integration. These PM areas can be tailored to any specific acquisition system such as DoD acquisition system [2, 3], NASA acquisition system [4], acquisition of commercial products and commercial services for US government agencies [5, 6, 7], and commercial procurement process for commercial systems acquisition for private companies [8]. Section 3 provides detailed description of the key PM areas. For defense and civilian acquisition systems, such as US Department of Defense (DoD) and NASA, a program manager can decompose these nine PM areas into at least 13 PM discipline areas [1, 2, 3, 4, 5, 6, 7], consisting of (i) system engineering related to the system being acquired; (ii) contracts and legal dealing with contractors, suppliers, and stakeholders; (iii) financial and cost management; (iv) schedule management; (v) system test and evaluation, (vi) logistics and supply chain management, (vii) production, quality, and manufacturing (PQM) management, (viii) program risk management, (ix) intelligence and security management, (x) software management, (xi) business and marketing practices, (xii) configuration management, and (xiii) information technology management. The program manager must have a good understanding of these discipline areas and integrate them to manage them and successfully execute the overall program. In the DoD and NASA, the program manager has the authority to accomplish program objectives for the development, production, and sustainment of systems to meet the user’s operational needs and is accountable to the acquisition decision authorities. Section 3 provides a generic model with a comprehensive list of 19 PM discipline areas that can tailored to fit any program types.

The main objective of this chapter is to present an innovative approach for integrating PM with emerging DDS technology enablers² (TEs) for improving the program execution and management of any program types. This approach is referred to as the PM-DDS integration throughout this chapter. The approach identifies the five key PM areas that are important to any program managers and a generalized approach to decompose these areas into multiple discipline areas and conducts an analysis of these (discipline) areas for PM-DDS integration. The goal of the analysis is to identify the discipline areas that a program manager can leverage DDS TEs to enhance the overall program planning, execution, and risk reduction. For each area, we will discuss potential ways in which DDS TEs can be used to support the program manager and his team in managing and executing the project more effectively. In addition, the chapter also discusses a simplified, flexible, and adaptable MLOps framework that can help any program managers to identify the desired program discipline areas and related DDS tools to support his program from the start to the end of the program. To limit the scope of work, the chapter only focuses on (i) the program management for acquiring a system, or a product, or a service, and (ii) five key PM areas, namely, program goals, schedule, cost, risk, and technical performance management.

The chapter is organized as follows: (i) Section 2 presents our innovative approach for PM-DDS integration; (ii) Section 3 provides a description of the nine key PM areas; (iii) Section 4 discusses a generalized approach on the decomposition of the multiple discipline areas and provides the decomposed discipline areas associated with the PM areas discussed in Section 3; (iv) Section 5 analyzes and selects a set of discipline areas for applying DDS; (v) Section 6 aligns the selected discipline areas with an appropriate DDS TE and provides some examples to demonstrate how the selected DDS TE can improve the program planning and/or reduce program risk; (vi) Section 7 describes our proposed simple, flexible, and adaptable MLOps framework for use by any program managers; and (vii) the chapter concludes with a summary and proposed way forward.

2. Proposed innovative approach for PM-DDS integration

Our proposed innovative PM-DDS integration approach includes a six-step approach as shown in Figure 1. These steps describe how any program manager, regardless of program types, can identify which PM discipline areas can leverage the emerging DDS TEs to improve the execution and management of their programs. A description of these steps is provided below.

Figure 1.

Proposed PM-DDS integration approach.

Step I: This step leverages existing PMBOK® Guide, NASA PM Guide and Processes, and DOD Guide for Program Managers to identify a set of generic PM areas that are the most important to any program managers. This set of PM areas is also referred to as the key PM areas that can be used for any program types. The detailed description of these key PM areas is provided in Section 3.

Step II: This step discusses a generalized approach to decompose PM areas into multiple discipline areas that any program manager is required to manage throughout the various phases of their program. The generalized decomposition approach can be tailored to any program types. Section 4 provides a detailed description of generalized approach and corresponding PM areas decomposition results.

Step III: To gain a deep understanding of existing DDS technologies, we have conducted a survey on the emerging DDS TEs and their applications on program management. Step III leverages the survey results and our own experience to perform the analysis of the decomposed discipline areas obtained from Step II above. The analysis helps us to select a set of discipline areas that can benefit from the integration of existing DDS TEs for improving the program planning and reduce overall program risk. As indicated in Section 1, the scope of work for this chapter is limited to the five key PM areas, including program goals, schedule, cost, risk, and technical performance management. We will focus our analysis on these five PM areas and related PM discipline areas decomposed from these five areas. Section 5 describes the analysis results on the selected set of discipline areas that can be beneficial from the PM-DDS integration.

Step IV: For each selected discipline area and/or a group of selected discipline areas, Step IV identifies corresponding DDS TE and/or a group of integrated TEs, respectively. The goal of this step is to align each selected discipline areas and/or a group of selected discipline areas with a specific DDS TE or a group of DDS TEs, respectively. The alignment will help us to identify which selected discipline area and/or a group of selected discipline areas can be beneficial by integrating DDS TEs for improved program planning and/or program risk reduction. In practice, this step is the most important step because it helps the program managers to address the question on the integration of DDS technology for enhancing the program execution and effectively reducing the overall program risk. Section 6 provides a summary of the survey results on existing DDS TEs, consisting of big data analytics (BDA), artificial intelligence (AI), machine learning (ML), deep learning, and neural networks. Additionally, Section 6 describes the alignment of the selected discipline areas with specific DDS TEs and/or a group of DDS TEs. Some examples are also provided in Section 6 to demonstrate the use of DDS TEs for improving PM execution and planning.

Step V: Leverages the above four steps and existing MLOps framework and processes to develop a simplified, flexible, and adaptable MLOps framework that can help any program managers to identify the desired program discipline areas and related DDS tools to support his program planning and execution from the start of his program. Section 7 describes the proposed MLOps framework.

3. Key PM areas identification

From our experience and review [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], as pointed out in Section 1, the PM areas for any program types that are the most important to any managers can be classified into nine key areas. These nine key areas can be generalized and organized as nine PM areas. Followings are a brief description of these nine key PM areas. Based on the description, this section provides a set of recommendations for the PM areas that should be beneficial from PM-DDS integration.

1. Enterprise, Organizational, and Program Goals Management: Program goals are usually derived from each organization’s goals within an enterprise’s overall strategic goals. The program goals should be clearly defined and provide obvious direction for the program team members to follow. The program’s goal may be to fix a problem or meet a need among customers (internal or external) or to provide a defense system to the warfighters. The program goals should not focus on fixing a human resource problem in an organization within the enterprise. Managing program goals require the program managers to look at each of the goal within the overall goals as an individual project that they need to execute and manage. For each of these goals (or projects), they need to decompose into sub-goals or project tasks that they need to accomplish to reach that program goal. In the context of program goals management, the program managers can consider each of these goals as an individual project objective and they need to achieve the overall program goals by successfully integrating these individual project objectives. To effectively achieve the program goals, the program managers need to clearly define the key performance indicators (KPIs) for the overall program and related projects. They need to track the projects’ KPIs progress and associated program’s KPIs to achieve the program goals [9]. It should be mentioned here that a project is usually focused on the development of a unique product, or a service and it has a short period of performance (PoP), while program is usually a much longer PoP that focuses on the integration of the outcomes of each individual projects to create a defense system that meets the overall program benefit to users, or goals or an enterprise service achieving overall program benefit to users.

2. Overall Financial and Program Cost Planning and Management: The program managers are responsible for planning and managing the overall program finances to ensure that they achieve their budget and program goals. One of the most important financial planning and management issues is the cost planning and management [1, 2, 3, 4, 5, 11]. The objective of the program cost planning is to estimate the costs and allocate required budget to the key program’s products, services, and tasks defined in the Work Breakdown Structure (WBS), Integrated Master Plan (IMP), and Integrated Master Schedule (IMS). For examples, the products can be a satellite system and related ground tracking subsystems or a residential building. The cost planning activities allow program managers to know where the money will be spent, and on what products, services, and tasks, as well as when those expenditures will occur. The budget planning allows them to know the limit of expenditure for each activity.

3. Overall Program Risk Management: In general, overall program risk is associated with a measure of future uncertainties related to all program activities preventing the program managers to achieve the program performance goals (or program KPIs), requirements, and objectives (or project KPIs) within defined cost, schedule, and performance constraints. Overall program risk can be associated with all aspects of a program, ranging from program team member’s safety issues to actual operational environment, as these aspects are linked to the tasks addressed in the WBS, IMP, and IMS. Program risk management requires the program managers to identify potential risks across the program and quantify these risks to assess and track the potential risk variation in the planned approach and its expected outcome [1, 2, 3, 4, 5, 6, 7]. It should be mentioned that the program risk can be classified into two categories, namely, (i) the technical risk associated with the technical requirements associated with a system or a product or a service, and (ii) non-technical risk associated with human resources, supplier, safety, etc.

4. Overall Program Schedule Planning and Management: The program managers are required to create a WBS and an associated program schedule management plan [1, 2, 3, 4, 5, 6, 7]. The overall program schedule plan captures the program start and end dates, program milestones, all individual projects and associated tasks identified in the WBS, timeline for completing individual tasks and related durations, resources for each task, identified predecessors, and dependencies for each task. The objective of the overall program schedule plan is to show the detail of how each individual projects and associated tasks are grouped together to achieve the overall program goals. The plan supports the program managers to effectively execute and manage their program activities through the program life cycle. For examples, some of the benefits of the plan include [1, 2]: (i) providing the basis for effective communications within the government team or stakeholders and with contractors or suppliers, (ii) identifying a baseline for overall program status monitoring, reporting, and program control, (iii) facilitating program management, and (iv) providing the basis for resource analysis and resource leveling, exploration of alternatives, and cost/time tradeoff studies.

5. Technical Performance Management: One of the key challenges to the program managers is to identify the technical risk associated with the system or a product performance that is being acquired by the program. This technical risk is related to the level of uncertainty associated with the performance requirements for the system or product. The level of uncertainty can be quantified in terms of the technology readiness level (TRL) and/or manufacturing readiness level (MRL). The higher the TRL/MRL, the less technical risk associated with the system or product being delivered by the contractor or a supplier to the program. For civilian and commercial programs using commercial of the shelf (COTS) products or services, the TRL and MRL are usually very high, and the risk is very low. But, for advanced development programs, the technical risk is very high, and the program managers are required to develop a technical risk management to manage and track the risk throughout the program phases.

6. Quality Assurance (QA) Management: In the context of a program that is intended for acquiring a system/product/service, the QA management involves with the approach and process to control and manage QA of the hardware and software products of a system/product being delivered by a contractor to the program. The program managers are required to (i) develop a QA management plan to address the required standardized QA models and related national and/or international standards, and (ii) create QA process for verifying and validating the quality of the delivered systems/products meeting national and international QA standards. As an example, the ISO/IEC 17025 model and related standards addressed general requirements for the competence of testing and calibration laboratories, which is the main ISO/IEC standard used by ISO certified testing and calibration laboratories [12]. This PM area is not the focus on this study and will not be addressed in the remaining sections.

7. Program Team Forming and Program Team Management: The program managers are responsible for developing, forming, and acquiring the program team of individuals who can carry out each individual project and related tasks [1, 2, 3, 4, 5, 6, 13]. The organization of the team can be based on individual project structure with project managers to handle projects’ objectives. In the US DOD, the team can also be organized into Integrated Product Team (IPT), where IPT team leaders are responsible for delivering the required products/subsystems. Just as the project team, this IPT team also has their own set of objectives, roles, and responsibilities, which will be aligned to the overall program objectives. This PM area will not be addressed in the remaining sections.

8. Internal and External Program Team Communications Management: Internal program team communications address the information, data, and ideas exchange within the program team members, project managers, IPT leads, and program manager. The internal communications allow everyone to keep track of their project and associated tasks progress and help the individual project manager and the IPT leads to address technical issues and problems timely. The internal communications also help the program manager to anticipate and mitigate program issues and problems before they occur. External communications with stakeholders, contractors, suppliers, and media are beneficial to program managers. Managing the external communications is very important to the program managers in terms of managing the stakeholders’ expectation, contractor’s work on achieving the system’s/product’s qualities, and media’s expectation on achieving the overall program goals on time. This PM area is not in the interest of this chapter, and it will not be addressed in the subsequent sections.

9. Program Integration Management: To achieve the overall program goals, the program managers are responsible for program integration that is required to integrate all the projects under their programs. The integration is required to be performed at the individual project integration level. At this level, the project manager coordinates tasks, resources, stakeholders, and any other project elements, in addition to managing conflicts between different aspects of a project, making trade-offs between competing requests, and evaluating resources. Integrated program management ensures related individual projects are not managed in isolation. This PM area is also not in the interest of this chapter, and it will not be addressed in the subsequent sections.

As mentioned in Section 1, due to page constraint and our focus on the application of DDS technology to program management, this chapter focuses on the four key PM areas that can receive the most benefits from DDS, including (i) program goals, (ii) schedule, (iii) cost, and (iv) risk management. These four PM areas are defined in the bullets above as i, ii, iii, and iv, which correspond to: (i) Enterprise, Organizational, and Program Goals Management using commonly used program KPIs, (ii) Overall Program cost estimate and cost management, (iii) Overall Program risk management, and (iv) Overall Schedule planning and management. Subsequent sections focus on the decomposition of these four PM areas into multiple discipline PM areas for PM-DDS integration.

4. PM area decomposition to multiple discipline areas

In practice, a program manager is the title that is assigned to an individual who is responsible for managing the nine PM areas described in Section 3. The program manager must have the knowledge and a good understanding of the required multiple discipline areas associated with the nine PM areas to successfully execute the overall program. The program manager has the full authority to achieve specific program objectives from the development phase to the sustainment phase. For the US DOD defense programs, the program manager is accountable to the Milestone Decision Authority (MDA). Based on our experience working on NASA, US DOD, commercial programs, and our review of the multiple PM discipline areas associated with the nine key PM areas [1, 2, 3, 4, 5, 6, 7, 8, 9], we propose a generalized model for the decomposition of the above nine PM areas into a set of multiple discipline areas. The program manager must fully understand these multiple PM discipline areas to effectively execute the program from the start to the end of the program. Below is a proposed generalized model consisting of 19 PM discipline areas, including:

1. Program goal management,

2. Systems engineering related to the systems/products/services being acquired,

3. Specialized engineering related to the products and services being acquired,

4. Contracts and legal dealing with contractors, suppliers, and stakeholders,

5. Program Financial management,

6. Business and marketing practices for the newly acquired systems/products/services,

7. System/product/service technical requirements and associated performance risk management,

8. System/product/service cost planning and management,

9. Program schedule planning and management,

10. Program cost planning and management,

11. System/product/service³ risk planning and management,

12. Program risk planning and management,

13. System test and evaluation,

14. Logistics and supply chain management,

15. Production, Quality, and Manufacturing (PQM),

16. Program and system intelligence and security management,

17. Program and system software management,

18. Program and system configuration management,

19. Program and system information technology, and

20. Other Specialty Program Planning and Management.

The above 20 PM discipline areas that can be tailored to fit with any program areas and types. Below is a list of four key PM areas and associated PM discipline areas:

• Program Area 1—Enterprise, Organizational, and Program Goals Management using Commonly Used KPIs: PM discipline area associated with this program area is:

• Program Goals Management: For this PM discipline area, let us assume that the program goals are to (i) Meet program budget on time, (ii) Acquire the system/products/service within the specified PoP with specified budget, and (iii) Meet technical performance requirements with acceptable risk. To manage these goals, we want to select DDS frameworks, processes, and tools to integrate them into existing program management processes and tools to support the program manager. These assumptions lead to an important question concerning program management: How the program manager can track and control the three key program areas, namely, cost, risk, and schedule, effectively using the PM-DDS planning processes and tools? Based on our experience and investigation of the existing program management frameworks, the system technical requirements and associated performance risk, cost planning, and risk management are intertwined, and they are the key factors to manage the overall program cost, risk, and schedule effectively. The subsequent sections will address the PM discipline areas related to these three program areas.

• Program Area 2—Overall Program Cost Estimate and Cost Management: PM discipline areas associated with this Program Area 2:

• Program Cost Planning and Management: This PM discipline area is a focus of Section 4.1.

• System (Product/Service) Cost Planning and Management: This PM discipline area covers the System Technical Requirements and associated cost planning and management. This PM discipline area is a lower level than the program cost planning and management and will not be covered in this chapter.

• Program Area 3: Overall schedule planning and management.

• Program schedule planning and management: This PM discipline area is a focus of Section 4.2.

• Program Area 4: Overall program risk management: PM discipline area associated with this program area is as follows:

• Program risk planning and management: This is PM discipline area the focus of Section 4.3.

• System (product/service) risk planning and management: Like PM discipline area 2, This PM discipline area covers the System Technical Requirements and associated risk planning and management. This PM discipline area is a lower level than the program risk planning and management and will not be covered in this chapter.

The following subsections provide detailed description of the above three key PM discipline areas related to program cost, risk, and schedule.

4.1 Program cost planning and management discipline area

Cost planning and management for a product being acquired by a program is critical for the success of the program, especially during the pre-acquisition phase, i.e., planning phase. The cost of a system and its risk depend on the technical requirements. The more uncertainty associated with the technical requirements, the more cost risk. This is especially true for acquiring an advanced state of the art system or when the program management team is not sure about the technical requirements on a specific system they are planning to acquire. In the following section, we discuss this challenge and identify existing DDS TE that can address it.

4.2 Program schedule planning and management discipline area

In Section 3, the program schedule planning provides a program schedule plan captures the program start and end dates for all activities defined in the WBS. The program activities include program milestones, individual projects with associated tasks, timeline for completing individual tasks, related durations, resources for each task, identified predecessors, and dependencies for each task. Based on our review of the existing schedule plan, development and management discipline area includes five steps, namely program activity definition (described in the WBS), activity sequencing, activity duration estimate, schedule development, and schedule control [1, 2, 3, 4, 5, 6, 7, 8]. Figure 2 captures these five steps of the schedule plan development and management, and their detailed descriptions are provided below.

Figure 2.

Five steps for schedule planning and management.

Step 1—Program Activity definition: This step identifies required activities specified in the WBS. This definition step also defines all WBS activities that must be accomplished to achieve the objectives of the overall program. The output of this step includes (i) a list of activities with a complete description of each of the activities and they are linked to the WBS. The list contains supporting details for each activity, including assumptions and constraints.

Step 2—Activity Sequencing: This step identifies the constraints and relationships among activities. It also defines the priority of the activities and the order of the tasks without causing bottleneck from one activity to the other. To determine the order, this step requires several inputs, including (i) activity list developed in Step 1, (ii) required constraints and related dependencies, discretionary constraints and related dependencies developed by the program management team based on “best practices” or specific sequences desired by management, (iii) external dependencies, and (iv) other constraints and assumptions. For instance, the required constraints and related dependencies can be a prototype must be fabricated before it can be tested, and external dependencies can be the availability of test sites.

Step 3—Activity Time Duration Estimate: It provides an estimate of the time duration required to complete the activities that make up the program. This is an important task that required SMEs who are most familiar with the activity to provide the estimates. At a minimum, there are two key required inputs to this step, namely, (i) the resources required and assigned for the activity, and (ii) the capabilities of the resources assigned. For improving the estimate, this step leverages historical information and lessons earned from past and similar programs and from commercial databases. In practice, the output of this step provides an estimate of the likely time duration to complete each activity. The estimates should include the mean values of the time duration estimate and 1-sigma value around the mean value, for instance, 1 month ±1 week, and corresponding assumptions made in the estimated time durations.

Step 4—Schedule Development: From the estimated time duration obtained in Step 3, this step develops realistic start and finish dates for each activity based on the specified program PoP. The schedule development process is an iterative process considering Step 2—activity sequencing, and Step 3—activity time duration estimates along with resource requirements and availability to display when the activities can be executed, constraints, assumptions, and associated risk. This step provides a set of schedules and associated information for the program, including (i) the IMS and the supporting detailed schedules, and (ii) the best balance possible between competing demands of time and resources. The schedules also consider the risk associated with time, cost, and performance tradeoffs and the impact on the overall program.

Step 5—Schedule Control: This step identifies potential schedule variations and manages actual changes to the developed schedules. The schedule change control system provides a well-defined procedure by which changes can be made and automatically be integrated into the program. The schedule change control system also provides mechanisms for (i) schedule performance tracking, and (ii) the approving and authorizing the required changes. Note that schedule changes come from various factors, including failure to achieve planned dates for specific activities, delayed tests, late delivery of required prototypes, internal program management assessment and replanning, and external direction, such as reallocation of funding.

4.3 Program risk management discipline area

As discussed above, program risk discipline required the program managers to define an approach to measure the future uncertainties in achieving overall program performance goals, requirements, and objectives within defined program cost, schedule, and performance constraints. More specifically, program managers need to address risk associated with cyber security threats, human safety, program safety, system risk and safety, technology maturity level (TRL), supplier capability, supply chain management, system design maturation, manufacturing maturity level (MRL), and performance against plan. These program and system risks are usually associated with the program tasks described in the WBS, IMS, and IMP. The program and system risks address the potential variations in the program planned approach and its expected outcomes [1, 2, 3, 4, 5, 6, 7]. The US DOD risk management framework is described in Ref. [10]. In general, the risk management process is shown in Figure 3 below.

Figure 3.

Program risk management process.

As shown in Figure 3, the risk management process consists of five key steps, namely, risk identification, risk analysis, risk mitigation planning, risk mitigation plan implementation, and risk tracking. The followings describe these five key risk management activities in detail.

Step 1—Risk Identification: This activity identifies program risks throughout the program life. The risk identification process includes the nine following steps:

• Step 1: Risk program meets with project managers and Integrated Product Team (IPT) to identify a list of potential risk items. There are various methods of identifying risks, including (i) lessons learned, (ii) SMEs, (iii) prior experiences, (iii) TRL determination, (iv) MRL determination, (v) programmatic constraints, (vi) brain storming, and (vii) WBS;

• Step 2: Risks are determined to be acceptable or not by the risk team. For a big program, the risk team usually consists of technical SMEs, risk Integrated Product Team (IPT) managed by the risk manager, project managers, and program manager. Note that all risk items identified in Step 1 above are not necessarily accepted by the program;

• Step 4: Only accepted risks should be recorded and placed into a risk register;

• Step 5: The risk team identifies root causes for each identified risk;

• Step 5: Risk analysis should examine each identified risk to refine the description of the risk, isolate the cause, determine the effects, and aid in setting risk mitigation priorities (Risk Reporting Matrix);

• Step 6: Risk Mitigation Planning should address each risk with action items and due dates.

• Step 7: The risk team meets regularly to (i) assess risks to determine if the risks are burnt down to acceptable levels, and (ii) add new risk items, if necessary.

• Step 8: Identified risks are closed when the risks are burnt down to acceptable levels. In practice, some risk items can be closed quickly; others can be opened until near the end of the program; and some are considered watch items with a pre-planned mitigation plan that only kicks in when a pre-defined negative event occurs.

• Step 9: Document closed risks in the database for lessons learned.

Step 2—Risk Analysis: This activity analyzes each identified risk to ensure the risk description is accurate, isolate the root cause, determine the effects, set risk and associated mitigation priorities. The analysis refines each risk item in terms of its likelihood, its consequence, and its relationship to other risk areas or processes.

Step 3—Risk Mitigation Planning: The objective of the risk mitigation is to reduce or eliminate the impact of risks on a program. The risk mitigation plan (RMP) activity identifies, evaluates, selects, and implements mitigation options to bring the identified risk from unacceptable levels to acceptable levels given program constraints and objectives. The RMP activity also provides detailed description of what mitigation technique should be used, when the risk mitigation should be accomplished, who is responsible for bringing the risk to acceptable levels, and associated cost and schedule. In general, the RMP strategy can be chosen from the four mitigation options, namely, risk avoidance (RAV), risk controlling (RCO), risk transfer and sharing (RTaS), and risk assumptions (RAS) [1, 10].

RAV approach is used when there is alternative activity that can be used for achieving the same outcome of the task without carrying the identified risk. This technique requires the risk team to reconfigure the project such that the identified risk in question disappears or is reduced to an acceptable level. RA approach is recommended when the risk team can control the identified risk by managing the root cause and/or related consequence. RCO approach can leverage the risk database along with a warning system that can provide required warning signs to assess more accurately the impact, likelihood, or timing of a risk. RTaS approach is preferred when the risk team can share the identified risk with a third party like a supplier or subcontractor or an insurance company. RAS approach is recommended as a mitigation strategy by the risk team when the identified risks are small risks. The small risk is defined as the risk that when it occurs the cost of insuring against the risk would be greater over time than the total losses sustained. The RAS strategy accepts the loss, or benefit of gain, from the identified risk when it occurs.

Step 4—Risk Mitigation Plan Implementation: The risk team is responsible for developing and implementing the RMP. The plan ensures successful risk mitigation occurs and the timing for the burnt down risks is based upon the RMP. In practice, the implementation plan (i) determines what planning and associated budget and requirements along with contractual changes are required to burn down the risks, (ii) provides a plan for coordination between program management team and all stakeholders, (iii) directs the program teams to execute the defined and approved RMP, (iv) provides a summary of the registered risk reporting requirements for on-going monitoring, and (v) documents the change history.

One of the key activities in the implementation step is the risk assessment activity. The risk assessment activity is performed by the risk team to identify and analyze the risk by its category. In practice, the key risk categories include performance, schedule, and cost risks. Thus, it is essential that the RMP implementation approach should also be accomplished by risk category, and it is important for this process to be worked through the risk IPT structure and and/or projects’ risk structure.

Step 5—Risk tracking: The risk tracking is also known as risk monitoring. It is defined as an activity that can track and evaluate the performance of risk mitigation actions against established metrics throughout the pre- and post-acquisition process. This objective of this activity is to (i) evaluate the performance of RMP actions against a pre-defined metrics, and (ii) execute the RMP or develop further risk mitigation choices, as appropriate. The results obtained from this activity provide required information on how the risks are burnt down ensuring the success of the RMP. The objective of the risk tracking activity is to ensure that the program team to (i) communicate risks status to all affected Stakeholders, (ii) monitor RMP, (iii) review RMP status updates, (iv) display RMP dynamics, (v) track RMP status within the risk reporting matrix, and (vi) alert management as to when RMP should be implemented or adjusted.

5. PM discipline areas analysis for DDS integration

This section provides a summary of our survey results and our experience on the analytical and simulation tools to support the three PM discipline areas discussed in subsections 4.1 (program cost management), 4.2 (program schedule management), and 4.3 (program risk management) above. The objective of this section is to (i) analyze these three PM discipline areas and the identified supporting tools, and (ii) select a set of the activities within each of the PM discipline areas that can benefit from the integration of existing DDS TEs. The objective of the PM-DDS integration is to improve the efficiency of the program planning and reduce the overall program risk for achieving the cost, schedule, and technical performance. The following subsections focus on the analysis of PM-DDS integration for program cost, schedule, and risk management discipline areas.

5.1 DDS integration with program cost management

Based on our survey of existing cost tools, the available cost tools implemented the four commonly used cost-estimating techniques [2, 5, 7, 14], including (i) Analogy technique that based on historical data for an analogous product or system or subsystem; (ii) Engineering Build-up technique, where a system or a product is broken into lower-level components (e.g., individual parts or assemblies), each of which is costed separately for direct labor, direct material, and other costs; (iii) Parametric technique that uses regression or other statistical methods to estimate the cost and its relationship between historical cost of a system and a product; and (iv) Actual cost estimation technique that leverages actual cost experience or trends from prototypes, engineering development models, and/or early production items used to project estimates of future costs for the same system. These cost tools can provide cost estimate with associated confidence level. The confidence level specification helps the program managers to understand the likelihood that actual costs would fall below estimated costs. This means that the greater the confidence level, the higher the estimated cost. Note that in the US DOD, it is mandatory for the program managers to conduct an independent cost estimate that is performed by different organization that is not part of the program organization. Some of the cost tools available are as follows:

• SEER: is an estimating cost tool used by civilian, commercial, and defense programs to generate independent cost estimates for manufacturing, sanity checks, and the analysis of contractor cost estimates [15].

• aPriori Cost Estimating Software Tools: aPriori provides a set of tools to generate manufacturing cost models for setting accurate cost targets accurately and timely. The tools also can provide the estimate model procurement costs for new designs without waiting for supplier quotes [16].

• DOD COCOMO Software: COCOMO software is a Constructive Cost Model consisting of a suite of tools focused on software cost estimation that was originally published in 1981 [17, 18, 19]. The COCOMO software tool is specifically designed for DOD defense program but can be extended to civilian and commercial applications.

For acquiring advanced systems with high uncertainty associated with the technical requirements, the cost and schedule estimates of the proposed system during the pre-acquisition phase become a challenge for the system design team, cost team, and risk management team. These three teams need to develop an optimal system solution based on multiple design criteria, including market uncertainty, technological uncertainty, technical risk, performance risk, cost risk, and schedule risk. The program manager needs to come up with a payoff-and-cost function that can balance out the performance, cost, and schedule risks with the market uncertainty and technological uncertainty. This is a multi-criteria decision problem that requires the designer to come up with a satisfactory and safe decision. Recently, a war-gaming concept using game theory was proposed to analyze alternative system solutions by playing out the Government’s acquisition objectives against the Contractor’s bidding motivations [20, 21]. As pointed out in Ref. [22], the game scenarios simulating various system solutions sometimes lead to conflicting and non-converging solutions. An advanced multiple-criteria decision mathematical model also proposed in Ref. [22]. This model employed the ELECTRE II model to resolve the non-convergence game scenarios encountered in the war-gaming model. Thus, the ELECTRE II model described in [22] when combined with proposed Advanced Game-based Mathematical Framework (AGMF), Unified Game-based Acquisition Framework (UGAF), and a set of War-Gaming Engines (WGEs) described [20] can address the cost estimate for an advanced system with low level of TRL (i.e., high technical requirements uncertainty). The recommended PM-DDS integration approach for cost planning includes big data analytics (BDA) approach with BDA data acquisition and data curation TEs, and artificial intelligent and machine learning (AI-ML) TEs. AI-ML TEs include (i) data mining techniques and tools (DMTT), (ii) data exploitation using multi-objective reinforce learning and adaptive neural network (MORL-ANN) tool, and (iii) predictive analytics techniques using MORL-ANN tool. For cost management, the recommended PM-DDS integration approach for performing the cost management includes three key components, BDA approach with BDA TEs listed above, AI-ML TEs listed above, and the Earn Value Management System (EVMS) to track and manage the cost [23, 24].

5.2 DDS integration with program schedule management

This section analyzes and discusses the five program schedule activity steps defined in Section 4.2.

Step 1—Program Activity Analysis: The techniques commonly used in activity definition are as follows: (i) decomposition process involved the successive breakdown of program elements into smaller, more manageable components, which eventually described the activities to be scheduled. This technique is essentially the same as the one used in the WBS development; and (ii) a template process that is an activity list or WBS element from another similar program that can serve as a model for the current program and provide a starting point for defining specific activities. Based on our current analysis of the existing techniques used for this Step 1 activity, it is difficult to integrate existing technique and tools with the current DDS technology and associated TEs.

Step 2—Activity Sequencing Analysis: Step 2 activity has been using several techniques and tools to develop the logic diagrams reflected the desired activity sequencing. Existing network scheduling techniques and tools include (i) Critical Chain Method (CCM), (ii) Critical Path Method (CPM), (iii) Precedence Diagram Method (PDM), and (iv) Program Evaluation and Review Technique (PERT) [1, 2, 3, 4, 5, 6, 7, 25, 26, 27]. The recommended PM-DDS integration approach for conducting the activity sequencing estimate includes three key components consisting of BDA TEs, AI-ML TEs, and existing activity sequencing tools (i.e., CCM, CPM, PDM, and PERT). The recommended integration approach is expected to improve the program planning efficiency.

Step 3—Activity Duration Estimate Analysis: The following techniques are commonly used in estimating activity durations: (i) Expert judgment guided by historical information, (ii) Analogous estimating based on the experience of similar programs, (iii) Parametric estimating based on formulas describing relationships among program parameters and time, and (iv) Use of simulation to develop distributions of the probable duration of each activity. Like the above Step 2 analysis, the recommended PM-DDS integration approach for conducting the activity time duration estimate also includes three key components consisting of BDA TEs, AI-ML TEs, and existing four activity duration estimate techniques described above. This recommended integration approach is also expected to enhance the program planning efficiency.

Step 4—Schedule Development Analysis: Several techniques and related tools are useful to developing schedules. These tools contain the capability to perform mathematical analyses calculating theoretical start and finish dates for each activity based on the overall sequencing of the program activities. Two of the more commonly known analysis techniques and related tools are: (i) CPM and (ii) PERT. Other scheduling development techniques and related tools that are also available to generate schedule plan using resource constraints, such as time, human resource, budget, and material. These tools provide another avenue to manage the effect of these constraints. A few of these techniques and related tools are schedule compression, and resource leveling. Like the above Steps 2 and 3 analyses, the recommended PM-DDS integration approach for generating a schedule plan also includes three key components consisting of BDA TEs, AI-ML TEs, and existing schedule development techniques and tools described above. This recommended integration approach when combined with the above integration approaches is also expected to increase the overall program planning efficiency.

Step 5—Schedule Control Analysis: The analysis of Step 5 on schedule control and management showed that the Line of Balance (LOB) process and related tools are currently used by program managers to manage the overall program control process [28, 29]. The LOB process and tools are used to collect program data, measure, and display the actual program status related to timing, phasing of the project activities, cost, related background, and accomplishments measured against a specific program management plan. The displayed results provide program management team desired program information that helps the team to (i) compare actual progress with a formal objective program plan, (ii) examine the deviations from the established plans and evaluate their degree of severity with respect to the remainder of the project, (iii) receive timely program information concerning potential trouble areas and indicate the areas that required immediate corrective action, and (iv) forecast future program performance. The recommended PM-DDS integration approach for schedule control and management also includes three key components consisting of BDA TEs, AI-ML TEs, and existing LOB tool. This recommended integration approach is expected to reduce the overall schedule risk and enhance the program execution by identifying and correcting potential trouble areas before they occur.

Figure 4 illustrates the proposed PM-DDS integration approach. This figure shows that the program schedule management can be integrated with existing DDS technology enablers for enhancing program planning and execution for risk reduction.

Figure 4.

PM-DDS integration approach for schedule planning and management.

5.3 DDS integration with program risk management

This section analyzes and discusses five program schedule activity steps defined in Section 4.2.

Step 1—Risk Identification Analysis: Our analysis shows that the commonly used techniques and tools for risk identification include (i) objectives-based risk identification (OBR-ID), (ii) scenario-based risk identification (SBR-ID), (iii) taxonomy-based risk identification (TBR-ID), and (iv) common-risk checking (CRC). The OBR-ID technique and related tools identify the risk associated with the objectives defined by organizations and project teams [1, 2, 3, 4, 5, 6, 7]. Any event that may endanger achieving an objective partly or fully is identified as risk. SBR-ID technique and related tool perform scenario analysis using different pre-defined scenarios. The scenarios may be the alternative ways to achieve a pre-defined objective, or an investigation of the interaction of external forces in, for example, a market or battle. An undesired scenario that may occur in the future is identified as a risk, and any event that may trigger it is considered a risk trigger.

The TBR-ID technique and related tools are used to breakdown possible risk sources. Leveraging the taxonomy and knowledge of best practices, a set of questionnaires is compiled for review by SMEs. The answers to the questions reveal potential risks and are compiled in the program risk registry. CRC tools can leverage industry databases to provide lists of known risks associated with known activities, products, program elements. Each risk in the program risk registry can be checked for application to a specific situation.

The recommended PM-DDS integration approach for conducting the risk identification activity includes three key components consisting of risk databases with data acquisition and data curation TEs, AI-ML TEs, and existing risk identification techniques and related tools. The recommended AI-ML TEs include (i) DMTT, (ii) data exploitation using MORL-ANN tool, and (iii) predictive analytics techniques using MORL-ANN tool. This recommended integration approach is expected to improve the program management planning by reducing the uncertainty associated with the risk identification process.

Step 2—Analysis of Risk Analysis Activity: Based on our analysis of Step 2 on the risk analysis activity, the current risk analysis processes and related tools are focused on (i) system performance risk analysis, (ii) schedule risk analysis, and (iii) cost risk analysis [30]. The output of these processes and related tools consists of (i) assigned likelihood (probability of occurrence) and related consequence (the environmental impact if a risk event occurs) results to each risk using the criteria in the risk reporting matrix, (ii) consequence results in terms of performance, schedule, and/or cost impact using defined criteria, (iii) the risk matrix reporting the risk results, and (iv) documented risk results in the program risk register.

The system performance risk analysis tools typically focus on analyzing the technical requirements related to operational environment, TRL and/or MRL associated with systems/products being acquired, standards, material readiness, etc. Section 5.1 discusses available tools for addressing technical requirements with low TRL (i.e., high technological uncertainty level) and/or low MRL (i.e., market availability is low). For technical requirements with low TRL and/or MRL, the recommended tools include ELECTRE II, AGMF, and WGEs.

Existing schedule risk analysis tools are focused on the analysis of the (i) baseline schedule inputs, including durations and network logic; (ii) technical and schedule uncertainty inputs to the program schedule model; (iii) risk impacts to program schedule based on the program technical SMEs’ inputs; (iv) IMS incorporating the potential impact from all contract and supplier schedules and associated stakeholders’ activities; and (iv) schedule excursions reflecting the effects of cost risks, including human resource, budget, and schedule constraints. Note that when the identified risk impacts the critical path, then this risk affects both schedule and cost, and this risk should be registered as a schedule risk. Section 5.2, Step 5, discussed required analysis tools using PM-DDS integration approach for efficient schedule control and management. The integrated tools can effectively identify potential trouble areas in the IMS and indicate the areas that required immediate corrective action.

Currently, the cost risk analysis tools available focused on the cost analysis of the life-cycle-cost (LCC) by (i) building on technical and schedule assessment results, (ii) translating performance and schedule risks into LLC, and (iii) deriving LCC estimates by integrating technical assessment and schedule risk impacts on resources. The cost analysis tools are also capable of creating budgetary requirements consistent with fiscal year planning and determining the adequacy and phasing of funding supports the technical and acquisition approaches. The tools can document the cost basis and risk impacts and provide program LCC excursions from near-term budget execution impacts and external budget changes and related constraints. The recommended PM-DDS integration approach for performing the cost risk analysis is identical to Section 5.1 above, i.e., also includes the cost analysis tools described above, BDA data acquisition and data curation TEs, and AI-ML TEs. AI-ML TEs include (i) DMTT, (ii) data exploitation using MORL-ANN tool; and (iii) predictive analytics techniques using MORL-ANN tool.

Step 3—Risk Mitigation Planning Analysis: Current risk mitigation planning (RMP) tools focused on the four mitigation techniques, namely, (i) risk avoidance (RAV), (ii) risk controlling (RCO), (iii) risk transfer and sharing (RTaS), and (iv) risk assumptions (RAS). Existing RMP tools are mostly customized to specific programs. Most of the existing RMP tools are stovepiped and do not leverage analytical tools that have recently been developed using BDA and AI-ML technologies and associated TEs. To effectively conducting the RMP activity, we recommend integrating existing BDA and AI-ML tools into existing mitigation techniques. BDA tools include data acquisition and data curation processes and tools. The AI-ML tools include (i) DMTT, (ii) data exploitation using MORL-ANN tool, and (iii) predictive analytics techniques using MORL-ANN tool. This recommended PM-DSS integration approach is expected to improve the program management planning and execution.

Step 4—Risk Mitigation Plan Implementation analysis: Our survey of the risk mitigation plan (MRP) implementation tools shows no tools available and the MRP implementation is usually conducted by the risk team with support from SMEs across the program related organizations. As discussed in Sub-Section 4.4, RMP captured the key risk categories including performance, schedule, and cost risks, and the risk team is responsible for implementing the plan with the support of the program.

Step 5—Risk Tracking Analysis: Our survey of the risk tracking tools showed that the tools are focused on the tracking of the performance of RMP actions against a pre-defined metrics. The tools are also capable of executing the RMP to generate alternative risk mitigation choices, as appropriate. The tools track the burnt down activity to ensure the success of the RMP. To effectively generate alternatives risk mitigation choices, we recommend integrating existing BDA and AI-ML tools into existing risk tracking tools. BDA tools include data acquisition and data curation processes and tools. The AI-ML tools include (i) DMTT, (ii) data exploitation using MORL-ANN tool, and (iii) predictive analytics techniques using MORL-ANN tool.

The recommended PM-DSS integration approach for conducting the risk tracking is expected to improve the program management planning and execution by providing alternative mitigation choices to burn down the risk before it occurs. Figure 5 depicts our proposed PM-DDS integration approach for conducting program risk management more effectively. For improving the program planning, we recommend incorporating BDA and AI-ML process and tools to support risk identification, risk analysis, and risk mitigation planning. To reduce the program risk, we recommend incorporating BDA and AI-ML process and tools to support risk tracking.

Figure 5.

PM-DDS integration approach for program risk management.

6. Selected DDS TEs alignment with PM discipline areas

This section provides the alignment between the selected sets of DDS TEs and the PM discipline areas identified in our analyses presented in Section 5 above. From the nine key PM areas discussed in Section 3, Section 4 decomposed them into a generalized model of 19 PM discipline areas and selected only three PM discipline areas that were aligned with the selected five PM areas (i.e., program goals, schedule, cost, risk, and technical performance management). Table 1 provides a summary of the proposed PM-DSS integration approach for the program cost management (PCM) discipline. The table aligns existing PCM framework/process/model with DDS framework/process/model for the recommended PCM integration.

PCM framework, process, models, and tool	Recommended DDS framework/process/tool	Recommended implementation approach	Remark
EVMS Framework	ML Library, BDA framework with Data Acquisition and Data Curation Models and Tools	Leverage related cost historical data bases for integrating EVMS models with BDA tools for managing cost	See [24, 31, 32] for basic EVM, [33] for implementation
EVMS Process and Models
Cost Analogy Model/Tool	Data mining technique & tool (DMTT); data exploitation using MORL-ANN tool; and Predictive analytics techniques using MORL-ANN tool	Implement MORL-ANN using MATLAB DDPG tool	See [34] for DDPG; See [35] for using ANN
Cost Engineering Build-up Model/Tool		For high technological risk with high market uncertainty: ELECTRE II model + AGMF + UGAF+ WGEs	See examples in [20, 22] for ELECTRE, AGMF /UGAF/WGEs
Parametric Cost Model/Tool		Implement MORL-ANN using MATLAB Deep Deterministic Policy Gradient (DDPG) tool	DOD COCOMO Software Tool [18]
Actual cost Estimation Model/Tool			See [34] for DDPG

Table 1.

DDS process and tool for PCM integration.

The use of artificial neural network tools to predict the actual cost of a project to enhance EVMS is discussed in Refs. [35, 36] and easily extended to the program cost prediction. Table 2 summarizes our recommended PM-DSS integration approach for the program schedule management (PSM) discipline. The table aligns existing PSM framework/process/tools with DDS framework/process/model for the recommended PSM integration.

PSM framework, process, models, and tool	Recommended DDS framework/process/tool	Recommended implementation approach	Remark
EVMS Framework	ML Library, BDA framework with Data Acquisition and Data Curation Models and Tools	Leverage-related schedule historical data bases with EVMS and BDA tools for managing Schedule	See [24, 31, 32] for basic EVM, [33] for implementation
EVMS Process and Models
Activity Sequencing: CCM, CPM, PDM, PERT methods and Tools	DMTT; Data exploitation using ML-AI tools; and Predictive analytics techniques using ML-AI tools, including MORL-ANN, decision tree, and support vector machine (SVM), cumulant calculator.	Implement MORL-ANN using MATLAB DDPG tool	See [34] for DDPG; See [25, 27] for CCM, CPM, and PERT. [37] addresses ML-AI methods for project; duration planning and forecasting. See [38] for cumulant calculator
Activity Duration Estimate: Analogous Estimation, Parametric Estimation, Monte Carlo Simulation methods and tools		Decision tree, SVM, and cumulant calculator
Schedule Development: CPM and PERT models and tools		Implement MORL-ANN using MATLAB DDPG tool	See [27] for CPM
Schedule Control: LOB process and tools			See [28] for LOB

Table 2.

DDS process and tool for PSM integration.

Like Table 2, Table 3 provides a summary our recommendation for the PM-DSS integration approach for the program risk management (PRM) discipline. This table aligns existing PRM framework/process/tools with DDS framework/process/model for the recommended PRM integration.

PRM framework, process, models, and tool	Recommended DDS framework/process/tool	Recommended implementation approach	Remark
Risk Identification: OBR-ID, SBR-ID, TBR-ID, CRC models and tools	ML Library, BDA framework with Data Acquisition and Data Curation Models and Tools; DMTT; Data exploitation using MORL-ANN tool; and Predictive analytics techniques using MORL-ANN tool	Leverage related schedule historical data bases with BDA tools for managing Schedule.; Recommend to implement MORL-ANN using MATLAB DDPG tool	See [24, 31, 32] for basic EVM, [33] for implementation; See [34] for DDPG. [39] addresses the differences between ML, AI, deep learning, and neural networks
Risk Analysis: system performance risk, schedule risk analysis, and cost risk analysis models and tools
Risk Mitigation Planning: RAV, RCO, RTaS, and RAS models and tools	DMTT; Data exploitation using MORL-ANN tool; and Predictive analytics techniques using MORL-ANN tool	Implement MORL-ANN using MATLAB DDPG tool	See [34] for DDPG; See [40, 41] for risk tracking tools
Risk Tracking Analysis: Periodic Risk Status Reporting, Periodic reporting of risk mitigation plans tools

Table 3.

DDS process and tool for PRM integration.

As discussed in Refs. [24, 31, 32, 33], EVMS is a systematic process that uses earned value as the primary tool for integrating program cost, schedule, technical performance, and risk to manage a program. Program managers can leverage EVMS tools to determine and track the actual program status at any given point during program PoP. This activity can be done very effectively if the tools have successfully implemented required program constraints, program rules and process, and organizational rules. The implementation of EVMS requires a disciplined approach. Recently, BDA and AI-ML processes and tools have been successfully integrated into EVMS. Current analysis results show that when these BDA and AI-ML tools are properly integrated with EVMS, the tools can certainly assist the program managers to execute their programs more effectively by anticipating the cost, schedule, program, and technical risks and mitigating these risks before they occur.

7. Proposed flexible and adaptable PM-DSS integration life cycle framework leveraging MLOps

Successful PM-DSS integration requires planning, structure process, and proper selection of DSS models and tools. This section focuses on how to leverage the concept of MLOps, and its existing framework described in [42, 43, 44, 45, 46] for the development a PM-DSS integration framework. The proposed framework should be easy to use and tailored to any program types.

As pointed out in Refs. [42, 43, 44, 45, 46], the objective of MLOps is to reduce the technical friction associated with the development of AI-ML models and tools from an idea into production in the shortest possible time to market with as little risk as possible. But the objective for the PM-DSS integration framework is to reduce the integration risk between a set of BDA, and AI-ML tools with the selected PM discipline areas. The framework focuses on the start of the program to the deployment of the integrated tools with the lowest possible risk. In addition, the framework should also address the operational phase where the program team can leverage the integrated PM-DSS products to execute the program effectively. More specifically, using the program data displayed by the integrated tools, the program team can use proactive risk management method to improve the program planning and execution by reducing overall program risk. Figure 6 depicts a proposed PM-DSS integration framework leveraging existing MLOps life cycle framework. This proposed integration framework is easy to tailor to any program types and can be adaptable to any PM discipline areas and any set of BDA and AI-ML models and tools.

Figure 6.

PM-DDS integration life cycle framework.

As shown in Figure 6, the proposed framework has a life cycle that consists of seven key stages, including (i) PM-DSS integration specification, (ii) related program data acquisition, (iii) related program data curation, (iv) BDA and AI-ML models selection and integration, (v) PM-DSS integrated models testing, (vi) integrated model deployment, and (vii) display, monitor, and control program data. A high-level description of these key stages is provided in Figure 6. The feed-forward and feedback arrows are shown to describe the transition of each stage and the dependent of each stage.

8. Conclusion

This chapter describes an approach for integrating existing DDS models and tools with any PM processes, models, and tools for improving program planning and more efficient program execution by reducing overall program risk. A detailed description of the nine key PM areas along with the decomposed generalized 20 PM discipline areas were provided. This chapter proposed a PM-DSS integration approach for three PM discipline areas that were aligned with the selected four key PM areas, including program goals, schedule, cost, and risk. For the integration of each PM discipline area, a list of BDA and AI-ML models and tools were identified and suggested for the integration. In addition, the chapter proposes a flexible and adaptable PM-DSS integration life cycle that can be used to deploy BDA and AI-ML models and tools for improving program planning and execution.

Finally, when writing this chapter, the authors were intentionally focused on the high-level PM discipline areas and DDS technology enablers without technical depth. Only common DDS technology enablers that are known to the authors were selected for the integration. In practice, each PM discipline area deserves a whole book to address it in technical detail. There are many open PM-DSS integration problems and technical relevance associated with each activity step described in this chapter. The authors hope that the program management experts, data scientists, decision scientists, and mathematicians would benefit from this paper and its applications to these open problems.

Acknowledgments

The authors would like to thank their CSUF colleagues for their support of this work, particularly Professor Sam Behseta.

Conflict of interest

The authors declare no conflict of interest. The proposed approach and opinions expressed in this chapter are those of the authors and they are not endorsed by CCAM and Mihaylo College of Business & Economics, CSUF.

Notes/thanks/other declarations

The first author would like to thank his wife, Thu-Hang Nguyen, for tremendous patience and support during the preparation and writing of this chapter.