Navigation
 

Information to Support Proposals


The NASA Applied Sciences Program seeks proposals in the ROSES-2008 solicitation for projects and feasibility studies to integrate NASA Earth science research results into decision support systems serving applications of national priority and to document improvements in the performance of the decision support systems.

The following information serves as reference materials to support the design and development of a proposal.

Key Reference Information, Documents, and Websites

The Program provides this information and websites as reference materials. The Program does not imply endorsement of any organization by including them on the list.

Information and Documents about NASA, NASA Earth Science, and the NASA Applied Sciences Program:

Information about Earth science research results:

Information about national, interagency, and international activities:

Information about open-consensus data standards:


Examples of National/Regional Organizations

The NASA Applied Sciences Program encourages solicited project proposals to include teams of organizations spanning organizational sectors (e.g., academia, private, Federal, public, nonprofit, etc.) and expertise (e.g., technical, management, scientific, etc.). The Applied Sciences Program supports projects involving organizations with national perspectives that have established networks to broad sets of end-users and established constituencies at national, regional, state, local, and tribal levels. The Program encourages teams to include national and regional organizations and associations, so broad sets of users are efficiently represented on teams, the results can be distributed widely, and the nation can benefit broadly from the project.

The links below provide examples of national or regional organizations and associations related to each of the twelve National Applications. The Program does not necessarily have established relationships with these organizations, and the Program does not imply endorsement of any organization on these lists. The lists are not exhaustive, and the lists are provided as examples only.


Systems Engineering Approach

The Applied Sciences Program employs an "end-to-end" systems approach in working with organization to assess and apply NASA Earth science products in their decision support systems. Where Earth science products are evaluate to have potential application, NASA and the organizations collaborate on the systems integration of the products into the decision support systems. In these collaborations, the Program promotes the use of a systems engineering approach. As the program defines it, this systems engineering approach involves three main phases - Evaluation, Verification and Validation, and Benchmarking. The terms associated with these three main phases are described below, and the Program encourages projects it sponsors to reference or adopt this language in their projects.

Evaluation

This phase identifies and initially evaluates opportunities for Earth science products to enhance organizations' decision support systems. Activities typically include: identifying decision support systems and organizations associated with a national application; examining the organziations' plans for developing its decision support systems; broadly assess the potential value of the Earth science products in the system; determining user defined requirements; and, assessing the partners' commitment to a project. Note: A proposal team essentially accomplishes this phase in the design and development of a proposal.

Verification and Validation

This phase devises and refines system integration approaches and resolves technical integration problems to meet the requirements and to achieve the intended outcomes. This phase focuses on determining how the and ensure the end-to-end system meets the intended objectives. Activities typically include: determining how the Earth science products will serve as inputs to the decision support systems; resolving technical issues related to the introduction of the Earth science products into the systems; developing prototype configurations to address requirements; assess, refining, and iterating on techniques to ensure the end-to-end system meets the intended objectives. Generally, verification refers to determining if and how well the Earth science product(s) can integrate into the system to meet the requirements (within a specified tolerance). Validation refers to determining if the decision support system can function with the Earth science products to achieve the intended outcomes.

Note: The Program's use of the phrase "verification and validation: refers to the activities associated with assuring the integration of the Earth science products in the decision support system. For projects the Program sponsors, this phrase does not refer to validation of NASA Earth science spacecraft measurements. Proposal teams should assume that spacecraft measurements distributed from NASA sources are validated.

Benchmarking

This phase quantifies the improved performance of the decision support system using the Earth science products and documenting the project to enable the user organization to continue using the products after the project's completion. Activities typically include: operating the decision support system "with" and "without" the Earth science products; comparing the performance of the enhanced version of the system using the Earth science products to the previous version; running different scenarios and conditions to compare and document performance ranges; comparing the system performance to other relevant benchmarks; using performance measures to assess the value of the Earth science products in the system; assessing the value of any improved decision support systems outputs to the actual decision making processes and decisions; determining or estimating the socioeconomic benefits of the improved decision support system and actual decisions; documenting the procedures, system configurations, integration approaches, and other techniques and steps the project team employed. The benchmark report is the primary report the project team will produce, and a primary purpose is to provide guidelines, technical information, and quantitative rationales to support the user organization in using the Earth science results in its decision support system on a sustained basis.

As part of the systems engineering approach, the Program encourages projects to recognize the uncertainties in Earth science products and represent the uncertainties to the user organizations. The Program encourages activities to support partner organizations in assessing and quantifying associated uncertainty in the decision formulation, decision making, and operational system enhancement as a result of incorporation NASA Earth science products.

The overall intent of this systems engineering approach is to provide a systematic rigor to determine potential Earth science results to extend, develop prototypes and resolve technical issues associated with use of Earth science results in decision support systems, and document value of the Earth science products to enable partners to adopt the Earth science products in a sustained manner. The purposes of this rigorous approach are to identify and resolve data exchange products, build partners confidence and reduce technical or perceived risks in using Earth science products, and strengthen partners' abilities to use the Earth science products in their decision making.


Suggestions for Proposals and Proposal Teams

Based on information gained and suggestions from past solicitations, the Program provides the following information and suggestions for the design and development of a proposal. The materials are grouped in four categories. Note: These suggestions serve merely as a guide, and the language of the ROSES-2007 Element A.20 supercedes any guidance listed below.

General

  • Project proposals should address and align with one or more of the twelve applications of national priority.
  • Projects should be focused, directed efforts using project management principles to identify and accomplish specific milestones and deliverables. Proposals should not be exploratory in nature.
  • Proposals should not seek to use NASA funds to create new decision support systems for partner agencies or end user organizations. (Partner agencies and end user organizations, who understand and are ultimately accountable for their decision making, are responsible for developing, owning, and operating their deicision support systems and decision processes. The Applied Sciences Program provides project funds for teams to assess if NASA's Earth science products can help improve and integrate into the agencies' and organizations' decision making.)
  • Projects should serve a broad set of users nationally and articulate their broader impact to many States, regions, localities, and tribal entities.
  • Proposals should present well-conceived approaches to transition results to the operational community, including plans and activities to help user organizations continue the use of Earth science results in the DSS after project completion. The Program suggests against a "build it and they will come" approach to the transition of results.
  • Projects should incorporate appropriate use of open-consensus standards (as delineated by ISO, Federal Geographic Data Committee, and Open GIS Consortium) toward interoperability of disparate systems.

Project Teams

  • Projects should include teams of organizations across public, private, academic, non-profit, etc. sectors. Projects should include on the teams the organizations that operate and employ the decision support system in their decision making process.
  • Projects can include commercial entities, such as geospatial firms, to participate on teams and integrate their products with Earth science results. The Program is interested in building skill and capacity throughout the nation and economy to use of NASA Earth science research and geosciences products to benefit society.

Earth Science Research Results

  • Proposed projects should make use of recent and upcoming Earth science missions, such as ICESat, GRACE, Aura, CALIPSO, CloudSat, Glory, OSTM, Aquarius, NPP, GPM, and OCO, as well as Earth science models. For upcoming missions, projects can use simulated data sets.
  • Projects should use an array of Earth science research results, including multiple sources and a variety of spacecraft observations, geophysical parameters, and model outputs.
  • Proposals should should assume that spacecraft measurements distributed from NASA sources are validated. Proposals should not propose to validate NASA Earth science spacecraft measurements.
  • Projects should pursue innovative uses of NASA Earth science observations, Earth science modeling capabilities, geosciences products, and other Earth science research results.
  • For organizations/proposals seeking operational mapping services, NASA encourages organizations to work directly with private sector geospatial companies and their extensive terrestrial mapping capabilities. Proposal project teams should not focus on routine mapping of land use/land cover with well-tested mapping techniques, and they should work directly with the private sector for those activities.

Proposal Content

  • Proposals may include letters of commitment from the end user organzations that will likely benefit from the proposed project. The Program suggests that proposals include these letters to indicate the users' interest, familiarity, commitment, level of involvement in the project as well as the expected benefits from the project. Proposers should emphasize the quality and content of these letters. Additional information on Suggestions for Letters of Commitment (link to section below)
  • Proposal teams should illustrate their project concept in an Integrated System Solution diagram (link to last section on this page). Teams should refer to this diagram to check if they reflect the appropriate system components and decision making activities in their proposals. The Program suggests that teams be specific when including information in the diagram.

Information on Performance Measures and Management Metrics

Section 4.4 of ROSES-2007 Element 20 specifies that projects need to articulate performance measures and project management metrics. In general, project management metrics refer to inputs, outputs, milestones, etc. internal to the project, and performance measures refer to means to assess and articulate the outputs, impacts, and benefits of the project. Used together, these measures and metrics should help a project track progress, identify issues and trends, make project adjustments to meet objectives, establish results, and communicate benefits. Based on knowledge gained from past solicitations and requests from project teams for guidance, the Program provides the following information and suggestions related to these measures and metrics.

Process metrics tend to be internally-focused to assess inputs, outputs, and productivity; Performance measures tend to be externally-focused to assess the value of a program or project and how the program or project serve its intended purpose.

Performance Measures

Generally, performance measures track and reflect changes in the value, effectiveness, and outcomes of project. The performance measures reflect changes in the decision making processes and the actual decisions as well as the outcomes of the improved decisions. Positive changes in the performance measures can help project teams articulate the impacts and socioeconomic benefits enabled by the project. The following provide some notional examples of performance measures:

  • forecast skill scores
  • change in planning time
  • change in response time (e.g. disaster response time)
  • percentage of population served
  • level or certainty/uncertainty in the decision support system
  • false negatives/false positives
  • exposure assessment time and accuracy
  • control measure effectiveness

Note: The Program suggests that a project at least include the performance measures that the end user organizations and other users of the decision support system employ to gauge their performance.

Project Management Metrics

Generally, project management metrics are internally-focused and refer to the inputs, outputs, quality, productivity, and other such factors typically within the project's control or influence. The team should use these metrics to identify and track problems and to determine if the project is on schedule, on budget, on task, etc. The following provide some notional examples of project management metrics:

  • Percent of tasks starting/ending as planned
  • Number (or percent) of deliverables meeting quality standards
  • Integration time of observations/model products into the decision support system
  • Delivery time of Earth science products
  • Costs for production of value-added products
  • Number of iterations to verify data product integration
  • Time to correct a problem
  • Milestones met ahead of schedule
  • Run-time for a decision support system

Note: Project meetings (number of, frequency of, etc.) and Project schedules not qualify as project management metrics.


Suggestions for Letters of Commitment

Section 4.4 of ROSES-2007 Element 20 allows for letters of commitment from end-user organizations that benefit from the projects. Based on knowledge gained from past solicitations and requests from project teams for guidance, the Program provides the following suggestions for information that an end user organization might include in a letter of commitment:

  • description of the end user organization including its purpose, functions, and mandated responsibilities (particularly those directly relevant to the project topic)
  • the organization's interest, role, and level of participation in the project;
  • the national importance of the project topic;
  • the relevance and importance of the project to the end user organization;
  • the potential of the project to improve the organization's decision making;
  • the value and benefit of any improved decision making to the community and beneficiaries that the end user organization serves;
  • the end user organization's commitment to adopt the use of the Earth science research results on a sustained basis in their decision support activities (if the project is successful).

Note: The suggested topics serve as a guide; this list is not mandatory, exclusive, or exhaustive.


Integrated System Solution Diagrams

The NASA Applied Sciences Program uses the figure below as a notional architecture to depict the contribution of Earth observations and models to organizations’ decision support, management, and policy-making activities that provide value and benefit to society. This figure relates to the architecture that the United States Group on Earth Observations and the International Group on Earth Observations have formally adopted.

Integrated System Solution Diagrams

On the right side of the architecture, organizations own, develop, and operate decision-support tools to inform their decision making processes. Decision support tools often refer to assessments and decision-support systems; various types of tools serve different purposes, such as planning, forecasting, and early-warning activities. Generally, decision support systems are interactive, computer-involved systems that provide organizations with methods to retrieve and summarize information, analyze alternatives, and evaluate scenarios to gain insight on critical factors, sensitivities, risks, and consequences of potential decisions. For example, Federal agencies use decision support tools to support their responsibilities to the public, such as resource management, planning, regulation, public health, and economic development.

On the left side of the figure, research and operational agencies develop measurement techniques, collect measurements, produce new knowledge, and extend Earth observations, environmental and climate data records, and model predictions and forecasts. Where the Earth science products are determined to have potential value to a decision support tool, NASA and organizations can collaborate to facilitate the flow of Earth science products into specific decision support systems. Given the copious volumes of Earth observation data and computationally-demanding scientific models, decision-support systems typically provide systematic mechanisms to incorporate data products and document the value derived from the inputs.

GEO, USGEO, and the NASA Applied Sciences Program promote the assimilation of Earth observations and model predictions to serve as inputs to decision-support tools. The outcomes of this architecture and approach are manifest in the organizations’ enhanced policy and management decisions, and the impacts are the resulting socioeconomic benefits from the improved decisions.

In this architecture, the outputs of NASA research (e.g., measurements, new knowledge, model products) serve as inputs to the partner organizations' decision support systems. The outcomes of this architecture are manifest in the organizations' enhanced policy and management decisions, and the impacts are the resulting socioeconomic benefits from the improved decisions.

Examples

Based on knowledge gained from past solicitations and requests from project teams for examples, the Program provides the following examples of Integrated System Solution diagrams from existing projects.

ROSES-2007 Element 20 Section 4.4 specifies that proposals must include a populated version of an Integrated System Solution diagram that depicts the project. The diagrams should be as detailed and specific as possible. Proposal teams may download a blank version that they can populate: