Objectives and approach

Monitoring and evaluation functions within GEO will be implemented in different ways depending on the Implementation Mechanism chosen.

GEO will implement a set of Core Functions essential for the realization of its Strategic Objectives. These Core Functions are accompanied by specific, measurable, and achievable targets designed to enable an effective implementation, including the monitoring of progress and evaluation of achievements.

The purpose of monitoring is to track the progress of the GPW. The GEO Secretariat will conduct monitoring on an on-going and systematic basis.

Evaluations inform improvement of GEO’s activities by conducting assessments of the progress towards achieving the expected results. These evaluations will take note of other contributing activities that are not included in the GPW, but are attributable, at least in part, to GEO’s actions. Results of the evaluation will help determine how effective GEO has been in terms of societal impacts. The evaluations previously performed by the Monitoring and Evaluation Working Group will be implemented through Foundational Tasks included in the GPW.

Two comprehensive evaluations should be conducted with one mid-way through the Strategic Plan period and the other near the end. The evaluations will address the full scope of the activities within GEO, although their primary focus should be on the Strategic Objectives and expected results. The mid-term evaluation will also include an assessment on the extent to which the priorities identified in the Mexico City Ministerial Declaration are being realized and re-orient / refine directions set out in the GEO Strategic Plan to take into account emerging trends and challenges.

Proposals for conducting additional evaluations on specific topics or Initiatives should be included in the GPWs. The results of the evaluations will be presented to the GEO Programme Board, the Executive Committee and Plenary. The GEO Secretariat will support the evaluations, including by providing the necessary links into GEO communities.

Prior Evaluations

The GEO Monitoring and Evaluation Working Group (M&E WG) was established by the GEO-V Plenary in November 2008. Its purpose is to monitor performance against defined requirements and the intended benefits of GEO/GEOSS.  Accordingly, the Working Group has developed evaluation plans for GEOSS implementation (2005–2015). To date, five evaluations have been conducted:

• 2009–2010: Midterm evaluation that extended across all GEOSS activities
• 2010–2011: Evaluation of GEOSS Architecture and Data Management
• 2011–2012: Evaluation of the Societal Benefit Areas (SBAs) Agriculture, Biodiversity and Ecosystems
• 2012–2013: Evaluations of the SBAs Disasters, Energy and Health
• 2013–2014: Evaluations of SBAs Climate, Water and Weather
• 2014–2015: Sixth and Summative evaluation

Purpose

The summative evaluation will assess the extent of GEO’s success during its first 10 years of implementing GEOSS (2005-2015) by addressing the following major questions:

• To what extent have the Strategic Targets [and their associated outcomes] been achieved?
• To what extent have the work plan tasks been completed as planned?
• To what extent were the strategic target outcomes (“To be demonstrated by…” bullets) achieved?
• What were the key gaps in implementation (which should be addressed in the next plan) and why did these gaps occur?

• To what extent has GEO provided added value to individual members and participating organizations?
• To what extent has GEO fostered increased collaboration across tasks and SBAs and between information providers and EO users.
• To what extent would the activities in the GEO Work Plan have occurred in the absence of GEO?
• To what extent do participants in GEO tasks and users of EO find benefit in GEO/GEOSS?
• To what extent has GEO articulated a clear role and mandate, and to what extent is this understood by GEO participants and EO users.

• Are there identifiable factors (people, resources, relationships, etc.) that have contributed to or hindered success?
• What common or systemic factors of GEO/GEOSS as identified in this or previous evaluations should be addressed in the next GEOSS Implementation Plan?

Approach

The plan for the Summative Evaluation of GEOSS can be found here. The Evaluation Team will use the following principal sources of information:

1. Review of GEO Documents (including, but not limited to, Ministerial Declarations; the GEOSS 10-Year Implementation Plan and the 10-Year Implementation Plan Reference Document; all Work Plans and Task Sheets; various versions of the GEOSS Roadmap; Progress Reports; meeting reports from Plenaries, Executive Committee and other GEO Committees; Interim Report of the IPWG: Towards the 10-Year Implementation Plan 2016-2025,)
2. Evaluations, audits, reviews or other external documentation (reposts, published paper and articles, etc.) by GEO Members or Participating Organizations that may be relevant to this evaluation
3. Any information (including performance measurement data) that task leads may have which is not available in the Task Component Sheets and which may be relevant to the assessment of Task implementation or to the assessment of progress
4. Case studies
5. Interviews with key persons
6. Web based surveys aimed at relevant GEO communities and other constituencies

Schedule

Evaluation Milestones

• July 15-16, 2014: Evaluation Team kick-off meeting in Geneva, Switzerland
• January 20-22, 2015: Evaluation Team mid-point meeting in Geneva, Switzerland
• February 2015: Deliver interim report to the M&E Working Group for sharing with the GEO Implementation Plan Working Group (IPWG)
• May 2015: Evaluation Team final meeting (location TBD)
• July 2015: Deliver final report to the M&E Working Group for sharing with the GEO Executive Committee

Meetings or conferences where the Evaluation Team plans to engage participants:

• October 28-30, 2014: CEOS Plenary in Tromso, Norway
• November 11-14, 2014: GEO-XI Plenary in Geneve, Switzerland
• December 15-19, 2014: American Geophysical Union (AGU) Fall Meeting in San Francisco, California
• More to be listed soon…

Team Members

As in the case of all previous GEOSS Evaluation Teams, the Summative Evaluation Team consists of persons volunteered by GEO Member states or organizations. The present members are:

Please contact any team member regarding input to and questions on the Evaluation.

The Co-Chairs of the GEO Monitoring & Evaluation Working Group are:

The GEO Secretariat's contact is Giovanni Rum (grum@geosec.org)

Reference Documents

• GEOSS 10-Year Implementation Plan
• GEOSS 10-Year Plan Reference Document
• GEO Work Plan 2012 2015
• GEOSS Strategic Targets

The 2016 Work Programme Activity