WG1: Data Rescue

WG Leader (Dr Rob Allan, ACRE Project Manager, Met Office Hadley Centre, United Kingdom - rob.allan@metoffice.gov.uk)

Terms of reference

Data Recovery (new sources all aspects);

Validation, uniform formats;

Imaging and digitization (SETI approach? Student involvements);

Best practice guidance;

Inventories - who has what, where is it?;

Check diverse archives;

Involve archivists - demonstrate the benefit of ACRE to archivists

Members:  Stefan Brönnimann (stefan.broennimann@giub.unibe.ch); Andrea Grant (andrea.grant@env.ethz.ch); Xiaolan Wang (Xiaolan.Wang@ec.gc.ca);

Catherine Marzin (Catherine.Marzin@noaa.gov); Martin Juckes (M.N.Juckes@rl.ac.uk); Phil Jones (p.jones@uea.ac.uk); Catharine Ward (catharinesi@aol.com);

Scott Woodruff (Scott.D.Woodruff@noaa.gov): Sylvie Jourdian (sylvie.jourdain@meteo.fr); Uwe Ulbrich (ulbrich@met.fu-berlin.de);  Gil Compo (compo@colorado.edu); Rajmund Przybylak(rp11@umk.pl); Joëlle Gergis (jgergis@unimelb.edu.au); Dennis Wheeler (dennis.wheeler@sunderland.ac.uk); Clive Wilkinson (c.w.wilkinson@uea.ac.uk); Juerg Luterbacher (Juerg.Luterbacher@geogr.uni-giessen.de); Birger Tinz (Birger.tinz@dwd.de)

ACRE-facilitated reanalyses database

The daily to sub-daily global historical terrestrial and marine surface observations needed for the ACRE-facilitated surface-input-only reanalyses (20th Century Reanalysis Project, Surface Input Reanalysis for Climate Applications (SIRCA) (1840s-2011), and North Atlantic-European Region mid 18th-early 19th Century Reanalysis) will be held in the GCOS AOPC/OOPC Working Group on Surface Pressure (WG-SP) International Surface Pressure Data bank (ISPD - terrestrial observations) and the International Comprehensive Ocean-Atmosphere Data Set (ICOADS - marine observations). These observations will be made available to the international scientific community.

Listings of terrestrial stations that are in or are going into the ISPD can be found at: 

                    NCDC (from all US sources of international data): ftp://ftp.ncdc.noaa.gov/pub/data/ispd/v2.2.3/ispd-history.txt

                    ACRE:  (access to all data in one EXCEL file for the ISPD are regularly revised and held on my Drop Box - contact me if you need     access - rob.allan@metoffice.gov.uk)

The ISPD version2 is currently available through our partners at the US National Center for Atmospheric Research (NCAR) at http://rda.ucar.edu/datasets/ds132.0/

Projects to improve availability of historical observations

ACRE is facilitating work to make more historical observations available for climate research and reanalyses; notably by organising projects to image and digitise instrumental weather observations in ship log and remarks books.

Archives and Climate Studies Meeting 11/7/2008

18th Century historical voyages with instrumental weather observations 

ACRE is collaborating with a number of international data rescue projects and activities

Inventory of historical Chinese and South China Sea instrumental weather observations

In 2008, ACRE was able to fund (from the Met Office Hadley Centre's Integrated Climate Program and an Arts & Humanities Research Council (AHRC) Knowledge Catalyst Scheme grant) an initiative in conjunction with Professor Robert Bickers, Department of Historical Studies, Bristol University and his Chinese Maritime Customs Project, to produce an inventory of all the sources of historical Chinese and South China Sea region terrestrial and marine daily to sub-daily meteorological data. In early 2008, Dr Catherine Ladds, a student of Prof Bickers, visited China to consult various archives and develop the inventory. A small digitisation effort is also underway in the Met Office Hadley Centre to key the twice daily pressure observations published in the China Coast Meteorological Register (1874-1932) and held as hard copy in the Met Office archives (1874-1893) or scanned on the NOAA Central Library WWW pages (1894-1932).  ACRE is looking to work with the China Meteorological Agency (under a Memorandum of Understanding with the Met Office  Hadley Centre) to image. digitise and archive the historical Chinese weather observations revealed by this inventory.  It is also looking for wide South East Asian collaborations with the joint CCl/CLIVAR/JCOMM Expert Team on Climate Change Detection and Indices (ETCCDI)

The Chinese meteorological inventory will:

Chinese Maritime Customs Report 2008    Chinese Maritime Customs Inventory 2008    China Coast Meteorological Register    

                                                                     Customs Data CMA    Hart's instructions

Mediterranean Region

ACRE is liaising and collaborating with data rescue and research projects focusing on the Mediterranean region: the MEditerranean climate DAta REscue (MEDARE) initiative, and the Mediterranean CLImate VARiability and Predictability (MedCLIVAR) initiatives. The aim is to work closely with the research communities and National Meteorological Services (NMS) that are part of MEDARE and MedCLIVAR in order to facilitate the recovery, imaging, digitisation, quality control and archiving of historical daily to sub-daily weather observations over the wider Mediterranean region.

ACRE is also collaborating with the Portuguese SIGN (Signatures of environmental change in the geophysical institutes) project; that aims to recover the 19th and 20th century data collected by the country's Geophysical Institutes and National Meteorological Service, including data from the former African and Asian colonies.

European Region

Collaboration between ACRE and the European Climate Support Network (ECSN) will focus on working together to extend and enhance the ECSN daily European data repository maintained by the European Climate Assessment & Dataset (ECA&D) project. Links to ECSN will also provide ACRE with direct access to the majority of National Meteorological Services (NMS) across the greater European domain. This infrastructure will again provide a regional focus for the coherent recovery, imaging, digitisation, quality control and archiving of historical European daily to sub-daily weather observations.

France and French colonies around the world

ACRE is using the revitalised Memorandum of Understanding between the Met Office Hadley Centre and MeteoFrance concerning exchange and collaboration on historical atmospheric pressure data for mainland France and French colonies around the world, to engender coordinated recovery, imaging, digitisation, quality control and archiving of daily to sub-daily pressure data.

African Region

Linkages have been established with the World Climate Research Programme (WCRP) Climate Variability and Predictability (CLIVAR) Variability of the African Climate System (VACS) Panel to explore joint activities with ACRE in Africa. ACRE is also looking to work with the AMMA (African Monsoon Multidisciplinary Analyses) initiative and the developing GCOS-WCRP-WCP-ICPAC World Bank Proposal "Climate Observations and Regional Modelling in Support of Climate Risk Management and Sustainable Development."

International

ACRE played a significant role in making possible a marine data exchange, which saw DWD (Deutscher Wetterdienst) provide ICOADS, and ultimately the 20th Century Reanalysis Project, with a block of digitised German merchant ship observations for the 1876-1902 period. There is hope that additional DWD marine data will be available in the near future.

ACRE is working with the International Environmental Data Rescue Organization (IEDRO) in order to coordinate international data rescue activities and share expertise and experience. IEDRO have ongoing data rescue projects in a number of countries in Africa and South America, and have worked closely with US agencies such as Climate Database Modernization Program (CDMP).

ACRE is also keen to see the development of optical methods to digitise charts and plots from instruments recording continuous traces of climate and weather variables (thermohydrographs, barographs etc.). This is being explored with IEDRO, Dr Theo Brandsma at KNMI, and Dr Alfred Dufter in Germany.

Data Homogenisation

ACRE is to make the daily to sub-daily atmospheric pressure data in the ISPD available to the COST Action ES0601: Advances in homogenisation methods of climate series: an integrated approach (HOME). This will be an important step towards extending the quality control procedures applied to these observations by exposing them to a wide range of data homogenisation techniques that HOME will be testing on climatic and weather data.

Links to library's, archives and various data repositories

Met Office Library & Archives        UK National Archives        British Library    How the Met Office Archive is helping ACRE data efforts    

                                                                      ARC Magazine article on ACRE February 2011

Links to scanned weather observations online

NOAA Central Library        University of Chile        SIGN       BADC     Hong Kong    Singapore  International Bull Simultaneous Met Obs    Salvá Sinobas    French Daily Weather Reports: 1857-1923 Bulletins climatiques quotidiens

French Academy of Medecine - ANR-CHEDAR          Old Dutch Data in Newspapers        Old Hong Kong Newspapers Collection         Macau Government Gazette            NCDC searchable online catalogue of their data holdings

Miscellaneous

1839 Memorandum - by the UK Colonial Secretary Lord Glenelg