Frequently Asked Questions

 Do you provide customised datasets?

 No. All SILO data products are now available through our self-service website in a range of customisable formats.


 Do I need an account?

 No, you do not need an account to access SILO data.


 Why do I need to provide my email address?

 We need your email address in case we need to contact you about the service. Your personal data are protected by our privacy policy.


 What are the differences between point data and gridded data?

 Point datasets are temporal datasets at a single location. In other words, they provide a time-series of data (usually daily time-step) at either a single grid cell or a single station.

Gridded datasets are spatial datasets for a given date. SILO grids cover the region 112°E to 154°E, 10°S to 44°S with resolution 0.05° longitude by 0.05° latitude (approximately 5 km × 5 km).


 What is the difference between point data at grid points and station locations?

 Station point datasets are a time series of data at a station location, consisting of station records which have been supplemented by interpolated estimates when observed data are missing. Station point datasets are available at approximately 8,000 station locations around Australia. These datasets were formerly known as SILO Patched Point datasets.

Grid point datasets are a time series of data at a grid point location consisting entirely of interpolated estimates. The data are taken from our gridded datasets and are available at any grid point over the land area of Australia (including some islands). The nominal grid location (where the interpolated surface is evaluated) is the centre of the corresponding grid cell. These datasets were formerly known as SILO Data Drill datasets.


 What happened to Patched Point and Data Drill datasets?

 Both datasets are still available, but as they can now be requested through the same interface, the distinction between them is no longer required. For further information, see the previous question.


 Why does the "comparison" predefined format change?

The comparison format is intended for people to use to decide what data they need, so when new variables are added to Silo they are included in the comparison format.


 Are point data interpolated or observed?

 Point datasets at grid locations consist entirely of interpolated data. Point datasets at station locations contain observed data (when available) and interpolated data (when observed data are not available or do not pass Quality Assurance tests).


 What tools can I use to view, convert or process NetCDF rasters?

 There are many open source and commercial tools available for working with NetCDF rasters. You might find the following tools helpful:

NetCDF files can be converted to a wide range of GIS formats using GDAL.

 Why can't I read SILO's NetCDF files?

 SILO rasters are packed into NetCDF files containing all 365 (or 366) daily rasters for a single year (and similarly, all 12 monthly rasters in the case of monthly rainfall). However the NetCDF file for the current year (for a given variable) only contains rasters for January 1 up to the previous day (or month), even though the NetCDF metadata indicate rasters are available for all timeslices within the year.

For example, on January 4 the NetCDF file for daily rainfall will contain rasters for January 1, 2 and 3. Inspection of the metadata however will show the time dimension is 365: 

> ncdump -h 2018.daily_rain.n
netcdf 2018.daily_rain {
dimensions
      lat = 681
      lon = 841
      time = UNLIMITED ; // (365 currently)

Some software packages (such as R and NCO) cannot handle files with this structure. To work around the issue you may be able to extract the desired timeslices into a new NetCDF file with the time dimension set to the true number of timeslices. For example, you could extract the rasters for January 1 - January 3 using ncks (part of the NCO package): 

> ncks -v daily_rain -F -d time,1,3 2018.daily_rain.nc 20180101-20180103.daily_rain.nc

 What time are observations made?

 For most daily climate variables the observations are recorded at 9 am. To assist in understanding how SILO datasets are constructed, it may be useful to see how data are assigned to a given day:

Observation times

 Why are evaporation data shifted to the day before?

 Evaporation (class A pan) is measured at 9am. In normal circumstances a large proportion of the observed evaporation would have occurred throughout the daylight hours (after 9am) on the previous day. Consequently, SILO shifts evaporation data to the day before the observation was made.

Note: the Bureau of Meteorology shifts maximum temperature data to the previous day for similar reasons. SILO uses the shifted temperature data provided by the Bureau.


 How are gridded rainfall datasets created?

 Daily rainfall gridded datasets are derived from interpolated monthly rainfall by partitioning the monthly total onto individual days. Partitioning requires estimation of the daily distribution throughout the month. The distribution is obtained by direct interpolation of daily rainfall data throughout the month. At the end of the month, the interpolated monthly rainfall is then partitioned onto individual days according to the computed distribution.

For further information, please read the journal article which documents many of SILO's processes, and also our metadata.


 Can I request point data for a large number of locations through the website?

 No, data can only be requested for one location when using the website to order data. If you wish to request data for a large number of locations, you may wish to consider using our API.


 How can I repeat a data request for the same set of stations (or grid points)?

 If you wish to repeat a previous request or automate requests, we suggest you try our API rather than manually entering requests via the interactive web page.


 SILO has data for 29-Feb-2000 but not for 29-Feb-1900, is this right?

 Yes. Under the Gregorian calendar 2000 was a leap year, but 1900 wasn't. Normally if the year can be evenly divided by 100 it is not a leap year, however it is a leap year if it can be evenly divided by 400.


 How are the grid locations selected?

 The nominal location for a given grid cell is the centre of the cell. For example, the value at 115.05° East, 34.00° South is intended to be representative of the area 115.025° - 115.075° East, 33.975° - 34.025° South.

You can request data for any grid cell that is not masked out. Grid cells over the ocean and some islands are masked out. If you request data at a location where the longitude or latitude is not a multiple of 0.05°, the location will be rounded to the nearest 0.05°. If you request data at a location that lies exactly on the edge of a grid cell, the longitude (or latitude) will be rounded up giving you data from the grid cell that is east of the cell edge (if the specified longitude lies on a cell edge), or north of the cell edge (if the specified latitude lies on a cell edge).


 Can you provide interpolated data at resolutions higher than 0.05° × 0.05°?

 No. The accuracy of the interpolated datasets is strongly dependent on the density of the input data (i.e. station density). In most regions the station density is not high enough to support higher resolution estimates.

Users should also note that the interpolated estimates are based on observations recorded in a standard Stephenson screen or equivalent, placed in an open and usually flat area. It does not represent the ground level climate of some areas (e.g. wooded systems) and does not address all the issues associated with slope and aspect. Users may need to implement their own microclimate adjustments.


 How accurate are the interpolated data prior to 1957?

 The number of stations recording climate variables significantly increased around 1957 – the International Geophysical Year. An anomaly interpolation technique is used to interpolate maximum and minimum temperature, radiation and vapour pressure for all years prior to 1957. The anomaly method is better able to interpolate sparse datasets than direct interpolation because much of the variance can be captured by a mean dataset constructed from a larger dataset.

The data quality throughout the pre-1957 period was examined in: Rayner, D.P., Moodie, K.B., Beswick, A.R., Clarkson, N.M., and Hutchinson, R.L. (2004), New Australian daily historical climate surfaces using CLIMARC. Queensland Department of Natural Resources, Mines and Energy Report QNRME04247. Available here.


 Are SILO data quality checked?

 SILO datasets are constructed from observational data collected by the Bureau of Meteorology. The Bureau has a quality assurance program which is progressively checking its observational collection. SILO does not use data which have been quality checked by the Bureau and classified as "wrong", "suspect" or "inconsistent with other known information". In addition, SILO implements a number of internal checks to identify data which may be erroneous. For example, SILO uses a "two-pass" interpolation technique to interpolate all variables except daily rainfall. Observed data are interpolated in a first pass and residuals computed for all data points. The residual is the difference between the observed and interpolated values. Data points with high residuals may be indicative of erroneous data and are excluded from a subsequent interpolation which generates the final surface from which the station-point datasets are constructed.


 Is it possible that minimum temperatures are higher than maximum temperature?

 No. Maximum temperature observations are shifted to the previous day (see above), so the observed maximum should always be lower than (or equal to) the minimum temperature.


 Why are the observed data and interpolated data different?

 The observed data for a given station should be similar to the corresponding interpolated data at the nearest grid cell. However differences can arise for several reasons:

  1. interpolated data are evaluated at the centre of the grid cell. If the station is a significant distance from the cell centre, and the interpolated surface exhibits a strong gradient in the area, there can be a significant difference between the value at the station and the cell centre. For example, under normal conditions the temperature usually decreases with elevation at approximately 5-7 °C/km. If the elevation at the cell centre is 500m higher than the station (quite possible in alpine regions), the temperature at the cell centre could be 2.5-3.0 °C lower than the temperature at the station.
  2. most of SILO's interpolated grids are constructed using a smoothing spline. If the input data are spatially homogeneous, the fitted surface will generally pass through (i.e. reproduce) the input data. However if the input data are highly variable or contain errors, the spline may smooth the data and consequently the fitted surface will not reproduce the input data in the affected area(s). Note: SILO uses kriging to interpolate daily and monthly rainfall. Kriging guarantees the input data are reproduced.
  3. SILO uses a "two-pass" interpolation technique to interpolate all variables except daily rainfall. Data rejected in the first pass are excluded from the dataset used to construct the interpolated grid in the second pass. Consequently, if a given datum has been rejected by SILO's interpolation system, the fitted surface may differ substantially from the observed datum at that location. Users should note that data rejected by the interpolation system are included in the point (station) datasets and can be identified by their source flag. Users wishing to exclude such observations can replace them with interpolated estimates by requesting the corresponding data at the nearest grid cell.

 Are the data suitable for determining the rates of changes in temperature, rainfall, evaporation, etc., caused by climate change?

SILO data are not intended for use in climate change detection studies. Small data movements caused by climate change can be easily be confounded by changes resulting from instrumental biases and relocating recording stations. For climate change detection we recommend using the Bureau's ACORN-SAT and High-Quality datasets.


 Can I mirror SILO's datasets?

 The gridded datasets are stored on Amazon Web Services' Public Data repository in NetCDF format in an S3 bucket. The NetCDF files can be mirrored using the AWS command line utilities for working with S3 datasets. For example, the monthly rainfall rasters can be mirrored to your current directory using the sync command: 

aws s3 sync s3://silo-open-data/annual/monthly_rain/ .

If you only require to mirror point files for stations, please see our mirroring page.

The point datasets cannot be easily mirrored because:

  • there are a large number of files, approximately 290,000.
  • every file changes every day, though changes are usually in the data for the most recent 12 months. (Data can change for a variety of reasons; see the next question below).

If you need to maintain a mirror of point datasets, please consider:

  • using our API (it is designed for repetitive or automated tasks).
  • mirroring the data for only the subset of locations that you need i.e. not all locations as that would require mirroring a large number of files as discussed above.
  • building your own point datasets from our gridded datasets. Point datasets at grid locations can be built by extracting the relevant pixel values from a time-series of gridded datasets. The gridded data can be efficiently downloaded because they are arranged in annual blocks, with each annual file containing all of the grids for the selected year and variable. Please note this approach is not possible if you are seeking point datasets at station locations.
  • an incremental approach. For example, you could occasionally download data for the entire time period required (e.g. every six months to capture major changes to the data), and frequently update only the most recent 3 months of each dataset (e.g. every week to capture nightly changes to the data).

Please remember SILO data are provided free of charge under the Queensland Government's Open Data program. In addition to the operational cost of maintaining the system, SILO also pays data egress charges for the data downloaded by our clients. We therefore ask you to carefully consider your data requirements before downloading large volumes of point data.


 Why do SILO data change?

 SILO data are constantly evolving. The changes can be a result of changes in the raw data or changes in SILO methodology. Recent datasets grow rapidly as new observational data are added (while some stations report in real-time, it may be many months or even years until data are received from all stations). The data are also subject to corrections and updates by the Bureau, so there can be ongoing changes to the raw data used to construct SILO datasets.

SILO actively seeks techniques for improving data quality and may implement changes which modify our interpolated estimates or derived variables. Users requiring static datasets (e.g. for model calibration) should archive their own copy of the data, as they should not rely on SILO supplying exactly the same dataset at different points in time.


 When does the nightly update occur?

 SILO is updated every night with new data from the Bureau of Meteorology. The update commences at 9:30 pm (AEST) and is usually completed by midnight.


 Do you have any metadata describing SILO datasets?

 Yes. Metadata are included in the NetCDF gridded datasets, and formal metadata are provided on our metadata page.


 How can I import SILO point datasets into Microsoft Excel?

 The data must be in a format that Excel can recognise. When requesting point data in:

  • a customised format, you should select the CSV (comma-separated values) option. Excel will be able to read the .csv file directly.
  • one of the standard (i.e. predefined) formats, the data are delivered as plain ASCII text. To import the data into Excel:
    1. In the ‘File Menu’, select 'Open'
    2. In the pop-up window, change the file filter (in the bottom-right corner) from ‘All Excel Files’ to ‘All Files’
    3. Select the SILO data file and click on ‘Open’
    4. In the ‘Text Import Wizard’ pop-up window, select ‘Delimited’ and click on ‘Next’
    5. Select 'Space' in the list of delimiters and click on ‘Next’
    6. Click ‘Finish’ to load the dataset.
    Please note you may need to slightly modify these procedures depending on your version of Excel.
Last updated: 18 September 2019