GovHack overview

GovHack is an annual event held simultaneously around Australia to provide an opportunity for teams to explore, mashup and use open government data over a 46 hour period.

For more information see GovHack.org. To register visit the GovHack Eventbrite page

LINC Tasmania datasets

LINC Tasmania has a total of 20 datasets available via data.gov.au.

The 20 datasets comprise of:

• 856,915 records from the popular Tasmanian Names Index
• 55,354 meta-data records of digitised item from the Tasmanian Archives Collection including references to their digitised resources.
• 20,896 meta-data records for series of items sored with in the Tasmanian Archives Collection.
• Geotagged location records for 74 LINC Tasmania locations

Here is a full list of all our open-data resources and their retrospective IDs:

• LINC Tasmania locations

  Tasmanian LINC, Library, Archives & Online Access Centre (OAC) locations by name, address, latitude and longitude. (http://www.linc.tas.gov.au/locations)
  
  

  Resources	Format	ID
  LINC Tasmania Locations - 12/04/2016	CSV	8318261a-0fcc-4794-9435-c0f76765d08a
  LINC Tasmania Locations - 12/04/2016	XML	e957f95d-9f24-48b4-9c44-b11c9d6bbd1d
  LINC Tasmania Locations - 12/04/2016	geoJSON	d0c08118-92b5-41ae-9cef-d12f1a1ca616
  LINC Tasmania GeoJSON Locations - 26/04/2016	geoJSON	05eda516-8afa-421c-a523-eb8e1fd4a212
  LINC Tasmania GeoJSON Locations - 27/06/2016	geoJSON	b5552515-5e65-4a03-81a4-37d9988e5bbf

• Tasmanian Hotels & Property Records (1818 - 1958)

  Tasmanian Hotels & Property Records available via the Tasmanian Names Index. Over 5,000 records.
  
  

  Resources	Format	ID
  Hotels & Properties	CSV	4152dd68-88df-4134-9a9a-47e142442762
  Hotels & Properties	XML	f891b16b-f864-47a2-8ef4-4ecc3e6c3130
  Hotels & Properties	JSON	c392c2fe-ef5e-4ea3-a642-b66bbfdfcd3a

• Tasmanian Births (1803-1933)

  Over 197,759 records (as at 14/4/2016) of people born in Tasmania including some baptisms collected by the Registrar General (1900-1933 baptisms only) and available from the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Births	CSV	8d710758-26ce-46ca-972d-5067db3a75d2
  Tasmanian Births	XML	a7c05fba-f620-4380-9a5e-b34bbb327fb2
  Tasmanian Births	JSON	4e30dd59-f38d-489a-8ee5-817ac70e5f1a

• Tasmanian Marriages (1803-1899)

  Over 103,524 records of people who married in Tasmania. Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Marriages	CSV	fabcdd72-4410-4d56-8abc-799fd33280c8
  Tasmanian Marriages	XML	37ce8ddb-858e-468e-a407-fcf9f54a873a
  Tasmanian Marriages	JSON	48867082-bfca-44d4-a5ef-4a99152bc426

• Tasmanian Departures (1817-1867)

  Over 112,500 records of people who people leaving Tasmanian ports, mainly Launceston. Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Departures	CSV	264a2c46-70a3-4dc7-80f4-13639014874a
  Tasmanian Departures	XML	e8b4a2b7-0d77-458a-8526-ebee523280de
  Tasmanian Departures	JSON	3c4c1621-5919-471b-82a0-d8ac33825d45

• Tasmanian Deaths (1803-1933)

  People who died in Tasmania including some burial records collected by the Registrar General (1900-1933 burials only). Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Deaths	CSV	d6c716f2-b66d-4096-bb8e-fccc454c49a3
  Tasmanian Deaths	XML	f4825fdb-17a5-47ab-bb4e-6b9b21bfe6b5
  Tasmanian Deaths	JSON	bcbed7d3-e0ad-4006-b9da-b4828a83268f

• Tasmanian Convicts (1803-1893)

  Convicts transported to Tasmania and those convicted locally through the convict system. Available from the Tasmanian Names Index.
  Over 78,504 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Convicts	CSV	8d3e12fa-a9fd-4268-9164-6240a97ff44d
  Tasmanian Convicts	XML	0107a43d-c30b-4398-9cbf-d44008ef8101
  Tasmanian Convicts	JSON	36a08ac4-c7a8-4f29-b383-161408a70ecb

• Tasmanian Wills (1824-1989)

  Wills and letters of administration of estates registered for probate. Over 104,122 records (as at 14/4/2016). Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Wills	CSV	3e4a55d2-554f-4577-a530-7fcb2d255060
  Tasmanian Wills	XML	880d70fb-10d3-4982-bf9c-313ac9f774c7
  Tasmanian Wills	JSON	752573d2-3c05-44c3-b8fa-fcc4b79ac4db

• LINC Tasmania Digitised Archives

  LINC Tasmania's collection of digitised archives including files, letters, manuscripts, maps, plans, photographs films and more.
  
  

  Resources	Format	ID
  Digitised Archives	CSV	ee8ec788-77ed-4faf-8ceb-17f046ab2a4f
  Digitised Archives	XML	b3bbc43c-d8ed-4515-8d13-a925a2e3bb0b
  Digitised Archives	JSON	2eb9e1bd-6e6c-4541-be0d-e3166b5bd84a

• Tasmanian Convicts - permission to marry (1829-1857).

  Convicts applying to marry free people or other convicts. Over 280,000 records (as at 14/4/2016). Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Convicts - permission to marry	CSV	fd0f27a5-e2af-4ec7-b782-ee791a56acd6
  Tasmanian Convicts - permission to marry	XML	c94a7da4-b2e9-411b-b091-015d3a348ca8
  Tasmanian Convicts - permission to marry	JSON	886d14db-11a9-40ed-9295-e35d267b9fb3

• LINC Tasmania Archives Series

  Descriptive records for series held by LINC Tasmania
  
  These records can also be viewed online here:
  https://linctas.ent.sirsidynix.net.au/client/en_AU/tas/search/results?te=ARCHIVES_SERIES
  
  

  Resources	Format	ID
  Archives Series	CSV	4ca7e432-e3f6-411d-adf3-1eab2efe2147
  Archives Series	XML	1b03a062-d376-4304-be64-125efc180539
  Archives Series	JSON	ec6d7702-ea1b-44ba-a85a-3b61d6082e68

• Tasmanian Arrivals (19th Century)

  Passengers and ships arriving, mainly in Hobart. Contains surnames A-K, L-Z to be added. Available from the Tasmanian Names Index. Over 52,918 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Arrivals	CSV	7d2c5531-94ec-474e-b60c-744b9d0cbbb8
  Tasmanian Arrivals	XML	bf619908-593a-44f2-9d7f-e14629d323a8
  Tasmanian Arrivals	JSON	6864c5f2-3489-4ef3-acb7-0b353e2ab5c1

• Tasmanian Newspaper Index

  The Tasmanian Index provides references to articles in the three metropolitan Tasmanian newspapers; the Mercury, the Examiner and the Advocate, and to most other Tasmanian journals and regional newspapers. The index was maintained from 1966 to 2010. This online version of the Tasmanian Index covers 1994 to 2010 with approximately 20,771 records. The records from 1966 to 1994 are available online as PDF files
  
  

  Resources	Format	ID
  Tasmanian Index	CSV	973d05c4-2f17-4609-bd98-1b96135d69c4
  Tasmanian Index	XML	37c714be-8ddd-4069-b131-bda352979a7f
  Tasmanian Index	JSON	f0feae8b-d02a-467c-9329-e37a9c2f38ac

• World War One Tasmanian photographs (1914-1919)

  Photographs and articles from the Tasmanian Mail and the Weekly Courier. Data indexed by Wendy Knolle and kindly supplied by the Tasmanian Family History Society, Hobart Branch The dataset is available via the Tasmanian Names Index.
  Over 8,250 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  World War One Tasmanian Photographs	CSV	2c27bb2e-c55f-4606-af72-85d3ed69049c
  World War One Tasmanian Photographs	XML	4c2a9c85-12dc-401a-a5b5-07d39f95dbae
  World War One Tasmanian Photographs	JSON	e747d968-905c-4b92-9910-95cc824a0fdf

• Tasmanian Naturalisations (1835-1905)

  People applying to become citizens. Available via the Tasmanian Names Index. Over 1,190 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Naturalisations	CSV	6e4a4678-9845-42da-a7e1-99f00aee2028
  Tasmanian Naturalisations	XML	a871cd24-9518-4768-9a44-4731cb582a10
  Tasmanian Naturalisations	JSON	e2060a5c-9b42-4f14-9a1b-bcc7bef88fe7

• Tasmanian Prisoners (1895-1902)

  Over 450 records (as at 14/4/2016) of prisoners in the Hobart gaol. Available via the Tasmanian Names Index.
  
  

  Resources	Format	ID
  Tasmanian Prisoners	CSV	2dc53c91-40a6-4eac-8b12-949bf0278bf2
  Tasmanian Prisoners	XML	1e1ef73d-b096-455b-95f1-0147549f9bb6
  Tasmanian Prisoners	JSON	441035ce-d276-4112-a871-94d51bb8714f

• Tasmanian Inquests (1828-1930)

  Inquests into people's deaths. Available via the Tasmanian Names Index.
  
  Over 13,750 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Inquests	CSV	59ca1b56-51a6-47ef-b1be-1d5064697599
  Tasmanian Inquests	XML	42e0e4a6-3539-48ea-bdb7-40165ab8dcc5
  Tasmanian Inquests	JSON	980aa7af-45ce-49a8-a813-013d20c52011

• Tasmanian Health & Welfare Records (1829-1873)

  Includes records from the New Norfolk Colonial Hospital and the Queen's Orphan School. Available via the Tasmanian Names Index.
  
  Over 386 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Health & Welfare Records	CSV	ba3d9d0f-df12-4016-b3b4-23fc84fb77b8
  Tasmanian Health & Welfare Records	XML	35ce31a8-2298-4085-bdcd-cef2f454d4ff
  Tasmanian Health & Welfare Records	JSON	ee76eb11-739e-4775-8186-33c7d1022427

• Tasmanian Divorces (1861-1920)

  Petitions for divorce in the Tasmanian Supreme Court. Available via the Tasmanian Names Index. Over 448 records (as at 14/4/2016).
  
  

  Resources	Format	ID
  Tasmanian Divorces	CSV	90996377-6eac-4d90-b72c-2eacde42a7d8
  Tasmanian Divorces	XML	7db2ad9b-5cd2-420f-83aa-c3b71a6d20e7
  Tasmanian Divorces	JSON	278fcf1b-5a21-48eb-82bf-7e7f1ddd3ef2

• Tasmanian Census (1837-1857)

  Householders in the 1840s and 1850s, not complete for all districts. Available from the Tasmanian Names Index. Over 13,750 records (as at 14/4/2016)
  
  

  Resources	Format	ID
  Tasmanian Census	CSV	3e8ac94c-0e5a-44dd-9a61-a8a08707650d
  Tasmanian Census	XML	80ea84b3-2983-41ec-9489-843f4d338c61
  Tasmanian Census	JSON	5ae7c468-b57f-4b8d-a057-94d749fe5042

Data formats

LINC Tasmania provide our open data in to 3 main formats.

• JSON

  JSON (JavaScript Object Notation) is a syntax for storing data and works well with opendata sets. It's very easy for the user to fetch and manipulate and is easily parsed in many different programming languages. JSON is a good choice for processing data at both the server side and with javascript on the client side.
  
  Note: Our datasets are very large and are probably too large to be loaded at the client side so we recommend only using JSON at the client side if you are pulling smaller sections of a dataset via the data.gov.au API.
  
  Here is an example of some JSON data that we provide:

  {
      "1449570": {
          "CATKEY": "(Sirsi) 1449570",
          "FORMAT": "VIEW",
          "INDEX": "Prisoners",
          "LINC_TAS_AVAIL": "Online",
          "NAME": "Allen, William",
          "NAME_FULL_DISPLAY": "Allen, William",
          "PERMA_LINK": "https://linctas.ent.sirsidynix.net.au/client/en_AU/all/search/detailnonmodal/ent:$002f$002fNAME_INDEXES$002f0$002fNAME_INDEXES:1449570/one",
          "PUBDATE": [
              "1900",
              "1901",
              "1902"
          ],
          "PUBDATE_RANGE": "1900-1902",
          "REFERENCE_URL": {
              "URL_TEXT": "GD128/1/1",
              "URL": "https://stors.tas.gov.au/GD128-1-1$init=GD128-1-1p155j2k"
          },
          "TASMANIAN": [
              "Published in Tasmania",
              "About Tasmania",
              "By a Tasmanian"
          ],
          "YEAR": "1900-1902",
          "YEAR_DISPLAY": "1900-1902"
      }
  }
  

• XML

  XML is a mark-up language and works very well at storing our open datasets due to their hierarchical nature. XML and JSON provide a basically the same data but you will most likely require an XML parser to navigate the XML document or possibly use regular expressions to extract the required data. Being that XML supports attributes the data may be represented in a slightly different way in XML than the same data in JSON.
  
  Here is an example of the same record in XML:

  <?xml version="1.0"?>
  <records>
  	<record id="1449570">
  		<PERMA_LINK>https://linctas.ent.sirsidynix.net.au/client/en_AU/all/search/detailnonmodal/ent:$002f$002fNAME_INDEXES$002f0$002fNAME_INDEXES:1449570/one</PERMA_LINK>
  		<REFERENCE_URL>
  			<URL_TEXT>GD128/1/1</URL_TEXT>
  			<URL>https://stors.tas.gov.au/GD128-1-1$init=GD128-1-1p155j2k</URL>
  		</REFERENCE_URL>
  		<TASMANIAN>Published in Tasmania</TASMANIAN>
  		<TASMANIAN>About Tasmania</TASMANIAN>
  		<TASMANIAN>By a Tasmanian</TASMANIAN>
  		<INDEX>Prisoners</INDEX>
  		<PUBDATE_RANGE>1900-1902</PUBDATE_RANGE>
  		<NAME_FULL_DISPLAY>Allen, William</NAME_FULL_DISPLAY>
  		<LINC_TAS_AVAIL>Online</LINC_TAS_AVAIL>
  		<NAME>Allen, William</NAME>
  		<FORMAT>VIEW</FORMAT>
  		<PUBDATE>1900</PUBDATE>
  		<PUBDATE>1901</PUBDATE>
  		<PUBDATE>1902</PUBDATE>
  		<YEAR>1900-1902</YEAR>
  		<YEAR_DISPLAY>1900-1902</YEAR_DISPLAY>
  		<CATKEY>(Sirsi) 1449570</CATKEY>
  	</record>
  </records>
  

• CSV

  CSV is just simple comma-separated values. We provide all of our data in CSV because data.gov.au is able to produce an API queryable dataset from CSV records. The issue with CSV is it's difficult to represent hierarchical data structures so we have used delimiters like pipe to separate values within a single value. Due to this we recommend using XML or JSON when possible for a more complete representation of the data.
  
  Here is an example of the same record in CSV:

  CATKEY,FORMAT,INDEX,LINC_TAS_AVAIL,NAME,NAME_FULL_DISPLAY,PERMA_LINK,PUBDATE,PUBDATE_RANGE,REFERENCE_URL,TASMANIAN,YEAR,YEAR_DISPLAY,NAME_SEE_ALSO
  "(Sirsi) 1449570","VIEW","Prisoners","Online","Allen, William","Allen, William","https://linctas.ent.sirsidynix.net.au/client/en_AU/all/search/detailnonmodal/ent:$002f$002fNAME_INDEXES$002f0$002fNAME_INDEXES:1449570/one","1900|1901|1902","1900-1902","GD128/1/1|https://stors.tas.gov.au/GD128-1-1$init=GD128-1-1p155j2k","Published in Tasmania|About Tasmania|By a Tasmanian","1900-1902","1900-1902",""

Using the open datasets

There are countless different ways of using the open datasets but here is an example of how to get started.

For this example I will use PHP to calculate the top 10 most popular baby names from 1803-1933.
I will start off by writing a function for fetching json data from data.gov.au

/*
Function for fetching JSON from CKAN installations
- LINC Tasmania (www.linc.tas.gov.au)
*/
function fetchJSON($id) {
	//CURL options
	$ckanApiAddress = 'http://data.gov.au/api/';
	$proxyHost = 'proxy.education.tas.gov.au'; //insert proxy host or false to not user a proxy
	$proxyPort = '8081';
	$proxyUser = false; //insert proxy host or false to not user a proxy
	$proxyPassword = '';
	
	//CURL init
	$ch = curl_init();
	curl_setopt($ch, CURLOPT_HEADER, 0);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	if ($proxyHost !== false) {
		curl_setopt($ch, CURLOPT_PROXY, $proxyHost);
		curl_setopt($ch, CURLOPT_PROXYPORT, $proxyPort);
		curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
		if ($proxyUser !== false) {
			curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyUser . ':' . $proxyPassword);
		}
	}
	
	//Fetch download URL
	curl_setopt($ch, CURLOPT_URL, $ckanApiAddress . 'action/resource_show?id=' . $id);
	$data = curl_exec($ch);
	
	$data = json_decode($data);
	$dlUrl = $data->result->url;
	
	//Downloading JSON
	curl_setopt($ch, CURLOPT_URL, $dlUrl);
	$data = curl_exec($ch);
	$data = json_decode($data);
	curl_close($ch);
	
	return $data;
}

Using this function I will fetch the births dataset and count the occurrence of every name.

$birthRecords = fetchJSON('4e30dd59-f38d-489a-8ee5-817ac70e5f1a');

$names = array();
$count = 0;
foreach ($birthRecords as $record) {
	$givenName = $record->NAME_FULL_DISPLAY;
	if (strpos($givenName,', ') === false) continue;
	if (strpos($givenName,'Not Recorded') !== false) continue;
	$count++;
	$givenName = substr($givenName, strpos($givenName,', ') + 2);
	if (isset($names[$givenName])) $names[$givenName]++;
	else $names[$givenName] = 1;
}
//Sort name data desc
arsort($names);

echo 'Top 10 most popular baby names from 1803-1933', "\r\n";
$i = 0;
foreach ($names as $name => $num) {
	echo $name, ' (', $num, ' of ', $count, ') ', number_format( $num/$count * 100, 2 ), '%', "\r\n";
	$i++;
	if ($i == 10) break;
}

After running our code we get the following output:

Top 10 most popular baby names from 1803-1933
John (3800 of 191615) 1.98%
William (3568 of 191615) 1.86%
James (2616 of 191615) 1.37%
Mary Ann (2359 of 191615) 1.23%
Thomas (2104 of 191615) 1.10%
Elizabeth (2024 of 191615) 1.06%
Mary (2021 of 191615) 1.05%
George (1827 of 191615) 0.95%
Henry (1365 of 191615) 0.71%
Charles (1207 of 191615) 0.63%

As you can see it's relatively easy to perform reports on JSON datasets using PHP.

Using the data.gov.au API

data.gov.au provides an API for querying datasets within their collection. Unfortunately only our CSV datasets are currently available via this API.
The API URL for Data.gov.au is http://data.gov.au/api/ and you can view the full API guide here.

The CKAN API is useful for front end ajax style calls.
One of the abilities made available by the API is the ability to perform SQL syntax queries against datasets
You just use the ID as the table name and field names from the first row of the CSV to perform your SQL query.
Note: you must wrap the table and field names in double quotes.

Here is an example of a SQL statement that I can use to query a dataset:

SELECT count("DEPARTURE_PORT") as count, "DEPARTURE_PORT" from "8d3e12fa-a9fd-4268-9164-6240a97ff44d" WHERE "DEPARTURE_PORT" != '' GROUP BY "DEPARTURE_PORT" ORDER BY count DESC LIMIT 10

To perform that SQL query I simply encode my SQL statement and pass it via a query string parameter like so:

https://data.gov.au/api/action/datastore_search_sql?sql=SELECT%20count(%22DEPARTURE_PORT%22)%20as%20count,%20%22DEPARTURE_PORT%22%20from%20%228d3e12fa-a9fd-4268-9164-6240a97ff44d%22%20WHERE%20%22DEPARTURE_PORT%22%20!=%20%27%27%20GROUP%20BY%20%22DEPARTURE_PORT%22%20ORDER%20BY%20count%20DESC%20LIMIT%2010

The API will respond in JSON which makes it very easy to parse the response.

Here we have an example using AJAX to fetch the JSON from the API and parse it in to a response

$.ajax({
    url: 'https://data.gov.au/api/action/datastore_search_sql?sql=SELECT%20count(%22DEPARTURE_PORT%22)%20as%20count,%20%22DEPARTURE_PORT%22%20from%20%228d3e12fa-a9fd-4268-9164-6240a97ff44d%22%20WHERE%20%22DEPARTURE_PORT%22%20!=%20%27%27%20GROUP%20BY%20%22DEPARTURE_PORT%22%20ORDER%20BY%20count%20DESC%20LIMIT%2010',
    dataType: "jsonp",
    success: function( response ) {
		var output = "Top 10 departure ports\n";
		$.each(response.result.records, function(k,v) {
			output += v.DEPARTURE_PORT + ": " + v.count + "\n";
		});
		alert(output);
    }
});

You can execute the example code with the button bellow:
Execute

Licence information

Our open data collection is covered by the Creative Commons Attribution 4.0 International licence.
More information on this licence can be found here.