|
Inter-Agency Committee on Geomatics and the Canadian Council on Geomatics Canadian Geospatial Data Infrastructure
Report On the CGDI Workshop
Data Integration and Tools for Framework Data
Center for Topographic Information
Sherbrooke
November 3 and 4, 1998
Timothy V. Evangelatos
CGDI National Secretariat Ottawa
December 21, 1998 |
|
Participants:
Background:
A Workshop on Core Data held in Ottawa
on October 15th, 1998 (Reference 1), re-examined some of
the IACG’s earlier conclusions on framework data and then modified
the proposed framework data so that the thrust would be more compatible
with provincial efforts to build regional geospatial data infrastructures.
One of the goals of that workshop was to move from a limited, federal
perspective on framework data, to a more national perspective. Part
of the workshop discussion had focused on how framework data would
be used, and this raised a number of issues of integrating data from
different sources and levels in a geospatial data infrastructure.
It was concluded that many of the data integration issues are independent
of data itself and it was decided to organize a separate workshop
to deal with the technical aspects of data integration and the tools
needed to support that task. Denis De Gagné offered to host
such a workshop at the Center for Topographical Information in Sherbrooke
which was subsequently held on November 4th and 5th,
1998. This report summarizes the discussion and conclusions of the
workshop.
Introduction:
The Chair, Sylvain Latour, introduced
the agenda (Attachment 1) and goals of the workshop. The objective
was to look at tools to support the use of CGDI framework data and
the integration of jurisdictional datasets that form part of that
data framework. Sylvain suggested that on the first day the workshop
should focus on the developing the vision, identifying the precepts,
and defining the problems we wish to solve. On the second day the
focus would be on developing a consensus on a common vision of what
is required, how will it be done and who will do it. The workshop
would conclude with identifying the priorities, summarizing the
conclusions, and agreeing on future actions.
Before asking the workshop participants
to introduce themselves and what they expected from the workshop,
Sylvain proposed a vision for the CGDI. In his view CGDI would deliver
"The Digital Canada" which would eventually provide a transparently
seamless and scaleless web-based access to geospatial data with
navigation capabilities to move horizontally across the country
and to vertically drill down in any area for more detailed information.
Roundtable: When the workshop was
advertised it was hoped to draw participants from areas such as
spatial data suppliers (public and private), users, academia, software
vendors, data integrators, as well as any interested parties who
deal with related data issues. Based upon the organizations represented,
the make up of the participants was roughly: Data Producers – 16,
Tools – 6 and Software Vendor – 3. Of course many of the data producers
are also tool developers and users of other data. The following
table summarizes some of the participants interests and hopes:
| Boyan
Brodaric: The Geological Survey of Canada is working on framework
data with geologists in the provinces and other federal agencies
provide to provide seamless coverage of geological data for
Canada. |
| Pierre
Lermusieaux: Elections Canada is also working with other federal
and provincial agencies. The focus is for a national road network
that will support the enumeration and census processes at both
levels of government. Besides having their data fit into the
overall process there are efforts to reduces duplication of
effort in collecting and maintaining this road data. |
| Paul Morin:
With National Defence, Paul is involved with the development
of tools and technology to suupport spatial data warehousing
and he is heavily involved with international standards activities. |
| Bill Lalonde:
His company CubeWerx Inc is developing new RDBMS components
that make it easy to manage and integrate spatial data with
other data in one database. Bill is interested in the tools
and technology to support such activities. |
| C. Douglas
O’Brien: With IDON Technologies, he works closely with various
geomatics standards groups including TC211 and is interested
in the standards required to support Framework Data. |
| Rick Nyarady:
An application specialist with Universal Systems Ltd., he is
interested in the tools needed to support Framework Data. |
| Marc LeMaire:
With CTI, he is interested in the infrastructure needed to support
the CGDI |
| Paula
Rojas: A member of the CGDI Secretariat, she is interested in
the tools and technology needed to build the CGDI |
| Sandra
Mackie: A member of the CGDI Secretariat, she has a focus on
Communications and would like to hear from the stakeholders
of the CGDI |
| Tim Evangelatos:
A member of the CGDI Secretariat, he has been leading some of
the discussions on Data Framework and is interested in the strategies
needed to create the CGDI |
| Richard
Post: With the Knowledge Integration Branch of DOE, his agency
is working to improve its management of information and make
it more accessible. |
| Ron Gauthier:
With the Knowledge Integration Branch of DOE, Ron is working
to on a Data Management Framework and associated standards and
protocols needed to support the infrastructure in DOE. Climate
change is of special interest. |
| Joseph
McIlhnney: With the Knowledge Integration Branch, DOE, Joe is
an Information Resource Specialist and member of the team. |
| Dale Swan:
With the Knowledge Integration Branch, DOE, Dale is a senior
systems analyst and member of the Knowledge Integration Team. |
| Peter
Paul: With GeoAccess, NRCan, he heads up the National Atlas
development. Being made available through CEONet, the National
Atlas is being integrated with DCW. |
| Denis
Boutin: With CTI, NRCan, he has been involved with building
a database to manage the NTDB data. |
| Rupert
Brooks: With Geo Acess, NRCan, he is working on the generalization
of National Atlas data and he would like to see data sets become
more compatible and easier to integrate. |
| Jacob
Sasson: A Consultant working with the Ontario Ministry of Natural
Resources who are leading a project to assemble the provinces
geospatial data into one system called NRVIS. |
| Peter
Schut: Agriculture Canada proposed the Data Alignment Layer
because it assists them in georegistering their agriculture
data. Peter is particularly interested in the integration of
various types of data. |
| Cathy
Caldwell: Many years ago Statistics Canada (SC) initiated the
development of a street network file to help collect census
data. It has evolved and grown to meet a number of related applications.
SC has partnerships with other departments such as Elections
Canada and NRCan for the extensions and maintenance of this
data. |
| André
Verville: A Consultant with DMR, he is involved in a project
to disseminate geospatial data over the Internet and is interested
in how this may be accomplished with the CGDI. |
| Dianne
Richardson: A scientist with CCRS, NRCan, she is in integrating
satellite and vector data. She is also investigating the generalization
of geospatial data. |
| Salvatore
Carboni: With Global Geomatics, a software vendor who worked
with National Defence to develop the Open Geospatial Datastore
Interface (OGDI), a public domain specification. In an on-going
partnership with National Defence they are continuing to enhance
OGDI as well as incorporating into their software tools. |
| Christine
Parent: With CTI, Christine has be working on the Data Alignment
Layer |
Presentations:
1. CGDI and Framework
Data: Tim Evangelatos gave a short introduction on CGDI and
its approach to framework data (Attachment 3). He discussed the
importance of framework data and illustrated how the ideas for it,
developed by the IACG in 1997 and which has a federal agency focus,
have being broadened to to encompass provincial geospatial infrastructure
requirements (Reference 1).
2. CEONet: Peter
Paul provided an overview of CEONet which is endeavoring to provide
a window for the discovery and access to geospatial data in Canada
and internationally. Standardized searches of catalogues and databases
either incorporated or linked to CEONet are possible. As of November
30, 1998 101 products can be reached and the number is
growing. CEONet also provides a number of on-line tools that are
listed below:
- GeoCommerce - On-line
Electronic Commerce; Includes sub-components for check-out, order-baskets,
reporting, catalogues and store-fronts.;
- GeoGratis - Hosting
of freely available geospatial data sets to provide on-line discovery,
visualisation and download;
- MetaManager - Toolkit
to enable RDBMS’s for FGDC/GEO distributed search and retrieval
without need to modify the database;
- Simple Search Toolkit
- Allows suppliers to make their catalogue/database searchable
through their existing web server;
- Earthscape - Pop-up
map and reference system tool that is available for reuse by suppliers
as part of their own web based search interfaces.
- WebAPI - Capability
for external web sites to invoke any CEONet service (e.g., list
advertisements, search remote databases) or content through a
web URL.
CEONet is not static and major
efforts are in progress to improve access to data. Some of the future
developments are:
Short to Mid Term Plans
- Total overhaul of
user-interface
- Callable client-side
interface
- Unified distributed
search
- Initial International
gateways
- Initial/custom GeoCommerce
Services
- Initial data access
capabilities including GeoGratis
Mid to Long Term Plans
- ISO TC211 support
- Unified external gateways
- Common GeoCommerce
Services
- Integrated data warehousing
- operational end-to-end
applications
Sylvan Latour noted the role
that standards and guidelines are playing in the success of CEONet
and what works there can also be applied to other data sets Also for
the workshop discussions he suggested the CGDI "Service Architecture"
shown in Figure 1. Users, buyers, and value adders would be able to
find data using standardized searches, extract what they want, if
necessary pay for it, and then use competing services to align it,
bring to common scales, projections, formats as well as conducting
other processing to generalize and integrate it.
Figure 1: Suggested
Service Architecture
3. Mercator Alliance - A
Partnership to Deliver Enterprise Level Systems: Bill Lalonde has
been involved in the Mercator alliance since its inception several
years ago. His slide presentation (Attachment 4) covered the history,
thrusts, accomplishments and plans of the Mercator I, II, and III
Projects which all dealt with creating tools and services for geospatial
data infrastructure. Bill mentioned several important technological
advances made by members of the Mercator Alliance. This included
standardized tools enabling RDBMS’ to effectively manage multi-key
information such as geospatial data and the Open Geospatial Datastore
Interface (OGDI) which is providing the keys to access and read
data in native and proprietary formats. One of Bill’s slides, reproduced
in Figure 2, illustrated how the various international standards
are related.
Figure 2: Relationship
of the two ISO standards efforts (TC211 and SQL) and OGIS (Attachment
4)
Sylvain Latour recognized
the importance of government and industry cooperating to solve problems
of data dissemination and interoperability but argued that the workshop
should not endorse any particular solution.
4. Data Interoperability
Research Institute (DIRI): The Open Geospatial Datastore Interface
(OGDI) was developed under the auspices of the Mercator I project
mentioned in the previous presentation. The success of this "Application
Program Interface (API)" has led to a public domain specification
and an initiative to create a non-profit organization to support
its implementation and enhancement. Paul Morin, who chairs this
organization gave a presentation (Attachment 5) on the evolution,
status, and plans for OGDI. Of particular interest is the cooperation
and integration of OGDI, RDA, and CLI that could provide users with
similar, transparent access to all types of data whether or not
it is geospatial. Figure 3 illustrates how these three specifications
provide interoperability for different platforms.
Workshop Precepts:
1. Data Alignment Layer:
For the development of CGDI
there are some fundamental assumptions. One of these is the Data
Alignment Layer. Many agencies, particularly at the federal level
have existing data that is not registered to NAD83, the national
geodetic standard and it is difficult to use the data with other
geospatial data sets. To alleviate this problem the IACG proposed
the construction of a data base of visually identifiable points.
This data base has been built from the NTDB and is now available
on the web at no charge or restrictions.
Figure 3: Merging
functionality of OGDI, CLI and RDA (From Attachment 3)
Sylvain gave a quick summary
of the Data Alignment Layer. It currently consists of over 6 million
points. The current version consists of an ASCII file but in 1999
they will be added to a database which will make it easier to find
and extract points of interest. Sylvain suggested that there are
several ways in which they should be accessed:
- Standard interface
using NTS Sheets
- Centroid and radius
- User defined areas
C. Douglas O’Brien suggested
a 4th option:
- A calculation based
upon a point in a rectangle.
Other suggestions were:
- Query by Attribute
- Query by selected
"frames of reference" or "reference set"
This initiated a discussion
for the need of a flexible system, with a quick response. It was felt
that there is a need for users to agree on a standardized set of points
and they should be identified as such in the database. Dianne Richardson
noted the importance of a seamless, scaleless site of coordinates
and the requirement for different generalization rules for different
data sets. Further, a validation check must be done on the source
data prior to generalization. In addition, as coordinates change due
to the availability of more accurate data, or because the road is
moved, information on the linage of the points must be kept. Questions
were raised on the attributes of the DAL points. For roads intersections
in this version of the DAL, the road numbers are included but names
are not. From Peter Schut’s suggestion that a value adder could provide
more detailed information around the DAL points., it was recognized
that the NTDB would be brought on-line as one of the CGDI layers and
there would be many opportunities for industry to provide new tools.
Other tools for visualizing
the DAL points that were suggested by Peter Schut:
- Link the points on a
given highway via splined arc to assist in visualization road
intersections
- Raster image of the
NTS topographic map area surrounding the identified point, by
single points or as a batch process for selected points, all viewable
in a local viewer.
The next topic to be discussed
was raised by Peter Schut. He asked "how does the user insert the
DAL points into his/her data sets?" Two of his suggestions were:
- Manual assistance using
either web or local tools to add points as a layer; then to click
on the alignment point to bring up the raster context image, and
then click on the correct intersection to tie user point to and
to delete all non-tied points.
- Automated routine to
assign points along a given highway arc.
- When using commercial
software use the two coverages where the alignment points have
been identified and rubber sheet them giving the RMS errors and
allow users to drop the worst points.
2. Standards: There
is broad agreement that international standards must be used whenever
possible. Furthermore standards need to be adopted and supported by
the commercial sector or else they won’t get broad acceptance. The
following summary is extracted from the Sylvain Latour’s Report (Attachment
2):
"All the efforts dedicated in Canada towards a sound Data
Framework should be based on existing standards and practices that
are relevant to our work. Funding should be allocated accordingly.
These standards and practices are (but not limited to):
- The work of the ISO TC-211 http://www.statkart.no/isotc211/
(items in Bold are viewed as particularly relevant for this group)
- Part 1: Reference
model
- Part 2: Overview
- Part 3: Conceptual
schema language
- Part 4: Terminology
- Part 5: Conformance
and testing
- Part 6: Profiles
- Part 7: Spatial
subschema and Spatial Operators
- Part 8: Temporal subschema
- Part 9: Rules for
application schema
- Part 10: Cataloguing
- Part 11: Geodetic
reference systems
- Part 12: Indirect
reference systems
- Part 13: Quality
- Part 14: Quality
evaluation procedures
- Part 15: Metadata
- Part 16: Positioning
services
- Part 17: Portrayal
of geographic information
- Part 18: Encoding
- Part 19: Services
- Functional standards
- Imagery and gridded
data
- The work of the OpenGIS
Consortium (OGC) http://www.opengis.org/
- Feature Geometry
(Topic 1)
- Spatial Reference
Systems (Topic 2)
- Locational Geometry
(Topic 3)
- Stored Functions
and Interpolation (Topic 4)
- Features and Feature
Collections (Topic 5)
- The Coverage Type
(Topic 6)
- Earth Imagery (Topic
7)
- Relations Between
Features (Topic 8)
- Quality (Topic
9)
- Transfer Technology
(Topic 10)
- Metadata (Topic
11)
- The OpenGIS Service
Architecture (Topic 12)
- Catalogs (Topic
13)
- Semantics and Information
Communities (Topic 14)
- ISO/TC-204 Transport
information and control systems http://www.iso.ch/meme/TC204.html
- ISO/IEC JTC1/SC32/WG4
SQL/MM
C. Douglas O’Brien noted that TC211 is
not conducting or planning any work to create standard feature catalogues.
Some individuals feel that they are important and will be needed to
build the CGDI while others feel that the task is too difficult. Similarly,
the data quality model proposed by TC211 is not very mature. The list
of items in the metadata specification is very long and it is important
to ensure that all those providing information on the web are using
harmonized specifications. The ISO TC-204 Committee is close to producing
a final standard. Unfortunately they are no longer harmonizing their
efforts with that of ISO TC-211 but C. Douglas believes that the two
solutions seem to fit at the spatial sub-schema level but the metadata
and catalogue services specifications are different, however he is
confident they will be aligned in due course. Doug also noted that
support for general spatial database querries are provided in SQL/MM
but the actual queries have not been set up.
3. Services: The part of the
discussion dealing with the services required to support framework
data was developed through an interactive process with the participants.
The following summary of this discussion is also extracted from
the Sylvain Latour’s Report (Attachment 2) with minor modifications:
"Both ISO and OGC, as well
as the CEOnet developers, are proposing a service architecture for
the deployment of technology. The following are high level services
that the workshop decided need to be detailed but are seen as the
set of tools for data integration.
Horizontal Alignment
(Done with the Data Alignment Layer (DAL)):
- Access
- By NTS Number
- Centroid with Radius
- Using different projection
system, Datum, Scale
- Define best points
- By sending a cluster
of points
- Visualization
- NTS Maplet
- Satellite image chip
- Insertion in User’s
Data set
- Alignment
- Document Technologies
- Document Issues
- Point Picker (based
in IDs)
- Optimizing approach
- Maintenance
- Projection conversion
- Frequently used points
- Align Horizontal
- Match Semantic
- Align Scale
- Conflate
- Translate Format
- Evaluate Quality
- Establish Link (permanent)"
Many of the proposed services
may be satisfied by tools that will be straightforward to develop
but one area that led to a major discussion was data quality. Discussion
on how to specify and measure it did not lead to any fruitful conclusions.
It was also unclear how useful the ISO TC211 quality standards may
be or how this critical aspect of CGDI should be dealth with. Boyan
Brodaric correctly concluded the discussion by describing the topic
of data quality as a "quagmire."
Pierre Lermusieaux suggested
that some of the issues will gradually be resovled as geosaptial
data warehouse are built. He envisions an evolution to common practices
from which standards for quality will emerge.
Vision of CGDI Evolution:
The second day of the workshop began
with a review of the vision for CGDI. Sylvain suggested we develop
a focus for further work by identifying areas where groups can be
formed to study the specific issues and propose solutions to resolve
them. However, it quickly became clear that the topics for study
were dependent on what particular vision was considered. As it had
been agreed that CGDI would evolve through several stages before
reaching the simple, ideal solutions that have been broadly expressed
and therefore a time-line illustrating this evolution would be useful
for determining priorities and deciding upon further effort. The
group foresaw that there were five distinct phases for CGDI and
these are described below:
1998-2002 - Version I: Discovery
and Access: Covers activities to describe data through standardized
metadata, the availability of tools and technology for interfacing
to proprietary and native formats (e.g. OGDI) and basic tools
for bringing data together in common datums and projections.
- Format Interfaces
- Transformations, Common Datums,
Projections
- Alignment Layer
- Common Terminology
- Standards
2002 – 2005 - Version II: Basic
Data Integration: By this time a preliminary data framework
would be established and accepted. Basic or national core data
sets are available and key data sets can be readily integrated.
The specifications for a new, more comprehensive data alignment
layer would be defined.
- CDAL Version 2.0 Specified
- Basic Core data Implemented
- Basic Conflagration tools available
- Advanced provincial infrastructures
- Generalization tools being implemented
2005 – 2008 -Version III: National
Framework: A national data architecture for CGDI would be
defined, broadly accepted and financed. National implementation
of this framework would be well advanced.
- National Data Framework defined
and accepted
- Implementations started, move
towards virtual data at smaller scales
2008 – 2012 - Version IV:
Interoperability: Data from all levels of government available
and readily integratable
- Substantial data interoperability
- National/Internatioanl Standards
implemented
- National Framework implemented.
2012 - Version V: National
CGDI: The beginning of a new "era."
- Full Data Integration
- Simple, transparent access
- National harmonization on data
policies
Next Steps:
A number of individuals were tasked
with the following actions"
- Peter Schut to prepare a list
of frequently asked questions (FAQ) on using the Data Alignment
Layer to supplement a summary description to be prepared by Christine
Parent
- Sylvain Latour and Tim Evangelatos
to work with the CGDI Web site team to prepare a description of
the Framework Data concept and of the Tools needed to support
that aspect of the CGDI.
- Valerie Hume and her colleagues
to work with the CGDI Web site team to provide more up-to-date
information on geomatics standards progress and issues. This is
to include information and links to the standards specifications
and to terminology.
- Paul Jolicoeur to make information
available on the CGDI Web site on the uses, users, partnerships
and funding available to use, extend and support the Data Alignment
Layer.
- The CGDI Secretariat to make information
available on the CGDI Web site on various related activities including
development opportunities and funding.
- Diane Richardson to organize a
meeting between CCRS and CTI to discuss evolving relationships.
Sylvain proposed that this group avoid
endorsing any commercial or public domain software. As no one objected
this policy was accepted. However, as C. Douglas O’Brien suggested,
there is a complementary need to deal with other aspects of CGDI such
as the relations of the Open Geospatial Datastore Interface (OGDI)
but it could be dealt with under some other group (e.g. The Mercator
Alliance).
Conclusions:
Twenty-four individuals with a diverse
range of backgrounds from federal and provincial government agencies
met with engineers and scientists from industry and had a very successful
workshop. In this and in other related workshops the idea of creating
a national geospatial data infrastructure has had complete acceptance.
Although the economic benefits of such an infrastructure is well
recognized by the geospatial community, major investments and commitments
are required. One of the more significant contributions of this
workshop was the identification of five phases and a proposed timetable
for their evolution. Hopefully this will help the community to focus
on realistic objectives that will eventually lead to an optimal
solution.
CGDI is a complex project but building
in stages will help ensure that the community moves collectively
in a constructive direction. The workshop covered many topics and
agreed upon a set of actions that are described later. Only one
topic was not adequately dealt with and that was geospatial data
quality, a complex, not very well understood issue that is of fundamental
importance. It is recommended that in 1999 a workshop be convened
to raise awareness, understand its place in CGDI and define, as
much as possible, the step needed to adequately deal with it.
References:
1. Evangelatos, T., IACG/CCOG Workshop
on Core Data, Ottawa, October 15th, 1998.
Attachments:
- Proposed Workshop Agenda
- Sylvain Latour, "Outcome
of the CGDI Workshop on Data Integration and Tools, Sherbrooke, November 4
and 5th, 1998.", Sherbooke, November 25, 1998.
- Presentation by Tim Evangelatos, " The
Canadian Geospatial Data Infrastructure (CGDI)."
- Presentation by Wiliam Lalonde, "Mercator
Alliance: A Partnership to Deliver Enterprise Level Systems."
- Presentation By Paul Morin, "Data
Interoperability Research Institute (DIRI)."
- Sylvain Latour, "Data
Integration Services." Sherbrooke, January 5, 1999.
TOP