Getting Started

This tutorial gives a brief overview of using OPALS. In a 15 minutes quick tour you will learn some basics about OPALS modules and their parameters and, finally, how to read ALS point cloud data, interpolate a Digital Surface Model (DSM) and derive a color coded raster map of the DSM. The ALS data used in this tutorial are located in directory demo.

For a more detailed description of the underlying concept, please refer to Section Software Concept.

Introduction

OPALS stands for Orientation and Processing of Airborne Laser Scanning data. It is a modular program system consisting of small, well defined components referred to as 'modules'. Each module is accessible in three different ways:

  • as commandline executable from within a DOS/unix shell
  • as Python module from within a Python shell
  • as C++ class via DLL linkage

Modern ALS sensors provide a huge amount of data (several Mio. points per kmĀ²). Thus, efficient management of point cloud data is a precondition for sucessful processing of ALS projects. Within OPALS, the administration of point and line related geometries is based on the OPALS Datamanager (ODM). The ODM acts as a spatial cache and features:

  • high performance spatial queries (nearest neighbour and range queries) using two different spatial indices (K-d-tree for points and R*-tree for all other objects)
  • an extensible adminstration scheme for storing arbitrary attributes (additional infos like echo width, echo number, amplitude, classification, surface normal vector, etc.) on a per point basis.

For a detailed description of the ODM, please refer to Sections OPALS Software Concept and OPALS Datamanager.

OPALS modules and parameters

The primary aim of OPALS is to provide an automatic processing chain for ALS data. Each OPALS module represents a small individual component within the overall process. In the majority of cases, the results of one module serve as input for the next module and so on. Thus, almost all OPALS modules provide input and output parameters (typically: inFile, iFormat, outFile, oFormat). Additional parameters either control the specific structure of the module output (e.g. gridSize, cellSize, tilesize for gridding/rasterization operations) or determine the processing behaviour (interpolation (method), neighbours (count) or searchRadius (for data selection), ...).

Each parameter belongs to one of the following classes:

  • mandatory: i.e., the parameter must be specified or else a static default value must be provided by the module, otherwise processing is not possible. Typical example: input file name (-inFile)
  • estimable: i.e., the parameter must be present, but if omitted, a reasonable value can be estimated within the module by deriving a value based on other parameters. Typical example: output file name (derived from input file name + output specific extension) or grid size derived from the mean point distance of the input data.
  • optional: i.e., the parameter can be skipped and the module will still produce reasonable results.
  • abortive: parameters of this type mostly output some informative text and abort afterwards. Typical example: help

One of the main benefits of OPALS is, that each module is available in three different implementations (commandline executable, python module, C++ class). However, calling the modules and specifying the parameters is slightly different for each variant. Therefore, the next section describes how to handle the different module implementations by means of a simple example.

First example: Deriving a colour coded DSM raster map

For the derivation of color coded raster map of a digital surface model, first, the raw ALS point cloud is read into the OPALS Datamanager (ODM). Subsequently, the surface model is interpolated using a simple interpolation method on the basis of the ODM. Finally, the resulting surface grid model is analysed and a colour coded raster map is derived by applying a colour table depending on the DSM heights. Thus, this rather simple task can be split up into three modules:

  • Module Import to import ALS point cloud data into the ODM
  • Module Grid to interpolate a DSM grid model based in the ODM data
  • Module ZColor to derive a colour coded visualization of the DSM

OPALS modules as commandline programs

The most straightforward way to use OPALS is to execute the command line programs from within a DOS/unix shell. Thus, first start a shell window and change into the proper directory containing the test data (demo).

To print a list of short usage examples, type:

opalsImport -examples

Each module presents a usage screen in case the -help parameter is specified. E.g. for Module Import the command:

will show the following usage screen.

opalsImport: Imports data into the opalsDatamanager format
c 2009-2013 Department of Geodesy and Geoinformation (GEO),
Vienna University of Technology, Austria.
--inFile Vector<Path> input files
--iFormat Vector<String> (=estim) file format [auto, xyz, bxyz, wnp, bwnp,
las, sdw, fwf, odm, gdal, <opals format
def. xml file>]
--trjFile Vector<Path> (=skip) trajectory files
--outFile Path (=estim) output manager file name
--filter String (=skip) modify the input using a (tree of)
filter(s)
--trafo TrafPars3dAffine (=skip) geometrically transform the data during
import
--storeOrder bool (=skip) store natural order of data
--tilePointCount int (=200000) average tile point count
--tileSize double (=skip) tile size of spatial index
--help [=String(=help)] print this usage screen: help on parameters
specific to this module
--helpCommon get help on parameters present in all
modules
--helpHtml open the HTML help specific to this module
--helpChm open the Compiled HTML Help specific to
this module
--version print program version
Type -helpCommon to get help on parameters present in all modules.
Type -helpHtml to open the HTML help page in your browser.
Type -helpChm to open the Compiled HTML help page (with full text search capabilities).
Type -help foo to get extended help on a parameter named 'foo'.

The usage screen contains a list of all module specific parameters (options) together with the respective data types and a short description. If available, default values are printed in parentheses (eg. default tilePointCount = 200.000). Parameters, whose values can be estimated are marked with =estim (e.g. -iFormat) and optional parameters with =skip (e.g. -filter).

Alternatively you can open the reference documentation of the module in the standard browser by typing

opalsImport -helpHtml

or as Compressed HTML Help containing a full-text search using the following command

opalsImport -helpChm

Note: In case you have problems viewing the Compressed HTML Help (CHM) in Windows 7, open the file properties of $OPALS_ROOT/doc/opals.chm and click on the Unblock button.

usr_gs_helpChm.png
Reference documentation of opalsImport viewed as CHM

To get a list of the parameters common to all modules, type:

opalsImport -helpCommon

The following screen will appear:

Parameters present in all modules:
--nbThreads unsigned int (=8) number of concurrent threads
--screenLogLevel LogLevel (=info) verbosity level of screen output
[error|warning|info|verbose|debug]
--fileLogLevel LogLevel (=verbose) verbosity level of log file output
[error|warning|info|verbose|debug]
--logFile Path (=opalsLog.xml) log file path
--cfgFile Path (=skip) configuration file
--paramMapping String mapping of parameters from file to own
parameters
--inParamFiles Vector<Path> (=skip) parameters to import from file
--outParamFile Path (=skip) final parameter export
--scope String (=skip) scope of execution

To get detailed help for a specific parameter (e.g. inFile), type:

opalsImport -help=inFile

or

opalsImport -help inFile

The follwoing detailed description of the parameter inFile together with a general description of the underlying data type (Vector<Path>) will show up:

Detailed help:
--inFile Vector<Path> input files
The specified input file are imported in the same order as defined. To append
data to an opalsDM file, specify the opalsDM file as input and outFile.
General help on type Vector<Path>: Holds a variable number of elements of type
Path.

The only mandatory parameter of Module Import is –inFile specifying the input data sources. Please note, that parameters can also be specified using single hyphens (-inFile), they are parsed case-insensitive (-inFile=-infile) and abbreviations like -inf are allowed. Although abbreviations are allowed, intermediate letters must not be omitted. Thus, the command:

opalsImport -ifil myFile.las

will provoke the following error message:

ERROR 1001: Unrecognized parameters -ifil myFile.las

-iFormat and -outFile are examples for parameters, whose values can be estimated. In case of -iFormat, the input file format is determined heuristically by analyzing the first few bytes of the data file. If no output file name is specified, the name of the resulting OPALS Datamanager file is constructed using the path and name of the input file and replacing the extension by '.odm' being the default extension of OPALS Datamanager files.

Thus, a short and valid command for importing the point cloud of ALS strip 11 is:

opalsImport -inFile strip11.laz

The file strip11.laz is automatically recognized as an ASPRS LAS file indicated by the log message (laz is the compressed version of las):

info: The binary file 'strip11.laz' was recognised as LAS (iformat=las)

Module Import accepts multiple input files as indicated by the container type vector<Path> of parameter -inFile. The data of all input files are merged into a single ODM file. In order to load the points of all ALS strips into a single datamanager file (alsPoints.odm), either specify multiple file paths separated by blanks, following the -inFile parameter:

opalsImport -inf strip11.laz strip21.laz strip31.laz -outf alsPoints.odm

or repeatedly specify parameter -inFile, followed by a single file path for each strip point file.

opalsImport -inf strip11.laz -inf strip21.laz -inf strip31.laz -outf alsPoints.odm

The following pair of files is created during data import:

alsPoints.odm

The OPALS Datamanager file (.odm) contains both, the actual coordinates and all attributes of the point data.

Furthermore, Module Import offers a series of additional parameters for controlling data import (filter, tilePointCount, tileSiz, storeOrder, ...) but all of them are optional. They are described in detail in Section Module Import.

Once the data are successfully loaded into the ODM, we are ready to interpolate a DSM with Module Grid. opalsGrid offers simple interpolation methods like nearest neighbour or moving planes interpolation to quickly create surface grid models. OPALS uses the GDAL - GeoData Abstraction Library for accessing grid/raster models. Thus, the whole variety of GDAL supported data formats (GeoTiff, ESRI ASCII Grid, USGS DEM, DTED, ...) is provided by Module Grid. Again, a usage screen is offered when executing opalsGrid with the -help or even without any further parameter.

--inFile Path input data manager file name
--outFile Vector<Path> (=estim) output gridfile name(s)
--oFormat String (=estim) grid file format [GTiff,AAIGrid,USGSDEM
,SCOP...]
--tileSize unsigned int (=64) tile (block) size
--gridSize double (=1) grid width x/y
--attribute String (=z) attribute for which a grid model shall
be derived
--interpolation GridInterpolator (=snap)
interpolation method
[snap|nearestNeighbour|movingAverage|mo
vingPlanes|movingParaboloid]
--neighbours int (=8) nr of nearest neighbours used for grid
point interpolation
--searchRadius double (=estim) maximum search radius for point
selection
--selMode SelectionMode (=nearest) data selection mode [nearest|quadrant|o
ctant]
--feature Vector<GridFeature> (=skip) additional output of feature grid
models [sigmaz|sigma0|pdens|pcount|exce
ntricity|slope|exposition|normalx|norma
ly]
--filter String (=skip) modify the input using a (tree of)
filter(s)
--limit GridLimit (=skip) 2D clipping window, Syntax:
[center|corner] [(left lower right
upper)]
--help [=String(=help)] print this usage screen: help on
parameters specific to this module
--helpCommon get help on parameters present in all
modules
--helpHtml open the HTML help specific to this
module
--helpChm open the Compiled HTML Help specific to
this module
--version print program version
Type -helpCommon to get help on parameters present in all modules.
Type -helpHtml to open the HTML help page in your browser.
Type -helpChm to open the Compiled HTML help page (with full text search capabilities).
Type -help foo to get extended help on a parameter named 'foo'.

As for Module Import, the only mandatory parameter is the input file name (i.e. the ODM file name in this case). All other parameters are either estimable, optional or dedicated default values are available. Thus, a DSM can simply be created using the following command:

opalsGrid -inFile alsPoints.odm -outFile dsm.tif

This results in a DSM in GeoTiff format (indicated by the .tif extension) with 1m resolution (=default gridSize). Since the interpolation method was not specified explicitely, the default snap interpolation was used. This is the fastest but poorest interpolation method: The ODM point cloud is traversed and the respective grid cell is determined for each ALS point, and the height of the point is mapped to the cell. Subsequent ALS points belonging to the same cell area override the grid cell height. Thus, the snap method is good in case the input points are already in regular structure (some ALS data vendors deliver a raster rather than the original point recordings) but disadvantageous otherwise. To obtain smoother results, we can specify a suitable interpolation method like movingPlanes. If we enter the command...

opalsGrid -inFile alsPoints.odm -outFile dsm.tif -interpolation moving

.. we will get confronted with the following error message:

ERROR 0001: std::exception: Ambiguities while matching value 'moving'.
Could be 'movingAverage', 'movingParaboloid' or 'movingPlanes'

What was wrong? There are two interpolation methods starting with 'moving' (movingAverage and movingPlanes). Thus, the entered abbreviation does not uniquely specify one of the available parameters as indicated by the error message above. The correct command (using movingPlanes interpolation) is:

opalsGrid -inFile alsPoints.odm -outFile dsm.tif -interpolation movingPlanes

Module Grid offers many more parameters to control the gridding process, like grid cell size (-gridSize), the number of neighbouring points used for grid point interpolation ( -neighbours), a maximum search radius for neighbouring points (-searchRadius), etc. All these additional parameters can be specified in one long parameter list, like:

opalsGrid -inFile alsPoints.odm -outFile dsm.tif -interpolation movingPlanes -gridSize 0.75 -neighbours 8 -searchRadius 3 -selMode nearest

However, it is also possible to prepare the parameters on a configuration file, which is especially useful, when surface models have to be calculated for multiple data sets (e.g. multiple ALS strips) in the same way. The simplest way is to create a file named opals.cfg in the current working directory and enter all common calculation parameters in parameter=value syntax. If we create a opals.cfg file with the following contents ...

# Module Grid configuration, project ALS_XX
oFormat=GTiff
gridSize=0.75
interpolation=movingPlanes
neighbours=8
searchRadius=3
selMode=nearest

... we can achieve the same result as in the example above by simply typing:

opalsGrid -inFile alsPoints.odm -outFile dsm.tif

As mentioned above, the use of cfg files allow processing multiple data sets using the same calculation parameters. Thus, to create three separate strip DSMs, we can simply write:

opalsImport -inFile strip21.laz
opalsImport -inFile strip31.laz
opalsGrid -inFile strip11.odm
opalsGrid -inFile strip21.odm
opalsGrid -inFile strip31.odm

Please note, that the point data (LAS files) of strip 21 and 31 have to be imported into the ODM prior to calling opalsGrid.

If you have tried out all the stated commands, you should find (at least) the following DSM grid files in the demo directory:

strip11_z.tif
strip21_z.tif
strip31_z.tif
dsm.tif

All these grid files are GeoTiff rasters containing a single 32-bit-float raster band (the geo-referencing is stored in the header of the Tiff file as well as in the respective .tfw files). Opening the dsm.tif file with an image viewer like IrfanView you should get an image like this:

usr_gs_dsm.png
DSM grid model, pixel values: 32-bit-float, pixel size: 0.75m (IrfanView v4.20)

Since the grid file contains the DSM heights as 32-bit float values, the image looks a bit strange. When using a GIS viewer, like QGIS, we can easily get e.g. a greyscale representation of the DSM. However, OPALS provides Module ZColor to create color coded raster maps (8-bit palette files). Z-colouring is basically a classification task. The translation of height values to colour classes is controlled by palette files. If no specific palette file is at hand, the built-in 'green-yellow-brown' palette is automatically used. The command:

opalsZColor -inFile dsm.tif -nClasses 50

produces the color coded raster map file dsm_zco.tif (GeoTiff) with 50 different height (=colour) classes. The result looks like this:

usr_gs_dsm_zco1.png
Colour coded DSM raster map, pixel size: 0.75m

If we prefer a greyscale representation of the DSM, we simply have to use the following predefined palette file (greyPal.xml) in XML syntax:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<?xml-stylesheet type="text/xsl" href="opalsPaletteStyle.xsl" ?>
<!-- Palette Definition for Opals -->
<OpalsPalette>
<name>Standard Grey Palette</name>
<comment>Black to White</comment>
<classes val="253" />
<values type="RGB255" />
<entry val="BG" R=" 0" G=" 0" B=" 0" comment="background color white" />
<entry val="0" R=" 1" G=" 1" B=" 1" comment="index_min" />
<entry val="100" R="255" G="255" B="255" comment="overflow color = index_max" />
</OpalsPalette>

Now, we can create the height dependent greyscale map by typing:

opalsZColor -inFile dsm.tif -pal $OPALS_ROOT\addons\pal\greyPal.xml -nClasses 50

Please note, that $OPALS_ROOT/addons/pal denotes the default palette file direcory of the OPALS distribution. Since all palette files are searched for in the current working directory and in the default palette file folder (in exactely this order) we can simply write:

opalsZColor -inFile dsm.tif -pal greyPal.xml -nClasses 50

We get the following result:

usr_gs_dsm_zco2.png
Greyscale DSM raster map, pixel size: 0.75m

For more details, please refer to Module ZColor.

OPALS modules as Python modules

The main advantage of using the OPALS command line executables is that they are fully self contained i.e. no other software needs to be installed in order to run the OPALS command line executables. However, there are also a series of disadvantages like limited error handling and scripting possibilities.

Python, instead, is a scripting as well as a full featured object-oriented programming language. Since every OPALS module is available as a python module, OPALS functionality can easily be integrated in the Python environment. OPALS modules can be invoked interactively within the Python shell or even embedded in Python scripts or programs. The following subsection describes how to load and execute OPALS modules within the Python shell. The details about embedding OPALS modules in Python scripts or programs are explained in Sections Using OPALS in a Scripting Environment and Using Python Bindings.

Assuming that you have not configured your OPALS installation to use an external Python interpreter (see Installation Guide), you can start an interactive Python shell e.g. in the following ways:

However, there are enhanced Python shells available, which provide useful features like code completion, syntax highlighting, and call tips. OPALS has an addon for the Python integrated development environment (IDE) PyCharm which, among other things, features such an interactive shell. This addon has to be downloaded from the Downloads section individually. Double-clicking on $OPALS_ROOT/addons/PyCharm/startPyCharm[.bat|.sh] opens up this IDE, where you can choose to open an exisiting project or to create a new one. For details on the first time startup, refer to First time start of PyCharm . In the main window, you can open the interactive shell at the bottom by pressing the "Python Console" button.

usr_gs_py_pycharm.png
PyCharm Python IDE with interactive shell at the bottom

To import ALS data files into the ODM within Python, just as in the previous subsection using the command line executables, the following steps are necessary. First, we have to make the respective OPALS module available. opals is the name of the Python package containing all available OPALS modules. Using PyCharm's tooltip feature, the list of available OPALS modules can be displayed by typing:

import opals
opals.loadAllModules()
opals.

The following tooltip window appears:

usr_gs_pycharm_tooltip_modules.png
PyCharm tooltip: list of available OPALS modules
Before starting with the actual example, we must change into the proper working directory. Within a Python shell this can be done using chdir from the built-in python module os.

from os import chdir
chdir('path/to/demo/directory')

To import the ALS point cloud data into the ODM, we first have to create a Module Import object. Let's call the import object imp.

imp = opals.Import.Import()

The next step is to specify the program parameters, but how? Again, the PyScripter tooltip feature is valuable to get an overview about the available methods of the Module Import module. Simply type ...

imp.

... and the list of available methods is displayed as a list in a tooltip. Besides that list, the documentation string of the currently selected method can be displayed by pressing Ctrl-Q. You can see by scrolling through the list that each parameter can be accessed via it's name as a python property. (see properties). For example, to set/get the input file name (which is the only mandatory parameter of Module Import), the following code snippet is sufficient:

# Set the parameter
imp.inFile = "/path/to/inFile"
# Query the status of the parameter
if imp.inFile is not None:
print "inFile has been set"
else:
print "inFile has not been set"
# Get the parameter
val = imp.inFile
print val

Thus, to set the input file name, select inFile from the list. Typing characters into the shell restricts the entries in the tooltip's list to those methods that start with or contain the characters typed so far. Thus, you may first type in a few characters until only a few items are left in the list:

usr_gs_py_tooltip_method.png
PyCharm tooltip: Import.inFile
The method's tooltip (opened by pressing Ctrl-Q) also informs about the required argument(s) and the(ir) respective internal data type(s). As the internal container type ('Vector') is represented as a list in Python (see Data type mapping), we can specify a Python list of opals::Path (i.e. a string representing a file path). Our Module Import object is told to import the three ALS strip LAS files contained in the demo directory into a new ODM using the following Python command:

imp.inFile = [ "strip11.laz", "strip21.laz", "strip31.laz" ]

Finally, to start data import, type:

imp.run()

Each OPALS python module provides the run() method in order to start the actual processing. So we can summarise that the general sequence for running an OPALS module within Python is:

  • import the OPALS module
  • create a module instance
  • set the module parameters
  • call the run() method

According to this general sequence, the entire process of loading ALS data, calculating the DSM and deriving the height (height dependent) greyscale raster map requires the following python commands:

from opals import Import, Grid, ZColor
imp = Import.Import()
imp.inFile = ["strip11.laz","strip21.laz","strip31.laz"]
imp.outFile = 'alsPoints.odm'
imp.run()
grid = Grid.Grid()
grid.inFile = 'alsPoints.odm'
grid.outFile = 'dsm.tif'
grid.run()
zco = ZColor.ZColor()
zco.inFile = 'dsm.tif'
zco.nClasses = 50
zco.palFile = r'$OPALS_ROOT\addons\pal\greyPal.xml'
zco.run()

Please note that the calculation parameters for the grid inpolation (grid size, nr. of neighbours, etc.) are read from opals.cfg, if available (see Specification of Parameters in Configuration Files). Furthermore, the creation of the greyscale raster map relies on the palette file greyPalette.xml as explained in the previous subsection.

Initialization with parameter values

As an alternative to setting parameter values one-by-one using the respective methods, opals Python modules may be instantiated with parameter values as arguments, allowing for most compact code. Thus, the above code is equivalent to the following:

from opals import Import, Grid, ZColor
Import.Import(inFile=["strip11.laz","strip21.laz","strip31.laz"], outFile='alsPoints.odm').run()
Grid.Grid(inFile='alsPoints.odm', outFile='dsm.tif').run()
ZColor.ZColor(inFile='dsm.tif', nClasses=50, palFile=r'$OPALS_ROOT\addons\pal\greyPal.xml').run()

For more details concerning OPALS Python bindings, please refer to Using Python Bindings. For instructions on how to debug Python scripts, see Debugging OPALS Python scripts.

OPALS modules as C++ classes

Each OPALS module is alo available as a C++ class via DLL linkage. Using the C++ bindings requires solid knowledge about the object-oriented programming language C++ and is only recommended for expert users. However, the C++ bindings offer highest performance and flexibility.

The following subsection demonstrates how to write a very simple C++ program covering the functionality of the above example. Please note, that mandatory preconditions like the availability of a C++ compiler and the necessary customisation steps (include directories, library directories, etc.) are not covered here.

#include "opals/Exception.hpp"
#include "opals/IImport.hpp"
#include "opals/IGrid.hpp"
#include "opals/IZColor.hpp"
#include <iostream>
int main(int argc, char** argv)
{
int errorCode = 0;
try {
//create import, grid and zcolor objects
opals::IImport *import = opals::IImport::New();
opals::IGrid *grid = opals::IGrid::New();
opals::IZColor *zcolor = opals::IZColor::New();
// local variables
inFile.push_back("strip11.laz");
inFile.push_back("strip21.laz");
inFile.push_back("strip31.laz");
dsmfile.push_back( "dsm.tif" );
opals::Path odmfile = "alsPoints.odm";
opals::Path palFile = "$OPALS_ROOT/addons/pal/greyPal.xml";
//set Module Import input parameters and run module
import->set_inFile(inFile);
import->set_outFile(odmfile);
import->run();
//set Module Grid input parameters and run module
grid->set_inFile(odmfile);
grid->set_outFile(dsmfile);
grid->run();
//set Module ZColor input parameters and run module
zcolor->set_inFile(dsmfile[0]);
zcolor->set_nClasses(50);
zcolor->set_palFile(palFile);
zcolor->run();
//delete module objects
import->Delete();
grid->Delete();
zcolor->Delete();
}
{
//in this case an error occured, output error message
errorCode = e.errorCode();
std::cout << e.errorMessage() << std::endl;
}
return errorCode;
}

Please note, that names of the module related methods are the same as for Python (e.g. run(), reset()). Setting and querying parameters in Python is achieved via parameters (e.g. imp.inFile = '/path/to/file'), however. The above example additionally demonstrates the general error handling concept of OPALS via exceptions (which is also available for Python). For more information concerning the C++ bindings, please refer to Using C++ Bindings.