Installation Guide

This document describes how to install a binary distribution of OPALS on your machine. The first step is to go to the download page to get the latest OPALS release.

Introduction

OPALS is a modular program system consisting of small components (modules) grouped together in packages. As a basic concept, OPALS modules are accessible in three different ways as:

  • Command line executables,
  • Python modules
  • C++ classes (via DLL API)

The overall installation of OPALS is described in section Installing OPALS on Windows, and Installing OPALS on Unix, respectively. A quick cookbook style installation guide for MS Windows is provided in section Quick Installation on Windows. If you are upgrading to a new version, consider a look at the Release notes.

Embedding OPALS modules as C++ classes requires a C++ compiler installation, which is not described in this manual. The structure and contents of an OPALS distribution are explained in section OPALS directory structure.

Quick Installation on Windows

To install OPALS with full functionality on MS Windows operating systems, perform the following main installation steps:

  • Download OPALS and save the self-extracting archive to a temporary folder
  • Extract OPALS to an appropriate $OPALS_ROOT directory (e.g.: C:\opals) for which the logged-on user has write permissions.
  • Copy your keyfile (opals.key) to $OPALS_ROOT/cfg/
  • Run the Redistributable Packages for Microsoft Visual C++ 2008, 2010 and 2015.
    • for 32-bit versions of OPALS:
      $OPALS_ROOT/addons/vcredist/2008/vcredist_x86.exe and
      $OPALS_ROOT/addons/vcredist/2010/vcredist_x86.exe and
      $OPALS_ROOT/addons/vcredist/2015/vcredist_x86.exe
    • for 64-bit versions of OPALS:
      $OPALS_ROOT/addons/vcredist/2008/vcredist_x64.exe and
      $OPALS_ROOT/addons/vcredist/2010/vcredist_x64.exe and
      $OPALS_ROOT/addons/vcredist/2015/vcredist_x64.exe
  • Optionally, install the Matlab runtime to enable opals modules that depend on it.

Following the above procedure results in a portable, fully functional OPALS installation that uses the internal Python interpreter that comes with opals. You may run opals command line executables from any working directory, but you will have to specify their full path. In order to run command line executables without specification of their relative / absolute file paths, - prepend $OPALS_ROOT/opals/ to your PATH environment variable.

Alternatively, you can run OPALS in the built-in OPALS shell ($OPALS_ROOT/opalsShell.bat). Within this pre-configured shell, all necessary environment variables (PATH, PYTHONPATH...) are set automatically and thus, no further installation steps are necessary. The shell additionally offers aliases for the command line version of the OPALS modules (cf here).

For more detailed information about configuring OPALS manually please refer to section Installing OPALS on Windows.

Installing OPALS on Windows

This section describes how to install OPALS on a computer running an MS Windows operating system (2000/XP/Vista/Win7).

Download either the 32bit or 64bit version of OPALS. If you run a 32bit version of Windows, you may only run the 32bit version of OPALS. Otherwise, you are free to install either the 32bit or the 64bit version of OPALS, with the advantage of the 64bit version being that you may access larger amounts of memory.

The OPALS distribution comes as a self-extracting archive containing binaries (executables, shared libraries), documentation, demo data, and demo scripts. Save the archive to a temporary folder, and execute it by double-clicking on the file. You will be prompted to enter a target directory. Enter a path of your choice serving as root directory for the OPALS installation (e.g. C:\opals ). This directory is referred to as $OPALS_ROOT in this manual. Please note that the extraction of the archive does not affect anything apart from $OPALS_ROOT - most notably, the Windows registry is left unchanged, no environment variables are altered, and no DLLs are installed outside of $OPALS_ROOT.

If the Redistributable Packages for Microsoft Visual C++ 2008, 2010 and/or 2015 have not already been installed on your machine, then it is necessary to install them by runnning

  • for 32-bit versions of OPALS:
    $OPALS_ROOT/addons/vcredist/2008/vcredist_x86.exe and/or
    $OPALS_ROOT/addons/vcredist/2010/vcredist_x86.exe and/or
    $OPALS_ROOT/addons/vcredist/2015/vcredist_x86.exe;
  • for 64-bit versions of OPALS:
    $OPALS_ROOT/addons/vcredist/2008/vcredist_x64.exe and/or
    $OPALS_ROOT/addons/vcredist/2010/vcredist_x64.exe and/or
    $OPALS_ROOT/addons/vcredist/2015/vcredist_x64.exe.

Installing Redistributable Packages multiple times will not compromise your machine - so if you are unsure whether they are already installed, then simply run them (again).

Copy your keyfile (opals.key) to $OPALS_ROOT/cfg/.

Optionally, install the Matlab runtime to enable opals modules that depend on it.

Now, OPALS is fully functional. To run OPALS command line executables from a working directory that is different from $OPALS_ROOT/opals, you need to specify their full (relative/absolute) file paths. To open an interactive Python shell where you can load OPALS modules, double-click on $OPALS_ROOT/opals/python.exe. To start the Python Integrated Development Environment (IDE) PyScripter that comes with OPALS, double-click on $OPALS_ROOT/addons/PyScripter/startPyScripter.bat. Kindly note that in order for PyScripter to start successfully its stabler remote Python engine, the logged-on user needs to have write permissions for that folder.

To run OPALS command line executables from any working directory, withouth specification of their relative or absolute file paths, you need to prepend $OPALS_ROOT/opals to your user's or system's PATH environment variable. This may e.g. be done using the Windows GUI:

  • on Windows XP: My Computer->Properties->Advanced->Environment Variables
  • on Windows 7: Start->Search programs and files->environment->Edit environment variables for your account

Prepending $OPALS_ROOT/opals to PATH also makes $OPALS_ROOT/opals/python.exe accessible from any working directory.

The following optional installation steps allow directly starting python scripts (like batch files) within the command line from any directory (There is an alternative way of calling OPALS python scripts that is similar convenient but does not require additional installation steps. Details can be found here):

  • Add .PY to the environment variable PATHEXT
  • Preprend $OPALS_ROOT/packages/python to PATH in addition to $OPALS_ROOT/opals
  • Associate the extension .PY with the Python distribution that comes with OPALS by e.g. double clicking a python file (e.g. $OPALS_ROOT/packages/python/opalsQuality.py) within the Windows explorer and choosing $OPALS_ROOT/opals/python.exe as corresponding program or by executing the following two commands in a command line (with administrator privileges).
    assoc .py=Python.File
    ftype Python.File="$OPALS_ROOT\opals\python.exe" "%1" %*
    Attention: This step possibly requires administrator privileges and may affect existing python installations since there is only one default file association for the overall Windows user session.

For instructions on how to install additional Python packages to the Python distribution that comes with OPALS (internal Python), see section Extending the internal Python installation.

For instructions on how to use an existing Python installation (external Python) in combination with OPALS, see section Configuring an external Python installation.

More information about the integration of Python within OPALS can be found in sections Using OPALS in a Scripting Environment and Using Python Bindings.

Installing OPALS on Unix

This section contains information on how to install OPALS on recent 64-bit Ubuntu systems (14.04 LTS, 16.04 LTS). It might work on other Unix distributions, but has not been tested. Download is provided as a .tar.gz archive, with python as an external dependency. To ensure/install python (Version 2.7), enter

sudo apt-get install python2.7

in the terminal.
To unpack the archive, execute the following command:

tar -xzf opals*.tar.gz

which will create a new folder in your current working directory called "opals". To start the interactive opals shell (a special instance of bash), change the directory and execute opalsShell.sh:

cd opals
./opalsShell.sh

From here, you can use the modules of opals as you can on Windows. However, Module names are case sensitive, e.g. opalsAddInfo can be abbreviated as AddInfo, but NOT as addinfo. The opals Shell also sets up the $PYTHONPATH environment variable for use of the opals modules within a python script or a shell.
Also, please note that the opalsShell does not source your ~/.bashrc file by default, since functionality cannot be guaranteed. If you know what you're doing, you can source it yourself (add source ~/.bashrc to the end of opals.bashrc)

Optionally, install the Matlab runtime to enable opals modules that depend on it.

OPALS bash scripts on Unix

The opalsShell is set up to be used as an interactive command line as well as an interpreter for bash (.sh) script files. You can create executable opals script files by putting a shebang at the first line of your script:

#!/path/to/.../opals/opalsShell.sh
# Any module calls, e.g.
Import -inFile demo/strip21.laz

By changing the path to the opalsShell script in the shebang, you can have a script choose one of multiple installations of opals. If you set the executable bit chmod +x script.sh, you can run the script with ./script.sh or by double-clicking on it in a file manager. It will get executed in the correct environment.
Finally, OPALS for Linux/Unix is still work in progress. If you encounter problems, we'd be happy about a detailed bug report sent to the OPALS developer team

Matlab runtime

Some opals modules require the MATLAB Compiler Runtime to be installed. Currently, they depend on version R2016b (9.1).

Extending the internal Python installation

OPALS comes with a fully functional internal Python interpreter. You may load OPALS modules as Python modules in an interactive Python shell that has been opened by double-clicking on $OPALS_ROOT/opals/python.exe, or in the Python IDE PyScripter by double-clicking on $OPALS_ROOT/addons/PyScripter/startPyScripter.bat. However, you may want to use additional Python packages that are not part of the OPALS distribution.

OPALS Python AddOns

Some Python packages have been found to complement OPALS especially well, namely the Python bindings for GDAL, NumPy, SciPy, and matplotlib. Therefore, the OPALS team provides these Python packages in a separate archive for download on the OPALS download page, section Python AddOns. To extend the internal Python installation with these packages, download a copy of that archive for your platform (32/64 bit), and extract the archive into $OPALS_ROOT.

Kindly note that if you want to use the Python bindings for GDAL using the internal Python installation, then you must load the Python package opals or any OPALS module before gdal - otherwise, either no or the wrong GDAL DLL would be loaded.

Arbitrary Python Packages

If you wish to use Python packages that are not contained in the OPALS Python AddOns, we recommend to use the PyPA recommended Python package manager pip. Python ships with the package ensurepip, which simplifies the installation of pip. Executing the following command in an OPALS shell:

python -m ensurepip

installs pip, which can then be used to install the package of your choice, e.g. PyContracts:

python -m pip install pycontracts

searches for that package on the Python Package Index, downloads and installs it, including all packages that it depends on. Notice, however, that the installation may fail if the required package contains code written in a compiled language. In that case, you may search yourself for a binary installation package in the Wheel format, e.g. on C. Gohlke's website, download the appropriate archive, and supply its file path to pip in place of the package name:

python -m pip install h5py-2.4.0-cp27-none-win_amd64.whl

If a binary installation package in the Wheel format is unavailable, you may instead find a legacy Windows installer (whose file name ends with .exe) for the wanted package. In that case, there is one difficulty: as mentioned in section Installing OPALS on Windows, installing OPALS by extracting the corresponding archive does not affect the Windows registry. This also means that the internal Python installation is not registered. However, legacy Windows installers for Python packages will search the Windows registry and let you choose only registered Python installations as the location where a package should be installed. Fortunately, those installers are also self-extracting archives, whose structure follow a common pattern, and hence, there exists a simple way how to use those installers for unregistered Python installations. To do so, you need to use a program that can inspect the contents of such archives. One such freely available program is 7-zip. E.g. using 7-zip, open the installer of the wanted Python package, and extract the contained directory PLATLIB to $OPALS_ROOT/opals/Lib/site-packages/.

Configuring an external Python installation

If, for some reason, you decide to use a Python installation other than the one that comes with OPALS, there are two prerequisites:

  • the platform (32/64 bit) of that Python installation must match the one of your OPALS installation.
  • currently, OPALS can be used with Python versions 2.6, 2.7, 3.0, 3.1, 3.2, 3.3, 3.4, 3.5, and 3.6

If these conditions are met, then you can re-configure OPALS to use an external Python installation by:

  • deleting or renaming the following files:
  • if using a Python version other than 2.7, then copy the contents of $OPALS_ROOT/opals/python/x.y/ (where x.y matches the version number of the Python installation to use) to $OPALS_ROOT/opals/, overwriting the existing files.
  • if using the Python package gdal, then make sure to import the Python package opals or any OPALS module before importing gdal

In order to make the OPALS Python package accessible to the external Python installation (which allows for loading OPALS modules in the Python interpreter), there are two options:

  • specification via the environment variable PYTHONPATH: if PYTHONPATH is undefined, then create it and set it to $OPALS_ROOT. If PYTHONPATH already exists, then prepend $OPALS_ROOT to it, OR
  • specification via opals.pth: create a text file named opals.pth within the subdirectory Lib/site-packages/ of your external Python installation. Open that text file in an editor, and insert the file path of your OPALS installation ($OPALS_ROOT), using the forward slash (/) as directory separator. As a reference, you may inspect $OPALS_ROOT/opals/Lib/site-packages/opals.pth.
    If you don't have write access to Lib/site-packages/ within your external Python installation's directory, then you may create opals.pth in the directory %APPDATA%/Python/PythonXY/site-packages/ (where "%APPDATA%" is an environment variable configured for your user account, and "XY" must be replaced by the version number of your external Python installation).

Note that your Python installation/IDE may clear PYTHONPATH upon startup, in which case the first option will fail. However, the second option should work in any case.

OPALS directory structure

This section describes the structure of an OPALS distribution. The main directory structure is as follows. If there are differences between Windows and Linux (e.g. opalsShell[.bat|.sh]) the Windows file is allways given first separated by a '|' character and followed by the Linux file:

$OPALS_ROOT/ // OPALS root (=installation) directory. Arbitrary name, referred to as $OPALS_ROOT
opalsShell[.bat|.sh] // Pre-configured OPALS shell
addons/ // Contains addons for OPALS
pal/ // Palette file directory
formatdef/ // OPALS format definitions directory
epsg/ // Spatial reference system definitions files provided
// by EPSG (European Petrol Survey Group)
PyScripter/ // Python integrated development environment (Windows only)
vcredist/ // Microsoft redistributable packages
c++_api/ // C++-Interface root directory
lib/ // C++ shared libraries of all OPALS modules
inc/opals/ // OPALS header files (modules and type declarations)
demo/ // C++ example source code and projects (makefiles)
cfg/ // Global configuration and license file
demo/ // Some test data
doc/ // OPALS documentation
html/ // HTML pages
opalsManual.html // HTML start page
opals.chm // Compiled HTML help file
licenses_3rdPartySoftware/ // Licenses of open source libraries that are used within OPALS
opals/ // OPALS modules command line executables, OPALS python modules and third party DLLs
DLLs/ // Directory used by the internal Python interpreter (Windows only)
Lib/ // Directory used by the internal Python interpreter (Windows only)
python/ // DLLs for using external Python installations (Windows only)
tools/ // common python scripts
python.exe // run a Python shell using the internal Python interpreter (Windows only)
python27.dll // core DLL for internal Python interpreter (Windows only)
pythonw.exe // run a Python shell with GUI support using the internal Python interp. (Windows only)
packages/ // Package scripts root directory
cfg/ // Standard configuration files for package scripts
python/ // Python scripts

directory: opals

The $OPALS_ROOT/opals/ directory contains the command line executables (*.exe|*.), the Python modules (*.pyd|*.so), and C++ shared libraries (opals*.dll|opals*.so) of all OPALS modules as well as a set of external shared libraries (e.g. gdal, xerces, mpfr...).

Furthermore, $OPALS_ROOT/opals/ contains a global python initialization file (__init__.py) responsible for loading all OPALS modules into the python environment.

Windows | Linux:

opals\
opals<Module1>[.exe|.]
opals<Module2>[.exe|.]
opals<Module3>[.exe|.]
...
<Module1>[.pyd|.so]
<Module2>[.pyd|.so]
<Module3>[.pyd|.so]
...
opals<Module1>[.pyd|.so]
opals<Module2>[.pyd|.so]
opals<Module3>[.pyd|.so]

directory: cfg

The cfg directory contains the following files:

cfg\
opals.cfg
opals.key

opals.cfg is the overall configuration file containing default values for a series of program parameters (options) common to all OPALS modules. opals.key is the key file containing licensing information. Please note, that the key file is not part of the OPALS distribution. In fact, if the opals.key file is missing, OPALS automatically runs in demo mode. OPALS supports a full featured demo mode, but the input is restricted to 1 Mio points. If you want to use OPALS for larger projects (in a commercial or educational environment), please contact the OPALS team for obtaining a license.

directory: packages

All OPALS package scripts (opalsPreprocess, opalsQuality, ...) are stored in the subdirectory python. These are python scripts providing standard workflows. A complete list of available OPALS packages can be found here.

directory:: c++_api

The contents are as follows:

Windows:

c++_api\
lib\
opals<Module1>.lib
opals<Module2>.lib
...
inc\
opals\
I<Module1>.hpp
I<Module2>.hpp
...
CustomType1.hpp
CustomType2.hpp
demo\
demo<Module1>.cpp
demo<Module1>.vcxproj
demo<Module2>.cpp
demo<Module2>.vcxproj
...

Each OPALS module is accessible as a C++ class via dynamic linkage. The corresponding (dynamic) link libraries are located in the lib subdirectory as separate module specific library files (on Windows: *.lib/*.dll). For including OPALS modules in custom C++ applications, the module interface header files (I<Module*>.hpp and the header files for custom (OPALS-specific) data types are required. These are located in the inc/opals subdirectory. Furthermore, the demo directory contains a demo source file (demo<Module*>.cpp) for each OPALS module demonstrating the use of the respective module in a C++ programming environment.

directory: demo

Finally, the OPALS distribution contains a number of test data along with some demo scripts. The test data also serve as basis for the examples given in the user manuals.

OPALS installation check

As a final step, it might be a good idea to verify the integrity of the installation.

Execution on the command-line

For a quick check whether opals executables run properly, please open a command window, change to the directory $OPALS_ROOT/demo/, and submit the following commands:

opalsImport -inFile test.xyz
opalsGrid -inFile test.odm -outFile test.tif -interpolation movingPlanes -grid 1
opalsZColor -inFile test.tif -nClasses 20

After running these commands, the following files should exist:

test.odm // OPALS data manager file
test.tif // Digital Surface Model (DSM) grid
test_zco.tif // Z-Coloring of DSM

The derived DSM Z-Coloring should look like like this:

install_test_zco.png
Color coded visualization of the Digital Surface Model

The pre-configured OPALS shell additionally offers aliases for the command line versions of the OPALS modules.

Module name Alias
opalsImport import
opalsExport export
opalsGrid grid
opalsCell cell
opalsZColor zcolor
... ...

Thus, the above commands can even be abbreviated and ...

import -inFile test.xyz
grid -inFile test.odm -interpolation movingPlanes -grid 1
ZColor -inFile test.tif -nClasses 20

... yields the same results as above.

Python bindings

To check your configuration of Python, please open a Python shell. In order to do so, you may open a command prompt, and enter python, or double-click on python.exe. Within that shell, issue the following commands:

import sys, ctypes
# print the absolute path of used python.exe
sys.executable
# on Windows OS only: print the absolute path of used Python DLL
b = ctypes.create_unicode_buffer('', 1024)
ctypes.windll.kernel32.GetModuleFileNameW(sys.dllhandle, b, 1024)
b.value
# print the effective search directories for Python packages
sys.path
# print all loaded Python packages
for module in sys.modules.itervalues():
sys.stdout.write(str(module)+'\n')

Please check if the correct Python executable python.exe has been loaded, the correct Python DLL is in use, and the search directories include $OPALS_ROOT.

To check if the OPALS Python package can be loaded, open a Python shell and issue the following commands:

from opals import Import
print Import.__file__ #python 3 syntax: print(Import.__file__)

If the Python interpreter is able to load the OPALS Python module Import, then its absolute file path is given, which should be $OPALS_ROOT/opals/Import.pyd.

Platform specific restrictions

The following platform specific restrictions apply:

  • MS windows
    • no restrictions
  • Linux
    • Module ICP not yet available
    • Module Rasterize not yet available
    • PyScripter not available under Linux
    • Python: Support for 2.7 versions only (2.6 and 3.x not supported yet)
    • GDAL library is linked without: SCOP DTM, MrSID, ECW