On my way to the FunGIS LiDAR workshop in Cairns,  I stopped in Adelaide to visit Airborne Research Australia. Flinders University recorded my talk on the capabilities for LiDAR processing with LAStools. In this introduction I use a recent project in the Canary islands, Spain, in Spring of 2012 as the example. We created canopy height maps, forest density grids with LAStools and showed that the LiDAR already owned by the municipal government agency could be used to significantly improve their photogrammetrically derived DTMs.

Often I get asked about the difference between las2dem.exe and blastdem.exe – the latter being part of the BLAST extension of LAStools. In the academic video from 2006 below, I outline the core technology details behind BLAST, which can seamlessly compute gigantic Delaunay Triangulations from billions of LiDAR points – using very little main memory. The blast2dem.exe module does not store the gigantic TINs that BLAST produces, but immediately rasters the finalized triangles streaming out of BLAST into equally massive DEMs. This avoids to ever having to store such huge triangulations – that are only needed temporarily – to disk. Currently blast2dem.exe can “only” process up to 2 billion points per file – a limitation that shall be lifted soon.

After attending Silvilaser 2012 in Vancouver and giving a keynote on “The Story of LAStools”, I had a stop over in Halifax and drove a cute little Fiat 500 convertible on a perfect Indian summer fall day to UNB Fredericton where Patrick Adda had organized a half-day workshop on LiDAR Processing with LAStools. Here is the first 54 minutes of my talk all the way through to LiDAR quality checking with LAStools. Special thanks to Houman for filming and to Patrick for editing the video.

Salzburg is a beautiful city in December. The European LiDAR Mapping Forum coincided with the days when the “Krampus” (= “Christmas monsters”) are roaming the Christmas markets in the old town to scare children and adults alike. One gave me a painful whipping in the legs with its leathery tail when I tried to protect a LAStools user … (-;

More to the point, here is my talk at ELMF 2012 on “LASindex – simple spatial indexing of LiDAR data”. I first give a little update on LASzip, then talk about spatial indexing with LAX, before sneak-previewing PulseWaves – our new and open LiDAR format for storing full waveform data.

Some more detail:
Airborne LiDAR surveys collect large amounts of elevations samples, often resulting in Terabytes of data. The acquired LiDAR points are typically stored and distributed in the LAS format or – its lossless compressed twin – the LAZ format. However, managing a folder of LAS or LAZ files is not a trivial task when a survey consists, for example, of 500 flight strips containing around 200 million points each. Even a simple area-of-interest (AOI) query requires opening all files and loading all those whose bounding box overlaps the queried AOI. One solution is to copy the survey into a dedicated data base such as Oracle Spatial or PostgreSQL. We present a much simpler alternative that works directly on the original LAS or LAZ files.

Our minimal-effort spatial indexing scheme has very small setup costs, avoids creating a second copy of the data, and is already in use in the LAStools software suite. For each LiDAR file we generate a tiny LAX file that resides in the same folder as the *.las or *.laz file and has the same name but with a *.lax extension. The LAX files are generally as small as 0.01 percent (for a LAS file) or 0.1 percent (for a LAZ file) of the file containing the LiDAR data and they can be generated as fast as the points can be read off disk.

The LAX files describe an adaptive quadtree over the x and y coordinates of all points. Each occupied quadtree cell stores a list of point index intervals that together reference all points falling into this cell. By merging all intervals of a cell that are less than 1000 apart in point index space we significantly reduce the number of intervals, the size of the LAX files, and the number of file seek operations.

Although individual cells typically reference too many points this is usually amortized as a typical AOI query will require returning a union of all intervals from many quadtree cells. However, our in-place spatial indexing relies on a certain degree of spatial coherency to be present in the point order. A simple measure of the efficiency of the existing order is to calculate the overhead factor when loading each quadtree cell individually from disk.

The source code for LASindex is part of the open source library LASlib of LAStools. It has been extensively field-tested in the LiDAR delivery pipeline of Open Topography (OT) where it is used to efficiently gather data from folders of LAZ files in accordance to area-of-interest queries that are generated by users via OT’s popular web-based LiDAR download interface. Another important use is on-the-fly point buffering. When batch processing, for example, 2km by 2km LiDAR tiles to create DTMs via rasterization of a temporary TIN, it is beneficial to load a 100 meter point buffer around each tile to avoid tile boundary artifacts. The presence of LAX files allows doing so efficiently on-the-fly.

PulseWaves is a new exchange format for full waveform LiDAR data. Below you see a simple visualization of a small sample of full waveform LiDAR data exported from RIEGL’s RiProcess to the PulseWaves format via a prototype of the PulseWaves DLL interface.

Illustrated in 3D are the waveforms of 50 pulses via a green to red shading of their intensity. Plotted in more detail in 2D from left to right are the waveforms of a single pulse that is in the very center of these 50 pulses. The outgoing waveform is red. The returning waveform – in this case consisting of two segments – is blue.

The 3D points (here mostly yellow apart from a few orange and blue ones) correspond to the first recorded waveform sample of each pulse. Through them a particular pulse (or a set of pulses) can be picked by pressing the ‘i’ hot key in the interactive 3D viewer – pulseview.exe – when hovering over a point. The color coding of these points indicates that different pulse descriptors are used for describing the waveforms of the corresponding pulse. For more details go to the discussion forum at http://pulsewaves.org/ where you find links to the specification, the tools, and some sample data.

A few weeks ago, I wanted to demonstrate how geometric compression can shorten download times for online dissemination of 3D archeological artifacts. The demo failed. My web page was gone. The web admin told me later that he had to delete it after receiving an email from a director at CyArk stating that I was “[...] hosting unauthorized content from CyArk [...] ” and that “[...] Dr. Isenburg gained unauthorized access to our information and the re-posted it to his webpage [...]“. Bummer. Unintentionally, I had become “the raider of the CyArk”. A point plunderer. A LiDAR looter. A scan scrounger. A laser pirate … arrr … (-;

How did I fall so low? After reading this LiDAR news article about CyArk’s new online 3D viewer I invested serious time into understanding their content delivery system and suggested how to shorten download times as I had done a lot of prior research on this particular topic (see this, this, this or this page). So, I created several interactive java-based web pages for them to demonstrate how – with some quantization, simple prediction, and clever scripting – more web-efficient 3D content might be possible. I did these experiments with their data sets to allow an apples-to-apples comparison: a point model (Ti’kal) and a mesh model (Mount Rushmore).

After a long technical exchange the person at CyArk suddenly demanded that I take down the compressed content. I was surprised and asked why I should have to delete these illustrative examples on 3D compression that represented a significant investment in volunteered time and energy.

Me: “What you mean with take down? Delete it from my webpages? But I am using it as a purely educational example for scanner precision and coordinate resolution. I am not promoting it in any context that would interfere with the mission of CyArk. I do not quite follow the imperative here. Aren’t you a non-profit site dedicated to science and education? And anyone could download those points clouds from your site just the way I did it. It’s not rocket science … (-:”

Person at CyArk: “And, yes, to clarify my request, I would like you to delete any content from your server or webpages. Sorry if I was vague. Thanks!”

Me: “I believe I am in accordance with both Ben (Kacyra)’s vision and the creative commons license with my educational use of the 3D content (see http://archive.cyark.org/copyright). Is there something I am missing?”

I considered myself well informed about CyArk’s mission on providing open access to 3D data for research, education, and virtual tourism through various media such as Wikipedia and Ben Kacyra’s visionary TED talk. I assumed that my creative commons argument had resonated because I did not hear back from them. I only realized that CyArk was not interested in explaining their licensing but simply had my pages removed when I tried to access this demo.

A few days ago I saw Tom Greaves, executive director at CyArk, commenting “Sweeeet use of CyArk data.” on their blog entry which describes the creation of a sugary fudge replica of Ti’kal – the very same data set that I had been using – for the launch event of a new sugar series by British-based multinational agribusiness Tate & Lyle.

Tik’al Fudge Cake made with golden caster sugar, over 80 cm tall

I like to have fun with LiDAR and appreciate the educational factor of such events. Yet I wonder whether the Guatemalan people would be that much happier to see their ancient cultural heritage presented as a piece of cake to promote a new line of sugars than to see it used as a demo on how to Web-optimize 3D content … (-;

I took this as an opportunity to – once more – inquire about the creative commons license of CyArk and I finally received an answer from Tom.

Dear Dr. Isenburg,

Please understand that only some of [...]     [...] have any questions.

Sincerely,
Tom Greaves
Executive Director
CyArk

Unfortunately Tom did “not recall giving” me his “permission to publish” his “private correspondence” as he pointed out shortly after this blog article went live, so I had to remove the reprint of his email. It essentially said that much of the data collected by CyArk remains property of the site owners and that Cake for Breakfast obtained permission to use the 3D scan as the secret ingredient for their Mayan bake job.

After inquiring with Tom “So which models are creative commons and which not?” I quickly got the surprising response from Tom that: “None of our 3D point cloud is available under Creative Commons. Only some of the 2D image data is covered by this.“

Now this is certainly not what I had been reading into their press releases and news articles. The data-generous openness in access to 3D data that is advertised for example on their mission statement: “Digital Preservation is ‘Preserving cultural heritage sites through collecting, archiving and providing open access to data created by 3D laser scanning, digital modeling, and other state-of-the-art technologies,’ the CyArk Mission.” is apparently not the practiced reality.

So I asked in my LAStools user forum about the experiences of others: What are the most and least permitting licenses for such data and what do they mean in practice? How do I know what is open and what not? Can you help clarifying what “creative commons” licensing means and what it allows and forbids so I don’t violate anyone’s license in the future. This sparked discussions with interesting outcomes:

1. the creative commons (non-commercial) license is useless
   The folks behind @OpenAccessArch picked up the story to provide their view of the particularities of the creative commons (non-commercial) license used by CyArk in a long blog post titled “Creative Commons Non-commercial A Cruel Joke.“
2. it is in general not possible to copyright a LiDAR scan
   Doug Rocks-Macqueen from @OpenAccessArch also started the fundamental discussion whether it is even possible to copyright a LiDAR scan in the first place. Apparently not – at least not for an object whose copyright has already expired and for details read this message thread. His closing argument was that in this legal battle between Meshwerks, Inc. and Toyota Motor Sales U.S.A., Inc. the Court of Appeals affirmed the district court’s opinion that “3D models of physical objects, if faithfully and accurately representing the original, are not original enough to warrant copyright protection.”
3. do not engage in open-washing … (-:
   In CyArk’s defense it needs to be said that there is probably a collision between their vision and the contract terms specified by the different site owners that they have to respect in order to get the permission to scan a site. What is lacking are clear terms of use in their communications and proper protection mechanisms. The academic pioneers of 3D scanning at Stanford university had to deal with similar issues during their Digital Michelangelo Project and created ScanView: a secure client / server rendering system that permits users to examine 3D models, but not extract the underlying data. In summary: do not claim your data is open and allow access to it when it is not.

With all the publicity I was worried that my little rabble rousing might be perceived as disruptive instead of constructive by the community until someone reassured me that: “I think we’re all quite happy that the discussion is happening. Between you and me, CyArk have a reputation as being rather less open with their data than their publicity would suggest.” … (-;

Martin @rapidlasso

PS: Be aware that all comments to this article will be considered “creative commons”. Or maybe not … (-;

The (loosely related) image shown above was obtained here and is courtesy of CyArk. I assume this use is allowed under their copyright and does not violate the creative commons (non-commercial) license … (-;

Addendum (May 1st, 2013): As a result of this article CyArk not only updated their copyright notice to exclude point clouds from the Creative Commons license but also added a very clear data use policy statement.

This will be the part one of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector deliverables such as DTMs, DSMs, CHMs, iso-contours, and TINs with multi-core batch processing.

To get started, download the latest version of LAStools and these 7 example flight strips: strip1, strip2, strip3, strip4, strip5, strip6, strip7 and put them into a folder called ‘.\lastools\bin\strips_raw’. For simplicity we will work inside the ‘.\lastools\bin’ directory, so open a DOS command line window and change directory to where this folder is on your computer, for example, with the command ‘cd c:\software\lastools\bin’ or with ‘cd d:\lastools\bin’ followed by ‘d:’.

I usually first run ‘lasview’ on newly received LiDAR to see if the data makes any sense. The command below will on-the-fly down-sample the data and display only around 5 million points:

C:\lastools\bin>lasview -i strips_raw\LDR*.laz -points 5000000

I usually inspect a few area-of-interests (AOIs) to get a few close-up looks at the data. If we do this several times for different areas it is best to first create a spatial indexing of the strips with ‘lasindex’ which it significantly speeds-up our spatial queries.

C:\lastools\bin>lasindex -i strips_raw\LDR*.laz -cores 3

Only add the ‘-cores 3′ option if you have a computer with at least 4 cores. If you have an 8 core machine you may even add ‘-cores 7′ and so on. Now start ‘lasview’ in the GUI mode either by double-clicking the executable and using the ‘browse …’ roll-out on the left side to load the seven flight strips or simply by adding ‘-gui’ to the command line.

C:\lastools\bin>lasview -i strips_raw\LDR*.laz -gui

Now you can maneuver up and down the file list to highlight the different strips (close the ‘browse …’ roll-out to see the header contents listed on the bottom left). You can again look at all the data by pressing the ‘VIEW’ button or you can inspect a smaller area by first picking a rectangular region as shown in the screenshot below.

In the GUI above you can see that the files have no projection information. We will add this later. The x, y, and z scaling factors are set to 0.001 which means that points are stored with millimeter resolution. Because airborne LiDAR is far from being that accurate we will later change the scaling factor to a more appropriate centimeter resolution. Finally, the creation day and year is missing, which – if we know when the data was flown – will be an easy fix.

After starting ‘lasview’ you can use the <c> hot-key to toggle through the different ways to color the points. Another way to change the point coloring is by presetting it in the GUI as shown above or via the ‘color by …’ menu that opens when you right click.

The coloring by flight-line which is supposed to be stored in the ‘point_source_ID’ field of each point seems rather odd. This field was probably used for some other purpose and not cleaned up before the data was distributed. Not nice and we will fix that later. There are a few other visualization shown below. Coloring points by return turns single returns yellow, first of many returns red, and last of many returns blue. Usually there are also a few green points corresponding to intermediate (neither first nor last) of many returns but there are none in this data set as the ‘lasinfo’ report will confirm later. Toggle through the colorings with the <c> key until it displays the intensities. You may need to press the <=> key a few times to increase the point size. Next, press the <t> key to triangulate the points and then decrease the point size with the <-> key until they have disappeared and you can see a hillside shading of the TIN. Use the <h> key to toggle through different shadings for the TIN triangles.

Before attempting to improve these LAZ files and prepare them for processing we want to make sure that our work will be worthwhile by running a quick visualization based of how well the flight strips fit together. We do this with the ‘lasoverlap’ tool.

C:\lastools\bin>lasoverlap -i strips_raw\LDR*.laz ^
                           -files_are_flightlines ^
                           -utm 19N ^
                           -step 1.0 -max_diff 1.0 ^
                           -o strip_overlap.png

I happen to know that the missing projection in the LAZ files is the UTM zone 19N. I can simply add this to the command line with ‘-utm 19N’. Then ‘lasoverlap’ will produce in addition to the PNG output also a KML file that allows visualizing the overlap and difference rasters within the geo-spatial context of Google Earth.

Serious troubles in the data would be evidenced by void areas in the overlap raster that are obviously caused by poor flight planning (not water bodies) and by deeply blue or deeply red saturated areas in open (=> non-forested) terrain in the difference raster. If either was the case you should send the data back to the vendor to have him fix it … (-:

Now we compare the LiDAR elevations with a set of 29 exactly known control points that were obtained through a ground survey that are listed below. If you want to follow this exercise along you should copy the text below and store it to a file called ‘strips_raw\cps.csv’.

C:\lastools\bin>more strips_raw\cps.csv
Type,Easting,Northing,Z
Open/Paved,273299.68,4715133.88,74.17
Open/Paved,273477.61,4714979.29,73.85
Open/Paved,274001.2,4714540.29,72.5
Open/Paved,273670.13,4714817.4,73.34
Open/Paved,273677.42,4715018.66,74.26
Open/Paved,273400.1,4714528.98,73.15
Open/Paved,274511.59,4714905.97,95.7
Open/Paved,275074.66,4714841.98,120.18
Open/Paved,275409.65,4714994.76,113.41
Field,273321.18,4714946.83,73.46
Field,273601.49,4715101.78,74.35
Field,273646.76,4714972.94,73.97
Field,273890.5,4714457.59,71.13
Field,274650.24,4714903.44,105.13
Field,274522.36,4714829.74,98.59
Field,275474.47,4714780.03,127.63
Field,275636.39,4714868.85,120.72
Field,274747.37,4714932.57,116.11
Field,272795.36,4714503.86,126.9
Forested,272547.02,4714623.09,127.71
Forested,273205.33,4714900.27,79.36
Forested,272530.52,4715045.46,113.48
Forested,275237.48,4715049.57,120.31
Forested,275268.37,4714543.82,104.99
Forested,274666.09,4714497.49,108.86
Forested,274403.56,4715053.43,93.17
Forested,274901.6,4714493.63,114.64
Forested,274658.37,4715072.74,104.49
Forested,274121.73,4714524.51,72.06

The first entry describes the location of the control point and the following three entries are its x, y, and z coordinate. We now compare the seven LiDAR strips against these 29 control points. You can also do thus via the GUI as shown below but you need to make sure to that the ‘keep ground (2) ‘ and the ‘keep keypoint (8)’ check-buttons are not checked since our LiDAR data is not yet classified.

C:\lastools\bin>lascontrol -i strips_raw\*.laz ^
                           -cp strips_raw\cps.csv ^
                           -parse sxyz
 diff,lidar_z,Type,Easting,Northing,Z
 0.1451,74.3151,Open/Paved,273299.68,4715133.88,74.17
 0.0190857,73.8691,Open/Paved,273477.61,4714979.29,73.85
 0.08872,72.5887,Open/Paved,274001.2,4714540.29,72.5
 -0.00419681,73.3358,Open/Paved,273670.13,4714817.4,73.34
 0.054868,74.3149,Open/Paved,273677.42,4715018.66,74.26
 0.0318439,73.1818,Open/Paved,273400.1,4714528.98,73.15
 -0.0720668,95.6279,Open/Paved,274511.59,4714905.97,95.7
 -0.0670072,120.113,Open/Paved,275074.66,4714841.98,120.18
 0.0756029,113.486,Open/Paved,275409.65,4714994.76,113.41
 0.00132683,73.4613,Field,273321.18,4714946.83,73.46
 0.0750162,74.425,Field,273601.49,4715101.78,74.35
 0.00228472,73.9723,Field,273646.76,4714972.94,73.97
 0.166194,71.2962,Field,273890.5,4714457.59,71.13
 -0.392266,104.738,Field,274650.24,4714903.44,105.13
 -0.0705893,98.5194,Field,274522.36,4714829.74,98.59
 -0.423622,127.206,Field,275474.47,4714780.03,127.63
 0.217528,120.938,Field,275636.39,4714868.85,120.72
 -0.12489,115.985,Field,274747.37,4714932.57,116.11
 0.0195411,126.92,Field,272795.36,4714503.86,126.9
 17.0342,144.744,Forested,272547.02,4714623.09,127.71
 9.20693,88.5669,Forested,273205.33,4714900.27,79.36
 26.355,139.835,Forested,272530.52,4715045.46,113.48
 20.2143,140.524,Forested,275237.48,4715049.57,120.31
 14.5363,119.526,Forested,275268.37,4714543.82,104.99
 0.173861,109.034,Forested,274666.09,4714497.49,108.86
 20.5742,113.744,Forested,274403.56,4715053.43,93.17
 0.00432622,114.644,Forested,274901.6,4714493.63,114.64
 23.8816,128.372,Forested,274658.37,4715072.74,104.49
 0.3645,72.4245,Forested,274121.73,4714524.51,72.06
 sampled TIN at 29 of 29 control points.
 avgabs/rms/stddev/avg of elevation errors are
 4.63438/9.61987/8.62324/4.55475 meter. skew is 1.47593.

The output of ‘lascontrol’ could be directed into a file using the ‘-o control.txt’ option. The tool prefixes two numbers to the beginning of every line: the elevation value that was computed from the LiDAR data at the xy location of the control point and the difference between the computed elevation and the elevation of the control point. At the end a summary reports the average absolute error, the relative mean-square error, the standard deviation, and the average error across all the differences. Of course, for control points of type ‘Forested’ we have terrible results because we use all LiDAR points in this computation and not just those that correspond to ground returns. Therefore we will have to repeat this exercise at a later point once we have ground-classified the LiDAR. Via the GUI of ‘lascontrol’ you can zoom in on different areas-of-interests and visually inspect how well the data matches the control points (see below).

Finally let us run ‘lasinfo’ on one one of the strips and also compute the density:

C:\lastools\bin>lasinfo -i strips_raw\LDR030828_211804_0.laz -cd
reporting all LAS header entries:
 file signature:             'LASF'
 file source ID:             0
 global_encoding:            0
 project ID GUID data 1-4:   0 0 0 ''
 version major.minor:        1.0
 system identifier:          'ALSXX'
 generating software:        'ALSXX_PP V2.56c'
 file creation day/year:     0/0
 header size:                227
 offset to point data:       5587
 number var. length records: 3
 point data format:          0
 point data record length:   20
 number of point records:    2404613
 number of points by return: 2210130 0 194483 0 0
 scale factor x y z:         0.001 0.001 0.001
 offset x y z:               0 4000000 0
 min x y z:                  272254.830 4714389.375 65.050
 max x y z:                  275391.197 4714711.445 169.208
 variable length header record 1 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1001
 length after header  5120
 description          ''
 variable length header record 2 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1002
 length after header  22
 description          'MissionInfo'
 variable length header record 3 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1003
 length after header  54
 description          'UserInputs'
 the header is followed by 2 user-defined bytes
 LASzip compression (version 2.0r2 c2 50000): POINT10 2
 reporting minimum and maximum for all LAS point record entries ...
 X  272254812  275391197
 Y  714389375  714711445
 Z      65050     169208
 intensity 0 255
 edge_of_flight_line 0 1
 scan_direction_flag 0 1
 number_of_returns_of_given_pulse 1 2
 return_number                    1 3
 classification      1     1
 scan_angle_rank   -13    13
 user_data           0     0
 point_source_ID     0   255
 WARNING: 2 points outside of header bounding box
 number of last returns: 2209098
 covered area in square units/kilounits: 728996/0.73
 point density: all returns 3.30 last only 3.03 (per square units)
 spacing: all returns 0.55 last only 0.57 (in units)
 overview over number of returns of given pulse: 2014615 389998 0 0 0 0 0
 histogram of classification of points:
 2404613  Unclassified (1)
 real max x larger than header max x by 0.000422
 real min x smaller than header min x by 0.017715

It seems Leica and Optech may have come a bit under “crossfire” (-: by the other big news (besides adding PulseWaves support to RiPROCESS) at the RIEGL LiDAR 2013 user conference in Vienna, namely the unveiling of the new LMS-Q1560 airborne laser scanner. Below you see the management team doing the ceremonial act.

The specification looks great and will especially make those happy looking for dense surveys in complex terrain or for more LiDAR returns from building facades in urban environments. By employing two units that “crossfire” at a particular angle while one unit looks forward and one backward, the combined system can provide a better point spacing on the ground and a better point coverage on vertical surfaces.

The new system combines two LMS-Q780 units each firing at rates of up 400 kHz via a shared rotating mirror in the center. The scan lines of the two laser-beam-emitting units are not perpendicular to the flight direction but angled at +12 and -12 degrees (actual number may be different) in the x-y plane to assure that they independently and consistently sample the covered terrain – unaffected from flying height or aircraft speed. Furthermore, the two fanning sheets of laser pulses are angled at +7 and -7 degrees (actual number may be different) in flight direction, meaning that one unit is forward-looking and the other backward-looking. This increases the likelihood that the sides of buildings are scanned as well. Colloquially put, those back-facing facades perpendicular to the flight direction that are not seen by the first round of laser pulses from the forward-looking unit will be hit by the second round of laser pulses from the  backward-looking unit and vice-versa. The above details may not be 100% correct but are what I (wrongly?) remember from the technical presentation that Dr. Ullrich gave (see below).

The overall ability to sample so rapidly is realized by combining the “crossfire” that I have just described with the “rapidfire” of the earlier Q780 units. Each unit is able to operate with up to 10 pulses simultaneously in the air by subsequently resolving ambiguities off-line with the multiple-time-around (MTA) processing technique introduced for earlier models (see the product descriptions for the Q680i or the Q780 for more on that topic).

Hence, this new scanner with “crossfire technology” promises to give your terrain a dense cross examination … and yes, it is a full waveform scanner. (-:

In August and September of 2013, I spent several weeks in the woods of Poland teaching LiDAR processing with LAStools to four groups of forestry students as part of the ForseenPOMERANIA camp. Wanting to be locally relevant, I switched my usual training data for a 500 by 500 meter tile of LiDAR from nearby forest that I randomly clipped from the massive amount of data that was handed to me. I did not do a test run before the workshop because I trust my algorithms. Imagine the shock when - during my live demo on the first day of teaching - the bare-earth points extracted with “lasground.exe” showed unexpected distortions: large, weird-looking bumps appeared when generating a hillshade with “las2dem.exe” for terrain that was completely flat.

Turns out we found remains of  WW II tank positions. We later drove to the site and verified our findings on the ground.

At that time I had generated a larger 6km by 6km bare-earth hillshade of the area which I showed to a local ranger who pointed out the defensive moat that was dug to stop tanks from advancing. It also became clear that the tanks were aiming towards the north-west, hence at Germany, suggesting they were Russian positions. When you follow the direction the line of tanks are pointing at you can find craters and evidence of the German trenches in the hills.

This was in August. On my return in September, I would meet a retired gentleman whose hobby was a variation of geo-caching: finding old German bunkers and pin-pointing their exact GPS coordinates starting from rough locations on old maps available in souvenir shops and historic records. He confirmed that the positions we found were from late WW-II when Russian forces were advancing fast towards the Eastern seaboard of Germany but were digging in whenever encountering pockets of strong resistance. The saga continues here …

This is part two of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector derivatives such as DTMs, DSMs, building footprints, and iso-contours with multi-core batch processing.

To get started, download the latest version of LAStools and these 7 example flight strips: strip1, strip2, strip3, strip4, strip5, strip6, strip7 and put them into a folder called ‘.\lastools\bin\strips_raw’. For simplicity we will work inside the ‘.\lastools\bin’ directory, so open a DOS command line window and change directory to where this folder is on your computer, for example, with the command ‘cd c:\software\lastools\bin’ or with ‘cd d:\lastools\bin’ followed by ‘d:’. It may be helpful (not mandatory) to follow the tutorial on quality checking first.

Create a new folder called ’.\lastools\bin\tiles_raw’ for storing the LiDAR tiles. Then run ‘lastile.exe’ in GUI mode and load the 7 example flight strips. You can either do this by double-clicking ‘lastile.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lastile -i strips_raw\LDR*.laz -gui

Set the GUI options as shown below to create a buffered tiling from the original flight strips. Check the button ‘files are flightlines’ so that points from different flight lines get a unique flight line ID stored in their ‘point source ID’ attribute. This makes it possible to identify later which points of a tile came from the same flight strip. Use the default 1000 to specify the tile size and request a buffer of 50 meters around every tile. This buffer helps to reduce edge artifacts at the tile boundaries in a tile-based processing pipeline.  Shift the tiling grid by 920 in x direction and by 320 in y direction so the flight strips fit in exactly 4 tiles. This is not mandatory but results in fewer tiles and therefore fewer cuts (experimentally discovered). Requests compressed output to overcome the I/O bottleneck. Set the projection to UTM zone 19 north and the vertical datum to NAVD88. When projection information is missing in the original flight strips it is a good idea to add this as early as possible so it is available in subsequent processing steps.

Press ‘RUN’ and the ‘RUN’ window will pop up. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this almost equivalent command here. It has been enhanced by one additional parameter shown in red. It quantizes the input on-the-fly to a more appropriate centimeter [cm] resolution because airborne LiDAR does not have the millimeter [mm] resolution that the raw flight strips are stored with. See the lasinfo report shown at the end of the tutorial on quality checking to see the excessive millimeter resolution that the additional parameter in red is fixing.

C:\lastools\bin>lastile -i strips_raw\LDR*.laz ^
                        -files_are_flightlines ^
                        -rescale 0.01 0.01 0.01 ^
                        -utm 19N -vertical_navd88 ^
                        -tile_size 1000 -buffer 50 ^
                        -tile_ll 920 320 ^
                        -odir tiles_raw -o fitch.laz

Create a new folder called ’.\lastools\bin\tiles_ground’ for storing the ground-classified LiDAR tiles. Then run ‘lasground.exe’ in GUI mode and load the 4 tiles. You can do this by double-clicking ‘lasground.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lasground -i tiles_raw\fitch*.laz -gui

Set the GUI options as shown below to classify the bare-earth points in all tiles. Select the ‘metro’ setting (which uses a step size of 50m) because there are very large hangars in the point cloud and the ‘ultra fine’ setting for the initial ground estimate. For more details on these parameters read the corresponding README.txt file. Specify the output directory as ’tiles_ground’ and select compressed LAZ output. Select 4 cores only if you have a computer with that many cores. This will assign tiles to different cores to process multiple tiles in parallel.

Press ‘RUN’ and the ‘RUN’ window will pop up. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here:

C:\lastools\bin>lasground -i tiles_raw\fitch*.laz ^
                          -metro -ultra_fine ^
                          -odir tiles_ground -olaz ^
                          -cores 4

Run ‘lasview.exe’ on ‘fitch_273920_4714320.laz’ in the ’tiles_ground’ folder to inspect the result of your bare-earth classification. Press ‘g’ on the keyboard to show only the ground points. After all points have loaded press ‘t’ on the keyboard to triangulate only the points shown. Press ‘-’ to make the points disappear and press ‘h’ multiple times to iterate over the possible ways to shade the triangulation. Now press ‘=’ to make the points reappear, press ‘u’ to show the unclassified (non-ground) points, and press ‘c’ a few times to iterate over the possible ways to color the points.

There are “dirt” points below the ground and “air” points far above the ground such as the one pointed at by the mouse cursor. We remove them with ‘lasheight.exe’. This tool computes for each point the vertical distance to the triangulation of the ground points and stores it in the ‘user_data’ field. We also need these heights for the following step of finding buildings and vegetation. Create a new folder called ’.\lastools\bin\tiles_height’ for storing the cleaned LiDAR tiles with height above ground information. Run ‘lasheight.exe’ in GUI mode and load the 4 ground-classified tiles by double-clicking ‘lasheight.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lasheight -i tiles_ground\fitch*.laz -gui

Configure the GUI with the settings shown below. By default ‘lasheight.exe’ uses the points classified as ground to construct a TIN and then calculates the height of all other points in respect to this ground surface. With ‘drop_above 30′ and  ’drop_below -2′ all points that are 30 meters above or 2 meters below the ground are removed from the compressed LAZ tiles output to the ’tiles_height’ folder. Run the process on multiple cores if possible.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>lasheight -i tiles_ground\fitch*.laz ^
                          -drop_below -2 -drop_above 30 ^
                          -odir tiles_height -olaz ^
                          -cores 4

Next, create a new folder called ’.\lastools\bin\tiles_classified’ for storing classified LiDAR tiles with building and vegetation points. Run ‘lasclassify.exe’ in GUI mode and load the 4 height-cleaned tiles by double-clicking or by entering the command below:

C:\lastools\bin>lasclassify -i tiles_height\fitch*.laz -gui

Set the GUI settings as shown below to identify buildings and trees. Mileage may vary on this step because automatic LiDAR classification is a hard problem. The default settings require a density of at least 2 pulses per square. Setting the step size for computing planarity (or ruggedness) to 3 meters has fewer false positives but misses some smaller buildings. For more details on tuning these parameters see the corresponding README.txt file.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this command-line:

C:\lastools\bin>lasclassify -i tiles_height\fitch*.laz ^
                            -step 3 ^
                            -odir tiles_classified -olaz ^
                            -cores 4

Run ‘lasview.exe’ on ‘fitch_273920_4714320.laz’ in the ’tiles_classified’ folder and be amazed about the result of your building and vegetation classification. Although not 100 percent correct, ‘lasclassify.exe’ identifies all man-made structures as class 6 and marks all vegetation above two meters as class 5.

Create the last folder called ’.\lastools\bin\tiles_final’ for storing the final LiDAR tiles without the 50 meter buffers that are ready to be shipped to the customer. Run ‘lastile.exe’ in GUI mode and load the 4 classified tiles with:

C:\lastools\bin>lastile -i tiles_classified\fitch*.laz -gui

Set the GUI settings as shown below to remove the buffers that were added in the very beginning and carried through all processing steps to avoid artifacts along the boundaries.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this command-line that has one additional parameter shown in red. This sets the user data field of each point back to zero that was set by ‘lasheight.exe’ to the height above ground (in decimeters). This number is meaningless to the customer and setting it to zero improves compression.

C:\lastools\bin>lastile -i tiles_classified\fitch*.laz ^
                        -set_user_data 0 ^
                        -remove_buffer ^
                        -odir tiles_final -olaz

Finally let us run ‘lasinfo’ on one of the generated tiles and also compute the density:

C:\lastools\bin>lasinfo -i tiles_final\fitch_271920_4714320.laz -cd
reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             0
  global_encoding:            0
  project ID GUID data 1-4:   00000000-0000-0000-0000-000000000000
  version major.minor:        1.0
  system identifier:          'LAStools (c) by Martin Isenburg'
  generating software:        'lastile (131011) commercial'
  file creation day/year:     0/0
  header size:                227
  offset to point data:       5689
  number var. length records: 4
  point data format:          0
  point data record length:   20
  number of point records:    2571486
  number of points by return: 2196212 0 375274 0 0
  scale factor x y z:         0.01 0.01 0.01
  offset x y z:               0 4000000 0
  min x y z:                  272044.80 4714466.57 82.34
  max x y z:                  272919.99 4715243.91 169.21
variable length header record 1 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1001
  length after header  5120
  description          ''
variable length header record 2 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1002
  length after header  22
  description          'MissionInfo'
variable length header record 3 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1003
  length after header  54
  description          'UserInputs'
variable length header record 4 of 4:
  reserved             43707
  user ID              'LASF_Projection'
  record ID            34735
  length after header  48
  description          'by LAStools of Martin Isenburg'
    GeoKeyDirectoryTag version 1.1.0 number of keys 5
      key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected
      key 3072 tiff_tag_location 0 count 1 value_offset 32619 - ProjectedCSTypeGeoKey: PCS_WGS84_UTM_zone_19N
      key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter
      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter
      key 4096 tiff_tag_location 0 count 1 value_offset 5103 - VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
the header is followed by 2 user-defined bytes
LASzip compression (version 2.2r0 c2 50000): POINT10 2
LAStiling (idx 8, lvl 2, sub 0, bbox 271920 4.71232e+006 275920 4.71632e+006)
reporting minimum and maximum for all LAS point record entries ...
  X   27204480   27291999
  Y   71446657   71524391
  Z       8234      16921
  intensity 0 255
  edge_of_flight_line 0 1
  scan_direction_flag 0 1
  number_of_returns_of_given_pulse 1 2
  return_number                    1 3
  classification      1     6
  scan_angle_rank   -13    21
  user_data           0     0
  point_source_ID     1     7
number of last returns: 2196301
covered area in square meters/kilometers: 408700/0.41
point density: all returns 6.29 last only 5.37 (per square meter)
      spacing: all returns 0.40 last only 0.43 (in meters)
overview over number of returns of given pulse: 1821027 750459 0 0 0 0 0
histogram of classification of points:
          286793  Unclassified (1)
          589019  Ground (2)
         1586840  High Vegetation (5)
          108834  Building (6)

This is the final part of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector derivatives from the LiDAR points such as DTMs, DSMs, contour lines and building footprints with multi-core batch processing and Tin streaming with BLAST.

It is maybe helpful to start with the tutorial on quality checking. But it is mandatory to first complete the tutorial on LiDAR preparation because that is where you generate the cleaned and classified LiDAR tiles in folders ’tiles_classified’ and ’tiles_final’ that we are using as input in this tutorial.

Generating matching DTM tiles without edge artifacts

Create a new folder called ’.\lastools\bin\tiles_dtm’ for storing the bare-earth digital terrain models (DTMs). Then run ‘las2dem.exe’ in GUI mode and load the 4 tiles from the folder ’tiles_classified’. You can either do this by double-clicking ‘las2dem.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz -gui

The las2dem tool creates rasters from LiDAR by triangulating the points and then sampling the resulting TIN at the center of each raster pixel. Remember, the tiles in folder ’tiles_classified’ have a buffer of 50 meter that was initially created with ‘lastile’ and that has been carried through ‘lasground’, ‘lasheight’, and ‘lasclassify’. The presence of these buffers allows us now to avoid artifacts along the tile boundaries because our TINs will be 50 meters wider in all directions than the raster that we are sampling. Rasterizing only the extend of the original tile without the surrounding buffer is accomplished with the ’tile_bb_only’ command line parameter.

Set the GUI options as shown below to create one DTM per tile. The DTMs will seamlessly fit together. Check the button ‘use tile bounding box’ for the reason explained above and check the button ‘extra pass’ which makes two passes over the data. In the first pass it only counts the number of points that pass through the filter so that in the second it can optimize the memory used to triangulate them. Choose ‘bil’ as the output format. It is much much much more efficient than the ‘asc’ format many people like to use. Select 4 cores if you have that many and set the output directory to ’tiles_dtm’. Use the ‘filter…’ menu to choose two filters: the first keeps only the ground points with filter ‘keep_classification 2′ and the second keeps only one point every 0.5 by 0.5 meter area with the filter ‘thin_with_grid 0.5′. The order is really important (excercise: explain why!). Because we create DTMs with the default ‘step’ size of 1 meter there is no sense in constructing a temporary TIN that is more detailed than one ground point per 0.5 by 0.5 meter area. Constructing a large TIN requires a lot of main memory. We should only use as many points as necessary to sample the TIN at our target resolution ‘step’. Adding a ‘thin_with_grid’ filter with half the ‘step’ size seems always a good choice for ‘las2dem’.

Press ‘RUN’ and the ‘RUN’ window will pop up. You may have to close the ‘output’ menu before you can actually see the ‘RUN’ button. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here.

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz ^
                        -keep_class 2 -thin_with_grid 0.5 ^
                        -extra_pass ^
                        -use_tile_bb ^
                        -odir tiles_dtm -obil ^
                        -cores 4

Generating matching DSM tiles without edge artifacts

Create a new folder called ’.\lastools\bin\tiles_dsm’ for storing the first-return digital surface models (DSMs). You could re-use ‘las2dem’ if you have not closed the application yet. Otherwise re-run it in GUI mode and load the 4 tiles from the folder ’tiles_classified’ again. Do this by double-clicking ‘las2dem.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz -gui

Set the GUI options as shown below. The only changes are that the output directory name is now ’tiles_dsm’ and that the filtering is now keeping first returns instead of ground points. If you re-used the GUI from the last step you need to delete the old filters and enter the new ones in the correct order (exercise: why is the order important?). You can delete old filter entries by double clicking them. Select 4 cores only if you have a computer with that many cores. This will assign tiles to different cores and process them in parallel.

Press ‘RUN’ and the ‘RUN’ window will pop up. You may have to close the ‘output’ menu before you can actually see the ‘RUN’ button. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz ^
                        -first_only -thin_with_grid 0.5 ^
                        -extra_pass ^
                        -use_tile_bb ^
                        -odir tiles_dsm -obil ^
                        -cores 4

Visualizing and processing DEM rasters as if they were points

Here a little trick. All LAStools can read the BIL format “as if” it was a point format. The other two raster formats that are supported this way are ESRI’s ASC and FUSION’s DTM. But if possible don’t use ASC because it stores the values as ASCII text which is terribly inefficient and much much slower to read than BIL or DTM. Run lasview in GUI mode and load the four BIL rasters from the folder ’tiles_dsm’. You need to check the button labelled ‘.bil’ in the ‘browse …’ menu to be able to select the BIL rasters as input. Alternatively you can use this command-line:

C:\lastools\bin>lasview -i tiles_dsm\fitch*.bil -gui

Choose the option ‘selected file only’ and pick one of the four tiles. Play with the ‘color_by’ or ‘render_only’ options. But these can also be changed once ‘lasview’ is displaying the point cloud.

Visualize the ‘fitch_273920_4714320.bil’ from the ’tiles_dsm’ folder. After all points have loaded press ‘t’ on the keyboard to triangulate them. Press ‘-’ to make the points disappear and press ‘h’ multiple times to iterate over the possible ways to shade the triangulation.

Using lasgrid to merge DEM tiles into one large DEM raster

Run ‘lasgrid.exe’ in GUI mode and load the 4 raster DSM tiles in BIL format from the ’tiles_dsm’ folder by double-clicking ‘lasgrid.exe’ and using the ‘browse …’ menu. Check the button labelled ‘.bil’ to be able to select the BIL rasters as input. Or use the following in the DOS command-line:

C:\lastools\bin>lasgrid -i tiles_dsm\fitch*.bil -gui

Configure the GUI with the settings shown below. Check the ‘merge files into one’ button, specify the output file as ‘dsm.tif’, and specify the northern UTM zone 19 and NAVD88 as the projection information because the BIL files we generated earlier do (currently) not store it.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>lasgrid -i tiles_dsm\fitch*.bil -merged ^
                        -utm 19N -vertical_navd88 ^
                        -o dsm.tif

Load the merged raster ‘dsm.tif’ into a GIS package such as QGIS or ArcGIS and repeat the exercise by creating – in the exact same way – a merged ‘dtm.tif’ from the 4 raster DTM tiles in the ’tiles_dtm’ folder.

Using the BLAST extension for seamless generation of rasters

Run ‘blast2dem.exe’ in GUI mode and load the 4 raster DTM tiles in BIL format by double-clicking ‘blast2dem.exe’ and using the ‘browse …’ menu.  You need to check the button labelled ‘.bil’ to be able to select the BIL rasters as input. Or enter this in the DOS command-line window:

C:\lastools\bin>blast2dem -i tiles_dtm\fitch*.bil -gui

Configure the GUI with the settings shown below. Check the ‘merge files into one’ button, specify the output file as ‘dtm_hillshade_raster.png’, select ‘hillside shading’, and choose ‘png’ as the output format. Use the ‘projection …’ menu to specify the northern UTM zone 19 and NAVD88 as the projection information because the BIL files we generated earlier do (currently) not store it.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>blast2dem -i tiles_dtm\fitch*.bil -merged ^
                          -hillshade ^
                          -utm 19N -vertical_navd88 ^
                          -o dtm_hillshade_raster.png

Whenever ‘blast2dem’, ‘las2dem’, ‘lasgrid’, or ‘lasoverlap’ generate rasters that are standard color or gray-scale PNG, TIF, or JPG that can be loaded as an image overlay into Google Earth, a KML is automatically generated – assuming there is projection information either available in the input or specified in the command-line. If you double-click the KML file that we just created you will see the hillshading overlaid and nicely aligned with Fitchburg airport.

Repeat the exercise by creating – in the exact same way – a merged and hillshaded ‘dsm_hillshade_raster.png’ with KML file from the 4 raster DSM tiles in the ’tiles_dsm’ folder.

Using the BLAST extension for seamless generation of contours

Run ‘blast2iso.exe’ in GUI mode and load the 4 BIL raster tiles from the ’tiles_dtm’ folder by double-clicking and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>blast2iso -i tiles_dtm\fitch*.bil -gui

Set the GUI as shown below to compute 1 meter contours from on-the-fly merged bare-earth DTM rasters in BIL format. Check the ‘merge files into one’ button, specify the output file name as ‘dtm_contours_raster_1m.shp’, set the isovalue extraction to every 1 unit (here: meter), and check the buttons for ‘simplify short segments’, ‘simplify small bumps’ and, ‘clean short lines’. Read the corresponding README.txt file if you want more details on what these options exactly do. Add the northern UTM zone 19 as the projection information because the BIL files we generated earlier do (currently) not store it. This is less important when generating SHP output but would be required to produce KML instead.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the command-line before you press START. Or use this command-line:

C:\lastools\bin>blast2iso -i tiles_dtm\fitch*.bil -merged ^
                          -iso_every 1 ^
                          -simplify_length 0.5 -simplify_area 1 -clean 5 ^
                          -utm 19N ^
                          -o dtm_contours_raster_1m.shp

Visualize the result with the convenient ShapeViewer.exe program and check if the 1 meter contours that ‘blast2dem’ has produced are what you expected:

Repeat the exercise by creating – in the exact same way – a merged and hillshaded ‘dsm_contours_raster_3m.shp’ with from the 4 raster DSM tiles in the ’tiles_dsm’ folder but with a spacing of 3 meters instead of 1 meter between the contour lines.

Extracting building footprints from on-the-fly merged tiles

Run ‘lasboundary.exe’ in GUI mode and load the 4 classified tiles with:

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -gui

Set the GUI as shown below to generate compute polygonal outlines around the buildings points classified in the previous tutorial. Check the ‘merge files into one’ button to create one seamless output file. Add a ‘keep_classification 6′ filter that filters out all points except those classified as building. Check the ‘disjoint’ button – otherwise a funny-looking single polygonal hull will be created connecting all buildings. Set the ‘concavity’ to 1.5 meters to specify the smallest feature that lasboundary can grow into. This value needs to be 2 to 3 times the average point spacing. If you set it too small, lasboundary will “eat” its way into the buildings. If you set it too high nearby buildings will be joined together and the outlines will become coarser. Play with a few different settings such as 1.0, 2.0, 5.0, and 10.0 to get an intuitive idea how this parameter affects the result.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the command-line before you press START. Or use the equivalent command-line below directly in the DOS window.

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -merged ^
                            -keep_class 6 ^
                            -concavity 1.5 -disjoint ^
                            -o buildings.shp

Visualize the result with the convenient ShapeViewer.exe program and check if the building footprints that lasboundary has produced are what you expected:

By generating KML output instead and overlaying the result on Google Earth imagery you can easily check which and how many buildings were missed and go back to adjust the parameters for ‘lasclassify’ in the previous tutorial to maybe achieve better results.

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -merged ^
                            -keep_class 6 ^
                            -concavity 1.5 -disjoint ^
                            -o buildings.kml

After accidentally finding Russian tanks in Polish forests I was curious to see if there was something else hiding under the forest canopy. Remember, I randomly picked a 500 by 500 meter LiDAR tile as example data to introduce a group of forestry students to LiDAR processing with LAStools during the ForseenPOMERANIA camp. After extracting ground points with “lasground.exe“, strange bumps appeared in the bare-earth hillshades generated with “las2dem.exe“ for terrain that was supposed to be completely flat … they turned out to be Russian WW-II positions.

I met Achim when returning to teach the next two groups of students. His hobby is a mix between geo-caching and conflict-archaeology: locating old German bunkers based on approximate coordinates available in historic records and tourist maps and then mapping them precisely with GPS. Achim had a list of longitude/latitude positions as KML files where he was planning to search for known bunkers. I used “lasboundary.exe” to create polygonal outlines in KML format for all areas where we had LiDAR from the forestry project. With Google Earth it was easy to find overlaps between his target areas and our LiDAR coverage.

I extracted the ground points and created bare-earth DTMs of the relevant area with a LAStools batch processing pipeline of “lastile.exe“, “lasground.exe” and “las2dem.exe” and used “blast2dem.exe” to create a seamless hillshading with proper KML referencing (here is a tutorial for such a pipeline). What I found was pretty amazing.

At first glance it looks like a maze of little creeks that are running alongside the ridges of the hillsides but we know that water flows downhill and not “along-hill”. What we see is a network of WW-II trenches that are connecting the bunkers Achim is looking for. A closer look also reveals the likely location of those bunkers.

I placed a pin on each of them and exported their longitude and latitude coordinates for upload into Achim’s GPS device. The next day we set out to verify our LiDAR findings on the ground.

It was a rainy day. Walking through this maze of green and overgrown trenches from one moss-covered bunker ruin to the next felt oddly quiet and peaceful. Achim explained that these bunkers were originally built to defend the border with Poland – long before the Second World War broke out. Only when Russian soldiers were advancing on Germany after the collapse of the Eastern front, the young boys of the Hitler Youth were commanded to dig this network of trenches in order to fortify the bunker and stop enemy lines from gaining ground. After the war the Russians blew up all bunkers that were facing East so that their troops would not ever have to face them again.

GRAFCAN has launched a new product based on LiDAR for the Canary Islands: a digital suface model (DSM) that is literally “on steroids”. It is a synthetic view (not image-derived) that lets the user intuitively understand the territory. The product combines standard hillshading with a height-based color-coding enabling the viewer to “see” where the trees are taller and to grasp height differences between buildings. Darker shades of green indicate high vegetation and lighter shades of green indicate low vegetation. For man-made structures, the shade of red is darker for taller buildings and lightest for one-story houses. The new product is available for all of the Canary Islands at a resolution of 2.5 meters/pixel via the GRAFCAN Web viewer and also as a WMS service.

Example of the new DSM on steroids: The viewer intuitively grasps the height and spatial distribution of trees in vegetation areas and the density and size of buildings in urban areas.

The product is complemented with a layer with the water areas and a layer of the main roads (both extracted from topographic maps). Around the airports the terrain is very flat and mostly empty. Therefore an intensity map was extracted from the LiDAR and fused into the imagery to provide additional details.

Comparison between orthophoto and DSM on steroids: The ridge is practically invisible in the ortho and the dark green colors of the vegetation are quite missleading to the untrained eye.

The LiDAR was flown between 2011 and 2012 with a Leica ALS60 with an average density of 1 pulse per square meter. Most of the LiDAR processing was done with LAStools batch scripts:

• clean-up of header with las2las.exe
• orthometric height correction with lasheight.exe
• tiling with lastile.exe
• bare-earth extraction with lasground.exe
• noise removal (clouds and low points) with lasheight.exe
• DSM rasterization at 2.5 meters per pixel with las2dem.exe
• flattening reservoirs, ponds, swimming pools with las2dem.exe
• normalize LiDAR to height above ground with lasheight.exe
• create height above ground map with lasgrid.exe
• generating intensity rasters for airports using LiDAR with lasgrid.exe

Other processing tasks involved ArcGIS and Photoshop:

• separate height above ground map into vegetation and building height map with ArcGIS
• generating hillshaded rasters from the DSM with ArcGIS
• rasterizing building footprints colored by height with ArcGIS
• rasterizing roads from topographic maps with ArcGIS
• rasterizing reservoirs, ponds, swimming pools with ArcGIS
• extracting water line of Ocean (contour at 1 meter) from DSM with ArcGIS
• some manual editing on complex areas (classification errors, ocean line…) with ArcGIS
• fusing airport intensity rasters into hillshade rasters with Photoshop
• blending all the layers into the final product with Photoshop

Comparison between bare earth DTM and DSM on steroids. The green houses appear as “low planar vegetatation”. They are made out of coarse maze fabric (instead of glass) that lets the laser through and does not deflect it (like glass would).

GRAFCAN is a company that  produces and publishes the geographic information of Canary Islands. The company has done LiDAR flights since 2010. You can explore the Canary LiDAR directly in 3D via their interactive Web viewer of IDECanarias (press the “Lidar” button).

Another beautiful example of a DSM on steroids.

Eric van Rees, the editor of the Geoinformatics magazine, met with rapidlasso GmbH at INTERGEO 2013 in Essen to have a quick chat about LAStools, LASzip, and PulseWaves as well as our LiDAR processing toolboxes for ArcGIS and QGIS.

In the three-page article we talk about the beginnings of rapidlasso’s software, about when and how the company got started, about our various open and closed source products, and about current projects. The interview was published in the most recent October/November edition of the Geoinformatics magazine that was distributed in print at the SPARELMF 2013 conference in Amsterdam this week. If you did not get a copy you can read the online version of the magazine here.

Recently rapidlasso GmbH gave a workshop on LiDAR processing with LAStools at SOPEC in Suva, Fiji following the 2013 Pacific GIS and RS user conference. In the exercises we used LiDAR from the Nadi and Ba basins that had been flown in early 2012 by Network Mapping using helicopters with funding from the World Bank.

For checking flight line overlap and alignment ‘lasoverlap.exe‘ needs flight line information present in the LiDAR that was delivered as 500 by 500 meter tiles in the Transverse Mercator 1986 Fiji Map Grid projection. For tiled LiDAR in LAS format the flight line information is meant to be stored in the ‘point source ID’ field for each point.

C:\lastools\bin>lasoverlap -i D:\fiji\FMG1986\*.laz -gui

Starting the GUI as shown above loads the 654 tiles (4 GB of LAZ or 30 GB of LAS) and setting the options as shown below creates overlap and a difference images for each of the 654 tiles on 8 cores in parallel. See the tutorial on quality checking for how to run lasoverlap directly on original flightlines.

Alternatively we can merge the 4GB of LAZ input on-the-fly by adding the ‘-merged’ option to the command line and by specifying an explicit output file name (although lasoverlap can only use a single core in this case).

C:\lastools\bin>lasoverlap -i D:\fiji\FMG1986\*.laz ^
                           -merged ^
                           -o D:\fiji\FMG1986\overlap.png

The resulting overlap image does not entire look like what we expected (note that the image below was downsampled from the original size of 8139 by 7074 pixels).

Although some of the helicopter trajectories are visible in the overlap image, the majority of the flight line information does not seem to be populated correctly in the LAZ files. To verify this we quickly generate a density image where flight lines become visible due to the increase in density in areas of flight line overlap.

C:\lastools\bin>lasgrid -i D:\fiji\FMG1986\*.laz ^
                        -merged ^
                        -last_only ^
                        -density ^
                        -false -set_min_max 0 25 ^
                        -o D:\fiji\FMG1986\density_25pts.png

Clearly … the flight line information in the point source ID fields of the LAZ files is incomplete. The points from most of the regular back and forth flights seem to have the same point source ID. One could argue that the lack of correct flight line information should be considered a defect in the delivered data because they prevent the recipient from checking the quality in the overlap and alignment of the flight lines and that the vendor should be held resposible to provide a better data set.

This was not the first time we have come across missing flight line information. So we added a new feature called ‘-recover_flightlines’ to the tools ‘lasoverlap.exe‘ and ‘lassplit.exe‘ that uses the GPS time stamps stored with each point in the LAS format to reconstruct missing information about the flight lines. It does, however, rely on the GPS time stamps to be available and correct (e.g. this will not work for the LAS point data format 0 that does not include GPS time stamps).

Above you see the GUI options we use to reconstruct the flight lines from the GPS time stamps and produce a merged overlap and difference image from the 4 GB of LAZ files. We can fix a setting in the ‘RUN’ window that will pop up by modifying the shown command before pressing START. Here is the equivalent command-line:

C:\lastools\bin>lasoverlap -i D:\fiji\FMG1986\*.laz ^
                           -merged ^
                           -recover_flightlines ^
                           -o D:\fiji\FMG1986\overlap.png

The resulting overlap image shows that the flightlines have been recovered.

And now the corresponding difference image also makes sense. It maps the elevation differences of the lowest point in overlapping flightlines per 2 by 2 grid cell to a color between blue (-0.5 meter), white (0 meter) and red (+0.5 meter) and allows to easily spot potential miss-alignments between flight lines.

After all this work it is also time for rapidlasso to recover. After all … we are in Fiji … (-:

The creators of the widely-used open source LiDAR compressor LASzip are issuing a “Call for Input” for extending the popular LAZ format to the new point types 6 to 10 that were introduced with the LAS 1.4 specification. Their LiDAR technology company rapidlasso GmbH invites all interested parties to “The LAS room” for an open discussion: ”As LAZ has gained significant traction in the community we want to involve stakeholders across industry, academia, and government,” so Dr. Martin Isenburg, founder of rapidlasso GmbH.

Recent conversations with large agencies suggest that LAS 1.4 is not yet being tendered and that there are no significant LAS 1.4 holdings. “So we have some time,” says Dr. Isenburg, “No need to rush and put out a half-baked solution. I have the feeling LAS 1.4 will stay with us for a while, so we better get it right.”

The LASzip compression scheme will not change for LAS 1.X files that only contain point types 0 to 5 to guarantee “forward-compatibility” with older, already existing LAZ readers. “The new point types,” so Dr. Isenburg, ”provide a unique opportunity to enhance the compression scheme further.”

Apart from the improvements mentioned in the LASzip journal paper, one change worth discussing is whether to encode GPS time stamps, RGBI colors, and WavePackets separately from other point attributes to allow selective reading of compressed files. Plain visualization, for example, does not need GPS time information. Another consideration is (optional) spatial indexing using LASindex. The new concept of “Appended VLRs” in LAS 1.4 allows appending the tiny LAX files of LASindex to the end of a LAZ file.

Dr. Isenburg asks especially those who do not yet use LAZ (for technical reasons) to join “The LAS room” and contribute suggestions: “Let’s make LASzip with LAS 1.4 support not only a better LAZ, but also your LAZ.”

The LAX files of LASindex contain a quadtree that describes which parts of the file need to be loaded to get all points within an area of interest.

Nifty! Am online Web viewer for LiDAR data in LAS or ASCII format that can load files from your local drive. Very handy in case you need to inspect or show off a LAS file but do not have any LiDAR viewer installed on your computer – especially in case it should run cross-platform in any browser (does it?). But no LAZ support yet … (-;

Below you see a screenshot of me testing the viewer after loading rapidlasso‘s decompressed “fusa.las” data set that is part of the LAStools distribution. Try it for yourself at this site: http://lidarview.com/ … who is behind this?

lidarview displaying the “decompressed “fusa.las” model that is part of the LAStools distribution

NEWSFLASH: update on Feb 5th and 6th and 17th (see end of article)

The year 2014 shapes to be a fun one in which the LiDAR community will see some major laser battles. (-: First, ESRI starts a “lazer clone war” with the open source community saddening your very own LAS clown here at rapidlasso with a proprietary LAZ clone. Then AHAB and Optech start to squirmish over the true penetration depth of their bathymetric systems in various forums. And now RIEGL – just having launched their Q1560 “crossfire” for battling Leica and Optech in the higher skies – is heading into an all-out “laser drone war” with FARO and Velodyne over arming UAVs and gyrocopters with lasers by rolling out their new (leaked?) RIEGL VUX-1 scanner … (-;

Some specs below, but no details yet on power-consumption.

• about 3.85 kilograms
• 225mm x 180mm x 125mm
• survey-grade accuracy of 25mm
• echo signal digitisation
• online waveform processing
• measurement rate up to 500 kHz
• field of view 300 degrees
• internal 360 GB SSD storage or real-time data via LAN-TCP/IP
• collects data at altitudes or ranges of more than 1,000 ft
• will be on display during ILMF 2014 (Feb 17-19, Denver, USA)

The new RIEGL VUX-1 laser scanner for UAVs

Below a proof-of-concept concept video by Phoenix Aerial on how such a system (here based on a Velodyne HDL-32E scanner) may look and operate.

A similar system introduced by Airbotix is shown below.

Aibot X6 with laser scanner from Velodyne HDL-32E

But before trusting these kind of drones with your expensive hardware you should make plenty of test flights with heavy payloads. Here a suggestion to make this more fun:

UPDATE (February 5th): It is reported by SPAR Point Group that the RIEGL VUX-1 LiDAR scanner for drones is to be shown at International LiDAR Mapping Forum in Denver later this month. Given the ceremony that the Q1560 “crossfire“ was introduced with, it surely seems as if Geospatial World Magazine spilled the news early …

UPDATE (February 6th): RIEGL officially press-released the VUX-1 LiDAR. Its weight is below 4kg and its range up to 1000 ft …

UPDATE (February 17th): Today the new VUX-1 finally made it’s much anticipated world-premiere during ILMF 2014 in Denver. For more images of the official unveiling see RIEGL’s blog.

The VUX-1 LiDAR scanner for UAVs unveiled at ILMF 2014.