I have a rather sizeable chunk of LiDAR data (a ~16-million point-cloud), which I have whittled down to X, Y, Z data (e.g):
436003.73 8758000.16 330.97
436007.17 8758000.40 330.49
436009.98 8758000.25 330.26
436011.71 8758000.16 330.10
I am wanting to create a DEM of this data, yet my computer is finding this difficult due to the huge dataset. I am not particularly well-versed in MATLAB/Octave, but I am learning.
Currently I'm using the griddata and meshgrid commands, as such:
cd ~/Octave
dem=load('436003_ed.txt');
x=dem(:,1);
y=dem(:,2);
z=dem(:,3);
minx=min(x);
miny=min(y);
minz=min(z);
maxz=max(z);
maxx=max(x);
maxy=max(y);
rijx=minx:2:maxx;
rijy=miny:2:maxy;
[XI,YI]=meshgrid(rijx,rijy);
ZI=griddata(x,y,z,XI,YI);
... and that's about as far I get as it runs out of memory whilst gridding the data.
I assume part of the problem is that as I am opening the entire dataset as a variable (dem=load('436003_ed.txt');) as ~500MB file, which of course is very memory-expensive. I have been told about an alternative way in which I can use the fopen command:
fid=fopen('436003_ed.txt');
XYZ=fscanf(fid,'%f %f %f');
However this seems to provide an odd format, as is itself not loaded into the RAM?
Is there possibly a way in which I can load it line by line, placing each Z value into the appropriate DEM cell, whilst counting the total elevation for that cell, and the number of hits (so I can then average the elevation afterwards) to produce a DEM? This will negate the use of the meshgrid and griddata commands.
Being entirely new at this I don't know which commands to look up and play around with!
Many thanks in advance. I have MATLAB R2013b, and I've been doing some of this via Octave too if this helps.
UPDATE
I solved the problem using some a command line tool library called GDAL (http://www.gdal.org/ GDAL grid, specifically). This helped me grid the data. Whilst this is an answer to my query, it is not the extact answer I was looking for.
Whilst my original method would potentially work, if my computer had enough memory, there are alternative ways to script this in MATLAB - using the fscanf and fopen commands - I have yet discovered how to do this properly.