Wednesday, May 28, 2008

How to convert a csv file to shapefile

Imagine you need to have a shapefile made of some points you have around from a sampling.
You want to create a shapefile out of it.

step 1: create a simple CSV file with the coords to add an attribute:

11.33260, 44.16922,test1

step 2: go under the file menu and select import

step 3: fill in the wizard parameters
  • the csv file to convert
  • the output shapefile
  • the description of the shapefile. The first is the geometry, which can be of 2 coordinates or 3 (need to check the box for that). Then every other attribute needs to have exactly one equivalent in the csv file
  • the coordinate reference system

step 4: press finish and drag the new created shape in the map

Tuesday, May 27, 2008

How to pass data from java to C with incredible ease

From the moment I started to develop in java, because of the interaction with academic environmental modelers, I often had the problem of retrieving data with java, or having a GUI in java, but needing to use the data in a supercomplex C or fortran algorithm.
So I spent days and nights in learing and applying JNI techniques, hating them every day.
Then one day SWIG came around and helped a bit, but I yet find pure java development much more fun and maintainable :)

And then one day this guy on the imagio mailinglist pointed me to JNA... and I gave it a first simple try... and I have to share this...
no nightmare...
no worries...
just the time to write the code... INCREDIBLE :)

Since I am currently working on porting the GRASS binary raster format to imageio, i will do an example with that. I will read a GRASS raster and print its content in java, in C as pointer of doubles and as array of doubles.

Step one: get the jni.jar from the link above

Step two: write some java code that reads an image or, as in my case a raster map

public class PrintImageInC {

public PrintImageInC( String filePath ) {

try {
// create the grass reader
GrassBinaryImageReaderSpi readerSpi = new GrassBinaryImageReaderSpi();
ImageReader reader = readerSpi.createReaderInstance();
File file = new File(filePath);
int imageIndex = 0;

// set the novalue to use
GrassImageReadParam imageReadParam = new GrassImageReadParam();

// read the raster
BufferedImage bi =, imageReadParam);

// get the databuffer out of the raster and wrap the data array
DataBufferDouble dataBuffer = (DataBufferDouble) bi.getData().getDataBuffer();
double[] ds = dataBuffer.getBankData()[0];

int rows = bi.getHeight();
int cols = bi.getWidth();

WritableRaster raster = bi.getRaster();
System.out.println("Printing array in Java");
for( int i = 0; i rows; i++ ) {
for( int j = 0; j cols; j++ ) {
System.out.print(raster.getSampleDouble(j, i, 0) + " ");

PrintLibrary.INSTANCE.printimageArray(ds, rows, cols);

DoubleBuffer wrapDs = DoubleBuffer.wrap(ds);
PrintLibrary.INSTANCE.printimagePtr(wrapDs, rows, cols);

} catch (IOException e) {

// here the magic is done
public interface PrintLibrary extends Library {
// load the C library
PrintLibrary INSTANCE = (PrintLibrary) Native.loadLibrary("imageioprint",

// define prototypes
void printimageArray( double[] imgdata, int rows, int cols );
void printimagePtr( DoubleBuffer imgdata, int rows, int cols );

public static void main( String[] args ) {

String infile = "/home/moovida/grass/grassdb/flangitest/prova/cell/flowy";
new PrintImageInC(infile);


Step three: write some simple C code to print out the contents (here my testfile imageioprint.c)

#include stdio.h

void printimagePtr(double *imgdata, int rows, int cols)
int i = 0;
int j = 0;
printf ("Printing pointer in C\n");
printf ("--------------------------------------------\n");
for (i = 0; i rows; i++)
for (j = 0; j cols; j++)
printf ("%f ", *imgdata);

void printimageArray(double imgdata[], int rows, int cols)
int i = 0;
int j = 0;
printf ("Printing array in C\n");
printf ("--------------------------------------------\n");
for (i = 0; i rows; i++)
for (j = 0; j cols; j++)
printf ("%f ", imgdata[ i*cols + j ]);

Step four: compile you C code to a shared library

gcc -shared -o imageioprint.c

Step five: supply the needed LD_LIBRARY_PATH to the java environment (i.e. the folder inside which the lib is located) and execute!

PS: JNA dynamically sets types, so it is less performant than JNI. So if you have to call a method millions of times, JNI will be the cross you have to carry, but in my case, with one single heavy raster to pass, the performance is no matter at all.

Monday, May 26, 2008

BeeGIS meets HP

BeeGIS, an extention for digital field mapping for uDig / JGrass, will be part of the Technology for Teaching Higher Education Award by HP.

My Prof. Mauro De Donatis at the University of Urbino has gained the award as only Italian University. The material will be used for teaching students on the field with the BeeGIS-JGrass-Udig GIS-engine.

The official announcement is here, but I prefer the rough cellphone picture from the department wall :)

BeeGIS is a very young project, but, hey... ARPA Piemonte is already sponsoring some development for it and now this award. Young and lucky!!

PS: we will first present BeeGIS at the national Geologists for IT meeting at Offida (this one will be Italian).

Wednesday, May 21, 2008

How to migrate from hsqldb to h2

Recently from a udig IRC meeting I got a real appetite to try out the H2 database, since the google summer of code this year will bring the geotools module to have spatial indexing...

Well, in JGrass we had based all out-of-the-box database needs (non spatial) on Hsqldb, so I wanted to try the substitution. Well, since they are all based on jdbc, the thing has been fairly easy. Some interface extracted, some new plugin, and everything was fine. Not much to say about that.

But can you note something from the code below?

// start the server
String[] args = {"-tcp", "-tcpPort", String.valueOf(_port)};
tcpServer = Server.createTcpServer(args).start();
args = new String[]{"-web", "-webPort", String.valueOf(_port + 1)};
webServer = Server.createWebServer(args).start();

// connect
url = "jdbc:h2:tcp://localhost:" + _port + "/" + database;
con = DriverManager.getConnection(url, user, _passwd);

With hsqldb I had a problem with not beeing able to access the database while JGrass being connected to it (which is nasty because of much development time lost). Also when bringing the application to client we usually had to ship also a db client like Squirrelsql to give a way to access the database.

H2 comes with a webserver instance with some nice application to browse the database tables and do queries:

Once JGrass is started I can access through the webbrowser my machine on the port I defined and I get the access login you can see in the picture above in the browser behind JGrass. JGrass in that case backs up the content of the annotationslayer in the H2 database booted together with JGrass.

Once entered, I get to see the following:

That is a great thing for keeping things under control while developing.
I think I will stay on H2 for a little while :)

Sunday, May 11, 2008

How to learn a new groovy friend Italian accents (and not only)

While updating my webpage-pieces-merger to be groovy, I noticed some odd problem on Italian accents (same would be for German umlaute and so on).

So what is going on with UTF-8?? Why is the encoding wrong, when my java encoding is always right?

When I have a look at the, I notice that the encoding is:

Alright, so I just have to change that one?
try 1) groovy -c UTF-8 myScript -> NOT WORKING

try 2) first set the encoding in the java options on the console:
export JAVA_OPTS=-Dfile.encoding=UTF-8
after that I get a nice file.encoding=UTF-8

and the text is ok:

so what about setting the property inside the script?

try 3) inside the myScript set System.setProperty("file.encoding","UTF-8")
Nice, I get file.encoding=UTF-8 but for some reason the accents are messed up again.

I really wonder why try 3 is not working, probably setting this inside the process is too late? Wrong JVM? Hmmm... really no idea about it. Try 2 is working ok, but I don't like the fact that I have to set a system variable before. I hope to find a solution soon.

How to get dictionary finally on windows commandline

Living in Southtirol we often need to write German as well as Italian. Given that we read tecnical stuff in English, we sometimes suffer from not knowing any word in any language any more. Well, here the beautifully leo dictionary comes into the game. Linux has this command that works from commandline, which makes things really fast without having to use a browser.

This command (apt-get install leo) is one of the many things I miss on Macosx and windows. So, since I wanted to get used to groovy, I tried to solve that. This first version is a bit rough, but does everything it is expected to:

#! /usr/bin/env groovy

if(!args || (args[0] != "ende" && args[0] != "itde" && args[0] != "frde" && args[0] != "esde")){
println "USAGE:"
println "groovy leo.groovy ende english or german words"
println "or"
println "groovy leo.groovy itde italian or german words"
println "or"
println "groovy leo.groovy frde french or german words"
println "or"
println "groovy leo.groovy esde spanic or german words"

def langToken = args[0]
args = args as List

def searchString = args.join(" ");
searchString = URLEncoder.encode(searchString);

def queryUrl = "${langToken}?lp=${langToken}"+

def parser = new org.cyberneko.html.parsers.SAXParser()
def page = new XmlParser(parser).parse(queryUrl)

def depth = page.BODY[0].TABLE

depth.each{ body ->
def tr = body.TR
tr.each{ bodyRow ->
def td = bodyRow.TD
td.each{ bodyCol ->
bodyCol.each{ content ->
def form = content
form.each{ formContent ->
def rows = formContent.TR
rows.each{ row ->
def totLine = row.value()
if(totLine.size() == 5){
// then it is a translation line
def first = totLine.get(1);
def second = totLine.get(3);

def firstKids = first?.children()
firstKids.each{ kid ->
if(kid instanceof String){
print kid + " "
}else if(kid instanceof Node){
def k = kid?.children();
if(it instanceof String){
print it + " "
print it.text() + " "
print "\t\t\t\t"

def secondKids = second?.children()
secondKids.each{ kid ->
if(kid instanceof String){
print kid + " "
}else if(kid instanceof Node){
def k = kid?.children();
if(it instanceof String){
print it + " "
print it.text() + " "
println ""
}catch(Exception e){

For this to work, you will need two additional librarires in your groovy libs folder in the groovy home:
  • the nekohtml.jar (which is not standard, you can find inside the zip downloaded here)
  • the xerces jar for xml parsing (do a locate *xerces*.jar and I would bet you have something there :))

Alright so let me try to see what the German word for groovy is:
>>> leo.groovy ende groovy
groovy Ê adj. fetzig
groovy Ê adj. handwerksmŠ§ig
groovy Ê adj. toll
groovy Ê adj. in Ordnung

Alright, some problems to solve with characterset on the macosx console, but hei, FETZIG!!!

Tuesday, May 6, 2008

How to load a mapgraphic layer programmatically

As Jesse states: "A mapgraphic is just a georesource. It just so happens that it is loaded into the catalog automatically. Search the catalog for MapGraphicService.SERVICE_URL."

And so we do:

List mapgraphics = CatalogPlugin.getDefault().getLocalCatalog().find(
MapGraphicService.SERVICE_URL, null);
List members = mapgraphics.get(0).members(null);
for( IResolve resolve : members ) {
if (resolve.canResolve(GeonotesMapGraphic.class)) {
ApplicationGIS.addLayersToMap(ApplicationGIS.getActiveMap(), Collections
.singletonList(resolve.resolve(IGeoResource.class, null)), 0);

Saturday, May 3, 2008

How to create a feature for an empty Udig layer

Assume you have a new created layer in your Udig workspace. Now you want to programmatically add a feature to it by just knowing its coordinates.

You will need to create a feature that adapts to the schema you layer will have.
I was used to take a feature from the layer, duplicate it and change attributes and geometry in the past. Oviously this isn't very nice and I finally found the limitation. So here is what to do if you have no feature to copy from.

FeatureSource source = layer
.getResource(FeatureSource.class, new NullProgressMonitor());
if (source == null) {
throw new LayerNotSupportedException(
"This layer is not supported, only feature layers are supported.");
// get the featurecollection from the layer, even if empty
FeatureCollection selectedFeatureCollection = source.getFeatures();
* in this case the feature has to be created on the blueprint of the layer's
* features, asking the user for the new attributes to use. Note that getFeatureType
* here doesn't supply hwat you want.
FeatureType featureType = selectedFeatureCollection.getSchema();
* try to create default features where possible
Object attributes[] = DataUtilities.defaultValues(featureType);
* geometry is not created as default, so add yours
attributes[0] = gFac.createPoint(new Coordinate(0.0, 0.0));
featureToUse = featureType.create(attributes);

Feature is created.