Scripts
I am writing scripts all the time. Here are a few that I have written that are useful. Many are Python scripts which require that the Python interpreter be installed before they will run. Python is a free, easy to learn and powerful scripting language. I encourage you to give it a try.
For users familiar with Python and working on the i386 architecture (PCs), you can get a significant speed boost by adding a just-in-time compiler to Python known as Psyco. Some of my scripts look to see if it is installed and will take advantage of it if possible. Check out the Psyco web site for the details (it's really easy to work with).
I get a lot of hits on my site looking for information on ESRI ASCII Grids. I put together a page explaining the ASCII GRID format and a utility to convert it to X,Y,Z triplets (grd2xyz.py below). This format is easy to understand, easy to read and widely supported. I use it whenever I can.
Surfer Scripts
I've only recently started using Surfer (Version 8.05) and I must say that I am completely impressed with this software. Over the years, I have used ArcGIS, GRASS, GMT, Fledermaus and a variety of proprietary griding packages sold with survey equipment. None of those packages compare with Surfer for its ease of use, flexibility and speed. Fledermaus can handle larger files, ArcGIS and GRASS are more complete GIS packages, and GMT can do stuff nothing else can do; but for the sweet spot of griding and plotting data sets with files <250 Mb in size, Surfer is the best tool I've used. A few things really shine in this package: (1) its very fast, (2) it can be completely automated with Python, (3) it has wonderful documentation, (4) it is super flexible, (5) it has nice graphic output capabilities.
If you are considering buying 3D analyst or Geostatistical Analyst for ArcGIS (at $2500 each per license), seriously consider using Surfer ($600) instead. I haven't used either module for ArcGIS since I purchased Surfer. I don't think it can replace Spatial Analyst (SA). Surfer doesn't have any of the modeling/GIS features that SA provides (it has far more powerful and efficient griding options than SA). If you need to work with very large files > 250 Mb, nothing beats Fledermaus (but get ready to spend some money).
Here are a few command line scripts that I've written to to convert surfer grids into and out of ArcGIS format (type the name of the script followed by -h to get a brief description of the options):
NOTE: In order for these Python scripts to work with Surfer, you must first register the Surfer COM Object Model with Python (it's easy): Open Pythonwin IDE (this is part of the Python for Windows package) and then select Tools > COM Makepy Utility > Surfer 8 Type Library (1.0) > OK.
Script | Description | Surfer Version | Dependencies |
Converts a binary Surfer grid to ESRI Ascii grid format | 8.0 | Python >= 2.4 | |
Converts an ESRI Ascii grid to binary Surfer grid to format | 8.0 | Python >= 2.4 | |
Grids an ASCII table (x,y,z) in Surfer. This an example of how to automate Surfer using Python | 8.0 | Python >= 2.4 |
GRASS GIS Scripts
These scripts are designed to run in GRASS GIS.
Script | Description | GRASS Version | Dependencies |
Creates a buffer around interpolation points that can be used as a MASK to prevent or clip excessive extrapolation artifacts. This works much nicer than a standard convex hull around the points. | 6.3 | grep, cut, bc | |
Creates a skeleton kml file from a GRASS raster, including a nice legend overlay. | 6.3 | Image Magick (needed to make nodata transparent) | |
Creates a vector map representing the sea bottom coverage of a multibeam (swath) sonar survey | 6.X | RDBM (any will do, I think) | |
Creates a raster map estimating the eucledian distance from known cells | 6.X | ||
Creates a composite of two rasters using a distance-weighted transition | 6.X | Requires also r.eucdist (above) |
ArcGIS 9.X Toolboxes & Scripts
These toolboxes contain scripts designed to integrate into the new ArcGIS 9.0 Geoprocessing framework. Download the toolbox. Add it to your ArcMap document and use the scripts in exactly the same way as the included tools in models, at the command line, or using a wizard.
Toolbox/Script | Description | Software Version | Dependencies |
An ArcGIS 9.0 toolbox with scripts designed to calculate fetch length, wave height and wave period for all water cells in a DEM. Fully documented in the help system. These scripts do not work on 9.2. I am working on a C-based version that does not rely on ArcGIS at all. | ArcGIS 9.0 | Spatial Analyst |
Data Converters
Useful scripts for converting data formats or pre-processing data prior to loading into GRASS or ArcGIS.
Script | Description | Environment/ | Dependencies |
Converts the ETOPO2 binary file ( ETOPO2.dos.bin) into an ESRI ASCII Grid. Leo Kreymborg sent me an optimized version of this script that runs 6x faster! | Python/All | None | |
Profile Coordinate Geometry. Converts a list of distances and angles from a benchmark to cartographic coordinates. This is a simple script that uses Euclidean distances and right-triangles. | Python/ALL | None | |
Converts a list of X, Y coordinates into distance and angles. Two options are available: (1) distances and angles are calculated from the first point in the file, (2) distances and angles are calculated from point-to-point along the list. | Python/ALL | None | |
Converts an ESRI ASCII Grid raster into X,Y,Z triplets. This is a simple script that is easy to understand. | Python/All | None | |
Converts an ESRI ASCII Grid raster into X,Y,Z triplets. This version has a full CLI interface for use in batch scripts. Type: grd2xyz.py -h for help on program options. | Python/All | None | |
Converts a GRASS raster (with optional intensity raster) into GMT format, converts the color map and builds a skeleton plot file which can be easily tailored for final map production. | Python/GRASS Shell | GRASS 6.0+, GMT 4.0+ | |
This program reads arbitrary date/time strings from standard input (or a file) and converts them to an arbitrary output format. It is very useful for working with time series data that has an inconvenient date/time format. For example, you can convert dates from YYYY-MM-DD HH:MM:SS format to Julian Day format or vis-a-vis. The program can work with date/time columns embedded in a delimited table without mangling the other fields. | Python/ALL | None |
System utilities for (mostly) Windows XP
These scripts give Windows XP functions normally found on Unix. I put these all in a "Scripts" directory in my search path so that I can execute them any time I am at the command line.
Script | Description | Environment/ | Dependencies |
Reports the free space on each drive connected to the computer. | Python/Windows | ||
Shows the top few lines of a text file. Very useful for peeking at the format of a very large table (that won't load into Excel for example). | Python/All | None | |
Concatenates a header file to the top of a data file. Useful for those large point files that Excel can't open. This isn't really necessary once you learn some Shell scripting. | Python/All | None | |
Shows the bottom few lines of a large file. | Python/All | None | |
This script published in Brian Knittel's, Windows XP under the hood. Allows you to "view" any file in your favorite editor. Type: view file.txt and your editor pops up with the file loaded. Exactly the kind of useful thing shell scripting makes easy. | Windows 2000/XP | None | |
Which [command] returns the location of the executable that answers the command call. Say, you have two versions of Java installed. 'which Java' would return the location of the first one in you path. This one is also straight out of Brian Knittel's Windows XP under the hood. | Windows 2000/XP | None |
Plotting Routines for R/S-plus
The R-project for Statistical Computing is an open source (free) implementation of the "S" language (sold commercially as S-plus). R is a popular environment for advanced statistical analysis and modeling. It also has a very flexible graphics system for making custom plots. Here are two plot functions designed for visualizing vector time series.
Script | Description | Environment/ | Dependencies |
Makes a circular histogram of directional data. These are often referred to as "wind roses" though any directional data can be plotted this way. | R/S-plus | None | |
A conventional method of plotting current time series is with a "stick" or "flag" plot. The "sticks" in this plot point towards the direction indicated. Oceanographic and atmospheric conventions are 180° different on this, so modify the script to suite your preference. I'm not satisfied with this the way it is. I still need to edit the results in a drawing program. | R/S-plus | None |
