What's New Facilities Utilities Links Spatial Data Catalog

NTL-LTER Spatial Data Catalog

Utilities

 

This page contains a collection of utilities and tips to use the spatial data contained in this catalog. It is structured as a FAQ (Frequently Asked Questions) page. You can use this page as a first stop when you encounter problems using the spatial data in the catalog.

If you can't solve your problem with the information found here, please contact the spatial data manager.

This page will be updated constantly based on the questions that data managers receive.

 

I    n    d    e    x

 

I'm new to spatial data analysis. Where do I start?

Confused about data formats? Here's the skinny

Compression formats: gz, zip, tar

Data formats

raster data: images (jpg, tiff, img, grid)

vector data: export (e00) files, ArcInfo coverages, and shapefiles

vector / raster conversions

Trouble with projections

I don't know the first thing about projections, coordinate systems, spheroids, datums, etc. Where do I start?

How do I find out about projection information for my data?

How do I project data from one coordinate system to another?

Problems with RS/GIS software

Arc/Info

The bare basics

Getting shapefiles in and out of ArcView

ArcView

Can I use arc/info coverages or grids in Arc/Info?

ArcView doesn't do what I need, or does it?

I got this great Avenue script. Now what?

 

 

I'm new to spatial data analysis. Where do I start?

First, you need to get familiarized with the basic concepts of spatial data analysis. The University offers several introductory courses. You can also get training at the Land Information and Computer Graphics Facility (LICGF) on campus, or do a course over the internet. ESRI, the company that makes ArcInfo and ArcView, has a virtual campus and offers a variety of courses. For a list of internet resources, go to the links page.

If you are of the self-teaching kind and want immediate hands-on experience, start with ArcView. Most installations include tutorials and come with example datasets (or you can use data from this catalog). On this page, you will also find basic information on data formats so you can get started.

For a list of facilities and software available to NTL-LTER researchers, visit the facilities page.

 

Confused about data formats? Here's the skinny. 

Geodata available from this catalog come in a variety of formats. Also, the data are often compressed so that multiple files can be delivered as one over the internet, and to minimized the size of the file, hence download speed.

 

Compression formats: gz, zip, tar

Most often you will find data compressed as a zip file, a format common in personal computers. You can use PKUNZIP, WINZIP or almost any other compression utility to uncompress them. The zip format can store multiple files, and also preserve directory structures.

Another format you may find is gz (GNU zip), sometimes associated with tar; these are two formats common in UNIX environments. gz is a compression format similar to zip; most programs that uncompress zip (e.g. winzip) will also uncompress gz files. tar is a UNIX format that allows to put multiples files into one file, but does not compress. Winzip will also extract files from a tar file.

If you encounter a file like "name.tar.gz', you'll have to uncompress it in two steps: first, uncompress gz, then extract files from the tar file. After the first step, make sure the resulting file has extension tar.

If you are working in UNIX, use the commands "gunzip" and "tar xvf". To get information on the syntax of these (or any) UNIX commands, type "man command".

 

Data formats

Spatial data comes in two fundamental types: raster and vector. Raster images are composed of a grid of contiguous pixels, just like in a digital photo or any scanned image. Vector data are stored as coordinates for nodes, arcs (lines) or polygons.

 

raster data: images (jpg, tiff, img, grid)

jpg and tiff are two common formats used for digital images. If associated with a so-called world file, they can also store geographic information that will be displayed in the appropriate coordinate system (e.g., in ArcView). One important difference between jpeg (jfif) files and tiff files is that the former are often heavily compressed to small size and have lower image quality.

World files are small ASCII files that contain pixel size and translation and rotation information for the appropriate display of the geographic data contained in the associated image file. World files for jpg images have the same base name as the image, and extension jgw. For tiff files, the extension is tfw.

You can display geographic data in jpeg or tiff format in ArcView. Make sure you have the appropriate extensions loaded (File/Extensions), and you select "Image file" in the "Add Theme" dialog box.

 

img files are, in this catalog, raster images stored in ERDAS Imagine format. Most satellite images in this catalog are stored in this format. You can display img files in ERDAS Imagine, or in ArcView (Make sure you have loaded the Imagine img extension).

 

The grid format is the ESRI's native raster data format, as used by the grid module in ArcInfo and by the Spatial Analyst module in ArcView. Just like ArcInfo coverages, grids are composed of multiple files stored in two folders: an info folder that is shared by all coverages and grids in the same root directory (an ArcInfo workspace); and a folder named as the dataset. Do not delete, copy, or rename grids using operating system commands: use the appropriate utilities in ArcView or ArcInfo. You can display grids in ArcInfo or in ArcView. In ArcView, you will need to load the "Spatial Analyst" extension.

 

vector data: export (e00) files, ArcInfo coverages, and shapefiles

 

Export format. An large ASCII file that contains all the files that compose an ArcInfo coverage. It has the extension .e00, although very large coverages may be stored as multiple export files, with consecutive extension .e00, .e01, .e02, etc. Before you can display, modify, or analyze the data, you will have to import the export file(s) into a coverage using ArcInfo (command IMPORT), or ArcView's utility Import71 (separate from ArcView).

 

ArcInfo coverage. Just like ArcInfo grids, coverages are composed of multiple files stored in two folders: an info folder that is shared by all coverages and grids in the same root directory (an ArcInfo workspace); and a folder named as the dataset. Do not delete, copy, or rename coverages using operating system commands: use the appropriate utilities in ArcView or ArcInfo. You can display grids in ArcInfo or in ArcView. In ArcView, you will need to load the "Spatial Analyst" extension.

 

Shapefile. Shapefile is the native format for vector data in ArcView. Just like an ArcInfo coverage, a shapefile is actually composed of multiples files, so it is easier to rename, copy, or delete shapefiles from within ArcView (File/Data Management) than to use OS commands. Shapefiles are composed of at least three files, with extensions .shp, .shx, and .dbf, but there may be others. Unlike ArcInfo coverages, shapefile do not build and store topology (essentially, contiguity information), so you have to be careful when editing shapefiles, because you may unknowingly end up with overlapping polygons. Shapefile and coverages also differ in that point, arc and polygon features for the same area can be stored in one coverage, but will require separate shapefiles. Finally, not all data structures supported by coverages are supported by shapefiles. On the bright side, shapefiles occupy less disk space and display faster than coverages.

 

Vector data conversions

Coverage to shapefile. You can convert a data layer (e.g., arcs, or polygons, or a region) from coverage to shapefile easily by simply displaying the layer in ArcView and selecting Theme/Convert to shapefile in the View menu. You can also use the Arc command ARCSHAPE.

Shapefile to coverage. Going from Shapefile to coverage isn't that easy. You will need to run arc and issue the command SHAPEARC, then use CLEAN or BUILD to create topology, if your are working with a polygon coverage. If you want to transfer also attribute data (and you probably will), you'll have to put all that data in a region, then convert the region to a polygon coverage with REGIONTOPOLY. Read about these commands in ArcInfo's on-line help.

 

Other vector formats. When looking for data on the internet, you may encounter other vector formats (e.g., STDS, DXF, CAD (dgn), etc.). Both ArcInfo and ArcView provide utilities and conversion programs to display and analyze data in the most common vector formats. Consult the on-line help systems form specific information on data conversions.

 

Vector / raster conversions

 

raster to vector. You can do a raster to vector conversion if your raster data is categorical and each patch is associated with a specific data attribute. If your raster data has polygons delimited by a black outline, for example, you will have to first eliminate that outline (that is, assign outline pixels to a valid class -- from one of the contiguous patches), otherwise you'll end up with zillions of polygons for individual outline pixels! You can use ArcInfo or ArcView to effect the raster to polygon conversion. Also make sure you filter you raster data to eliminate patches smaller that the minimum patch size you want to preserve. Do not convert raster to vector if your data are continuous.

vector to raster. You can use ArcInfo or ArcView to convert a coverage or shapefile into a raster (grid) data set. You will have to make decisions about the pixel size you want in the resulting raster file. You will want to choose the largest pixel size that will preserve the detail that you are interested in -- otherwise you end up with an unnecessarily big raster file (remember that file size increases very rapidly as pixel size decreases).

 

Trouble with projections

 

I don't know the first thing about projections, coordinate systems, spheroids, datums, etc. Where do I start?

Learning the basics of cartographic projections shouldn't be daunting.

Hum. Take that back. It ain't easy. Fortunately, you can do a lot of work with spatial data without knowing much about projections. Just be careful -- very careful --,  when you convert from one projection to another. Here you will find the essential information to work with the projection systems commonly used in Wisconsin, the gist on what to pay attention to.

If you really want to understand projections, there are several resources easily available short of taking a full-fledged course:

ArcInfo's and ArcView's online help contain a wealth on information on projection systems and how to convert from one to another.

The Wisconsin State Cartographer's Office has published a short book on Wisconsin Coordinate Systems with the basic information about coordinate systems. You can read it on the web at http://feature.geography.wisc.edu/sco/

 

How do I find out about projection information for my data?

If you obtained the data set from this catalog or from a source that maintains metadata (data about data), the easiest way to find out about coordinate system information is reading the metadata. If there is no metadata available (it happens more often than you'd think, even in this catalog!), you can still do several things:

If your data are in a raster ERDAS Imagine img or lan file, run Imagine or ArcView's Image Analyst, load the image, and look at the image information. If no coordinate system is defined, try overlaying  a vector layer in the coordinate system that you suspect your data may be.

If your data are in a vector ArcInfo coverage, run ArcInfo, navigate to the appropriate workspace, and issue the DESCRIBE command. Alternatively, you can look at the files in the data folder of the data coverage, and look for a file named prj (or prj.adf). It's an ASCII file with projection information.

If the data set is a shapefile, look for a file called [shapefilename].prj. Most pre-ArcView 3.2 shapefiles wont have that file, though. You'll have to resort to overlaying a coverage in a know, suspected coordinate system, or talk to someone familiar with the data set.

 

How do I project data from one coordinate system to another?

Once you know the coordinate system of your dataset, you can project your data into another coordinate system. Find out whether you transformation involves a datum conversion. If so, make sure you provide datum information for both the input and output projection. A datum is a mathematically defined reference surface used to represent the size and shape of the Earth. The two datums most commonly used in North America are the North American Datum of 1927 (NAD27) and the North American Datum of 1983 (NAD83), but other datums are also used. If in doubt, consult with a knowledgeable person.

In ArcInfo, use the PROJECT command:

project cover <input> <output> [projection file]

for a vector coverage. For a grid:

project grid <input> <output> [projection file]

where [projection file[ is an ASCII file that contains the parameters of the input projection (the 'origin' coordinate system) and the output projection (the 'target' coordinate system).

For a list of conversion files commonly used with Wisconsin data, look at the following list of projection files, where file names follow the convention:

 [input_coordinate_system]_[output_coordinate_system].prj

For example, to convert from GEOGRAPHIC, NAD 27 to Wisconsin Transverse Mercator, NAD83, with 1992 corrections, use the file geo27_wtm8391.prj.

You generally cannot project data in ArcView 3.1 or earlier versions, but ArcView 3.2 includes that capability, including datum conversions.

Note that if your data is in GEOGRAPHIC coordinate system, then you can project it on the fly in ArcView 3.x. All your datasets should be in geographic coordinate system, though, and you cannot project grids this way.

 

Common problems with software

Having problems using one of the Remote Sensing or GIS software packages? Here's a list of frequent problems.

Remember that you can find a list of software and facilities for RS/GIS available to the NTL-LTER researchers in the facilities page.

 

Arc/Info

 

The bare basics

So you started Arc/Info and gazed at the prompt for a while, not know what to do? It's happened to all of us. Here are the essential things that you should know:

(i) Use the on-line help. To invoke it, type help. Note that you will not be able to see the help if you are using Arc/Info over a telnet connection to a UNIX computer (unless you are running an X-emulator). In that case, use asciihelp.

(ii) Use arctools, a set of menus to manage and display coverages and grids). To use arctools, type arctools. See also note above.

(iii) Print and use this Arc/Info Cheat Sheet, a list of the most useful commands.

 

Getting shapefiles in and out of arc/info

ArcView is a powerful GIS program, and keeps gaining in functionality with every version. Still, there are thing you just can't do in ArcView. If you need to use ArcInfo and your data are in shapefile format, you will need to convert your dataset into an ArcInfo coverage. That's easy for point and line shapefiles, but a bit more complicated for polygon shapefiles:

(First, use SHAPEARC with the following syntax:

SHAPEARC <in_shape_file> <out_cover> {out_subclass}

this will create a polygon coverage with a REGION named {out_subclass} (you choose the name). The region will contain the attributes. Run CLEAN on this coverage, and then convert the REGION to a polygon coverage with REGIONPOLY. You will now have a polygon coverage with all the attributes you had in the shapefile.

Going from coverage to shapefile is easier. You can either do it in arc/info or in ArcView. In arc/info, use the command ARCSHAPE. In ArcView, add the coverage to a View, then use "Theme/Convert to Shapefile".

 

ArcView

Can I use arc/info coverages or grids in ArcView?

AV can display many data formats, including ArcView's native format -- shapefile --, and Arc/Info's native format -- coverage. You can edit shapefiles, but not coverages. However, you can easily convert a coverage into a shapefile. Just click on the coverage name to make it active, then go to Theme/Convert to shapefile. Note that if you have any features selected, the resulting shapefile will only contain those features, which is a quick and dirty way to subset a shapefile or coverage.

Other formats may required that you load and extension into AV. This applies, for example, to tif, jpeg, and ERDAS Imagine images. To see a list of extensions available for you to load, from the project window, click on File/Extensions.

AV will only work with GRIDS if you first load the "Spatial Analyst" extension.

 

ArcView doesn't do what I need, or does it?

ArcView does more that you think it does. Maybe you just don't have the appropriate extension loaded. To see a list of extensions available to you, click on File/Extensions.

ESRI, the company that make AV, maintains a searchable catalog of free extensions and scripts written by AV (and arc/info) users. It's an extremely useful resource.

 

I got this great Avenue script. Now what?

So you found the script that you where looking for, but now you don't know where in the world to put it. No sweat. Open you ArcView Project, where you want to use the scrip, and open a new Script window. Paste the Avenue script in the window and compile it. If it compiles well (it should, if you got it from ESRI's catalog), you are ready to run it. Depending on how the script was written, it may ask you which View contains the data that you want to process, and the target theme(s). Try it. Click on the "Run" button (the one with the person running), and see what happens. If you only get an error message, it probably means that the script is trying to find themes in the active window, which may not be a View. The script will run on the last active window, so just click on the View bar to make it active, then go back to your script window and try again.

You can also add a button to your view to run any particular script. From the project window, go to Project/Customize.

 

 

 

Last updated: 24 April 2000