Installation Question - what is my path directory?

ISIS ISIS Support
#1

Hello! I am trying to download ISIS on my Mac (10.10.15) following the steps outlined on the USGS webpage (GitHub - USGS-Astrogeology/ISIS3: Integrated Software for Imagers and Spectrometers v3. ISIS3 is a digital image processing software package to manipulate imagery collected by current and past NASA and International planetary missions.). I am using python version 3.9.7.

I have a question regarding step 8 (environment variables). When the input asks for the user to specify the path here:

conda env config vars set ISISROOT=$CONDA_PREFIX ISISDATA=[path to data directory]

What should I be putting in for ā€œpath to data directoryā€? Would that be the the directory to the ISISDATA, like this, for example:

rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/base .

So then, would I be inputing the following into python like this:

conda env config vars set ISISROOT=$CONDA_PREFIX ISISDATA=rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/base .

Or is this the directory where the ISISDATA ( rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/base .) has been saved on my computer? (Not sure where that would beā€¦)

#2

It should be where the rsync command copied the data. If you copy and pasted the command from the installation guide, then you downloaded it into the same location you ran the command.

#3

Thanks for your response. So my understanding is that the rsync command should be run before the conda env config vars set command? How do I specify the location for the ISISDATA? Iā€™m not very familiar with using Python so I donā€™t understand how to set directories and path names.

#4

I would recommend running the rsync command before setting the conda environment variables. This is all working through general shell stuff. ISISDATA is just an environment variable thatā€™s a path to where you want to put your data. For example, I generally keep a copy of the base data area in my home directory so I set ISISDATA to /Users/jmapel/isis_data_base. I use bash so I just run export ISISDATA=/Users/jmapel/isis_data_base. The actual download in the rsync command depends on how you run it. The last argument to rsync is where to copy to.

If you copy and paste it will download to the directory you run it in because the last argument is just . or the current directory. So, for my case if I was setting up the base area from scratch I would do this:

mkdir /Users/jmapel/isis_data_base
export ISISDATA=/Users/jmapel/isis_data_base
cd $ISISDATA
rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/base .

Then for the conda env setup I would do

conda env config vars set ISISROOT=$CONDA_PREFIX ISISDATA=/Users/jmapel/isis_data_base

From then on, whenever I active the conda environment, it will set the ISISDATA environment variable properly.

#5

Thank you! This worked! I have ISIS running now.

However, I have run into a problem with the spiceinit function. I get the following error:

spiceinit from=M103977388LE_lev0.cub

**ERROR** No existing files found with a numerial version matching [kernels.????.db] in [/Users/Dave/Documents/ISISDATA/base/kernels/lsk].

I have downloaded the LRO ISISDATA, but but I do not have the directory ./base/kernals/lsk. I only have ./lro/kernals, and in the kernals folder I do not have the ā€˜lskā€™ folder. Is there some other dataset that I need to download?

#6

You need to also download the base area as it has things like leapsecond kernels and some formatting files that are required for all missions.

#7

Ah, yes. I forgot to do that part.

Iā€™m wondering if there is some way to save my cam2map final product directly on an external HD. Right now, all my ISISDATA and .IMG files are on my Macā€™s HD and all my .cub files save there as well, but Iā€™d like the final product (i.e., M#########_lev2.cub) to save directly to my external HD so I donā€™t have to keep moving files.

#8

You can specify any path for application outputs. For example, you could do cam2map ... to=/mounts/some/external/drive/projected_stuff.cub

1 Like
#9

That worked! Thank you. Iā€™m very close to having everything work, but I am running into some errrors.

Error 1

The NAC is successfully projected, but there is no data on it. It is completely black. Tried stretching and changing the histogram, but the high and low values are very large and it just gives me a black NAC:

Picture1
Picture1720Ɨ326 133 KB

Error 2

For some NACs, I get the following error:

cam2map from=M140136780LE_lev1.echo.cub to=M140136780LE_lev2.cub

**USER ERROR** The cube you are attempting to create [M140136780LE_lev2.cub] is [50GB]. This is larger than the current allowed size of [12GB]. The cube dimensions were (S,L,B) [106475, 69505, 1] with [4] bytes per pixel. If you still wish to create this cube, the maximum value can be changed in your personal preference file located in [~/.Isis/IsisPreferences] within the group CubeCustomization, keyword MaximumSize. If you do not have an ISISPreference file, please refer to the documentation "Environment and Preference Setup". Error.

I did go into the IsisPreferences document, but I do not want to increase the max size because 50 GB is too large to work with in ArcMap. I tried changing the max/min LONG and LAT and resolution settings in the map template, but that did not work. Is there anyway to reduce the file size so it can be processed?

#10

Reducing the resolution is going to be your best option. If you want to use the resolution in your map file you also need to set pixres=map. It defaults to the camera resolution which will get you a very large high resolution projected image for nac.

#11

Error 1: What about the error with the black NAC? Would pixelresolution = map work for that error? I am able to process the NAC with no errors and it gives me a 7.9 GB cube file. It just appears black in ArcMap.

Error 2: Unfortunately changing pixelresolution = map did not change the error Iā€™m getting. I am still getting the error that its too big.

#12

For the black image, run the isis stats program on it to see if it has any valid data in it. If it doesnā€™t then check that you are projecting to the correct region and that the image actually covers that region.

For the images with the too big error, keep reducing the resolution until it becomes small enough.

#13

Ok, running the stats program for the black image returned this:

From                    = /Volumes/CosetteHD/NACs/M103977388LE_lev2.cub
  Band                    = 1
  Average                 = N/A
  StandardDeviation       = N/A
  Variance                = N/A
  Median                  = N/A
  Mode                    = N/A
  Skew                    = N/A
  Minimum                 = N/A
  Maximum                 = N/A
  Sum                     = N/A
  TotalPixels             = 1965893736
  ValidPixels             = 0
  OverValidMaximumPixels  = 0
  UnderValidMinimumPixels = 0
  NullPixels              = 1661787035
  LisPixels               = 0
  LrsPixels               = 0
  HisPixels               = 0
  HrsPixels               = 304106701
End_Group

One thing I noticed is that the ProjectionName = Sinusoidal is listed after running cam2map, but in my .map file I have set ProjectionName = PolarStereographic.

#14

Oh! I managed to get it to output ProjectionName = PolarSterographic! But that didnā€™t solve the problem of the black image.

#15

Hi @cosine92, for the sake of helping you get through projecting, hereā€™s what my map file looks like for M103977388LE

Group = Mapping
TargetName = MOON
EquatorialRadius = 1737400.0
PolarRadius = 1737400.0
ProjectionName = PolarStereographic
CenterLongitude = 0.0
CenterLatitude = -90.0
LongitudeDirection = PositiveEast
LongitudeDomain = 180
PixelResolution = 10.0 <meters/pixel>
End_Group

In order to get cam2map to use whatā€™s in there, your command should look like the following:

cam2map from=M103977388LE.cub to=M103977388LE.lev2.cub map=spola_clon0_10mmp.map pixres=map

If you were creating an isis mosaic you would use the same map file for each image so they are projected to the same resolution, projection name, etc. You donā€™t have to do this for ArcMap, but I recommend it.

It took this image <10 minutes to project at this resolution and the output image is about 11M in size, so you could certainly go with a higher resolution (I have often projected lroc nac to 3 meter/pix to get more but without going nuts), but it depends on what your needs are. 3m/pix will take long and require more output space.

I did not calibrate the data (lronaccal), but you might want to, again depends on your needs.

I did minimal processing and used camstats to ā€œseeā€ where the data lived.

lronac2isis from=M103977388LE.IMG to=M103977388LE.cub
spiceinit from=M103977388LE.cub spksmithed=true
camstats from=M103977388LE.cub attach=true linc=100 sinc=100
cam2map from=M103977388LE.cub to=M103977388LE.lev2.cub map=spola_clon0_10m.map pixres=map

Have you viewed the lev0.cub image using qview to make sure there is something there (there is, but make sure you see something). You can also run stats on the lev0.cub file like you did on the lev2.cub file.

Let us know how it goes.

Update: oh! Iā€™m terrible w/ ArcMap and remembering itā€™s longitude preference, so you may want/need to change LongitudeDomain=360 and CenterLongitude=180 to get the output to be in a 0-360 longitude system. The map file I shared was for LROC NAC mosaics we created and our output matched the IAU coordinate system. @dpmayerUSGS, tips?

1 Like
#16

Can you paste your exact command line for cam2map? It sounds like you arenā€™t passing the map to cam2map

1 Like
#17

IT WORKED!!! THANK YOU!!!

I have a few follow up questions:

  1. I need my NACs processed at the highest resolution. So Iā€™m looking at NACs with resolutions between 0.5 and 1.2 m/px. How can I ensure I get the best resolution? I know this will take longer to process and will be a larger file. Do I just put the resolution of the NAC in the .map file?

  2. Iā€™m curious as to what spksmithed=true does in the spiceinit command.

#18

Hi @cosine92, I would go with something in middle of your resolution range for your images, maybe 1m/pix. Either way you are up sampling some images, down sampling the others. Iā€™ve always been taught to go with the highest res image for the entire group, but I donā€™t always follow that rule.

I would put the value in the map file and use the same map file for all images via cam2map and pixres=map, like I have in my example.

Or, since you are bringing these images into Arc, you let each one project and pick up their native resolution, so you get the best for each. You would run cam2map like this:

cam2map from=M103977388LE.cub to=M103977388LE.lev2.cub map=spola_clon0_10m.map

Notice I left out pixres=map. The latter tells cam2map to use the resolution found in the map file. Excluding that tells cam2map to use the image resolution. You can check the PixelResolution keyword in the output file to confirm the differing values for your images. If you take this route, you can not mosaic the images together in isis because it requires all the images have the same resolution and projection information.

At some point improved spacecraft position information became available for the mission and that is only picked up by explicitly asking for it via spksmithed=true. Iā€™m not sure all images get it or maybe just certain time periods. It looks like the commissioning phase image I tested (M103977388LE) picked it up. I confirmed by looking at the image label using less (or maybe you use more) and looking for the keyword InstrumentPositionQuality. Here it is along with the camera information as a comparison:

InstrumentPositionQuality = Smithed
InstrumentPointingQuality = Reconstructed

If the image time was not included in the smithed kernel it would say Reconstructed.

You can also run the program and find out:

getkey from=M103977388LE.cub grpname=Kernels keyword=InstrumentPositionQuality
Smithed
1 Like
#19

Thank you @lwellerastro! You have really helped me. This is my first time using ISIS so it is a steep learning curve!

I have a few more questions I hope you donā€™t mind answering.

  1. Following your instructions above to get the best resolution possible, the output file says that the resolution for the NAC is 0.66 m/px but on the LROC webpage it says that the resolution is 1.01 m/px. Iā€™m curious as to why the resolutionā€™s donā€™t match. Which one should I trust?

  2. I have other NAC files processed by colleagues. Is there a function in ISIS that tells you the resolution of level 2 cube files?

#20

You are very welcome @cosine92. And yes, the learning curve can be very steep. I certainly did not learn all of this on my own. One thing you might check out is our Wiki which has some tutorials (which were transferred from a different website, so some things didnā€™t move well) and examples. See the right panel for links to various topics:

There is also lots of documentation on particular programs via this site:
https://isis.astrogeology.usgs.gov/Application/index.html

Well, youā€™re going to have to trust ISIS since you are working with an isis cube file. The software uses the camera model and kernel information to determine the image resolution when not explicitly set by a user during projection. I donā€™t know what the LROC webpage based their calculations on, but I would expect the kernel information was involved. But I guess it really depends on the actual product on that page.

You can run the program camstats on the unprojected, spiced image to get some basic geometry and illumination information. The default linc/sinc=1 which will take a lifetime with LROC data so I set it to something bigger (the defaults are telling camstats to use the entire image to calculate things, but you donā€™t really need to know about every pixel to get a idea of whatā€™s going on).

camstats from=M103977388LE.cub attach=true linc=100 sinc=100

camrange gets to the point quicker when it comes to resolution and lat/lon range, again this is run on your level 1 unprojected images

camrange from=M103977388LE.cub

Here is some of the output:
Group = PixelResolution
Lowest = 0.71397111767264
Highest = 0.66491847808392
End_Group

Itā€™s not clear how a resolution of 1.01 m/pix is even possible.

  1. I have other NAC files processed by colleagues. Is there a function in ISIS that tells you the resolution of level 2 cube files?
    If they are isis cubes you can look at the label in your unix environment by using less:

less M103977388LE.lev2.cub

Scroll down until you come across Group = Mapping and there will be PixelResolution - this is what the image was projected to which might not be the highest resolution available.

You can also run getkey to get this information:

getkey from=M103977388LE.lev2.cub grpname=Mapping keyword=PixelResolution
10.0

This image was projected to 10 m/pix.

1 Like