Distillation from the NEON Data Institute
/So much to learn! Here is my distillation of the main take-homes from last week.
Notes about the workshop in general:
- Making participants do organized homework and install software is necessary for complicated workshop content: http://neondataskills.org/workshop-event/NEON-Data-Insitute-2017
- NEON used tips from software carpentry workshops, particularly the green-and-pink sticky tag tip. When you put up a pink sticky on your computer, you need help; when you put up a green sticky on your computer, you are all good. This makes everything go smoother and means participants don't have to hold up their hand to get attention.
- Having lots of helpful, friendly faces around to help during the coding bits, and having access to the code when you got behind was critical.
- The workshop content and daily schedule:
NEON data and resources:
- http://www.neonscience.org/resources/data-tutorials
- All the NEON airborne data can be found here: http://www.neonscience.org/data/airborne-data
- For more on when data rolls out, sign up for the NEON eNews here: http://www.neonscience.org/
Other misc. tools:
- For cleaning messy data - check out OpenRefine - a FOS tool for cleaning messy data http://openrefine.org/
- Excel is cray-cray, best practices for spreadsheets: http://www.datacarpentry.org/spreadsheet-ecology-lesson/
- Morpho (from DataOne) to enter metadata: https://www.dataone.org/software-tools/morpho
- Pay attention to file size with your git repositories - check out: https://git-lfs.github.com/. Git is good for things you do with your hands (like code), not for large data.
- Markdown renderer: http://dillinger.io/
- MIT License, like Creative Commons for code: https://opensource.org/licenses/MIT
- There is a new project called "Feather" that allows compatibility between python and R: https://blog.rstudio.org/2016/03/29/feather/
- All the NEON airborne data can be found here: http://www.neonscience.org/data/airborne-data
- Information on the TIFF specification and TIFF tags here: http://awaresystems.be/, however their TIFF Tag Viewer is only for windows.
- All NEON point cloud classifications are done with LASTools. Go LASTools! https://rapidlasso.com/lastools/
- Check out pdal - like gdal for point clouds. It can be used from bash. Learned from my workshop neighbor Sergio Marconi https://www.pdal.io/
- Reflectance Tarps are made by GroupVIII http://www.group8tech.com/
- ATCOR http://www.rese.ch/products/atcor/ says we should be able to rely on 3-5% error on reflectance when atmospheric correction is done correctly (say that 10 times fast) with a well-calibrated instrument.
- NEON hyperspectral data is stored in HDF5 format. HDFView is a great tool for interrogating the metadata, among other things.