Wrap-up from ESRI User Conference 2019 Day 3

/

Day 3 Highlights

Note to Self: Remember the User Types: Viewer, Editor, Field Worker, Creator, GIS Professional

Wednesday morning I went to ArcGIS Apps for the Field so that I could get my head around all the different and new tools that can be used to collect field data. And sister, there are a lot of them. There are of course the capture apps that we know and love - Survey123 and Collector - but there are also apps for all the steps of the field operations: planning, coordination, monitoring and navigation to sites. These apps include: Explorer, Workforce, Tracker, Operations Dashboard, QuickCapture, and Navigator. Excellent session.

Apologies to the NRC for borrowing the tagline…

Apologies to the NRC for borrowing the tagline…

I then went to two Women in GIS events: the “Ask Me Anything” panel of women leaders, and then the Women in SIG lunchtime session where I was the guest speaker. The panel was super inspiring and included Whitney Kotlewski, Nicole Franklin, Dierdre Bishop, Wan-Hwa Cheng, and Miriam Olivares. These passionate and smart women shared very diverse experiences in life, in their careers, and in GIS. We saw a hot-off-the-presses video from Black Girls MAPP, which was so great. Why haven’t BGMAPP been featured at the ESRI Plenary Session? My session was a lot of fun, with a great turnout and good questions. I talked about my ideas around “Mapping for Impact” and what that means in terms of #people, #data, and #tools. Here is one of my slides. This is what I think about all day. Very pleased that this kind of networking is going on, and that panel in the morning session was extraordinary. 

Afternoon sessions had me try to ingest as much as possible about cutting-edge tools. I focused first on QuickCapture. This looks like a promising app for classroom teaching, field sampling, and citizen science. And I am not the only one who thought so - the room was full and so I watched the workshop livestream from another room. Some points: fully customizable collection forms via Designer, can collect points, paths, photos, text, maybe voice in the future, and can connect to external high accuracy GPS antenna. More to do! Then, to close out the day, it was a toss-up between Architecting my ArcGIS Online or more Kenneth Field showing off 3D Cartographic Techniques. Of course I chose 3D. This was an immersive presentation with Kenneth Field (the self-proclaimed “3D skeptic”) and Nathan Shepard guiding us through some of their favorite 3D visualizations and how they make cartographic choices in 3D. The good news is most of the things you know from 2D cartography are relevant in 3D. The bad news is that there are some vis challenges like perspective distortion and hidden content with 3D. But also, 3D might be overkill. When do you use 3D? When is 2D good enough? After all, 2D mapping is awesome. To discuss this (and provide lots of friendly banter back and forth) they showed us an example of 2D vs 3D by mapping the arctic fox on her journey from Norway to Canada. I liked the 2D version myself. They also looked at the classic Minard map of Napoleon’s journey to and from Moscow in 2D and 3D. Lots of good stuff to dig in to. Thanks to Nathan Shepard and Kenneth “I quite like spinny globe thingys” Field. Good stuff.

Wrap Up from Day 2
Wrap Up from Day 1

Wrap-up from ESRI User Conference 2019 Day 2

/

Day 2 Highlights

Morning sessions included using ArcGIS Pro for Lidar analysis. This was a great overview of working with LAS files within ArcGIS Pro: data formats, including LAS related data structures multipoints and multipatches, data diagnostics to examine data quality and coverage, and basic processing of Lidar data in ArcGIS Pro. Very useful. Thanks to ccrawford. Next Shane and I checked out Data Science in ArcGIS Pro: Using R and Python. This is an update from last year’s session on R and Python in ArcGIS, and again we started with the classic battle of the bands. The python demo showed PySAL, and how to do a quick spatial econometric analysis in PySAL within Pro. Need more time, obvi. The R demo was a nice walk through of data input - vector and raster - via the R-bridge, and showed how easy the framework is. Very useful. 

Afternoon sessions included a bunch of stuff, but the highlight was the always funny, always useful, always inspiring wizards of cartography Kenneth Field, John Nelson, and Edie Punt and Mapping with Style. Another great session! Edie discussed the excellent Styles capacity in Pro, which I am aware of but not an expert at. She helpfully pointed out some key things to pay attention to: USE STYLES FOR COLOR CONTROL! because of the new graphics engines used in Pro, TRANSPARENCY is available on all colors, and will be transferred to your PDF! Also, color locking is great! And color brewer schemes are available in Pro Styles. So much love! Interspersed with Edie’s slide were John and Kenneth delivering their usual hilarious take on making beautiful maps: John showed off some nice and creative new published styles: Firefly, Imhof, and Lego; and Kenneth presented a whimsical case study bringing to life a 1930s map of Redlands using the watercolor style from John. See other styles here. Good day.

Wrap-up from Day 1

Wrap-up from ESRI User Conference 2019 Day 1

/

ESRI UC 2019 Day 1 Highlights - The Plenary Sessions

The morning session as usual was a flood of high-level vision about GIS (this year’s catchphrase is “see what others can’t”); as well as key shout-outs of new tech wizardry we can expect in the next software update. Jack spent some time discussing the concept of the human nervous system as an analogy for GIS: an intelligent nervous system to respond to, analyze, and use complex spatial data. The goals of this network remind me of the goals of cooperative extension: Learning, Sharing, Collaborating, and Engaging Communities. We also heard about the new NatureServe biodiversity/imperiled species database. This gathers field work from biologists on critical imperiled species and fills in the gaps using predictive species modeling (RF) on jupyter notebooks. The maps are validated via a network of scientists. Seems like they should connect with iNaturalist.org stat.

ESRI2019_2.jpg

Lunch was kick-ass fish tacos, FYI.

New Tools in ArcGIS Pro 2.4 include some coooooool stuff. What I am excited about:

  • Story Maps updates are rad. Gotta get after it. Already available.

  • I can customize my basemaps using the Vector Tile Editor;

  • Calendar heat maps are now provided;

  • YAY! Through the “History” tab we can capture analysis workflow and turn into processing model automatically;

  • Pixel editor – OMFG. What new devilry is this. I want in.

  • Slice multipath tool to cut, clip and visualize 3d objects;

  • Viz of real material properties in 3D (e.g. light reflection, waves in water, etc.);

  • Parcel fabric: cadaster parcel processing: topology, editing, COGO measurements; helpful for deed changes and deed edits;

  • ArcGIS Monitor: This is like a dashboard for administrators of ArcGIS Enterprise to monitor web performance times, diagnostics, and utilization analytics, plus others.

The afternoon session usually packs in important examples of groups using GIS internationally. 2019 was no exception. The speakers included Ambassador Stefano Toscano and Olivier Cottray from the Geneva International Centre for Humanitarian Demining. They showed the Mine Action program, that uses GIS to find and destroy mines. This involves using community input from Survey123 to target searches for mines and mine destruction. Olivier Cottray says “webapps and webmaps have been game-changers for us”. He closed with this critical example: modern warfare – such as in Mosul and elsewhere - has led to mass-scale urban destruction and mines being left not only in the ground, but in buildings, above ground. Imagine what this means for search and destroy: it is now a 3D problem.  We heard from Naftali Honig, Evan Trotzuk, Geoff Clinning who work in the Garamba National Park in the Congo basin about their Conservation Intelligence framework to combat illegal poaching.  In the last panel, we heard from Jane Goodall and E. O. Wilson on an informal panel moderated by Jack. Wilson laid out his top three challenges for human kind: climate change, fresh water shortages, and mass extinctions. He talked about his HalfEarth program, and also gave us a primer on biodiversity from global to microbial scales. We need to discover, characterize, and map all remaining species on earth, creating a true science of ecosystem organization in aid of conservation and earth sustainability. Finally, we got to hear from Jane Goodall. Her work is so inspiring. At the heart of all her conservation and science work, are the importance of empowering women, the role of traditional knowledge in biodiversity conservation, using local decision making, and supporting education. She offered us passion, humor, hope, while frankly discussing the stark realities of a changing climate and the rise of the political right and widespread corruption. Her work and voice still instructs us on the power of the human intellect and indomitable human spirit to make positive change.

ESRI2019-1.jpg


Inspiring stuff.

DroneCamp 2019 Student Scholarships

/

We are very pleased to announce the recipients of the 2019 student scholarship awards for DroneCamp 2019. We were able to leverage seed support from the open access journal Drones (dedicated to research on the design and applications of drones) to support three wonderful young scholars to attend DroneCamp 2019 in Monterey CA in June. Here is a little bit about each of them.

Reyes headshot.JPG

Melinda Reyes is the recipient of the Drones scholarship. A computer science student at Northeastern University, Melinda aims to improve precision agriculture tools that measure and advance soil health and soil-based carbon capture via data collected by UAVs. Her ultimate goal is to ensure food security in a changing climate. She is passionate about UAVs and their applications in agriculture, and is a future leader in agricultural innovation.  She comes to DroneCamp to learn to fly drones, to learn more about current applications in the field, and to meet others interested in UAVs' agricultural applications and applications for good.

zackdinh_portrait.JPG

Zack Dinh is the recipient of the IGIS student scholarship. He is a M.S. student in the Department of Landscape Architecture and Environmental Planning at the University of California, Berkeley. He is working with Professor Iryna Dronova at UC Berkeley on wetland restoration monitoring in the Sacramento-San Joaquin Delta. Together, they are interested in using UAV-captured optical imagery and point clouds to detect vegetation patch heterogeneity, and plant growth. He comes to DroneCamp to learn about best practices for using UAVs for field research as well as networking with others who are engaged in this type of research. He has some experience flying UAV surveys and processing imagery, and is interested in learning more about fixed-wing vehicles, sensors, and data processing techniques.

Anish Photo.jpg

Anish Sapkota is the recipient of the IGIS student scholarship. He is a Ph.D. student in the Department of Environmental Sciences at the University of California, Riverside. He is working in the Irrigation and Water Management Lab under the supervision of Dr. Haghverdi. His research focuses on the effect of water-stress conditions in landscape and agricultural crop species, and he seeks to develop irrigation management strategies to save water while increasing production, and soil and plant health. His research involves collection of data from a range of handheld sensors that measure stomatal conductance, biomass growth, and soil moisture. These data will be combined with NDVI and NDRE data from field and UAV-based sensors to determine the stress level in plants and their response to varying rates of irrigation. He comes to DroneCamp to continue to gather skills in the geospatial domain, and to learn how to fly drones, and process UAV data.

Congratulations to each of you!

Sabbatical in China in the spring...

/

Sabbatical report April 2019

I’ve been on sabbatical now for a few months, and it’s time to report. I’ve been working on updating all my course materials: slides, reading and labs, for the fall. This has been a blast, and a lot of work! Especially the labs. We are finally moving to ArcGIS Pro, people! It’s been scary, but thanks to some excellent on-line resources, including this list of excellent tutorials from Jarlath O’Neil-Dunne and from ESRI (Getting started with Pro) we are making progress. Shane Feirer and Robert Johnson from #IGIS are helping here too, and we’ll likely be using some of the new material in IGIS workshops soon. 

Currently, I am in China, visiting former PhD student Dr Qinghua Guo and my “grandstudent” Dr Yanjun Su at their lab set in the bucolic Institute of Botany northwest of Beijing (just outside the 5th ring, for those of you in the know). It has been a blast. I came to catch up on all the excellent UAV, lidar, remote sensing, and modeling work going on in the Digital Ecosystem Lab at the Institute of Botany (part of the Chinese Academy of Sciences). These students are serious Data Scientists: they are working on key spatial problems and remote sensing data problems using ML, classification, spatio-temporal algorithms, data fusion tools. They routinely work with lidar, hyperspectral, multispectral and field data, and focus on leaf-scale to landscape-scale processes. One of the big experiments they are working on uses a new instrument, dubbed “Crop3D”. It is a huge frame installed over an ag field with a movable sensor dock. The field is about 30m x 15m, and the sensor can move to cover the entire field. Here is my summary in graphic form:

CROP3D. Very cool.

CROP3D. Very cool.

This season’s experiment focuses on mapping corn plant phenotypes using hyperspectral, RGB, and lidar data by classifying leaf-scale metrics such as leaf angle and branching angles, along with spectral indices. VERY COOL STUFF. I am eager to hear more about the results of the experiment and see what is yet to come. 

I gave a couple of talks, one on “big” (serious air quotes here) data and ecology (to the Institute of Botany at the Chinese Academy of Sciences) and one on UAVs (to the Institute of Geographical Sciences and Natural Resources Research, CAS). In both I highlighted all the excellent work done by students and staff in my various research and outreach groups. In the first I focused on our Lidar work in the Sierra Nevada (with Qinghua Guo, Yanjun Su, Marek Jakubowski); the VTM work and FAIR data (with Kelly Easterday); and UAV/water stress (with Kelly again plus Sean Hogan and Jacob Flanagan). In the second talk I got to gush about all the IGIS work we are doing across our “Living Laboratories” in California. We have flown ~30 missions (total 25 km2) on and around the network of research properties in California (see the panel below for some examples). I talked about the recent CNN work with Ovidiu Csillik; the BORR water stress experiment with Kelly, Sean and Jacob; the fire recovery work at Hopland with Shane Feirer and the rest of the IGIS crew; and the outreach we do like DroneCamps. I also talked about UAV Grand Challenges: Scaling, Sampling, and Synergies. Those ideas are for another post. 

IGIS UAV Missions in California

IGIS UAV Missions in California

My hosts took great care of me: Showing me the sites, making sure I tried all the regional delicacies, and indulging me in my usual blather. Below are some pics of us on our adventures, including in the bus on our way to a distant portion of the Great Wall. Walking the Wall was: 1) awesome (in the real sense of the word – it really is mind-blowing); 2) STEEP (calves were screaming at the end of the day); and 3) windy. Plus there are snakes. I was told that there are other sections of the Wall that are called “Wild Wall” which I think is extremely cool. And speaking of walls, GOT starts again this weekend. China in springtime is BURSTING with flowers. And being housed at the Institute of Botany means all of them are on show in a concentrated area. Finally, you can get all over this huge country on trains. Trains that go really fast (220 MPH), and are on time, and are comfortable! I went to Shanghai (800+ miles away) for the weekend by train! My current joke: “In China, it takes 4 hours to get from Beijing to Shanghai. In California, it takes 4 hours to get from Berkeley to Sacramento.” (Thanks Dad!).   

GroupPeeps2.jpg

Off to Tokyo. But not before a final panel of pics that remind me of this trip: Technology, Art, Food, Flowers, Shopping, History. Here in China, Red = Happiness + GoodLuck, not the Cardinal.

ChinaFavs2.jpg

Wrap-up from the Geospatial Software Institute (GSI) Workshop: “Towards a National Geospatial Software Ecosystem”

/

My wrap-up from a very engaged and provocative 1.5 day workshop on geospatial technology futures, hosted by the CyberGIS Center: “Towards a National Geospatial Software Ecosystem”. First: great group of cool peeps all hyper-engaged in geospatial data, tools, use cases, science, and community. Second: fun to be involved in big-picture thinking on what a geospatial software institute might look like if it was to be built from scratch. Finally, I was on the panel discussing core questions bridging use cases and core technical capabilities, and I share my reflections of the workshop here.

  • Question 1. Are there any significant gaps between the use cases and core technical capabilities that GSI should address?
    • Training needs: beyond GIS training – “spatial data science” training, for K-12; undergrad; graduate; veterans; professionals
    • Easy ways to get access to cloud storage and computation, and for different datasets like UAVs. There are examples like CyVerse (from Tyson Swetnam) and others
    • Data integration: Data assimilation, Data fusion, Sensor triangulation.
      • Whatever you want to call it – this remains a challenge for geospatial experts and beginners alike. And it is especially a challenge when you work across disciplines (e.g. the work of SESYNC from Mary Shelley and Margaret Palmer, SESYNC, University of Maryland)
    • Dynamics: Spatio-temporal and real-time data streams: sensor networks, social media, cube sats
    • Resolution:
      • in space (e.g. the new Antarctic DEM from Paul Morin, University of Minnesota);
      • in time (e.g. cubesats, sensor networks; social media);
      • in depth?: going under-ground (from Debra Laefer, NYU)
    • We love FAIR for data. What about FAIR for tools: make tools Findable, Accessible, Interoperable, and Re-usable
  • Question 2: What does the CyberGIS Geographic Software Institute (GSI) need to do to address community needs and contribute to the national CyberInfrastructure ecosystem?
    • Link strongly with existing diversity-supporting frameworks: HBCU; community colleges; tribes; networks such as @WomenWhoCode, @LadiesOfLandsat, @BlackGirlsCode, @500womensci, @RLadiesGlobal, etc.
    • More of these workshops! Multi-disciplinary meetings of people with tight/packed agendas and make use of workshop attendees between workshops; what can we do to spread the word
    • Create GSI Data Institute or Bootcamp or Faculty Education Mentoring Network
    • Support standards for data and software standards to promote interoperability
    • Support frameworks for data and software discovery and interoperability: FAIR for data; FAIR for tools

Conclusion: Super Fun. Learned a Ton. Plus parting words from Michael Goodchild: It is not location that matters, it is context. Location provides context; context allows integration: with data, between disciplines, between people, between tools. "Let's get above the layers".

ESRI User Conference 2018 wrap-up

/

As always, the Plenary session was an immersive and emotional showcase of the power of mapping. Running through Monday’s talks was a sense of urgency for we GIS people to save the world. This is what JD calls “societal GIS”, or “embracing the digital transformation and leverage the science of where”. Shane and I had a great time. Some key news from the Plenary:

  • ESRI is in every K-12 school in the US; JD announced it will be offered to every K-12 school in the world. JD gave a special award to two inspirational teachers - Mariana Ramirez and Alice Im from the Technology Magnet Academy at Roosevelt High School in LA. Not a dry eye in the house: starts at 22.21 on this video. I hope they can hook up with @strtwyze. 
  • The work of Thomas Crowther, Professor of Global Ecosystem Ecology at ETH Zürich (@crowthelab) is inspirational. His talk here. They estimate 3T trees globally, with room for 1T more. (See paper here.)  Gonna be checking out his tree data on the Living Atlas (global maps of tree density, diversity, carbon uptake, and reflectance).
  • A great demo from JD Irving, a private Canadian forestry, transportation and products company heavy into sustainability and GIS. All there properties are managed using ArcGIS + R. Demo here. 
  • ESRI is showcasing some key "Solution Configurations" that are bundled software products focused on high-priority areas such as: 1) community engagement ("Hub"); 2) interior spaces ("Indoors") and, 3) smart cities ("Urban"). The highlighted snazzy urban planning 3D vis tools (demo here) will be giving UrbanSim a run for their money. Might we work RUCS2.0 into a "Solution Configuration" for working landscape planning? 

Plus some highlights of what I learned overall: 

Data updates

  • Wow. ESRI's Living Atlas of the World has some amazing resources. Living Atlas is ESRI’s curated web data portal that links seamlessly with Pro. It has tons of data on environment and imagery. Want Sentinel-2 imagery, NAIP, or MODIS thermal? Want global climate and weather data? Want to easily play with Open Street Map or other vector tiles within your GIS project? It is all in the Living Atlas. This will be a game changer for class. Plus TC’s tree data. Gonna be checking this out.
  • Unstructured data can be added to your workflow now, this is text, etc. This is big. 
  • ESRI is offering editable access to Open Street Map within Pro. 

Software updates (mostly about Pro)

  • Pro is the way to go, but ESRI will continue to support ArcMap “for years to come”
  • New stuff in ArcGIS Pro related to Image Analysis:
    • Sensor support has been expanded; plus new formats supported, eg. netcdf. Pro supports mosaic datasets, they call mosaics the optimum data model for image management. 
    • ESRI is now supporting “oriented” imagery - StreetView Imagery, oblique imagery, etc. Easily integrate things like iPhone photos within your Pro project. They call this working in “image space” rather than “map space”.
    • Ortho Mapping within ESRI has 3 solutions: Drone2Map (stand-alone software), within ArcGIS Pro (using the Image Server license), and OrthoMaker (web interface).
    • New release of Pro has full motion video support. (Upcoming releases will have more deep learning algorithms, multi-patch editing in stereo, and pixel editing.)
    • There are so many cool things going on on the imagery front in Pro, makes me excited.

  • New stuff in ArcGIS Pro in general:

    • Adding an unstructured data format - e.g. text!
    • 3D editing and 3D voxel support.
    • Machine Learning is increasingly embedded in ESRI workflows, and when that is not enough, ML is also possible via linkages with external resources (via R, TensorFlow, MXNET, AWS tools, etc.).
    • ESRI increasingly recognizing that people work in and outside of ESRI software: R-Bridge, Python API, Jupyter Notebooks makes external linkages super easy. 
  • ESRI is working to support cloud-based storage and computing with support via AWS and Azure; Optimizing raster storage and caching in multiple formats; and the ability to point to existing cloud storage
  • Plus, for your inexpensive GPS needs, consider the new Trimble Catalyst antenna + ESRI Collector might be the way to go, but it is windows/android specific for now. iOS compatibility is "on a horizon" as of now.
  • A quick note about ArcGIS online (ESRI's complete mapping and location intelligence platform). It has 6M subscribers (!), making 1B maps a day (!!). (Did I get those numbers right?)

Notes for classes/workshops

  • GIS-stat-analysis-py-tutor on GitHub. 
  • ESRI provides many Learning templates for us who are dreading converting all our ArcMap labs to Pro: https://www.esri.com/training/ and https://www.esri.com/training/learning-plans/. 
  • ESRI is also working on providing templated best practice workflows to help teach concepts. They call them, at least in Image Analyst "Imagery workflows". Might be useful in class/workshops. 

The new ESRI terminology might be a useful organizing structure for class: A GIS is a system of: 

  • Record: storing spatially indexed information
  • Insights: via analysis
  • Engagement: through mapping and visualization

As always a great conference!

#DroneCamp2018 is in the bag!

/

We've just wrapped up #DroneCamp2018, hosted at beautiful UC San Diego. 

This was an expanded version from last year's model, which we held in Davis. We had 52 participants (from all over the world!) who were keen to learn about drones, data analysis, new technology, and drone futures.  

Day 1 was a flight day from half our participants: lots of hands-on with takeoffs and landings, and flying a mission; 
Day 2 covered drone safety and regulations, with guest talks from Brandon Stark and Dominique Meyer;
Day 3 covered drone data and analysis;
Day 4 was a flight day for Group 2 and a repeat of Day 1. 

We had lots of fun taking pics and tweeting: here is our wrapup on Twitter for #DroneCamp2018.

NASA Data and the Distributed Active Archive Centers

/

I’ve been away from the blog for awhile, but thought I’d catch up a bit. I am in beautiful Madison Wisconsin (Lake Mendota! 90 degrees! Rain! Fried cheese curds!) for the NASA LP DAAC User Working Group meeting. This is a cool deal where imagery and product users meet with NASA team leaders to review products and tools. Since this UWG process is new to me, I am highlighting some of the key fun things I learned. 

What is a DAAC?
A DAAC is a Distributed Active Archive Center, run by NASA Earth Observing System Data and Information System (EOSDIS). These are discipline-specific facilities located throughout the United States. These institutions are custodians of EOS mission data and ensure that data will be easily accessible to users. Each of the 12 EOSDIS DAACs process, archive, document, and distribute data from NASA's past and current Earth-observing satellites and field measurement programs. For example, if you want to know about snow and ice data, visit the National Snow and Ice Data Center (NSIDC) DAAC. Want to know about social and population data? Visit the Socioeconomic Data and Applications Data Center (SEDAC). These centers of excellence are our taxpayer money at work collecting, storing, and sharing earth systems data that are critical to science, sustainability, economy, and well-being.

What is the LP DAAC?
The Land Processes Distributed Active Archive Center (LP DAAC) is one of several discipline-specific data centers within the NASA Earth Observing System Data and Information System (EOSDIS). The LP DAAC is located at the USGS Earth Resources Observation and Science (EROS) Center in Sioux Falls, South Dakota. LP DAAC promotes interdisciplinary study and understanding of terrestrial phenomena by providing data for mapping, modeling, and monitoring land-surface patterns and processes. To meet this mission, the LP DAAC ingests, processes, distributes, documents, and archives data from land-related sensors and provides the science support, user assistance, and outreach required to foster the understanding and use of these data within the land remote sensing community.

Why am I here?
Each NASA DAAC has established a User Working Group (UWG). There are 18 people on the LP DAAC committee, 12 members from the land remote sensing community at large, like me! Some cool stuff going on. Such as...

New Sensors
Two upcoming launches are super interesting and important to what we are working on. First, GEDI (Global Ecosystem Dynamics Investigation) will produce the first high resolution laser ranging observations of the 3D structure of the Earth. Second, ECOSTRESS (The ECOsystem Spaceborne Thermal Radiometer Experiment on Space Station), will measure the temperature of plants: stressed plants get warmer than plants with sufficient water. ECOSTRESS will use a multispectral thermal infrared radiometer to measure surface temperature. The radiometer will acquire the most detailed temperature images of the surface ever acquired from space and will be able to measure the temperature of an individual farmer's field. Both of these sensors will be deployed on the International Space Station, so data will be in swaths, not continuous global coverage. Also, we got an update from USGS on the USGS/NASA plan for the development and deployment of Landsat 10. Landsat 9 comes 2020, Landsat 10 comes ~2027.

Other Data Projects
We heard from other data providers, and of course we heard from NEON! Remember I posted a series of blogs about the excellent NEON open remote sensing workshop I attended last year. NEON also hosts a ton of important ecological data, and has been thinking through the issues associated with cloud hosting. Tristin Goulden was here to give an overview.

Tools Cafe
NASA staff gave us a series of demos on their WebGIS services; AppEEARS; and their data website. Their webGIS site uses ArcGIS Enterprise, and serves web image services, web coverage services and web mapping services from the LP DAAC collection. This might provide some key help for us in IGIS and our REC ArcGIS online toolkits. AppEEARS us their way of providing bundles of LP DAAC data to scientists. It is a data extraction and exploration tool. Their LP DAAC data website redesign (website coming soon), which was necessitated in part by the requirement for a permanent DOI for each data product.

User Engagement
LP DAAC is going full-force in user engagement: they do workshops, collect user testimonials, write great short pieces on “data in action”, work with the press, and generally get the story out about how NASA LP DAAC data is used to do good work. This is a pretty great legacy and they are committed to keep developing it. Lindsey Harriman highlighted their excellent work here.

Grand Challenges for remote sensing
Some thoughts about our Grand Challenges: 1) Scaling: From drones to satellites. It occurs to me that an integration between the ground-to-airborne data that NEON provides and the satellite data that NASA provides had better happen soon; 2) Data Fusion/Data Assimilation/Data Synthesis, whatever you want to call it. Discovery through datasets meeting for the first time; 3) Training: new users and consumers of geospatial data and remote sensing will need to be trained; 4) Remote Sensible: Making remote sensing data work for society. 

A primer on cloud computing
We spent some time on cloud computing. It has been said that cloud computing is just putting your stuff on “someone else’s computer”, but it is also making your stuff “someone else’s problem”, because cloud handles all the painful aspects of serving data: power requirements, buying servers, speccing floor space for your servers, etc. Plus, there are many advantages of cloud computing. Including: Elasticity. Elastic in computing and storage: you can scale up, or scale down or scale sideways. Elastic in terms of money: You pay for only what you use. Speed. Commercial clouds CPUs are faster than ours, and you can use as many as you want. Near real time processing, massive processing, compute intensive analysis, deep learning. Size. You can customize this; you can be fast and expensive or slow and cheap. You use as much as you need. Short-term storage of large interim results or long-term storage of data that you might use one day.

Image courtesy of Chris Lynnes

Image courtesy of Chris Lynnes

We can use the cloud as infrastructure, for sharing data and results, and as software (e.g. ArcGIS Online, Google Earth Engine). Above is a cool graphic showing one vision of the cloud as a scaled and optimized workflow that takes advantage of the cloud: from pre-processing, to analytics-optimized data store, to analysis, to visualization. Why this is a better vision: some massive processing engines, such as SPARC or others, require that data be organized in a particular way (e.g. Google Big Table, Parquet, or DataCube). This means we can really crank on processing, especially with giant raster stacks. And at each step in the workflow, end-users (be they machines or people) can interact with the data. Those are the green boxes in the figure above. Super fun discussion, leading to importance of training, and how to do this best. Tristan also mentioned Cyverse, a new NSF project, which they are testing out for their workshops.

Image attribution: Corey Coyle

Image attribution: Corey Coyle

Super fun couple of days. Plus: Wisconsin is green. And warm. And Lake Mendota is lovely. We were hosted at the University of Wisconsin by Mutlu Ozdogan. The campus is gorgeous! On the banks of Lake Mendota (image attribution: Corey Coyle), the 933-acre (378 ha) main campus is verdant and hilly, with tons of gorgeous 19th-century stone buildings, as well as modern ones. UW was founded when Wisconsin achieved statehood in 1848, UW–Madison is the flagship campus of the UW System. It was the first public university established in Wisconsin and remains the oldest and largest public university in the state. It became a land-grant institution in 1866. UW hosts nearly 45K undergrad and graduate students. It is big! It has a med school, a business school, and a law school on campus. We were hosted in the UW red-brick Romanesque-style Science Building (opened in 1887). Not only is it the host building for the geography department, it also has the distinction of being the first building in the country to be constructed of all masonry and metal materials (wood was used only in window and door frames and for some floors), and may be the only one still extant. How about that! Bye Wisconsin!

Mapping fires and fire damage in real time: available geospatial tools

/

Many of us have watched in horror and sadness over the previous week as fires consumed much of the beautiful hills and parts of the towns of Napa and Sonoma Counties. Many of us know people who were evacuated with a few minutes’ notice - I met a retired man who left his retirement home with the clothes on his back. Many other friends lost everything - house, car, pets. It was a terrible event - or series of events as there were many active fires. During those 8+ days all of us were glued to our screens searching for up-to-date and reliable information on where the fires were, and how they were spreading. This information came from reputable, reliable sources (such as NASA, or the USFS), from affected residents (from Twitter and other social media), and from businesses (like Planet, ESRI, and Digital Globe who were sometimes creating content and sometimes distilling existing content), and from the media (who were ofen using all of the above). As a spatial data scientist, I am always thinking about mapping, and the ways in which geospatial data and analysis plays an increasingly critical role in disaster notification, monitoring, and response. I am collecting information on the technological landscape of the various websites, media and social media, map products, data and imagery that played a role in announcing and monitoring the #TubbsFire, #SonomaFires and #NapaFires. I think a retrospective of how these tools, and in particular how the citizen science aspect of all of this, helped and hindered society will be useful.  

In the literature, the theoretical questions surrounding citizen science or volunteered geography revolve around:

  • Accuracy – how accurate are these data? How do we evaluate them?  

  • Access – Who has access to the data? Are their technological limits to dissemination?

  • Bias (sampling issues)/Motivation (who contributes) are critical.

  • Effectiveness – how effective are the sites? Some scholars have argued that VGI can be inhibiting. 

  • Control - who controls the data, and how and why?

  • Privacy - Are privacy concerns lessened post disaster?

I think I am most interested in the accuracy and effectiveness questions, but all of them are important.  If any of you want to talk more about this or have more resources to discuss, please email me: maggi@berkeley.edu, or Twitter @nmaggikelly.

Summary so far. This will be updated as I get more information.

Outreach from ANR About Fires

Core Geospatial Technology During Fires

Core Technology for Post-Fire Impact

 

Wrap up from #DroneCamp2017!

/

UC ANR's IGIS program hosted 36 drone enthusiasts for a three day DroneCamp in Davis California. DroneCamp was designed for participants with little to no experience in drone technology, but who are interested in using drones for a variety of real world mapping applications. The goals of DroneCamp were to:

  • Gain an broader understanding of the drone mapping workflow: including
    • Goal setting, mission planning, data collection, data analysis, and communication & visualization
  • Learn about the different types of UAV platforms and sensors, and match them to specific mission objectives;
  • Get hands-on experience with flight operations, data processing, and data analysis; and
  • Network with other drone-enthusiasts and build the California drone ecosystem. 

The IGIS crew, including Sean Hogan, Andy Lyons, Maggi Kelly, Robert Johnson, Kelly Easterday, and Shane Feirer were on hand to help run the show. We also had three corporate sponsors: GreenValley Intl, Esri, and Pix4D. Each of these companies had a rep on hand to give presentations and interact with the participants.

Day 1 of #DroneCamp2017 covered some of the basics - why drone are an increasingly important part of our mapping and field equipment portfolio; different platforms and sensors (and there are so many!); software options; and examples. Brandon Stark gave a great overview of the Univ of California UAV Center of Excellence and regulations, and Andy Lyons got us all ready to take the 107 license test. We hope everyone here gets their license! We closed with an interactive panel of experienced drone users (Kelly Easterday, Jacob Flanagan, Brandon Stark, and Sean Hogan) who shared experiences planning missions, flying and traveling with drones, and project results. A quick evaluation of the day showed the the vast majority of people had learned something specific that they could use at work, which is great. Plus we had a cool flight simulator station for people to practice flying (and crashing).

Day 2 was a field day - we spent most of the day at the Davis hobbycraft airfield where we practiced taking off, landing, mission planning, and emergency maneuvers. We had an excellent lunch provided by the Street Cravings food truck. What a day! It was hot hot hot, but there was lots of shade, and a nice breeze. Anyway, we had a great day, with everyone getting their hands on the commands. Our Esri rep Mark Romero gave us a demo on Esri's Drone2Map software, and some of the lidar functionality in ArcGIS Pro.

Day 3 focused on data analysis. We had three workshops ready for the group to chose from, from forestry, agriculture, and rangelands. Prior to the workshops we had great talks from Jacob Flanagan and GreenValley Intl, and Ali Pourreza from Kearney Research and Extension Center. Ali is developing a drone-imagery-based database of the individual trees and vines at Kearney - he calls it the "Virtual Orchard". Jacob talked about the overall mission of GVI and how the company is moving into more comprehensive field and drone-based lidar mapping and software. Angad Singh from Pix4D gave us a master class in mapping from drones, covering georeferencing, the Pix4D workflow, and some of the checks produced for you a the end of processing.

One of our key goals of the DroneCamp was to jump start our California Drone Ecosystem concept. I talk about this in my CalAg Editorial. We are still in the early days of this emerging field, and we can learn a lot from each other as we develop best practices for workflows, platforms and sensors, software, outreach, etc. Our research and decision-making teams have become larger, more distributed, and multi-disciplinary; with experts and citizens working together, and these kinds of collaboratives are increasingly important. We need to collaborate on data collection, storage, & sharing; innovation, analysis, and solutions. If any of you out there want to join us in our California drone ecosystem, drop me a line.

Thanks to ANR for hosting us, thanks to the wonderful participants, and thanks especially to our sponsors (GreenValley Intl, Esri, and Pix4D). Specifically, thanks for:

  • Mark Romero and Esri for showing us Drone2Map, and the ArcGIS Image repository and tools, and the trial licenses for ArcGIS;
  • Angad Singh from Pix4D for explaining Pix4D, for providing licenses to the group; and
  • Jacob Flanagan from GreenValley Intl for your insights into lidar collection and processing, and for all your help showcasing your amazing drones.

#KeepCalmAndDroneOn!

AAG Boston 2017 Day 1 wrap up!

/

Day 1: Thursday I focused on the organized sessions on uncertainty and context in geographical data and analysis. I’ve found AAGs to be more rewarding if you focus on a theme, rather than jump from session to session. But less steps on the iWatch of course. There are nearly 30 (!) sessions of speakers who were presenting on these topics throughout the conference.

An excellent plenary session on New Developments and Perspectives on Context and Uncertainty started us off, with Mei Po Kwan and Michael Goodchild providing overviews. We need to create reliable geographical knowledge in the face of the challenges brought up by uncertainty and context, for example: people and animals move through space, phenomena are multi-scaled in space and time, data is heterogeneous, making our creation of knowledge difficult. There were sessions focusing on sampling, modeling, & patterns, on remote sensing (mine), on planning and sea level rise, on health research, on urban context and mobility, and on big data, data context, data fusion, and visualization of uncertainty. What a day! All of this is necessarily interdisciplinary. Here are some quick insights from the keynotes.

Mei Po Kwan focused on uncertainty and context in space and time:

  • We all know about the MAUP concept, what about the parallel with time? The MTUP: modifiable temporal unit problem.
  • Time is very complex. There are many characteristics of time and change: momentary, time-lagged response, episodic, duration, cumulative exposure
    • sub-discussion: change has patterns as well - changes can be clumpy in space and time. 
  • How do we aggregate, segment and bound spatial-temporal data in order to understand process?
  • The basic message is that you must really understand uncertainty: Neighborhood effects can be overestimated if you don’t include uncertainty.

As expected, Michael Goodchild gave a master class in context and uncertainty. No one else can deliver such complex material so clearly, with a mix of theory and common sense. Inspiring. Anyway, he talked about:

  • Data are a source of context:
    • Vertical context – other things that are known about a location, that might predict what happens and help us understand the location;
    • Horizontal context – things about neighborhoods that might help us understand what is going on.
    • Both of these aspects have associated uncertainties, which complicate analyses.
  • Why is geospatial data uncertain?
    • Location measurement is uncertain
    • Any integration of location is also uncertain
    • Observations are non-replicable
    • Loss of spatial detail
    • Conceptual uncertainty
  • This is the paradox. We have abundant sources of spatial data, they are potentially useful. Yet all of them are subject to myriad types of uncertainty. In addition, the conceptual definition of context is fraught with uncertainty.
  • He then talked about some tools for dealing with uncertainty, such as areal interpolation, and spatial convolution.
  • He finished with some research directions, including focusing on behavior and pattern, better ways of addressing confidentiality, and development of a better suite of tools that include uncertainty.

My session went well. I chaired a session on uncertainty and context in remote sensing with 4 great talks from Devin White and Dave Kelbe from Oak Ridge NL who did a pair of talks on ORNL work in photogrammetry and stereo imagery, Corrine Coakley from Kent State who is working on reconstructing ancient river terraces, and Chris Amante from the great CU who is developing uncertainty-embedded bathy-topo products. My talk was on uncertainty in lidar inputs to fire models, and I got a great question from Mark Fonstad about the real independence of errors – as in canopy height and canopy base height are likely correlated, so aren’t their errors? Why do you treat them as independent? Which kind of blew my mind, but Qinghua Guo stepped in with some helpful words about the difficulties of sampling from a joint probability distribution in Monte Carlo simulations, etc. 

Plus we had some great times with Jacob, Leo, Yanjun and the Green Valley International crew who were showcasing their series of Lidar instruments and software. Good times for all!

Google Earth Engine @ the GIF!

/

Students, researchers, mappers, and big data enthusiasts took place in an exciting 2 day Google Earth Engine workshop this last week hosted by the GIF and the Google Earth Engine Team. We had an exiting overview of the latest and greatest research adventures from Google by Kelly lab alum Karin Tuxen-Bettman including advances in some of what Google Earth Outreach team is involved in...

As well as new/upcoming ventures

The Earth Engine team led some great tutorials getting people well versed in JavaScript and using the Earth Engine playground, and Earth Engine API. Having beginner and advanced workshop tracks during the two day event allowed for both broad and deep participation from researchers across the Berkeley campus. Take a look at the packed agenda and more

here!

We also had a stellar panel of UC Berkeley professor Jeff Chambers and graduate students Sophie Taddeo, Alexander Bryk, and Lisa Kelley who shared an intimate view of how they were using Earth Engine in their research. The panel shared stories of using Earth Engine to evaluate disturbance in tropical forests, map the movement of wetlands, and meandering rivers, as well as looking at agroforestry systems in Indonesia through a socio-ecological lens.

Thanks to Google and the Earth Engine Team for guiding, the GIF for hosting, and all of the participants for engaging in an action packed two days!

GIS Day 2014!

/

Discovering the World Through GIS

November 19, 2014, 5PM-8:30PM

UC Berkeley, Mulford Hall

GIS Day took place in Mulford Hall Wednesday Nov 19th from 5-8:30pm. We had about 200 attendees who participated in workshops, listened to talks, saw posters, and networked with other like-minded GIS-enthusiasts.

Some of the activity at 2014 GIS Day in Mulford Hall

See the agenda here: http://gif.berkeley.edu/gisday.html.

2014 Western Section of the Wildlife Society Meeting wrap-up

/

I recently attended the 2014 annual meeting of the Western Section of the Wildlife Society in Reno CA. The focus of the conference was on harnessing citizen science toward greater conservation.

I saw some interesting talks in my session (I was clearly the odd-talk-out in a session dominated by animal tracking (I spoke about our SNAMP website evaluation)). For example:

  • Peter Bloom discussed red-tailed hawk movements from banded bird recovery. The birds are banded as juveniles and observed by citizens and scientists. In this way their movements can be mapped: across southern California, across the Pacific flyway, and across the US.
  • Joe Burnett presented on the use of GSM transmitters to track California condor (the largest flying bird in north America) movement patterns. He caught us up on condor recovery and current threats (lead poisoning from foraging on wild game) to condors. He showed some very nice visualizations of wild condor flights between Ventana and the Pinnacles (including some stops for water and dead animal chomping) from the GSM transmitters and Google Earth. 
  • Shannon Rich looked at migration patterns of flammulated owls using light-level geolocators. "What is a flammulated owl”? you say: I will tell you. They are super cute tiny owls, with neat flame-like markings on their face and body. Geolocators are small (~1g) that record ambient light levels during the day, and from timing of sunrise and sunset, you can get latitude and longitude. These are not sending out signals, and you need to recapture the owl to download data. As always, I am stunned by the dedication and time it takes for wildlife biologists to gather their careful data on animal movement.
  • Russ Bryant talked about native honeybee habitat in North Dakota. He talked about the important services that bees give us: 95 agricultural plants benefit from pollination services (estimated at $15b). I did not know that ND is the top honey producer in the US. Colony collapse across the US has been profound. They used INVEST to explore the role of land cover and bee pollination to produce a pollinator habitat index, and a habitat connectivity for areas where bees had been captured. 

In the climate change session, I heard from a range of speakers on practical adaptation strategies, curriculum for climate change education (Whitney Albright), new tools and reports for grassland bird species conservation (Ryan Diguadio), landscape-scale conservation planning for bobcats in the San Diego area (Megan Jennings), and some neat genetics of the SF Bay’s salt marsh harvest mouse (Mark Statham). Also, Curtis Alling talked about local, regional and state climate preparedness planning, and dedicated a slide to cal-adapt.org. Nice!

I also got to catch up briefly with ESPM grads Sarah Sawyer who is now at the Forest Service and Tim Bean, who is thriving at HSU. Alice, he suggested a trip up to Redwood State Park to check out the dark figure of crime in the tall trees.