Recurrent Neural Networks applied in the time-series classification over a high resolution data

[EvandroCT]

Idea (Proposed by student)

The increasing number of sensors orbiting the earth is systematically producing larger volumes of data, with better spatiotemporal resolutions. To deal with that, better accurate machine learning approaches, such as Deep Learning (DL), are needed to transform raw data into applicable Information. Several DL architectures (e.g. CNN, semantic segmentation) rely only at spatial dimension to perform, for example, land-cover/land-use (LCLU) maps, disregarding the temporal dependencies between pixels observations over the time. Also, high-res remote sensing data (e.g. Planet, Sentinel) may provide more consistent time-series, that can be use in the identification of important LCLU classes, like crop, pastureland and grasslands.

This potential can be explored using Recurrent Neural Networks (RNN), a specific family of DL approaches which can take temporal dimension into account. A promising project idea would be implement a RNN approach (e.g. LSTM) to classify a Sentinel time-series, that will organize and preprocess an input data set (e.g. labeled time-series), calibrate and evaluate a RNN model, and finally classify an entire region (i.e. 2 or 3 scenes) to produce a map for one or more LCLU classes and ship the final version of the model into the Pycoal library as part of the COAL project. It will be great evaluate the accuracy and the spatial consistency of a map produced with a RNN approach.

A simple example on classifying LCLU with two classes (pastureland and non-pastureland):

Target region (input)

Generated LCLU map (output)

Skills Needed

• Good understanding of Machine Learning and Deep-Learning;
• Python and Tensorflow;
• QGIS;
• Desirable: TensorBoard, scikit-learn, Shell Script, GDAL and RIOS.

Mentors
Seeking for mentors. Interested ones, please, email me: [email protected]

[Prakash2403]

@EvandroCT Is there some eligibility criteria for becoming a mentor?

[EvandroCT]

@Prakash2403 Thank you for being interested! The main restriction is that a mentor cannot participate also as a student, as appointed in the clause 5.2 of the GSoC Program Rules. If you would like to become a mentor, please, ask @abburgess to send you an invitation.

[EvandroCT]

@Prakash2403, I suppose not. If you choose to mentor my project idea, you should not submit at least the project idea (which is what normally happens), but I don't know if there's a mentor's application asked by the organization.

[esip-lab]

Hi all - As ESIP is the mentoring organization, the project mentor would have to be affiliated with an ESIP Member Organization.

…

On Thu, Mar 22, 2018 at 1:50 PM, Evandro Carrijo Taquary < ***@***.***> wrote: @Prakash2403 <https://github.com/prakash2403>, I suppose not. If you choose to mentor my project idea, you should not submit at least the project idea (which is what normally happens), but I don't know if there's a mentor's application asked by the organization. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG_XwPsn8R8QL4oQ9dZYnODeSZHdQnaZks5thA6TgaJpZM4S1l25> .

[Prakash2403]

@abburgess Is there a standard procedure to join ESIP member organization?

[lewismc]

Hi @EvandroCT I would be interested in taking this on as a mentor however I have a few suggestions

1. I would also highly encourage you to consider using Apache MXNet simply due to the fact that I am already a member of the Apache Software Foundation therefore we would have much more traction if and when we were either looking for help or attempting to contribute code to the MXNet Project.
2. I already have an ongoing project called Coal and Open-pit surface mining impacts on American Lands (COAL) where we implement various types of classification. Currently we use the primitive Spectral Angle Mapper classification algorithm for undertaking pixel classification however I would VERY interested to see if a DL approach could improve classification over SAM.

Would you be interested in adapting this issue to work on # 2 above? If so, then I would happily mentor it.

[EvandroCT]

@lewismc Thank you for being interested! I took a look at the COAL project and it sounds very interesting. I have already played around with MXNet within R, but at that time, the "port" to the R environment was a little lacky and I end up using Tensorflow at all. But, as we're going to Python coding (its native environment), it's a great opportunity for me to take a step back and give MXNet another trial.

For now I have few questions:

1. What is, approximately, the temporal resolution of the available data?
2. Is there already a reliable data set (ground-truths) in which I can start training right away, or, will I have to do some data pre-processing? I ask that because I'm not familiarized with the kind of targets used in the COAL project and never have worked with hyperspectral imagery; this could become a time constraint due to GSoC deadlines.

Nevertheless, I'm already studying COAL's documentation to yield insights on my proposal.

[adi07wad]

Hi All,

I am interested in contributing to this project and wanted to understand about the project in further detail. Can someone please guide me on where I can find further documentation about this project?

Thanks and regards,
Aditi

[EvandroCT]

Hi @adi07wad! I'm about to publish a first version of the project's proposal and, as soon as finish it, I'll share the draft here with you, ok?

Thank you for being interested!

[lewismc]

@EvandroCT

What is, approximately, the temporal resolution of the available data?

It depends on which flight lines from AVIRIS-C and AVIRIS-NG we use. The instrument is flown as a suborbital mission hence there is no predictale repeat period e.g. no temporal repeat.
This does not however meant that we could not still progress with the project. I would be VERY interested in extending COAL to accommodate other (possibly obital) datasets which demonstrate repeat periods.

Is there already a reliable data set (ground-truths) in which I can start training right away, or, will I have to do some data pre-processing? I ask that because I'm not familiarized with the kind of targets used in the COAL project and never have worked with hyperspectral imagery; this could become a time constraint due to GSoC deadlines.

I have enough experience to get us off of the ground. ALL of the datasets for AVIRIS-C and AVIRIS-NG are available from

https://aviris.jpl.nasa.gov/alt_locator/, and

https://aviris-ng.jpl.nasa.gov/alt_locator/

[EvandroCT]

@lewismc I have looked at the AVIRIS datasets and already noticed the observations you made. Even though there's no repeat periods, there are some regions with big density of flight lines (namely Hawaii and California) which have good - but irregular - temporal resolutions. But, for sake of simplicity, at the beginning we could work with Sentinel or other open high-res imagery.

I already have seen the COAL documentation and suppose that's OK incorporate to the library the routines that will be made.

Here is the draft proposal I'm about to finish: https://docs.google.com/document/d/1uzK09fEVrrLfNJ4MHJ_AvkR0_7E0ZslBumOC_hYIbV0/edit?usp=sharing

Please, provide me feedback, if necessary. I already consider your mentorship :)

[lewismc]

Hi @EvandroCT the proposal is looking reasonable but it is missing some key criteria such as reporting.
If possible, I would like to set up a WebEx or GHangout to iron out the issues with you before you submit the proposal. Would that be OK?

[EvandroCT]

@lewismc Yes, it is ok! I sent you an email to schedule the conference.

[lewismc]

@EvandroCT an you please populate the report at https://github.com/ESIPFed/GSoC/wiki/ISSUE-10-Reporting#week-1-community-bonding-period1st-may--7th-may, thank you

[EvandroCT]

@lewismc Done! Thank you for remembering.

[muskanchugh5]

@EvandroCT Where can I get the further details of the project.

[lewismc]

@muskan6769 all of the code was baked into the pycoal toolkit.
The final pull request for the project is located at
capstone-coal/pycoal#159
Documentation for the classes and functions you can use to classify AVIRIS-C and AVIRIS-NG imagery using DNN can be located at https://pycoal.readthedocs.io/en/latest/
Any questions, please tag me and I can assist.

[lewismc]

@EvandroCT @abburgess I am closing this issue off as it is essentially done :)