Cloud masking
From AWF-Wiki
(Difference between revisions)
(→Applying cloud and cloud shadow mask) |
(→Graphical Processing Tool Model) |
||
| (71 intermediate revisions by one user not shown) | |||
| Line 1: | Line 1: | ||
| − | = Sentinel-2 | + | = Inspect Sentinel-2 quality indicator data = |
| − | The exclusion of clouds and cloud shadow is an important processing step which is usually done in an early | + | The exclusion of clouds and cloud shadow is an important processing step which is usually done in an early pre-processing stage. |
| − | The Sentinel-2 products are annotated with Quality Indicators (QI_DATA). | + | The Sentinel-2 products are annotated with Quality Indicators (QI_DATA). |
| − | + | ||
| − | + | If you use a Sentinel-2 Level 1C product, you'll find a rough dense cloud and cirrus mask (GML vector format) in the folder '''QI_DATA''' with the file name '''MSK_CLOUDS_B00.gml'''. The vector attribute column ''masktype'' distinguishes two classes: OPAQUE (dense clouds) and CIRRUS. | |
| + | |||
| + | In case of a Sentinel-2 Level 2A product you may find a file in the folder '''QI_DATA''' with the extension '''_CLD_20m.jp2''' (data type: 8bit unsigned integer, spatial resolution 20m) which might be used for masking clouds. Data range is from 0 for high confidence clear sky to 100 for high confidence cloudy (for more details see [https://earth.esa.int/documents/247904/685211/S2+L2A+Product+Definition+Document/2c0f6d5f-60b5-48de-bc0d-e0f45ca06304 L2A Product Definition Document]). | ||
| + | |||
| + | * Load a multiband Sentinel-2 satellite image (Level 2-A product) [[Changing Raster Layer Style]] to display a true color (RGB = B4,B3,B2) and a standard false color composite (RGB = B8,B4,B3) | ||
| + | * Zoom in to a cloudy part of the image and compare the extent of cloud and cloud shadow in both composites. Which composite is better for displaying clouds? | ||
[[File:Qgis_cloud_tcc.png|400px]] [[File:Qgis_cloud_fcc.png|400px]] | [[File:Qgis_cloud_tcc.png|400px]] [[File:Qgis_cloud_fcc.png|400px]] | ||
| − | + | * Overlay the QI Cloud band on top of the color composites. Use the {{tool|text=Identify Feature Tool}} for requesting pixel values of the QI band file inside a cloud. Which values represent clouds? | |
| − | + | * Change '''Render type''' {{button|text=Singelband pseudocolor}} | |
| − | + | * '''Interpolation''' is {{button|text=discrete}}'''. | |
| − | + | * Choose '''Color ramp''' {{button|text=spectral}} | |
| + | * '''Mode''' is {{button|text=equal}} with | ||
| + | * Interval '''Classes''' is {{typed|text=10}} as shown in the screenshot. | ||
| + | * Double click on a color field of some classes {{mitem|text=Layer --> Properties --> Style) to open the change color window. Change the opacity from 100% to 0%. Find the threshold for clouds. | ||
[[File:Qgis_cloud_qi_pseudo.png|400px]] [[File:Qgis_cloud_qi_opacity.png|300px]] | [[File:Qgis_cloud_qi_pseudo.png|400px]] [[File:Qgis_cloud_qi_opacity.png|300px]] | ||
| − | = | + | = Convert quality indicator band to a binary cloud mask = |
| − | + | * Create a binary cloud mask (bitmask) where clouds = 1 and non-clouds = 0. Use the Processing tool {{mitem|text= OTB --> Image Manipulation --> Band Math}}. Tick the single band QI layer ('''_CLD''') in the '''Input Image list''' list. In the Parameter tab write a conditional ''ifelse expression'' using lazy operators. | |
| − | [[File:Qgis_cloud_bandmath.png| | + | Type into the '''Expression''' field: {{typed|text=(im1b1>30)?1:0}} |
| − | + | If pixel values of band 1 of multiband image 1 are larger than 30 then replace with 1 otherwise replace with 0. Adjusot labels for NoData {{typed|text=3}} and undecided {{typed|text=2}} he output pixel type is '''uint8'''. | |
| − | [[File:Qgis_cloud_majority.png| | + | |
| − | + | [[File:Qgis_cloud_bandmath.png|400px]]] | |
| − | [[File:Qgis_cloud_dilate.png| | + | |
| − | + | * Apply a circular majority filter with a radius of 16 pixels. Open the Processing tool {{mitem|text=OTB --> Learning --> ClassificationMapRegularization}}. The input classifcation image is the Output Image of the band math operation step. Adjust the Label for NoData {{typed|text=3}} and Undecided {{typed|text=2}} class. Structuring element radius is 16 pixels. Output pixel type is '''uint8'''. | |
| + | |||
| + | [[File:Qgis_cloud_majority.png|400px]] | ||
| + | |||
| + | * Open the Processing tool {{mitem|text= OTB --> Feature Extraction --> BinaryMorphologicalOperation}}. Choose the '''Regularized image''' as input image. The structuring element type is '''ball''' with a X and Y radius of 21 pixels. The morphological operation is {{button|text=dilate}}. Output pixel type is {{button|text=uin8}}. | ||
| + | |||
| + | [[File:Qgis_cloud_dilate.png|400px]] | ||
| + | |||
| + | * Convert the Feature Output Imge from raster to vector format. {{mitem|text= Raster --> Conversion --> Polygonize}}. Select an output file name for vector polygons in format GPKG. | ||
| + | |||
[[File:Qgis_cloud_polygonize.png|400px]] | [[File:Qgis_cloud_polygonize.png|400px]] | ||
| − | + | ||
| − | + | * Change the display of the output vector polygon layer {{mitem|text= Layer properties --> Symbology --> Single Symbol --> Simple Fill --> Fill Style}} from ''Solid'' to ''No Brush''. Change the '''Outline color''' to bright green and increase the '''Outline width'''. As background layer use a Standard False Color Composite (RGB = B8,B$,B3) of the Sentinel-2 image. | |
| + | |||
| + | * Toggle editing [[File:Qgis_vector_edit.png]] of the resulting vector polygon layer. Activate {{tool|text=Select polygons by dragging a box or by single click}} [[File:Qgis_vector_select.png]]. select all polygons with attribute '''DN''' = 0 and delete them by pressing {{button|text=Del}} on your keyboard. Save your edits [[File:Qgis_vector_save.png]]. | ||
= Creating cloud shadow mask = | = Creating cloud shadow mask = | ||
| − | + | * Prepare a False color composite (RGB = B8,B4,B3) of the Sentinel-2 image as background layer in the canvas. | |
| − | + | * Right click on the cloud polygon layer in the [[TOC]]. Choose {{button|text= Export --> Save features as...}}. Fill the form as shown below. | |
[[File:Qgis_vector_saveas.png|400px]] | [[File:Qgis_vector_saveas.png|400px]] | ||
| − | + | * Mark the new created polygon layer in the [[TOC]] and toggle editing [[File:Qgis_vector_edit.png]] of the vector file. Select all polygons using [[File:Qgis_vector_select_all.png|200px]] or press {{button|text=Ctrl + A}}. Activate {{mitem|text=Edit --> Move feature(s)}} [[File:Qgis_vector_move.png]]. Left click on a selected yellow polygon, move the polygon that shadows are completely covered, and confirm with left click. Save your edits and [[File:Qgis_vector_save.png]]and stop editing [[File:Qgis_vector_edit.png]]. | |
| − | + | ||
| − | + | = Merge cloud and cloud shadow mask = | |
| − | = | + | * {{mitem|text= Vector --> Data management tools --> Merge vector layers}}. as '''Layer to merge''' select the ''cloud'' and ''cloud_shadow'' polygon layers. {{button|text=Run}}. |
| − | + | [[File:Qgis_vector_merge.png|400px]] | |
| − | [[File:Qgis_vector_merge.png|400px]] | + | = Apply the cloud and cloud shadow mask to a Sentinel-2 multiband file = |
| − | + | * Create a polygon with the extent of the Sentinel-2 image: [[Create image extent polygon]]. | |
| − | + | * {{mitem|text= Vector --> Geoprocessing Tools --> Symmetrical difference}}. Input layers are the image extent and the merged cloud and cloud_shadow polygons. | |
[[File:Qgis_vector_symdiff.png|400px]] | [[File:Qgis_vector_symdiff.png|400px]] | ||
| − | + | * Apply the output vector layer '''cloud_cloudshadow''' as mask to the multiband Sentinel-2 image file. {{mitem|text= Raster --> Extraction --> Clip Raster by Mask Layer}}. | |
| − | [[File: | + | |
| − | # | + | =Graphical Processing Tool Model= |
| + | Use a prepared graphical processing toolbox model. Download the file from Stud.IP: {{typed|text=S2cloudDetection.model3}}. | ||
| + | # Open QGIS and go to the Processing Toolbox; if the Toolbox is not opened click {{mitem|text= Processing --> Toolbox}} to activate it. Once the Toolbox is opened go to [[File:qgis-processing_model_iconExample.png]] {{mitem|text= Models --> Add model to Toolbox...}}. Open the downloaded model. | ||
| + | # The model should appear in the Toolbox under {{mitem|text=Models --> S2 --> S2 Cloud Detection}}. Double click to execute. | ||
| + | # A window showing the different input and outputs should appear, just as if you were using any other geo-algorithm of the Processing Toolbox. | ||
| + | # Just provide a multispectral Sentinel-2 image file with 10 bands with the rank (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12). Click {{button|text=Run}}. | ||
| + | # {{mitem|text=Models --> S2 --> S2 Cloud Detection}} . Right click {{button|text=Edit Model}}. In the Model Designer double click on the graphical objects to display parameters of processing components. It is easy to change parameter settings of input, output or processing steps. Save changes in the Model Designer {{mitem|text=Model --> Save Model as...}}. | ||
| + | |||
| + | [[File:Qgis_s2_cloud_detection_model.png|700px]] | ||
| + | |||
| + | Find out how to use the [https://docs.qgis.org/3.16/en/docs/user_manual/processing/modeler.html?highlight=graphical%20model graphical modeler] to program your own workflow. | ||
| + | |||
| + | {| class="wikitable" | ||
| + | | style="border: 0pt" | [[file:Qgis_cloud_fcc.png|thumb|left|400px|'''Figure A:''' False color composite RGB=8,4,3]] | ||
| + | | style="border: 0pt" | [[file:Qgis_cloud_fcc_masked.png|thumb|left|400px|'''Figure B:''' False color composite RGB=8,4,3 masked with cloud and cloud shadow]] | ||
| + | |} | ||
| + | |||
[[Category:QGIS Tutorial]] | [[Category:QGIS Tutorial]] | ||
Latest revision as of 12:05, 19 November 2020
Contents |
[edit] Inspect Sentinel-2 quality indicator data
The exclusion of clouds and cloud shadow is an important processing step which is usually done in an early pre-processing stage. The Sentinel-2 products are annotated with Quality Indicators (QI_DATA).
If you use a Sentinel-2 Level 1C product, you'll find a rough dense cloud and cirrus mask (GML vector format) in the folder QI_DATA with the file name MSK_CLOUDS_B00.gml. The vector attribute column masktype distinguishes two classes: OPAQUE (dense clouds) and CIRRUS.
In case of a Sentinel-2 Level 2A product you may find a file in the folder QI_DATA with the extension _CLD_20m.jp2 (data type: 8bit unsigned integer, spatial resolution 20m) which might be used for masking clouds. Data range is from 0 for high confidence clear sky to 100 for high confidence cloudy (for more details see L2A Product Definition Document).
- Load a multiband Sentinel-2 satellite image (Level 2-A product) Changing Raster Layer Style to display a true color (RGB = B4,B3,B2) and a standard false color composite (RGB = B8,B4,B3)
- Zoom in to a cloudy part of the image and compare the extent of cloud and cloud shadow in both composites. Which composite is better for displaying clouds?
- Overlay the QI Cloud band on top of the color composites. Use the Identify Feature Tool for requesting pixel values of the QI band file inside a cloud. Which values represent clouds?
- Change Render type Singelband pseudocolor
- Interpolation is discrete.
- Choose Color ramp spectral
- Mode is equal with
- Interval Classes is 10 as shown in the screenshot.
- Double click on a color field of some classes {{mitem|text=Layer --> Properties --> Style) to open the change color window. Change the opacity from 100% to 0%. Find the threshold for clouds.
[edit] Convert quality indicator band to a binary cloud mask
- Create a binary cloud mask (bitmask) where clouds = 1 and non-clouds = 0. Use the Processing tool OTB --> Image Manipulation --> Band Math. Tick the single band QI layer (_CLD) in the Input Image list list. In the Parameter tab write a conditional ifelse expression using lazy operators.
Type into the Expression field: (im1b1>30)?1:0 If pixel values of band 1 of multiband image 1 are larger than 30 then replace with 1 otherwise replace with 0. Adjusot labels for NoData 3 and undecided 2 he output pixel type is uint8.
]
- Apply a circular majority filter with a radius of 16 pixels. Open the Processing tool OTB --> Learning --> ClassificationMapRegularization. The input classifcation image is the Output Image of the band math operation step. Adjust the Label for NoData 3 and Undecided 2 class. Structuring element radius is 16 pixels. Output pixel type is uint8.
- Open the Processing tool OTB --> Feature Extraction --> BinaryMorphologicalOperation. Choose the Regularized image as input image. The structuring element type is ball with a X and Y radius of 21 pixels. The morphological operation is dilate. Output pixel type is uin8.
- Convert the Feature Output Imge from raster to vector format. Raster --> Conversion --> Polygonize. Select an output file name for vector polygons in format GPKG.
- Change the display of the output vector polygon layer Layer properties --> Symbology --> Single Symbol --> Simple Fill --> Fill Style from Solid to No Brush. Change the Outline color to bright green and increase the Outline width. As background layer use a Standard False Color Composite (RGB = B8,B$,B3) of the Sentinel-2 image.
- Toggle editing
of the resulting vector polygon layer. Activate Select polygons by dragging a box or by single click 
. select all polygons with attribute DN = 0 and delete them by pressing Del on your keyboard. Save your edits 
.
[edit] Creating cloud shadow mask
- Prepare a False color composite (RGB = B8,B4,B3) of the Sentinel-2 image as background layer in the canvas.
- Right click on the cloud polygon layer in the TOC. Choose Export --> Save features as.... Fill the form as shown below.
- Mark the new created polygon layer in the TOC and toggle editing of the vector file. Select all polygons using
or press Ctrl + A. Activate Edit --> Move feature(s) 
. Left click on a selected yellow polygon, move the polygon that shadows are completely covered, and confirm with left click. Save your edits and and stop editing .
[edit] Merge cloud and cloud shadow mask
- Vector --> Data management tools --> Merge vector layers. as Layer to merge select the cloud and cloud_shadow polygon layers. Run.
[edit] Apply the cloud and cloud shadow mask to a Sentinel-2 multiband file
- Create a polygon with the extent of the Sentinel-2 image: Create image extent polygon.
- Vector --> Geoprocessing Tools --> Symmetrical difference. Input layers are the image extent and the merged cloud and cloud_shadow polygons.
- Apply the output vector layer cloud_cloudshadow as mask to the multiband Sentinel-2 image file. Raster --> Extraction --> Clip Raster by Mask Layer.
[edit] Graphical Processing Tool Model
Use a prepared graphical processing toolbox model. Download the file from Stud.IP: S2cloudDetection.model3.
- Open QGIS and go to the Processing Toolbox; if the Toolbox is not opened click Processing --> Toolbox to activate it. Once the Toolbox is opened go to
Models --> Add model to Toolbox.... Open the downloaded model.
- The model should appear in the Toolbox under Models --> S2 --> S2 Cloud Detection. Double click to execute.
- A window showing the different input and outputs should appear, just as if you were using any other geo-algorithm of the Processing Toolbox.
- Just provide a multispectral Sentinel-2 image file with 10 bands with the rank (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12). Click Run.
- Models --> S2 --> S2 Cloud Detection . Right click Edit Model. In the Model Designer double click on the graphical objects to display parameters of processing components. It is easy to change parameter settings of input, output or processing steps. Save changes in the Model Designer Model --> Save Model as....
Find out how to use the graphical modeler to program your own workflow.
| |
 |
