Sentinel-2
- Joe Ferdinando
- Namrata Kolla
- Andrew Howe
Skylight now provides vessel detections using imagery from the European Space Agency’s Sentinel-2A and 2B electro-optical satellites. This provides immense value for countering illegal, unreported, unregulated fishing by detecting smaller vessels, more frequently, and with more detail, more often and across wider areas, including many not currently served by Sentinel-1. We believe this model is the first of its kind, allowing detection of vessels at scale in freely available optical imagery.
Overview
Sentinel-2 is a pair of electro-optical (EO) earth observation (EO) satellites operated by the ESA’s Copernicus program.
Sentinel-2 operates in 13 spectral bands, or individual color channels: https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/resolutions/spectral
Skylight’s model uses natural color images, where red=red, green=green, and blue=blue. Other combinations called “false color” can assign other observed bands, such as infrared, to visible color channels in the image. True color images are taken at 10 meter resolution, and provide substantially more precise detail than Sentinel-1 images at the same resolution.
Schedule
Together, Sentinel-2A and 2B image areas every 5 days. https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/resolutions#:~:text=combined%20constellation%20revisit%20is%205%20days
Skylight does not control the imaging operations of Sentinel-2, and provides vessel detections as bonus data from standard images. Schedules of upcoming acquisitions are publicly accessible here: https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2/acquisition-plans Please note that Sentinel-2A and Sentinel-2B acquisitions are shared as separate KML files.
Sentinel-2A and 2B operate in sun-synchronous orbits, imaging the earth close to 10:30am local time. This is intended to provide relatively consistent image quality and lighting.
Because Sentinel-2 is affected by cloud cover, to avoid excessive false positives from small clouds, we process images where the overall cloud percentage is below 30%.
Latency
In practice, we have observed ESA posts Sentinel-2 L1C image products approximately 4-7 hours after sensing. However, it should be noted that ESA’s standard is delivery in under 24 hours from sensing.
To reduce the time from sensing to vessel detection, Skylight uses Level 1C products. This is a “top of atmosphere” image. For the end user, this means images may sometimes appear foggy or hazy. L1C products are released by ESA as a patchwork or plaid-appearing set of tiles, squares that have a slight overlap at their edges. This overlapping grid is normal, and splits the satellite pass into more manageable chunks.
Capabilities
Skylight’s Sentinel-2 models can detect vessels of all sizes, including those that are quite small, particularly when a wake or bottom disturbance is visible. These small vessel detections may not offer much detail, but give a good indication of vessel presence and direction of travel at the time of the image.
The color, deck configuration, and certain activities like vessel-to-vessel encounters may be clear in Sentinel-2 imagery, particularly for larger vessels.
Unlike Sentinel-1 synthetic aperture radar, Sentinel-2 is affected by cloud cover. However, you will note when exploring the platform, that the model is still capable of detecting larger and more colorful vessels through sparse clouds.
False Positives
While Skylight is committed to constant improvement of our machine learning models, we believe the current outputs are extremely valuable despite the occurrence of false positives. We are aware of, and actively working to omit, the occasional surfacing of wispy clouds, whitecaps, icebergs, and sun glint off the ocean surface. Luckily, most of these false positives are readily identified by human eyes.
Clouds
Sometimes, a cloud will be evident because the detected object is somewhat transparent.
Other times, a small cloud will be evident because it is accompanied by a shadow. These can be found as a dark spot located the same distance as other shadows in the image.
Whitecaps
Whitecaps form from breaking wind-driven waves. The absence of a wake may make it evident that the detection is not a vessel. Visiting SentinelHub and viewing in the infrared false color modes might also clarify that a whitecap is not a vessel. While the white tops of waves may appear similar in the visible portion of the spectrum, versus infrared where sea spray will not be as distinct.
Icebergs
Large, white objects, occasionally with edges that appear light blue, appearing at extreme latitudes. Iceberg detections will usually be evident from their lack of wakes, the distinctive way waves break around them, and typically too large and irregularly shaped to be mistaken for vessels.
Cloud and sea surface diffraction/refraction
Other strange optical phenomena might occur with bright sunlight, reflections, icy clouds, that because of satellite and ocean surface motion, will produce bright rainbow effects that catch the model’s attention, but should be fairly recognizable as not vessels to human eyes.
Aircraft
Sentinel-2 uses a “push broom” sensor with a refracting mirror which causes a slight delay in the time certain color channels are captured and arrive at the sensor. While this is typically not noticeable with stationary or slow-moving objects, the altitude of the aircraft, movement of the aircraft, and movement of the satellite all work together to separate the red, green, and blue separation of the aircraft detected in the image, as seen below.
Ocean Surface Features
Plankton, currents, and litter windrows are sometimes visible and detected in Sentinel imagery.
Specular Reflection
Currently, the model can generate false positives from specular reflection off the ocean surface when alignment is right to reflect sunlight directly into the sensor. Large scale ocean features, such as the curvature of the earth, internal waves, upwelling, and ground swells cause a break-up of the sunlight reflection.
More information and examples can be found here:
https://earthobservatory.nasa.gov/images/81814/ocean-revealed
https://earthobservatory.nasa.gov/images/84333/the-science-of-sunglint