[[searchwing-bilderkennung]]

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
searchwing-bilderkennung [2019/05/30 23:22]
wf68spef
searchwing-bilderkennung [2021/05/31 22:03] (current)
beckmanf added SE2 Projektarbeit Kamera
Line 3: Line 3:
 By analysing the image with we receive from the camera, we can detect boats on the sea. To achieve this, different algorithms and approaches from the image processing and deep learning domain can be used. By analysing the image with we receive from the camera, we can detect boats on the sea. To achieve this, different algorithms and approaches from the image processing and deep learning domain can be used.
  
-===== Approaches ​=====+===== Camera ​=====
  
-==== Boatdetector by using a simple edgedetector and a 2d tracker in world coordinates ====+{{ ::​g2-picam.pdf | S. Keller, K. Dierig, L. Range, "​SearchWing - Messung des Energieverbrauchs eines Raspberry Pi und Charakterisierung des Kamersystems",​ Projektarbeit,​ WS 2020/21}}
  
-Features\\Ìý+===== Compression effects on detection =====Ìý
-Bootdetektion / Bilderkennung\\Ìý+Ìý
-Erkennung von beliebigen Objekten im Wasser\\Ìý+Does JPEG image compression affect detection? SeeÌý
-Klassische Bildverarbeitung\\Ìý+Ìý
-RGB → HSV → Sobel-Kantendetektion je Kanal → Addition → Konturen/​Boundingboxes\\Ìý+[[https://​doi.org/​10.1117/​1.JMI.6.2.027501|Farhad Ghazvinian ZanjaniSvitlana Zinger, Bastian Piepers, Saeed Mahmoudpour,​ Peter Schelkens, and Peter H. N. de With "​Impact of JPEG 2000 compression on deep convolutional neural networks for metastatic cancer detection in histopathological images,"​ Journal of Medical Imaging 6(2), 027501 (24 April 2019). https://​doi.org/​10.1117/​1.JMI.6.2.027501]]Ìý
-Bestimmung der genauen 3D Position der Objekte mittels kalibrierter Kamera\\Ìý+Ìý
-Code für ARM Platform optimiert\\Ìý+===== Detection by using edgedetector and a 2d tracker in world coordinates =====Ìý
-Laufzeit0,8 sek @ 8 MegaPixel(3240×2480 Pixel@ Raspberry Pi 3\\Ìý+Ìý
-Tracking\\Ìý+{{:​ros.jpg?​nolink&​1311x744|ros.jpg}}Ìý
-Wiedererkennung der Boote über mehrere Bilder\\Ìý+Ìý
-Dadurch kann sich ein genaueres Lagebild geschaffen werden\\Ìý+==== Code ====Ìý
-Falsch-Positiv Detektionen werden verringert\\Ìý+Ìý
-Speicherung\\Ìý+[[https://​gitlab.com/​searchwing/​development/​boatdetectorcpp|https://​gitlab.com/​searchwing/​development/​boatdetectorcpp]]
-Abspeicherung der Detektionen als Bilddaten auf dem Flieger\\Ìý+
-GPS Positionsangabe im Bild als EXIF Datenblock\\Ìý+
-Verwendete Software\\Ìý+
-ROS\\Ìý+
-Kommunikation zwischen den einzelnen Modulen\\Ìý+
-Koordinatensystemtransformationen\\Ìý+
-Aufnahme und abspielen von Flugdaten\\Ìý+
-Visualisierung der Detektionsergebnisse\\Ìý+
-MAVROS\\Ìý+
-Zur Kommunikation mit der Fliegerhardware via MAVLINK\\Ìý+
-OpenCV\\Ìý+
-Implementierung der Bootdetektion\\Ìý+
-ARM Compute Library\\Ìý+
-Alternative für ARM optimierte Implementierung der Bootdetektion\\Ìý+
-Ausblick\\Ìý+
-Deep Learning basierte Objekterkennung\\Ìý+
-Testen verschiedener Ansätze\\Ìý+
-Laufzeitevaluation auf den embedded Systemen\\Ìý+
-Programmierung von Interfaces für die Übertragung der Detektionen mittels MAVLINK zur Basisstation+
  
 ==== Algorithms ==== ==== Algorithms ====
  
-=== Assumptions ===+=== Assumptions ​for the algorithms ​===Ìý
 +Ìý
 +  * By flying over the sea with 50-80km/h boats can be assumed to stand almost still in relation to the droneÌý
 +  * Waves appear and dissapear over timeÌý
 +  * Approach to detect boatsÌý
 +      * Detect parts in the image which dont change over timeÌý
 +      * Redect parts by checking the same position in consecutive framesÌý
 +      * If parts look the same, save themÌý
 +      * If something is redetected over 3 frames, we can assume this could be a boat
  
 === Proposal / boat detector === === Proposal / boat detector ===
Line 57: Line 46:
       * Alternative:​ Compute Library for even faster processing       * Alternative:​ Compute Library for even faster processing
   * Runtime   * Runtime
-      * 0,8 sec @ 8 MegaPixel(3240x2480 ​Pixel) @ Raspberry Pi 3+      * 0,8 sec @ 8 MegaPixel(3240×2480 ​Pixel) @ Raspberry Pi 3
  
 {{:​0cae1334da2eb44438be608612fe6628.png}} {{:​0cae1334da2eb44438be608612fe6628.png}}
Line 63: Line 52:
 | |
  
-| | |Ìý+=== Tracking ===
-| | |Ìý+
-| | |+
  
-Proposal ​/ boat detector+  * Redetect boats in consecutive framesÌý
 +      * Get more information/images about the same boatsÌý
 +        * See image belowÌý
 +      * Reduce false-positive rateÌý
 +        * Valid detections only if we redetect the same boatÌý
 +      * AlgorithmsÌý
 +        * Association ProblemÌý
 +          * Euclidean distance based cost matrix between each possible track and detectionÌý
 +          * Solve global neigherest neighbor assignement problem by using Hungarian AlgorithmÌý
 +        * TrackingÌý
 +          * Different trackingmodels possibleÌý
 +          * Constant positionÌý
 +          * Kalman constant positionÌý
 +          * Kalman constant velocity
  
-  * asfÌý+{{:​5ee30e4d3dc8729b80b2ca22e4b9f3af.png}}
-      * asf+
  
-<​code>​Ìý+=== Output ===
- ''​Ìý+
-  * Detection of randomly looking objects in the waterÌý+
-    * Classical image processing ( no deep learning)Ìý+
-    * Processing chain depicted in the imageÌý+
-    * There false positives after inital detectionÌý+
-  * Calculation of the 3D Position of the ObjectsÌý+
-    * In world coordinatesÌý+
-    * By using a camera calibrationÌý+
-  * Code optimized for ARM PlatformsÌý+
-    * By using ARM related flags for OpenCVÌý+
-    * Alternative:​ Compute Library for even faster processingÌý+
-  * RuntimeÌý+
-    * 0,8 sec @ 8 MegaPixel(3240x2480 Pixel) @ Raspberry Pi 3+
  
-''​Ìý+  * Save boat images on harddiskÌý
-</​code>​+  * Metadata for each detection is saved in exif-datablock of each detected boatÌý
 +      * GPS PositionÌý
 +      * GPS TimeÌý
 +  * Visualization of the detection in digikam
  
-||+{{:​digikamdetections.png?​nolink&​878x498|asd}}
  
 | |
  
-|{{:​5ee30e4d3dc8729b80b2ca22e4b9f3af.png}}|+==== Used software ====
  
-| | |+  * ROSÌý
 +      * Module communicationÌý
 +      * Coordinatesystem transformationsÌý
 +      * Recording and playback of datasetsÌý
 +      * VisualisationÌý
 +  * MAVROSÌý
 +      * Communication with the drone via MAVLINKÌý
 +  * OpenCVÌý
 +      * ImageprocessingÌý
 +  * ARM Compute LibraryÌý
 +      * Imageprocessing
  
-  * *+==== Outlook ====
  
-  * *+  * Deep LearningÌý
 +      ​Test different approachesÌý
 +      * Runtime evaluation for embedded hardwareÌý
 +  * Interfaces to send detections via MAVLINK to basestation
  
-==== Imagerecognition by using a OpenCV Haar Cascade Classifier ====+==== Presentation on the topic ====Ìý
 +Ìý
 +[[https://​media.freifunk.net/​v/​35c3oio-77-detection-of-refugee-boats-on-the-mediterranean-with-a-drone-by-using-foss|Click to go to the video]]Ìý
 +Ìý
 +===== Anomaly detection using FFT =====Ìý
 +Ìý
 +{{:​screenshot_from_2020-05-18_09-05-03.png?​600|}}Ìý
 +Ìý
 +Ìý
 +Paper: https://​cloud.hs-augsburg.de/​s/​KNiMLrY5P6H4JXJÌý
 +Ìý
 +===== Imagerecognition by using a OpenCV Haar Cascade Classifier ​=====
  
 I composed a few images to train a Haar Cascade Classifier. The code is over here: I composed a few images to train a Haar Cascade Classifier. The code is over here:
Line 108: Line 121:
 [[https://​git.etech.fh-augsburg.de/​friedrich.beckmann/​bilderkennung|https://​git.etech.fh-augsburg.de/​friedrich.beckmann/​bilderkennung]] [[https://​git.etech.fh-augsburg.de/​friedrich.beckmann/​bilderkennung|https://​git.etech.fh-augsburg.de/​friedrich.beckmann/​bilderkennung]]
  
-Die Bilderkennung markiert ​in dem Bild mögliche BooteDas sieht dann so aus:+The detected boats can be seen in the following image. They are marked by a blue rectangle.
  
-{{:​bilderkennung-beispiel.jpg|Bilderkennung Beispiel}}+{{:​bilderkennung-beispiel.jpg?1101x1044|Bilderkennung Beispiel}}
  
-Und in der Vergrößerung so:+Recified:
  
-{{:​bilderkennung-beispiel-detail.jpg|Bilderkennung Beispiel Detail}}+{{:​bilderkennung-beispiel-detail.jpg?1082x1027|Bilderkennung Beispiel Detail}}
  
-In diesem Beispiel wird das zweite Boot mit der blauen ​Persenning ​nicht erkanntDas Boot darüber ist mit dem blauen Rechteck markiert.+In this example the second boat with Persenning ​does not get detectThe boat above get detected.
  
 ===== Datasets ===== ===== Datasets =====
Line 128: Line 141:
   * [[:​searchwing-flug-virus-20181007|Flug am 7.10.2018 mit der Pipistrel Virus zum Bodensee]]   * [[:​searchwing-flug-virus-20181007|Flug am 7.10.2018 mit der Pipistrel Virus zum Bodensee]]
   * [[https://​captain-whu.github.io/​DOTA/​|DOTA Airial Image Dataset (Wuhan Univ./​Cornell/​DLR)]]   * [[https://​captain-whu.github.io/​DOTA/​|DOTA Airial Image Dataset (Wuhan Univ./​Cornell/​DLR)]]
- 
-===   === 
- 
-\\ 
  
  
  • searchwing-bilderkennung.1559251370.txt.gz
  • Last modified: 2019/05/30 23:22
  • by wf68spef