I am mostly interested in images and in their mathematical representation and interpretation. My work covers diverse topics such as statistical analysis of visual tasks, segmentation and recognition algorithms, learning, and recently, also 3D point cloud analysis. A selection of my projects over the years is described in the Research page.
My recent projects are described here. The whole research field was transformed in the last 10 year with the introduction of the powerful deep learning algorithms. While I am not fully converted, part of my work goes in this direction as well.
I would like to develop several topics with the help of research students. Currently I am interested mostly in image patches and in the the information that may be available from them, in tools for interpreting image classification decision, and in in using vision tools for analyzing images of water waves.
Reconstructing water surface waves from polarimetric images
Accurate and cost-effective tools, providing water surface elevation, are of great interest for scientific research and engineering applications. We propose a new tool, based on inferring the surface slope from the polarimetric images using learned deep neural network. The DNN is trained on laboratory-collected supervised datasets of prescribed mechanically generated monochromatic wave trains and tested on a stochastic wave field of JONSWAP spectral shape. The proposed method provides a dense estimate of the water surface.
Recognition under occlusion
Objects in real images are often partially occluded, and deep categorization algorithms usually fail for them. The usual approach is to train on occluded object, but we have shown that keeping the filters narrow, by specialized regularization, is as good.
Increasing cnn robustness to occlusions by reducing filter suppot, E Osherov, M Lindenbaum, CVPR, 550-561, 2017
Generic (non-semantic) segmentation
Learning region representation (with an example to its embedding shown on the right) and adding it to edge data improved the state the art in generic edge detection.
Recognition from partial minimal data
Can we recognize an object from a single small patch? The human visual systems can do it. Here we proceed a computational mechanism that can do it as well with similar characteristics. (In the example the green patch is of the minimal size.)
On the Minimal Recognizable Image Patch, M Fonaryov, M Lindenbaum, ICPR, 6734-6741, 2021.
Strcture from motion with a-contrario
Looking at randomly placed points, the remarkable human visual system can detect a subset of points that coherently move. We analyze this (well known) phenomena and provide quantitative predictions regarding the visibility using the statistical a contrario methodology.
Seeing Things in Random-Dot Videos, T Dagès, M Lindenbaum, AM Bruckstein, ACPR, 195-208, 2020.
3D point cloud data
The 3D modified Fisher vector (3DmFV) representation
Representing a point cloud in a structures way is challenging. We proposed a method that is based on the classical Fisher vector representation of a data set with respect to a distribution, and a deep neural algorithm.
3dmfv: Three-dimensional point cloud classification in real-time using convolutional neural networks, Y Ben-Shabat, M Lindenbaum, A Fischer, IEEE Robotics and Automation Letters 3 (4), 3145-3152, 2018.
The Nesti-Net surface normal estimator
Estimators for the surface normal and the effective scale, that is based on the 3DmFV.
Nesti-net: Normal estimation for unstructured 3d point clouds using convolutional neural networks, Y Ben-Shabat, M Lindenbaum, A Fischer, CVPR, 2019.
Comparing point clouds
An accurate estimator for comparing two point clouds, using the estimated distance from a point to the underlying point cloud surface that is based on local (and hence fast) modeling
DPDist: Comparing point clouds using deep point cloud distance, D Urbach, Y Ben-Shabat, M Lindenbaum, ECCV, 545-560, 2020.
