Real time image denoising on GPU

During a three-month research internship at Karlsruhe Institute of Technology (KIT), I worked on a method to reconstruct images rendered with the path-tracing algorithm on a GPU. I was under the supervision of Johannes Hanika.

The path-tracing algorithm is a Monte Carlo algorithm that can generate images of 3D scenes. In real-time conditions, it is usually not possible to perform enough iterations for the algorithm to converge: the output image is very noisy. It is possible to apply machine learning techniques to reconstruct an estimation of the original image from the noisy image, but these machine learning methods are computationally expensive: it is not yet possible to use them at real-time rates.

During my internship, I first generated several images of 3D scenes using a path-tracer that I created. Two versions of each image are generated: a noisy image with a low number of samples per pixel that could have been obtained at real-time rates, and the final image that is obtained after the convergence of the algorithm. I trained a convolutional neural network with the U-Net architecture to denoise noisy images, and worked on a low-level method to infer the prediction of the network as fast as possible on a Nvidia GPU using its tensor cores efficiently.

Python Data Processing on Supercomputers for Large Parallel Numerical Simulations

Six-month Master's Project carried out in the DataMove team of Inria Grenoble, advised by Bruno Raffin.

As the world's most powerful supercomputers have reached a new computational power threshold, now exceeding the exaflop, it has become possible to run large-scale simulations of complex phenomena with unprecedented precision. These simulations typically follow an iterative process, generating a large amount of data at each iteration. The data produced is then used by scientists for visualization and/or analysis.

In situ analytics consists of performing the analysis on the fly, directly on the nodes where the simulation runs. In situ solutions can be complex to implement since simulation and data analysis programs rely on fundamentally different technologies and paradigms. Simulation codes are performance-critical and are developed using efficient technologies such as C++ with MPI. On the other hand, data analysis pipelines are often designed using high-level programming languages such as Python, relying on a rich ecosystem of efficient libraries.

This Master's project introduces Doreisa (Dask-on-Ray Enabled In Situ Analytics), a scalable and efficient system for in situ analysis of simulation data. Doreisa offers a Dask-based interface, allowing a simple definition of analysis tasks. Experiments show that Doreisa scales well to hundreds of nodes, allowing it to run on the Jean Zay supercomputer.

Fake hands generation

Using a diffusion algorithm, this program generates images of hands. This AI model is a convolutional neural network (CNN) with the U-Net architecture. It is trained on 11K Hands, a dataset of images of hands [1].

The slider above can be used to show images generated during the training of the network. The network is trained during 500 epochs, and six new images are generated every ten epochs. We can see that as the training progresses, the images get increasingly realistic, even if the number and the position of the fingers may be wrong.

Since the network is fully convolutional, it has only access to local information when it generates the image. This can explain why it struggles to build more realistic images. This was expected and obtaining these strange images was actually the goal of the project :)

[1] Mahmoud Afifi, "11K Hands: Gender recognition and biometric identification using a large dataset of hand images." Multimedia Tools and Applications, 2019.