This PhD project in theoretical neuroscience was conducted as a collaboration between the Hector Institute for AI in Psychiatry at the Central Institute of Mental Health and Heidelberg University. The objective is to develop a tool that enables objective and precise diagnosis and longitudinal tracking of psychiatric disorders by capturing their dynamic and complex nature, with the ultimate goal of enabling personalized, effective therapy and treatment. The work focused on modeling individual cognitive behavior and its neural correlates using longitudinal, high-resolution behavioral and neuroimaging data.

A central component of the project was the design and implementation of a gamified smartphone application containing a cognitive task battery of twelve tasks spanning decision making, executive functioning, relational processing, and working memory. A one-year longitudinal experiment was conducted with five self-recruited patients who engaged with the application on a near-daily basis. This study was carried out in compliance with the German Medical Device Regulation and resulted in dense, individualized behavioral datasets.

Task difficulty and complexity were adapted on a monthly basis according to each participant’s performance, enabling the systematic capture of inter-individual differences and intra-individual learning dynamics. The tasks yielded detailed behavioral measures, including reaction times as well as qualitative and quantitative error patterns across varying task demands. In parallel, five MRI sessions per participant were conducted throughout the study, and the resulting neuroimaging data were preprocessed using adapted and extended versions of established pipelines.

To support model-based analyses, an extensive preprocessing pipeline for smartphone-based cognitive data was developed, allowing flexible encodings of trial structure and task information tailored to different modeling approaches. Building on this, a complete machine learning pipeline for Artificial Neural Network (ANN) training was implemented in TensorFlow. This pipeline supported a wide range of network architectures, loss and activation functions, initialization schemes, regularization strategies, and other tunable hyperparameters. Extensive hyperparameter grid searches were performed across participants and datasets, enabling the successful fitting of highly complex, real-world human cognitive behavior in a multitask setting.

Beyond predictive performance, the internal representations of the trained networks were systematically analyzed from multiple methodological perspectives to reduce model opacity and identify general organizational principles. Network robustness, internal structure, and autocorrelative dynamics were examined in detail. The analyses revealed robust within-subject results and significant between-subject differences. High test performance was consistently associated with biologically plausible internal representations, particularly under moderate to high sparse L1 regularization. Under appropriate conditions, significant similarities emerged between global organizational principles of cognitive artificial neural networks and brain imaging–based biological models, providing converging evidence for shared structural constraints across artificial and biological systems.

https://github.com/oliver-frank/beRNN_v1
https://github.com/oliver-frank/beRNN_ex
Manuscript in preparation.

A study, in which I mainly contributed as a statistical advisor during my time as a research scientist at the Hector Institute for AI in Psychiatry, investigated social interactions between seven psychotic adolescent patients and 18 psychiatric staff members and examined how these interactions were associated with disorder severity over a three-month period. Participants wore time-synchronized sensors that exchanged information every four seconds via radio signals, including participant IDs, proximity, signal strength, and precise timestamps. Based on these data, social interactions were modeled as individual adjacency matrices representing weighted social network graphs, in which nodes corresponded to all involved participants and edges reflected the duration of interactions aggregated over a defined time period. From these networks topological markers of social structure were derived.

To examine the relationship between social interaction variables, network topological measures, and disorder severity, assessed using the Positive and Negative Syndrome Scale (PANSS), were fitted into a linear mixed-effects model with nested participant structure. The analysis revealed that relative daily interaction time, average interaction duration, and the clustering coefficient were each significantly associated with PANSS total scores.

https://academic.oup.com/schizophreniabulletin/article/51/1/236/7634440

The project was conducted in the scope of my Master thesis at TU Berlin at the Institute for Biopsychology and Neuroergonomics as part of the Human Factors Data Science track. The examined data was collected from 14 participants using a virtual reality setup with integrated eye tracking (HTC Vive Pro Eye). Participants observed a virtual grid-based environment in which an avatar moved randomly either toward or away from a fixed object. For each movement, participants were instructed to label the avatar’s behavior as either goal-directed or non-goal-directed.

The objective of the study was to determine whether these subjective labels could be differentiated based on gaze behavior fitted to Hidden Markov Models (HMMs). The gaze behavior was specified separately for each participant and label, and model parameters (Priors, Emissions, Transition Matrices) were initialized based on theoretically motivated assumptions.

Parameter optimization was performed using the Expectation Maximization (EM) algorithm, which iteratively maximizes the marginal likelihood of the model given the observed gaze sequences. In this process, posterior state probabilities (computed from the product of prior probabilities and likelihoods) were used to update model parameters according to the expected transition and emission frequencies until convergence.

Finally, the resulting HMM parameters served as features for classification. A Linear Discriminant Analysis (LDA) was conducted using repeated 5-fold cross-validation (10 repetitions) to distinguish between goal-directed and non-goal-directed evaluations. The best average classification accuracy achieved was .63, indicating a measurable differentiation between the two conditions.

https://github.com/oliver-frank/gzHMM

During a semester project at TU Berlin in my master’s program, I developed and trained a Convolutional Neural Network (CNN) to classify whether a person was wearing a face mask. The model enabled real-time image classification and was integrated with a Raspberry Pi connected to a webcam and an electronic lock. When the camera detected a person wearing a mask, the system unlocked the mechanism, and it automatically locked again when a person was detected without a mask. The trained model was compressed using TensorFlow Lite to enable efficient deployment on the Raspberry Pi, which operated locally under limited computational resources.

https://github.com/oliver-frank/eCNN

As a Data Scientist, I designed and prototyped frontend features for a smartphone-based AR application and analyzed user behavior data-driven to enable more intuitive, efficient, and faster human-device interactions. 3DQR develops immersive and interactive augmented reality (AR) experiences for simulation and training, primarily in the medical and industrial machinery sectors.

https://3dqr.de/